My favorites | Sign in
Project Home Wiki Issues Source
READ-ONLY: This project has been archived. For more information see this post.
Search
for
ExperimentResults  
index
Updated Nov 7, 2011 by Standing...@gmail.com

Introduction

We did an experiment to validate our approach. First, we employed a real synchronization for runtime model, which could keep a JOnAS system model (A-side) and a C2 architecture model (B-side) consistent, to constructed a pair of synchronized models. Then, five postgraduate students in our lab, who have at least two years experiences in software modeling, were asked to modify the B-side model freely. Then, we check if the modified model can be synchronized successfully.

Four situations are as follows: 1) the modified model does not violate any sc-constraints and can be synchronized correctly; 2) the modified model does not violate any sc-constraints but it cannot be synchronized; 3) the modified model violates some sc-constraints, yet it can be synchronized when all inconsistencies are fixed; 4) the modified model violates sc-constraints, and it cannot be synchronized even though all inconsistencies were fixed. Case 2) and case 4) mean that our approach fails to find out all inconsistencies.

We totally collected 153 modified B-side models, where 7 models fall in case 2) and 4 models in case 4). It means that only 11 updates didn’t our approach detect all inconsistencies. Hence, the experiment result implied that in most cases (92.8% in this experiment) our approach is sufficient for detecting all inconsistent updates.

Details

5 persons took part in this experiment and their test data are listed below.

Person 1:

Modified : 31 Fall in case 2) : 1 Fall in case 4) : 1

Testdata1

Person 2:

Modified : 31 Fall in case 2) : 0 Fall in case 4) : 1

Testdata2

Person 3:

Modified : 31 Fall in case 2) : 3 Fall in case 4) : 1

Testdata3

Person 4:

Modified : 30 Fall in case 2) : 2 Fall in case 4) : 0

Testdata4

Person 5:

Modified : 30 Fall in case 2) : 1 Fall in case 4) : 1

Testdata5

Powered by Google Project Hosting