We provide a docker image that contains the script checkers in Vibranium and all the applications we use in the paper. Below are steps to run it and reproduce the experiments on docker.

Pull and run the docker image

  • Login to you docker hub account and pull the docker image:
    $ docker pull managedataconstraints/data-constraints-analyzer:first
    
    $ docker run -it -p 127.0.0.1:3000:3000 managedataconstraints/data-constraints-analyzer:first
    

Reproduce experiments in the paper.

  • Figure2
    $ cd main278/formatchecker/ 
    $ ruby run_apps.rb --tva
    

    The output will be stored at the ../log/ouput_#{app_name}.log file for each corresponding app.

    The data is also stored in the excel file.

  • Table 4: Data constraints in web applications
    $ ruby run_apps.rb  --latest-version
    

    The data is presented in the excel file under the latest-version #constraints tab.

  • Table 5: # Constraints in DB but not in Application

    Go to the main278/formatchecker/ script folder and run:

    $ ruby run_apps.rb -s 
    

    Extract the results from the log file:

    $ grep “db_present_model_absent” log/output.log
    

    Detailed data is presented in the excel file.

  • Table 6: # Constraints in Application but not in DB

    Go to the main278/formatchecker/ script folder and run:

    $ ruby run_apps.rb -s 
    

    Extract the results from the log file:

    $ grep “model_present_db_absent” log/output.log
    

    Detailed data presented in the excel file.

  • Table 7: Top 5 popular types of different layer

    Go to the main278/formatchecker/ script folder and run:

    $ ruby run_apps.rb  --api-breakdown
    

    This will generate a single log file for each application under log/api_breakdown_#{app_name}.log

    $ ruby api_breakdown_spread_sheets.rb 
    

    The summarized breakdown will be written to output/api_total_breakdown.xlsx.

    Details presented in the summary tab of the excel file

  • Table 8: app versions vs constraint changes

    Details presented in the constraint-evolution tab of the excel file

    Go to the main278/formatchecker/ script folder and run:

    $ ruby run_apps.rb --tva 
    

    The output will be stored at the ../log/ouput_#{app_name}.log file for each corresponding app.

  • Table 9: Data-constraint issues in real-world apps

    Raw issues in the issue file

    Go to the main278/formatchecker/ script folder and run:

    $ cd issues
    
    $ python extract_breakdown.py
    

    The breakdown will be both printed out on the console and stored in the output.xlsx file.

  • Table 10: # Mismatch constraints

    Details presented in the excel file

    Go to the main278/formatchecker/script folder and run:

    $ ruby run_apps.rb -s 
    

    Then, to extract the results from the log file:

    $ grep “mismatch_constraint” log/output.log
    
  • User study

    results can be accessed through google drive or here

    User study questionnaire can be found here.

    Both file can be found under the folder main278/user-study on the docker image.

  • The table in the Discussion section of issues in Django

  • Issues we report to developers and their feedback (Section 7)

  • Source Code for better error message gem.

Apply on other applications

  • Prepare your application

    Go to main278/formatchecker/apps/ folder, clone your application there.

    $ git clone git_repo_link app_name
    

    It’s not required to put the file under th app folder, just for convenience to next step.

  • Run our scripts.

    Go to main278/formatchecker/constraint_analyzer folder.

    $ ruby main.rb -a ../apps/app_name -h