Objective

ScriptCover is a Javascript coverage analysis tool which is implemented as a Chrome extension. It provides line-by-line (and per instructions) Javascript code coverage statistics for web pages in real time without any user modifications required. The results are collected when the page loads and then when users interact with the page. These results can be viewed with general overall coverage scores and for each external/internal script by highlighting those lines that were run.

Background

The tool’s broad scope is attractive for both testers and developers by providing data used in a variety of common techniques for debugging, analysis, and exploration. The most useful applications are the following: * Manual testing. The coverage results provided by the tool help one understand how much of the application has been covered while exploratory testing. * Automated testing. Javascript coverage by automated integration UI tests are hard to get. ScriptCover can be used to collect these statistics automatically. The use of Javascript coverage in automated UI tests such as WebDriver provides some sort of indication of how well an application is tested. * Debugging issues and code understanding. ScriptCover provides a mechanism for dynamic tracking which portions of JavaScript code have been executed in a given page. The tool can be also used for identifying dead/obsolete code and verifying that code does not ship with hidden or deprecated functionality. * Code reduction/isolation. ScriptCover helps developers to isolate code that contributes to a bug. The detailed coverage report for internal and external scripts is useful for identifying an application’s components and functionalities that were not covered while testing and can be safely removed while debugging.
In comparison with several other tools, https://code.google.com/p/script-cover/'>ScriptCover has a number of advantages:

1. because of built-in code formatting, in most cases it provides coverage for logical JavaScript statements rather than physical lines of original code;
   
2. it instruments not only external scripts, but internal scripts as well;
   
3. user doesn't have to save web pages locally in a predefined directory and then modify them (e.g. changing absolute URLs to external scripts to relative URLs).
   

Overview

ScriptCover consists of two main components - Google Chrome extension and instrumentation HTTP proxy server: * Chrome extension is the client part of the tool which is implemented in JavaScript and has three parts: background script (including popup), content script and the script which is injected into the inspected web page. * Proxy server is an optional component. It requires the use of a special version of the extension that is compatible with the proxy. The proxy server and compatible Chrome extension will be checked in to the open source project later.
In its basic (non-proxy) implementation, https://code.google.com/p/script-cover/'>ScriptCover performs code analysis by means of introducing markers inside the code loaded in the browser.
As a result, part of JavaScript code from a page is executed twice: once as intended to be used by the page and once as instrumented by the tool.
Because of this there may be cases when the tool doesn't provide absolute accuracy.
However, a proxy implementation of https://code.google.com/p/script-cover/'>ScriptCover extension allows the instrumenting of code before it's loaded in the browser.
The proxy approach eliminates the first execution of non-instrumented JavaScript code, and only the instrumented bits are executed within the page.
This provides accurate coverage statistics for internal and external Javascript code.

Detailed Design

Chrome extension component

Injected script

This script is injected into the webpage and executed in its context, having access to all variables and functions of the page. Because coverage data is initially collected in the context of the webpage, injected script is required to send this data to background script.
http://code.google.com/p/script-cover/source/browse/src/backgroundInteraction.js'>backgroundInteraction.js - Implements communication between background and content scripts. The coverage data is put into the special hidden <div> container in the web page. After that, event notifications are sent to the content script which extracts data from the hidden <div>.

Content script

Content script is executed on every page, but it is unable to use page's variables because of the "isolated worlds" concept. Nevertheless, it has access to page's DOM.
http://code.google.com/p/script-cover/source/browse/src/scriptLoader.js'>scriptLoader.js - Contains functions that wire listeners for event notifications produced by injected scripts, insert injected scripts and initialise extension-wide variables.
http://code.google.com/p/script-cover/source/browse/src/instrumentation.js'>instrumentation.js - Implements loading of external scripts, instrumentation and generation of an accessory/helper script to collect coverage. It loads all external scripts (<script src=”...”>) (sending request to background page, which is able to load external files via XMLHttpRequest), sets their innerHTML property to their content, whilst the src attribute is removed. All scripts are formatted and instrumented using third-party library UglifyJS by Mihai Bazon (see details of instrumentation below). Finally, the accessory/helper script that initialises a data structure composed of data about the script objects and coverage is generated and added to the webpage.
http://code.google.com/p/script-cover/source/browse/src/startTool.js'>startTool.js - Initiates inserting of injected scripts into the web page. After the injected scripts are loaded, instrumentation is started.

Detailes

Instrumentation is performed in the following way:
A “semantic block” is defined as any part of code that is body of function, or body of cycle, or body of “if” or “else” statements. The whole script is a semantic block. Blocks can be nested.
For semantic block #i in script with index index, labels are added:

"//BRT_BLOCK_BEGIN:i" at its beginning;

"//BRT_BLOCK_END:i" at its end;

also the instrumentation instruction is added at its beginning:

"scriptObjects[index].executedBlock[i] = (scriptObjects[index].executedBlock[i] ? scriptObjects[index].executedBlock[i] + 1 : 1);".

This instruction checks whether the execution counter for this block was initialized. If it was, it increments, otherwise sets to 1. When a block of code is executed in a browser, corresponding execution counter is updated.

For example:

if (a == 0) {

 //BRT_BLOCK_BEGIN:12

 scriptObjects[2].executedBlock[12] = (scriptObjects[2].executedBlock[12] ? scriptObjects[2].executedBlock[12] + 1 : 1);

 b = 1;

 c = 2;

 //BRT_BLOCK_END:12

}

Background script

Background script is a script that is executed on the extension’s background page. As only the background page of Chrome extension can perform cross-domain XMLHttpRequest, it handles all requests to other domains (for example, loading of external JavaScript files).
http://code.google.com/p/script-cover/source/browse/src/showCoverageHelper.js'>showCoverageHelper.js - Performs aggregation of coverage data and constructs the coverage report. There is a full coverage report which opens in a new browser tab, or short report displayed in Chrome extension popup.

Coverage data structure has the following format:

• scriptInfo is array-like object of tabScriptInfo
  • tabScriptInfo is array of pageScriptInfo
    • pageScriptInfo is object with fields:
      
      • url (string) is URL of the webpage
        
      • scriptObjects is array-like object of scriptObject
        • scriptObject is object with fields:
          
          • src (string) - source of the script
            
          • counter (number) - number of lines in beautified script
            
          • commands (1-based array of strings, the last element has index counter) - lines of formatted script (in URL-escaped form)
            
          • instrumented (string) - whole instrumented body of script
            
          • blockCounter (number) - number of semantic blocks
            
          • executedBlock (1-based array of numbers, the last element has index blockCounter) - execution counters for blocks

In the full coverage report, executed lines of code are colored green and the execution counter is displayed for every line of code. User can show and hide report for every script clicking on its header.

http://code.google.com/p/script-cover/source/browse/src/background.js'>background.js - Contains functions that load external files on demand, handle requests coming from content scripts, initialise extension-wide objects.
http://code.google.com/p/script-cover/source/browse/src/popup.js'>popup.js - Initializes popup instance, displays total coverage statistics on Chrome extension badge, displays short coverage report in popup on user demand, initiates displaying of full report if user clicks on appropriate link. It also provides possibility to track/untrack selected scripts and recalculate total coverage for only tracked scripts. The coverage statistics are received by popup from background script.


Proxy component (to be checked in later)

HTTP proxy component intercepts all communication between the web browser and the web server, performing instrumentation on-the-fly. Browser can be configured to use this proxy and then transparently get instrumented scripts. Below is the scheme of instrumentation process when a JavaScript file is requested by the browser:

• An HTTP request from the web browser is intercepted by the proxy.
  
• The proxy creates a new HTTP request and sends it to the destination.
  
• The website responses with the corresponding file.
  
• The proxy determines whether or not the file contains JavaScript and uses the parser to parse the source code.
  
• The instrumentation code is injected into original script.
  
• The resulting instrumented JavaScript file is sent to the web browser.