My favorites | Sign in
Project Home Downloads Wiki Issues Source
Search
for
TestingServiceAPI  
API Standards for The Testing Service
Featured
Updated Apr 18, 2011 by PatMee...@gmail.com

Introduction

These are the interfaces that need to be supported by a service that implements Web Performance Testing.

Operations

These are the operations that the interfaces will need to support

Location/Configuration Enumeration

We need a way to query for the different test machine configurations and capabilities. This will need to be a filterable query for a flat list rather than a hierarchy because some models may make sense to say "give me all of the physical locations" and then list the configurations in each location but it is also possible to build a UI that would want the list of tester pools that have a particular browser or that are on a particular carrier.

There will need to be a collection of entry-points: - Query for the list of parameters that can be used for filtering and their possible values - Query for the actual tester pools (and identify fields of data you'd like back about each pool)

Script Registration

For complex or larger test scripts there needs to be an interface for registering a script bundle (including any referenced external files) and referencing it in the actual test request.

Method: POST

URI: script

Parameters:

Response:

Example:

Request

POST http://www.webpagetest.org/wtf/test
multipart-form
script_file=...

Response

Test Submission

The ability to submit a test request to a given tester pool/configuration. If a test can be completed synchronously then the results are returned directly in the response, otherwise the status of the test is returned (the response is the same as the Status Check API).

Not all features will be supported by all services so be sure to check the capabilities of a service before assuming it can support the features you are requesting (particularly when requesting multiple iterations, urls, etc).

Method: POST

URI: test

Parameters:

  • url: URL to be tested (if the test is for an url). If multiple URLs are supported in a single test they should be tab-delimited.
  • config: Configuration to use for the test (see Location/Configuration enumeration above). If multiple configurations are supported for a single test they should be tab-delimited.
  • connection: Predefined connectivity for the given configuration or or custom parameters if supported. For custom configurations, bandwidth is in Kbps (1024 Bps), latency is in ms and packet loss is in percent and each supports fractional (decimal) values. The values are preceded with d for download bandwidth, u for upload bandwidth, l for round-trip latency and p for packet loss (additional parameters may be avaliable in the future).
    • custom:<parameters>: custom:d1500,u768,l50,p0
  • iterations: Number of times to run the test
  • end: End condition for the test
    • onload: Test ends when the browser indicates the page has finished loading
    • activity: Test ends when the browser has stopped requesting resources for a defined period (2 seconds?)
  • script: Script to use for the test (sent with the test request). The actual scripting language is vendor-specific
  • script-id: Script to use for the test (reference to script uploaded separately)
  • screen-shots: Grab screen shots of the test. Images will be included in the HAR file, base-64 encoded. Can be a comma-delimited list of:
    • all: All available screen shots
    • error: Only when there is a test error
    • start-render: Initial render of the test
    • end: The final image at the end of the test
    • onload: The page at the time of the onLoad event
  • video: Capture video of the page load (output format is vendor-specific). If multiple video modes are available we can extend the values to support them.
    • 1: Capture video
    • 0: Do not capture video
  • login: Authentication (if supported). User name and password need to be supplied separately. It is recommended that https be used for the API for any tests where credentials are supplied.
    • basic: HTTP-basic auth
  • login-user: User name for authentication
  • login-pass: Password for authentication
  • post-result: Post the result to the supplied URI. This will allow for direct integration with the data API's. The result will be deleted locally upon successful post. Multiple URIs may be supported by tab-delimiting them.
  • beacon-result: Send an abridged beacon to the supplied URI with summary information (and retain the test for a short period to allow for asynchronous fetching of the result). Multiple URIs may be supported by tab-delimiting them.

Response:

Example:

Request

POST http://www.webpagetest.org/wtf/test
url=...

Response

Check Status/Retrieve Test Result

Returns the details of a test. If the test isn't complete yet then the status would be returned

Method: GET

URI: test/<id>

  • id: Test ID returned when the test was submitted

Parameters:

Response:

Example:

Request

GET http://www.webpagetest.org/wtf/test/20110325_XX_YYYSA

Response

Cancel/Delete Test Result

The testing service isn't expected to provide long-term storage of results. At a minimum you'll need to be able to cancel a pending test but it wouldn't hurt to allow for an explicit delete of ephemeral results as well.

Method: DELETE

URI: test/<id>

  • id: Test ID returned when the test was submitted

Parameters:

Response:

Example:

Request

DELETE http://www.webpagetest.org/wtf/test/20110325_XX_YYYSA

Response

Notification/Beaconing of Results

In addition to polling for status/result we should allow for the testing service to push results (or just notification of results being available) using a standard interface (HTTP Post to a supplied URL).

Comment by markp...@gmail.com, Mar 27, 2011

I would like to propose to support both "real-time" tests and "polling for result" type of tests. In a real-time test the test result will be part of the response of the "Test submission" call. So maybe we can add a flag in "Test submission" for that (or make it part of the response for that call).

Mark

Comment by adrian...@gmail.com, Apr 7, 2011

Not an expert on REST interfaces, but from what I've seen aren't id's typically passed in via the URI? Eg:

GET http://www.webpagetest.org/wtf/test/20110325_XX_YYYSA

Comment by ian.t.wi...@gmail.com, Apr 8, 2011

I also agree that we should distinguish between time sensitive/real time tests and best effort requests. There will be some requests that if that can't be completed right away aren't needed. In theory on the back-end these requests will be handled by different equipment and possibly billed for at a different rate by providers.

This could be done with a separate call or simply a variable though.

This also raises the issue of scheduled tests, i.e. do a test at or as close to a specific day and time in the future. Taking this a step further there is the issue of scheduling recurring tests. Sounds like we need a position on whether or not that is in scope here.

Finally perhaps it's best to tackle this near the end, but there is the small matter of error handling.

Ian

Comment by project member PatMee...@gmail.com, Apr 18, 2011

@ian, I'd prefer not to tackle scheduled tests directly as part of this API (or at least this iteration) because I think the consumption model is different enough that it justifies it's own interface. I expect you'd want to managed the schedules and "jobs" but the result access would be more through something like the data api. For most users I expect they'll be using a vendor's UI for managing the schedules and then use the data API as the integration point for the results.

@adrian - thx, I'll adjust the interface appropriately.


Sign in to add a comment
Powered by Google Project Hosting