Running webkit layout tests on Linux

1. Compile DumpRenderTree (see LinuxBuildInstructions)
2. Checkout the layout tests
• If you have an entry in your .gclient file that includes "LayoutTests", you may need to comment it out and sync.
• You can run a subset of the tests by passing in a path relative to src/third_party/WebKit/LayoutTests/. For example, run_webkit_tests.sh fast will only run the tests under src/third_party/WebKit/LayoutTests/fast/.
3. When the tests finish, any unexpected results should be displayed.

See Running WebKit Layout Tests for full documentation about set up and available options.

Pixel Tests

The pixel test results were generated on Ubuntu 10.4 (Lucid). If you're running a newer version of Ubuntu, you will get some pixel test failures due to changes in freetype or fonts. In this case, you can create a Lucid 64 chroot using build/install-chroot.sh to compile and run tests.

Fonts

Make sure you have all the necessary fonts installed.

sudo apt-get install apache2 wdiff php5-cgi ttf-indic-fonts \
   msttcorefonts ttf-dejavu-core ttf-kochi-gothic ttf-kochi-mincho \
   ttf-thai-tlwg

You can also just run build/install-build-deps.sh again.

Plugins

If fast/dom/object-plugin-hides-properties.html and plugins/embed-attributes-style.html are failing, try uninstalling totem-mozilla from your system:

sudo apt-get remove totem-mozilla

Running webkit layout tests under valgrind on Linux

As above, but use tools/valgrind/chrome_tests.sh -t webkit instead. e.g.

sh tools/valgrind/chrome_tests.sh -t webkit LayoutTests/fast/

This defaults to using --debug. Read the script for more details.

If you're trying to reproduce a run from the valgrind buildbot, look for the --run_chunk=XX:YY line in the bot's log. You can rerun exactly as the bot did with the commands

cd ~/chromium/src
echo XX >  valgrind_layout_chunk.txt
sh tools/valgrind/chrome_tests.sh -t layout -n YY

That will run the XXth chunk of YY layout tests.

Configuration tips

• Use an optimized DumpRenderTree when rebaselining or running a lot of tests. (bug 8475 is about how the debug output differs from the optimized output.) make -j16 BUILDTYPE=Release DumpRenderTree
• Make sure you have wdiff installed: sudo apt-get install wdiff to get prettier diff output
• Some pixel tests may fail due to processor-specific rounding errors. Build using a chroot jail with 32-bit user space to be sure that your system matches the checked in baselines. You can use build/install-chroot.sh to set up a hardy 32 chroot.

Getting a layout test into a debugger

There are two ways:

1. Run DumpRenderTree directly rather than using run_webkit_tests.sh. You will need to pass some options:
• --no-timeout to give you plenty of time to debug
• the fully qualified path of the layout test (rather than relative to WebKit/LayoutTests).
2. Or, run as normal but with the --startup-dialog flag. That makes DumpRenderTree bring up a dialog before running, which then would let you attach to the process via gdb -p PID_OF_DUMPRENDERTREE .

Using an embedded X server

If you try to use your computer while the tests are running, you may get annoyed as windows are opened and closed automatically. To get around this, you can create a separate X server for running the tests.

1. Install Xephyr (sudo apt-get install xserver-xephyr)
2. Start Xephyr as display 4: Xephyr :4 -screen 1024x768x24 -ac
3. Run the layout tests in the Xephyr: DISPLAY=:4 run_webkit_tests.sh

Xephyr supports debugging repainting. See the Xephyr README for details. In brief:

1. XEPHYR_PAUSE=$((500*1000)) Xephyr ...etc... # 500 ms repaint flash
2. kill -USR1 $(pidof Xephyr)

If you don't want to see anything at all, you can use Xvfb (should already be installed).

1. Start Xvfb as display 4: Xvfb :4 -screen 0 1024x768x24 -ac
2. Run the layout tests in the Xvfb: DISPLAY=:4 run_webkit_tests.sh

Tiling Window managers

The layout tests want to run with the window at a particular size down to the pixel level. This means if your window manager resizes the window it'll cause test failures. This is another good reason to use an embedded X server.

xmonad

In your .xmonad/xmonad.hs, change your config to include a manageHook along these lines:

test_shell_manage = className =? "Test_shell" --> doFloat
main = xmonad $
  defaultConfig
    { manageHook = test_shell_manage <+> manageHook defaultConfig
    ...