My favorites | Sign in
Project Home Downloads Wiki Issues Code Search
New issue   Search
for
  Advanced search   Search tips   Subscriptions
Issue 98636: Express all test dependencies to run it as a target in gyp
2 people starred this issue and may be notified of changes. Back to list
 
Project Member Reported by maruel@chromium.org, Sep 29, 2011
Some work has already be done towards that but nothing is in production at the moment.

For each test executable that needs to be run on testers, note all the input files that needs to be present on the tester to recreate the tree.

This includes sources files (like test data) and build artifact. For example, browser_tests needs chrome to be present.

Once this is done and stored inside gyp, this can be used as a building block for further build infrastructure improvement.
Sep 29, 2011
#1 maruel@chromium.org
(No comment was entered for this change.)
Blocking: 98639
Oct 20, 2011
#2 maruel@google.com
(No comment was entered for this change.)
Owner: maruel@chromium.org
Jan 6, 2012
#3 maruel@chromium.org
(No comment was entered for this change.)
Blocking: 109173
Jan 7, 2012
#4 maruel@chromium.org
(No comment was entered for this change.)
Labels: -InstantCQ TaskForce-DevLatency
Jan 9, 2012
#5 maruel@chromium.org
More background.  The idea is to implement the step as a gyp target. For example, add
ui_tests_run which defines all the needed dependencies to run ui_tests outside of a checkout.

In this particular case, ui_tests_run needs to depend on chrome_run, since it needs to be able to start chrome executable.

A *_run target is basically a target:None gyp target with a single rule that starts a python script. The python script does:
- For each dependency, recursively map the file and directories as symlink to a temporary directory.
- Run the test
- Tear down the temporary directory

So the python script as on its command line all the files to map.

Having this as a gyp target has a few benefits:
- The target will behave the same on the slave or on the dev machine
- The dev can configure this at the same place as other build related dependencies.
- This can be easily reused for  issue 98640 .
Cc: szager@chromium.org csharp@chromium.org
Jan 9, 2012
#6 maruel@chromium.org
(No comment was entered for this change.)
Blocking: 98640
Jan 9, 2012
#7 maruel@chromium.org
(No comment was entered for this change.)
Blocking: 98637
Jan 11, 2012
#8 maruel@google.com
(No comment was entered for this change.)
Summary: Express all test dependencies to run it as a target in gyp
Mar 6, 2012
#9 lafo...@google.com
Available + Owner == Default to Assigned
Status: Assigned
Mar 14, 2012
#11 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=126660

------------------------------------------------------------------------
r126660 | maruel@chromium.org | Wed Mar 14 09:35:47 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate_test.py?r1=126660&r2=126659&pathrev=126660
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate.py?r1=126660&r2=126659&pathrev=126660

Converts absolute paths in the command to relative path.

It is a workaround for 'make' which is sending absolute paths for actions. To be
reverted once all build tools are verified to not use absolute paths.

R=rogerta@chromium.org
BUG=98636
TEST=


Review URL: http://codereview.chromium.org/9704001
------------------------------------------------------------------------
Mar 15, 2012
#12 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=127100

------------------------------------------------------------------------
r127100 | maruel@chromium.org | Thu Mar 15 21:18:23 PDT 2012

Changed paths:
 A http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_child_process.py?r1=127100&r2=127099&pathrev=127100
 M http://src.chromium.org/viewvc/chrome/trunk/src/testing/test_env.py?r1=127100&r2=127099&pathrev=127100
 A http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=127100&r2=127099&pathrev=127100
 D http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/strace_inputs.py?r1=127100&r2=127099&pathrev=127100

Implement dtrace log parsing for MacOSX.

Rename strace_inputs.py to trace_inputs.py since it's now using both strace and
dtrace, depending on the platform.

R=rogerta@chromium.org
BUG=98636


Review URL: http://codereview.chromium.org/9706058
------------------------------------------------------------------------
Apr 29, 2012
#13 maruel@chromium.org
(No comment was entered for this change.)
Blockedon: 125513
Apr 29, 2012
#14 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134507

------------------------------------------------------------------------
r134507 | maruel@chromium.org | Sun Apr 29 21:16:36 PDT 2012

Changed paths:
 A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=134507&r2=134506&pathrev=134507
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=134507&r2=134506&pathrev=134507

Add unit_tests_run that (mostly) works for all of linux, OSX and Windows.

The dependencies were retrieved with trace_inputs.py and merged with merge_isolate.py.

Manual edits:
- Added <(EXECUTABLE_SUFFIX) to two executables: unit_tests and xdisplaycheck.
- Removed testserver.log and unit_tests.log.
- Added copyright.
- Removed *.db-journal files.

R=rogerta@chromium.org
BUG=98636
TEST=

Review URL: http://codereview.chromium.org/9693023
------------------------------------------------------------------------
Apr 29, 2012
#15 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134508

------------------------------------------------------------------------
r134508 | brettw@chromium.org | Sun Apr 29 22:06:57 PDT 2012

Changed paths:
 D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=134508&r2=134507&pathrev=134508
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=134508&r2=134507&pathrev=134508

Revert 134507, broke the main builders:
http://build.chromium.org/p/chromium/builders/Linux%20x64/builds/28236/steps/compile/logs/stdio

- Add unit_tests_run that (mostly) works for all of linux, OSX and Windows.

The dependencies were retrieved with trace_inputs.py and merged with merge_isolate.py.

Manual edits:
- Added <(EXECUTABLE_SUFFIX) to two executables: unit_tests and xdisplaycheck.
- Removed testserver.log and unit_tests.log.
- Added copyright.
- Removed *.db-journal files.

R=rogerta@chromium.org
BUG=98636
TEST=

Review URL: http://codereview.chromium.org/9693023

TBR=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/10270005
------------------------------------------------------------------------
Apr 30, 2012
#16 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134580

------------------------------------------------------------------------
r134580 | maruel@chromium.org | Mon Apr 30 13:14:13 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate_smoke_test.py?r1=134580&r2=134579&pathrev=134580
 M http://src.chromium.org/viewvc/chrome/trunk/src/build/common.gypi?r1=134580&r2=134579&pathrev=134580
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate.py?r1=134580&r2=134579&pathrev=134580

Add --mode=noop so progress can be made while the builders are being reconfigured

R=rogerta@chromium.org
BUG=98636
TEST=committing r134507 again would work


Review URL: http://codereview.chromium.org/10261010
------------------------------------------------------------------------
Apr 30, 2012
#17 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134604

------------------------------------------------------------------------
r134604 | maruel@chromium.org | Mon Apr 30 14:46:58 PDT 2012

Changed paths:
 A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=134604&r2=134603&pathrev=134604
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=134604&r2=134603&pathrev=134604

Add unit_tests_run that (mostly) works for all of linux, OSX and Windows.

Revert r134508 "Revert 134507, broke the main builders:"
This reapplies r134507.

The dependencies were retrieved with trace_inputs.py and merged with merge_isolate.py.

Manual edits:
- Added <(EXECUTABLE_SUFFIX) to two executables: unit_tests and xdisplaycheck.
- Removed testserver.log and unit_tests.log.
- Added copyright.
- Removed *.db-journal files.

R=rogerta@chromium.org
BUG=98636
TEST=


Review URL: http://codereview.chromium.org/10271012
------------------------------------------------------------------------
Apr 30, 2012
#18 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134607

------------------------------------------------------------------------
r134607 | kkania@chromium.org | Mon Apr 30 15:06:18 PDT 2012

Changed paths:
 D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=134607&r2=134606&pathrev=134607
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=134607&r2=134606&pathrev=134607

Revert 134604 - Add unit_tests_run that (mostly) works for all of linux, OSX and Windows. Reverted because of build failures.
E.g.:
make: *** No rule to make target `chrome/test/data/osdd/bogus.xml', needed by `out/Release/unit_tests.results'

Revert r134508 "Revert 134507, broke the main builders:"
This reapplies r134507.

The dependencies were retrieved with trace_inputs.py and merged with merge_isolate.py.

Manual edits:
- Added <(EXECUTABLE_SUFFIX) to two executables: unit_tests and xdisplaycheck.
- Removed testserver.log and unit_tests.log.
- Added copyright.
- Removed *.db-journal files.

R=rogerta@chromium.org
BUG=98636
TEST=


Review URL: http://codereview.chromium.org/10271012

TBR=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/10274016
------------------------------------------------------------------------
Apr 30, 2012
#19 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134661

------------------------------------------------------------------------
r134661 | maruel@chromium.org | Mon Apr 30 18:42:03 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/tools/build/masters/master.chromium/master_full_cfg.py?r1=134661&r2=134660&pathrev=134661
 M http://src.chromium.org/viewvc/chrome/trunk/tools/build/masters/master.chromium/master_linux_cfg.py?r1=134661&r2=134660&pathrev=134661

Disable all isolate step logic on full builders.

Remove crosstool_prefix and arm_gclient_env left over.

R=nsylvain@chromium.org
BUG=98636
TEST=Builders would compile fine r134507

Review URL: http://codereview.chromium.org/10263009
------------------------------------------------------------------------
Apr 30, 2012
#20 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134667

------------------------------------------------------------------------
r134667 | maruel@chromium.org | Mon Apr 30 19:16:49 PDT 2012

Changed paths:
 A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=134667&r2=134666&pathrev=134667
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=134667&r2=134666&pathrev=134667

Add unit_tests_run that (mostly) works for all of linux, OSX and Windows.

Revert r134607 "Revert r134508 "Revert 134507, broke the main builders:""

Overall, this reapplies r134507.

The dependencies were retrieved with trace_inputs.py and merged with merge_isolate.py.

Manual edits:
- Added <(EXECUTABLE_SUFFIX) to two executables: unit_tests and xdisplaycheck.
- Removed testserver.log and unit_tests.log.
- Added copyright.
- Removed *.db-journal files.

TBR=rogerta@chromium.org
BUG=98636
TEST=


Review URL: http://codereview.chromium.org/10269019
------------------------------------------------------------------------
Apr 30, 2012
#21 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134670

------------------------------------------------------------------------
r134670 | estade@chromium.org | Mon Apr 30 20:55:22 PDT 2012

Changed paths:
 D http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=134670&r2=134669&pathrev=134670
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=134670&r2=134669&pathrev=134670

Revert 134667 - Add unit_tests_run that (mostly) works for all of linux, OSX and Windows.
it broke the linux x64 bulder.

> Revert r134607 "Revert r134508 "Revert 134507, broke the main builders:""

> Overall, this reapplies r134507.

> The dependencies were retrieved with trace_inputs.py and merged with merge_isolate.py.

> Manual edits:
> - Added <(EXECUTABLE_SUFFIX) to two executables: unit_tests and xdisplaycheck.
> - Removed testserver.log and unit_tests.log.
> - Added copyright.
> - Removed *.db-journal files.

> TBR=rogerta@chromium.org
> BUG=98636
> TEST=


> Review URL: http://codereview.chromium.org/10269019

TBR=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/10264027
------------------------------------------------------------------------
May 1, 2012
#22 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=134689

------------------------------------------------------------------------
r134689 | maruel@chromium.org | Tue May 01 06:32:12 PDT 2012

Changed paths:
 A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=134689&r2=134688&pathrev=134689
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=134689&r2=134688&pathrev=134689

Add unit_tests_run that (mostly) works for all of linux, OSX and Windows.

Revert r134670 "Revert 134667 "Revert r134508 "Revert 134507, broke the main builders:""

This CL's going to get in at whatever cost. I moved more optional files to be untracked.

The dependencies were retrieved with trace_inputs.py and merged with merge_isolate.py.

Manual edits:
- Added <(EXECUTABLE_SUFFIX) to two executables: unit_tests and xdisplaycheck.
- Removed testserver.log and unit_tests.log.
- Added copyright.
- Removed *.db-journal files.

TBR=rogerta@chromium.org
BUG=98636
TEST=

Review URL: https://chromiumcodereview.appspot.com/10263018
------------------------------------------------------------------------
May 11, 2012
#23 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=136601

------------------------------------------------------------------------
r136601 | maruel@chromium.org | Fri May 11 10:25:02 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate_smoke_test.py?r1=136601&r2=136600&pathrev=136601
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=136601&r2=136600&pathrev=136601
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate.py?r1=136601&r2=136600&pathrev=136601
 A http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_test_cases.py?r1=136601&r2=136600&pathrev=136601
 A http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/list_test_cases.py?r1=136601&r2=136600&pathrev=136601
 A http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/read_trace.py?r1=136601&r2=136600&pathrev=136601

Add scripts to list or trace all test cases in a gtest executable.

list_test_cases.py quickly outputs the test cases in a gtest executable. For browser_tests is takes ~100ms.

trace_test_cases.py traces each test individually to be able to generate pretty graphs.

read_trace.py is a debugging tool to read a single trace.

Refactor trace_inputs.py to make it possible. I changed the semantic for it so that stdin is closed and stderr is redirected to stdout. I also have the logger not look at the logging level to determine if output should be shown.

R=rogerta@chromium.org
BUG=98636
TEST=

Review URL: https://chromiumcodereview.appspot.com/10377105
------------------------------------------------------------------------
May 11, 2012
#24 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=136655

------------------------------------------------------------------------
r136655 | maruel@chromium.org | Fri May 11 13:44:35 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=136655&r2=136654&pathrev=136655

Add .clean_trace() to the APIs to delete the stale traces.

It'll become soon more complex for strace so it's preferable to have a proper
interface.

R=mad@chromium.org
BUG=98636
TEST=

Review URL: https://chromiumcodereview.appspot.com/10381112
------------------------------------------------------------------------
May 13, 2012
#25 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=136814

------------------------------------------------------------------------
r136814 | maruel@chromium.org | Sun May 13 18:17:19 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate_smoke_test.py?r1=136814&r2=136813&pathrev=136814
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs_test.py?r1=136814&r2=136813&pathrev=136814
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/isolate.py?r1=136814&r2=136813&pathrev=136814

Fix mode=hashtable without --outdir to create the directory at the right place.

Improve isolate_smoke_test.py to be more representative of what's happening in
chromium's gyp files.

Fix tests on windows. No need to run Strace tests on Windows, they fail because
of os.path.sep differences.

TBR=mad@chromium.org
BUG=98636
TEST=

Review URL: https://chromiumcodereview.appspot.com/10383154
------------------------------------------------------------------------
May 15, 2012
#26 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=137127

------------------------------------------------------------------------
r137127 | maruel@chromium.org | Tue May 15 07:56:33 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/net/net.gyp?r1=137127&r2=137126&pathrev=137127
 M http://src.chromium.org/viewvc/chrome/trunk/src/build/common.gypi?r1=137127&r2=137126&pathrev=137127
 M http://src.chromium.org/viewvc/chrome/trunk/src/base/base.gyp?r1=137127&r2=137126&pathrev=137127
 M http://src.chromium.org/viewvc/chrome/trunk/src/third_party/cacheinvalidation/cacheinvalidation.gyp?r1=137127&r2=137126&pathrev=137127
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_tests.gypi?r1=137127&r2=137126&pathrev=137127

Add a new gyp define to be used by the swarm master.

Rename tests_run to test_isolation_mode.

TBR=mad@chromium.org
BUG=98636
TEST=
NOTRY=true

Review URL: https://chromiumcodereview.appspot.com/10392073
------------------------------------------------------------------------
Blockedon: -125513 chromium:125513
Blocking: -98639 -109173 -98640 -98637 chromium:98639 chromium:109173 chromium:98640 chromium:98637
May 16, 2012
#28 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=137397

------------------------------------------------------------------------
r137397 | maruel@chromium.org | Wed May 16 04:38:22 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs_test.py?r1=137397&r2=137396&pathrev=137397
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=137397&r2=137396&pathrev=137397
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_test_cases.py?r1=137397&r2=137396&pathrev=137397

Refactor trace_inputs.py strace implementation to use -ff instead of -f.

So each process has its own log file, reducing the race conditions inside
strace.

R=mad@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10356129
------------------------------------------------------------------------
May 18, 2012
#29 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=137912

------------------------------------------------------------------------
r137912 | maruel@chromium.org | Fri May 18 11:34:20 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=137912&r2=137911&pathrev=137912

Remove '<(PRODUCT_DIR)/extensions/' from the dependencies, it doesn't exist anymore.

R=thakis@chromium.org
BUG=98636
TEST=ninja unit_tests_run with test_isolate_mode!=noop works again

Review URL: https://chromiumcodereview.appspot.com/10409028
------------------------------------------------------------------------
May 18, 2012
#30 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=137914

------------------------------------------------------------------------
r137914 | maruel@chromium.org | Fri May 18 11:49:39 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/build/common.gypi?r1=137914&r2=137913&pathrev=137914

Change the test_isolation_outdir default from PRODUCT_DIR to PRODUCT_DIR/isolate

Otherwise a lot of spew is generated when using test_isolation_mode=hashtable.
Do not use hashtable since outdir is also used with _run and _remap.

R=mad@chromium.org
BUG=98636
TEST=

Review URL: https://chromiumcodereview.appspot.com/10382238
------------------------------------------------------------------------
May 30, 2012
#31 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=139546

------------------------------------------------------------------------
r139546 | maruel@chromium.org | Wed May 30 08:58:03 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=139546&r2=139545&pathrev=139546

Improve tracing on OSX and Windows

OSX:
- Make the execve more strict to be able to extract more arguments later.
- Fix the initial trace to be less hacky, add strict handling of the initial
  process.

Windows:
- Only handle files that were opened by the relevant processes. Keep a
  threadid->processid map to be able to figure which process generated the
  Create event.
- Only handle files that were actually opened. Detect this implicitly by
  looking if the file has a Cleanup event and tracking the kernel file object
  instances.
- Add strict handling of the initial process.
- Properly process \\?\ header.

NOTRY=true
R=mad@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10448034
------------------------------------------------------------------------
May 31, 2012
#32 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=139786

------------------------------------------------------------------------
r139786 | maruel@chromium.org | Thu May 31 07:58:01 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs_test.py?r1=139786&r2=139785&pathrev=139786
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs_smoke_test.py?r1=139786&r2=139785&pathrev=139786
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=139786&r2=139785&pathrev=139786
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_test_cases.py?r1=139786&r2=139785&pathrev=139786

Add Results class to hold structured data with details about each of the child processes.

This permits to know which process opened which file, and permits to add more
metadata in the future.

For example with this, it's possible to know which child process opened which
files. Eventually, executable name, command arguments and timestamp data will
be added to make it even more useful.

NOTRY=true
R=mad@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10444052
------------------------------------------------------------------------
Jun 1, 2012
#33 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140113

------------------------------------------------------------------------
r140113 | maruel@chromium.org | Fri Jun 01 15:24:49 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs_test.py?r1=140113&r2=140112&pathrev=140113
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs_smoke_test.py?r1=140113&r2=140112&pathrev=140113
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=140113&r2=140112&pathrev=140113

Add Results.Process.executable and command.

Permits to know _what_ was executed, which is really needed to understand which
child process is using what. Most of the time was spent in normalizing accross
OSes. Some things where more difficult, especially because of python usage,
since on OSX the /usr/bin/python is a stub that reads /usr/bin/python2.7 symlink
and calls the linked executable manually. Also tries to 'guess' the initial
directory on Windows since the event log is lossy.

Fix yet again NTFS' EXTEND. Will have to redo the blacklisting logic later.

NOTRY=true
TBR=mad@chromium.org
BUG=98636
TEST=manually tested on each OS


Review URL: https://chromiumcodereview.appspot.com/10446052
------------------------------------------------------------------------
Jun 2, 2012
#34 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140220

------------------------------------------------------------------------
r140220 | maruel@chromium.org | Sat Jun 02 20:19:27 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=140220&r2=140219&pathrev=140220
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser_tests.isolate?r1=140220&r2=140219&pathrev=140220

Add files to unit_tests.isolate and browser_tests.isolate.

R=thakis@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10505006
------------------------------------------------------------------------
Jun 2, 2012
#35 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140221

------------------------------------------------------------------------
r140221 | rniwa@chromium.org | Sat Jun 02 22:12:48 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=140221&r2=140220&pathrev=140221
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser_tests.isolate?r1=140221&r2=140220&pathrev=140221

Revert 140220 - Add files to unit_tests.isolate and browser_tests.isolate.

R=thakis@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10505006

TBR=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/10497011
------------------------------------------------------------------------
Jun 3, 2012
#36 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140232

------------------------------------------------------------------------
r140232 | maruel@chromium.org | Sun Jun 03 06:55:47 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=140232&r2=140231&pathrev=140232
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser_tests.isolate?r1=140232&r2=140231&pathrev=140232

Reapply 140220 - Add files to unit_tests.isolate and browser_tests.isolate.

Move more files to the untracked section, since they aren't present in all the time.

NOTRY=true
TBR=thakis@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10473017
------------------------------------------------------------------------
Jun 3, 2012
#37 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140234

------------------------------------------------------------------------
r140234 | maruel@chromium.org | Sun Jun 03 07:43:20 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser_tests.isolate?r1=140234&r2=140233&pathrev=140234

Remove extraneous directories.

Looks like the directory removal code in isolate.py uses os.path.sep instead of
'/', causing invalid directories to be kept on Windows, will be fixed in a
separate CL.

TBR=thakis@chromium.org
BUG=98636
TEST=
NOTRY=true


Review URL: https://chromiumcodereview.appspot.com/10496015
------------------------------------------------------------------------
Jun 3, 2012
#38 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140235

------------------------------------------------------------------------
r140235 | maruel@chromium.org | Sun Jun 03 08:03:08 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser_tests.isolate?r1=140235&r2=140234&pathrev=140235

More trailing slashes.

TBR=thakis@chromium.org
BUG=98636
TEST=
NOTRY=true


Review URL: https://chromiumcodereview.appspot.com/10503008
------------------------------------------------------------------------
Jun 4, 2012
#39 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140421

------------------------------------------------------------------------
r140421 | maruel@chromium.org | Mon Jun 04 16:19:53 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_child_process.py?r1=140421&r2=140420&pathrev=140421
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=140421&r2=140420&pathrev=140421

Use trace_child_process.py on Windows and save information about the trace in json.

Removes much of the guesswork in the log parsing code, now the child process id
is known.
Remove references to logman.exe in the log parsing script since it's now called
by the grand-parent process.

NOTRY=true
R=cmp@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10448044
------------------------------------------------------------------------
Jun 4, 2012
#40 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140425

------------------------------------------------------------------------
r140425 | maruel@chromium.org | Mon Jun 04 16:31:01 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=140425&r2=140424&pathrev=140425

Reduce the amount of log generated with dtrace.

Remove extraneous ppid and errno logging.
Have better variable cleanup characteristics to reduce the amount of kernel
memory leaked while tracing.

Add more comments explaining how the process tracking is done

NOTRY=true
R=cmp@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10440073
------------------------------------------------------------------------
Jun 4, 2012
#41 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=140427

------------------------------------------------------------------------
r140427 | maruel@chromium.org | Mon Jun 04 16:34:49 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs_test.py?r1=140427&r2=140426&pathrev=140427
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_inputs.py?r1=140427&r2=140426&pathrev=140427
 M http://src.chromium.org/viewvc/chrome/trunk/src/tools/isolate/trace_test_cases.py?r1=140427&r2=140426&pathrev=140427

Add more consistent exception generation and handling

Convert many assert into proper exceptions. This makes the code more usable as
a library. This is important as tracing in some case will fail, e.g. with
strace race conditions, so they need to be handled properly.

NOTRY=true
R=cmp@chromium.org
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10442068
------------------------------------------------------------------------
Jun 15, 2012
#42 bugdro...@chromium.org
The following revision refers to this bug:
    http://src.chromium.org/viewvc/chrome?view=rev&revision=142437

------------------------------------------------------------------------
r142437 | maruel@chromium.org | Fri Jun 15 12:03:03 PDT 2012

Changed paths:
 M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=142437&r2=142436&pathrev=142437

Update unit_tests.isolate on linux.

Should improve unit_tests_swarm success rate on linux, in theory it could
become green.

R=thakis@chromium.org
NOTRY=true
BUG=98636
TEST=


Review URL: https://chromiumcodereview.appspot.com/10559009
------------------------------------------------------------------------
Nov 13, 2012
#43 maruel@chromium.org
(No comment was entered for this change.)
Blockedon: -chromium:125513 chromium:125513
Blocking: -chromium:98639 -chromium:109173 -chromium:98640 -chromium:98637 chromium:98639 chromium:109173 chromium:98640 chromium:98637 chromium:98637
Mar 9, 2013
#44 bugdro...@chromium.org
(No comment was entered for this change.)
Labels: -Area-Build Build
Apr 12, 2013
#45 bugdro...@chromium.org
------------------------------------------------------------------------
r194010 | maruel@chromium.org | 2013-04-12T20:58:30.662248Z

Changed paths:
   A http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome.isolate?r1=194010&r2=194009&pathrev=194010
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_exe.gypi?r1=194010&r2=194009&pathrev=194010

Add chrome.isolate and chrome_run gyp target.

This permits a user to archive a working chrome build on isolateserver for
downloading it elsewhere. Note that the actual list of files listed
chrome.isolate hasn't been verified but I prefer to commit it as-is and check
each OS afterward. In particular, this new chrome_run target is not included in
chromium_swarm_tests.

Command to archive is:
  ninja -C out/Release chrome_run
  python tools/swarm_client/isolate.py hashtable \
      -s out/Release/chrome.isolated \
      -o https://isolateserver-dev.appspot.com/
  sha1sum out/Release/chrome.isolated

Command to download is:
  python tools/swarm_client/run_isolated.py \
      -r https://isolateserver-dev.appspot.com/content/retrieve/default-gzip/ \
      -H <hash got from sha1sum command above>/ \
      --download <path to download files>

Note that the same command can be used on all OSes, except for sha1sum on
Windows.

R=csharp@chromium.org
BUG=98636

Review URL: https://chromiumcodereview.appspot.com/13913025
------------------------------------------------------------------------
Apr 23, 2013
#46 bugdro...@chromium.org
------------------------------------------------------------------------
r195786 | maruel@chromium.org | 2013-04-23T13:46:54.228930Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser_tests.isolate?r1=195786&r2=195785&pathrev=195786
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=195786&r2=195785&pathrev=195786

Remove unnecessary files from unit_tests and browser_tests in windows/isolate.

The trace_inputs.py implementation on Windows is a bit fuzzy and occasionally
includes files that were not accessed. The Builder/Testers setup on the CI
caught these.

NOTRY=true
TBR=csharp@chromium.org
BUG=98636

Review URL: https://chromiumcodereview.appspot.com/14359005
------------------------------------------------------------------------
Jun 28, 2013
#47 benhenry@chromium.org
(No comment was entered for this change.)
Labels: -Build-Infrastructure Infra
Sep 19, 2013
#48 maruel@chromium.org
The meta task is done. We are now in doing it for each single target.
Status: WontFix
Dec 5, 2013
#49 bugdro...@chromium.org
------------------------------------------------------------------------
r238989 | maruel@chromium.org | 2013-12-05T17:50:12.504606Z

Changed paths:
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/interactive_ui_tests.isolate?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/content/content_browsertests.isolate?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/net/tools/testserver/testserver.py?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/net/test/python_utils.cc?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/net/test/spawned_test_server/local_test_server_win.cc?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/unit_tests.isolate?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/net/net_unittests.isolate?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/net/test/spawned_test_server/local_test_server.cc?r1=238989&r2=238988&pathrev=238989
   M http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser_tests.isolate?r1=238989&r2=238988&pathrev=238989

Stop using third_party\python_26 for many tests.

Migrate testserver.py from using embedded python in third_party\python_26 to
the python version installed on the system.

Kill many python_26 references.

It reduces the amount of data that needs to be transfered for Windows swarm
slaves.

The remaining uses of third_party\python_26 are:
1. chrome_frame.
2. pyauto needs the python SDK. As such, it needs a "known" python installation.
3. gn.

R=phajdan.jr@chromium.org,rsleevi@chromium.org
BUG=98636
BUG=321703
TEST=

Review URL: https://codereview.chromium.org/74253002
------------------------------------------------------------------------
Sign in to add a comment

Powered by Google Project Hosting