Changes chrome ebuild to use --jobs 4 for gclient sync.
Comment #1
Posted on Nov 8, 2010 by Massive OxWhen syncing my personal tree, I usually use 8. Have we benchmarked it? Is 4 really the inflection point, where adding more parallelism stops really helping?
Comment #2
Posted on Nov 8, 2010 by Grumpy Wombat(No comment was entered for this change.)
Comment #3
Posted on Nov 8, 2010 by Grumpy HorseWe may want to leverage MAKEOPTS="-jN" defined in the make.conf since that is a single source of how many parallel jobs to use. It may currently not be set right today, and it may need to be autodetected and setup during seutp_board time.
For a quick fix we could detect number of CPUs from /proc/cpuinfo and then set the right -J flag.
Comment #4
Posted on Nov 8, 2010 by Massive HippoThis is for gclient fetching - its almost all network based.
Making this too high will cause a DOS against GIT if we kick off all builders at once.
In talking with folk earlier (forget who but was either Anush or David) I think we settled on four as "gives improvement without killing git server.
Comment #5
Posted on Nov 8, 2010 by Massive OxI'm aware of what the --jobs flag is for in this case, as I use it almost every day in my development :-)
Also, the chromium repo isn't in git, and it's certainly not in the chromium os git server, so I figure it would make more sense to talk to the chromium infrastructure people. They're the ones who maintain the svn server we'd be syncing against to fetch the chrome source, so they'd probably know better than anush or david in this case.
Comment #6
Posted on Nov 11, 2010 by Grumpy Wombatbulk edit. the block label was set by a bulk edit with the wrong query.
Comment #7
Posted on Nov 11, 2010 by Happy BirdBulk reassigning chrome build issues to Anush
Comment #8
Posted on Nov 16, 2010 by Happy GiraffeCommit: ced29a8be8f58308168f56c799ebfcd907183d4c Email: cmasone@chromium.org
Sync chrome source in parallel
We add the --force flag to gclient sync to make it cope with a race condition. The main chrome tree has some stuff checked in under src/third_party/ On Chrome OS, we have a special dep that checks out from a git repo to src/third_party/cros
When doing gclient sync --jobs N, with N>1, the latter will complete before the former, meaning that svn will complain that an unversioned directory exists at src/third_party.
The --force ignores this.
BUG=8859 TEST=emerge chrome with SERVER_SOURCE
Change-Id: Ie69d0b040fb8f7b1830b6875bca56c6775489672
Review URL: http://codereview.chromium.org/5009005
M chromeos-base/chromeos-chrome/chromeos-chrome-9999.ebuild
Comment #9
Posted on Dec 10, 2010 by Helpful OxBulk edit: Can we please update status and close the bugs if appropriate?
Comment #10
Posted on Dec 13, 2010 by Grumpy Horse(No comment was entered for this change.)
Comment #11
Posted on Feb 16, 2011 by Helpful OxR10->R11
Comment #12
Posted on Feb 16, 2011 by Helpful OxR10->R11
Comment #13
Posted on Mar 23, 2011 by Grumpy Horse(No comment was entered for this change.)
Comment #14
Posted on Mar 24, 2011 by Helpful Hippo(No comment was entered for this change.)
Comment #15
Posted on Oct 9, 2012 by Quick RabbitCommit: c0232e95690f17b07868daf08736ddd0244c9429 Email: cmasone@chromium.org
Sync chrome source in parallel
We add the --force flag to gclient sync to make it cope with a race condition. The main chrome tree has some stuff checked in under src/third_party/ On Chrome OS, we have a special dep that checks out from a git repo to src/third_party/cros
When doing gclient sync --jobs N, with N>1, the latter will complete before the former, meaning that svn will complain that an unversioned directory exists at src/third_party.
The --force ignores this.
BUG=8859 TEST=emerge chrome with SERVER_SOURCE
Change-Id: Ie69d0b040fb8f7b1830b6875bca56c6775489672
Review URL: http://codereview.chromium.org/5009005
M chromeos-base/chromeos-chrome/chromeos-chrome-9999.ebuild
Comment #16
Posted on Mar 7, 2013 by Grumpy Hippo(No comment was entered for this change.)
Comment #17
Posted on Mar 10, 2013 by Quick Rabbit(No comment was entered for this change.)
Comment #18
Posted on Mar 14, 2013 by Happy HorseMoved to: Issue chromium:191257
Status: Moved
Labels:
Type-Feature
Pri-1
Effort-1
OS-Chrome
M-11
Build