New workflow

This page is obsolete, see:

http://code.google.com/p/chromium/wiki/UsingNewGit

Old workflow

Some people working on Chromium prefer to use the Git version control system. This document offers advice and tips on using Git.

Warning: If you just git clone the Chromium repo, your checkout will not work. A full checkout requires checking out around 100 other subprojects.

• New workflow
• Old workflow
• Getting the code
• First checkout
• Advanced Tricks
• Staying up to date
• Contributing
• Configure your checkout
• Making local changes
• Presubmit tests
• Code Reviews
• Trybots
• Renames and Copies
• Committing
• Initial Setup
• Really Committing
• Windows
• Workflow
• Troubleshooting
• Why NOT TortoiseGit
• Using Pageant as SSH agent on Windows
• Tips & Tricks
• Finding Files Fast
• Checking out and committing to a branch
• Checking out trunk/tools
• Checking out LKGR

Getting the code

First checkout

1. First, check the prerequisites for your platform: Windows, MacBuildInstructions, LinuxBuildInstructionsPrerequisites.
2. Make a new directory for working with Chromium and cd into it.
3. Download the depot tools, which are used to fetch external dependencies of the code base. Here are Linux instructions; see the install info for other platforms. Be sure to add the depot_tools directory to your PATH.

svn co http://src.chromium.org/svn/trunk/tools/depot_tools

4. Grab the main tree with Git. You should be using git 1.6.6 or above in order to leverage "smart HTTP transport." Using this exact name of output directory (src) is important! This is a very large download, more than a gigabyte. You might get a smaller download via the --depth argument to git clone.

git clone http://git.chromium.org/git/chromium.git src

5. Create a .gclient config file. (If you are a committer, use should probably use svn://svn.chromium.org/chrome/trunk/src instead. This will check out dependencies in a way that's compatible with git cl and gcl.)

-NOTE: Do this from the parent of the src directory (the one you created in step #2), otherwise it will create a new src/ directory under your current src directory.

gclient config http://src.chromium.org/svn/trunk/src

1. Optional. Edit your .gclient file to avoid checking out the enormous set of WebKit layout tests. Find the line that has an example of excluding a component, and add beneath it (to the "custom_deps" dictionary):

"src/third_party/WebKit/LayoutTests": None,

Other things that are large and that you probable won't need that you can put there:

"src/chrome/tools/test/reference_build/chrome": None,
"src/chrome_frame/tools/test/reference_build/chrome": None,
"src/chrome/tools/test/reference_build/chrome_linux": None,
"src/chrome/tools/test/reference_build/chrome_mac": None,

Specifying a safe-sync URL can be very helpful. Add the following to your .gclient to ensure that only green builds will be synced:

"safesync_url": "http://chromium-status.appspot.com/lkgr",

1. Have gclient pull Chromium's dependencies as normal svn checkouts:

gclient sync

If all went well, gclient will recognize you're managing your src directory yourself, and print the following at the very beginning.

      ________ found .git directory; skipping src

Note that it will still check out other files such as tools and third party files under the src directory.

If you are using Linux, you may need to set certain environment variables before building. See LinuxBuildInstructions for more information.

Advanced Tricks

• WebKit and git. See UsingWebKitGit and http://trac.webkit.org/wiki/UsingGitWithWebKit.

Staying up to date

1. Run git pull or whichever command is appropriate to update your checkout. The Git community book is a good resource to learn git from.
2. Run gclient sync if any of the DEPS files change to update dependent subdirectories.

Contributing

Configure your checkout

Since contributing code using git requires git-cl, read git-cl's README.codereview and README.git-cl to learn how to use it. Since you already have depot_tools in your PATH, git-cl will "just work."

You must add the junk to your ~/.subversion/config that's specified on http://dev.chromium.org/developers/coding-style under "Subversion properties". Else git won't set svn properties correctly and people might yell at you. That is not necessary if you use the commit queue

Making local changes

The first step to contributing a patch is to write it. Create a branch and make your changes:

git checkout -q -b mywork origin
vi ...
git commit -a -v

Presubmit tests

After committing your changes locally, use git-cl to run all of the presubmit tests associated with your change.

git cl presubmit

This runs both upload and dcommit presubmit tests. They will alert you to any problems with your change that you should address before completing or dcommitting your patch.

There is currently a bug in the presubmit checks which will cause it to warn about missing BUG and TEST fields even when they are present. These can be safely ignored for now assuming the commit message contains those fields.

Code Reviews

All code changes are reviewed using http://codereview.chromium.org. Use git-cl to easily upload a branch as a proposed change.

You can associate each git branch you work on with a code review by running the following:

git checkout mywork
git cl upload

Uploading your change will also run the upload presubmit tests on your change. To skip the presubmit tests, use the flag --bypass-hooks.

Subsequent invocations of 'upload' on the same branch will add updates to the same code review. See git cl help for more.

Trybots

git-try, part of depot_tools, will run git branches through the try bots.

If your patch depends on specific branches in Chromium and WebKit, you can run git-try with:

git try -b BOT --webkit WEBKIT_BRANCH [CHROMIUM_BRANCH]

where WEBKIT_BRANCH and CHROMIUM_BRANCH are branches to compare with. In most cases, they will be master and trunk, respectively.

If you are running the patch on try bots, use BOT as win_layout,mac_layout,linux_layout.

Renames and Copies

Git has a strange notion of how copies and renames work; see the Git FAQ about it. If git diff -M -C shows that git knows a file was renamed, then git cl upload and git cl dcommit will include the proper metadata for reviewing and committing the rename. If not, ask evan@chromium.org about it -- this code isn't tested well so there are surely bugs.

Committing

To commit from your checkout, you must (obviously) first have commit access.

• It is also important that you have a fairly recent version of Git -- 1.6.1 or so will do. Support for committing from a repository that is kept up to date with git clone was added only recently.
• Make sure you have git-svn package installed.
• Make sure to setup svn auto props, as described here: http://dev.chromium.org/developers/coding-style#TOC-Subversion-properties

Initial Setup

Make sure you are in src/ folder when you run these commands:

git svn init --prefix=origin/ -T trunk/src svn://svn.chromium.org/chrome
git svn fetch

(If you're a git ninja, you might notice that refs/remotes/origin already points at the git server, and we're telling git-svn that it should also use refs/remotes/origin/trunk -- we're instructing plain git and git-svn to use the same git branch.) fetch rebuild its revision database.

If any of your git svn commands appear to hang, you may have run the git svn init without the --prefix=origin/ and -T trunk/src arguments.

Really Committing

Should Just Work. Always use git cl dcommit. It runs git svn dcommit for you along with running the very important presubmit tests. To bypass the presubmit tests (eg. to committ on a closed tree), use the flag --bypass-hooks.

Windows

Use cygwin with the following packages (discussion against msysgit):

• curl (for git-cl-patch)
• git
• git-svn
• gitk (optional)
• subversion-perl
• python

After installation, sanity check your git-svn binaries by running "git svn info". If you see an error message like below, you'll need to remap your binaries.

Be sure to turn off git's crlf munging AND disable file mode detection:

git config --global core.autocrlf false
git config --global core.filemode false

Binaries may not work out of the box:

cygwin 1.7 (and also 1.5 on Windows 7) may throw error message like

$ git svn info
6 [main] perl 4760 C:\cygwin\bin\perl.exe: *** fatal error 
- unable to remap <library> to same address as parent

while using git. You'll need to rebase your dlls with ash.exe and 'rebaseall'.

If this doesn't work, you may need to get serious; see CygwinDllRemappingFailure for details

gclient may not work from cygwin shell:

"gclient runhooks" may fail when run from cygwin. Try this instead from cygwin:

cmd /c gclient runhooks

Workflow

Here's a thread on chromium-dev where people discuss Windows-specific workflow issues. In particular due to the slower filesystem and Visual Studio getting confused about files changing out from underneath it, you might want to avoid switching branches if possible, and instead use either only one branch or git-new-workdir. See also the VanillaMsysgitWorkflow which takes a different approach, running all Chrome-specific git tools on a Linux box.

Troubleshooting

Sometimes (it seems to happen on Windows, but not other platforms?) git-svn will get confused about things. Symptoms include it trying to refetch commits you've already fetched, or printing "transaction out of date" errors on unrelated files.

Try this recipe:

rm -rf .git/svn   # clobber git-svn metadata
git svn find-rev r1   # run a no-op command to rebuild metadata

If you see errors about being unable to unlink old files when running git commands on Windows, it's quite likely that you have the chrome solution open in Visual Studio and it's updating IntelliSense, which will briefly lock files at random. The workaround is either to disable IntelliSense entirely, or to be sure to close the solution temporarily while performing any git operations which could touch a large number of files.

Why NOT TortoiseGit

The primary reason not to use TortoiseGit is speed. Chromium and Webkit repositories are too large for TortoiseGit to handle efficiently. TortoiseGit is too wasteful in polling status and the UI is too often unresponsive in a workable manner.

msysgit got a lot better, and is considered by many to be much faster than cygwin git. All the known issues we had with it were also fixed. The chrome infrastructure team recommends the use of msysgit. If you use msysgit and have problems, please let the chrome-infrastructure-team know.

msysgit did appear to be functioning correctly after configuring autocrlf=false and filemode=false (pay attention during installation for that option to make life easy on yourself).

Note: install msysgit to a location that is not controlled by antivirus software, e.g. c:\src (set not to be scanned). symlinks to git.exe in the libexec\git-core folder may trip up scanning packages and cause the msys GUI to hang.

Note that msysgit (as of 1.7.4-preview20110204) has a bug where it doesn't follow config if it's a shortcut when changing it, breaking the shortcut. That means that the working directory style of using multiple git branches (use the new 'winsymlinks' CYGWIN setting, see http://cygwin.com/cygwin-ug-net/using-cygwinenv.html, or see https://github.com/joero74/git-new-workdir for a .cmd port that works when run as admin) has extra caveats associated with it. Cygwin git is fine with working directories.

Mixing Cygwin and msysgit can be done, but is fragile and unsupported. Not recommended. The basic tactic is to isolate the two worlds by jumping back and forth using cmd.exe, but quoting issues are deadly here. Email 'scottbyer@chromium.org' if you're determined. With an SSD for sources, the speed difference between the two isn't dramatic enough to put up with the workflow interruptions.

Using Pageant as SSH agent on Windows

Some of those using git on Windows will constantly git pull and git push from/to a git repo on our Linux box (hint: you can even commit to running all try jobs and git cl commands from there). Having an SSH agent makes this a lot easier since you don't have to provide your password every time git needs to SSH into your Linux box.

If you're using Cygwin, it's reasonably easy to just use the normal ssh-agent, you log in once after opening your Cygwin shell and that's it.

On the other hand, for those using msysgit from a regular Windows command prompt (not from the bash shell that comes with msysgit), using Pageant (the SSH agent that comes with PuTTY) is a great solution. Here are the steps, roughly:

1. Set up PuTTY along with plink, puttygen and Pageant.
2. Follow the steps in chapters 8 and 9 of the PuTTY documentation to generate your public/private key pair using puttygen, paste the public key into your ~/.ssh/authorized_keys2 file (man sshd is your friend), and add the key to Pageant. Every time you start your machine (or when you want the SSH agent running), you will need to run Pageant, add the key, and provide the password to decrypt the key.
3. Set an environment variable GIT_SSH to the full path of your plink.exe, e.g. GIT_SSH=c:\tools\putty\plink.exe. Remember that you may need to restart your command prompt so that it picks this new variable up.
4. You're good to go. You can test by using PuTTY to SSH into your Linux box; PuTTY will automatically use a key from Pageant if available. Then run e.g. git pull to pull code from the git repo on your Linux box and it should just work.

Tips & Tricks

Finding Files Fast

Use git-gs located in src/depot_tools to search the source tree.

Since git keeps an index of all files it knows about, it can filter the master list of files by file extension and only grep through those files. It doesn't have to crawl your tree (and all those layout test results) to grep over source code. Performing a git gs [searchterm] from /src/ gets results in a couple seconds with a cold disk. It's often instant with a warm disk.

Note this won't search dependencies pulled by gclient.

Checking out and committing to a branch

Instructions per shess@. These instructions assume that you already have a clone of the trunk repo. If these destroy your git repo, please fix it yourself!

BRANCH=552  ## Put your branch number here.
git config --add svn-remote.svn_$BRANCH.url svn://svn.chromium.org/chrome
git config --add svn-remote.svn_$BRANCH.fetch branches/$BRANCH/src:refs/remotes/origin/$BRANCH
git svn fetch svn_$BRANCH
git checkout -b my_$BRANCH origin/$BRANCH

From there, git cl dcommit commits to the right branch magically! Unfortunately, due to {bikeshed discussion omitted}, the DEPS file in the branch doesn't do the right thing. You need a per-release DEPS file to build correctly. Head over to http://omahaproxy.appspot.com/ and pick your release, then:

RELEASE=8.0.552.18
cd ..  ## cd to directory ABOVE src/.
gclient config http://src.chromium.org/svn/releases/$RELEASE --gclientfile=.gclient_$RELEASE
## Consider editing .gclient_$RELEASE custom_deps to cut down on test data.

Then, to checkout the branch:

cd src/
git checkout -b my_$BRANCH origin/$BRANCH
gclient sync --gclientfile=.gclient_$RELEASE --jobs 8

Note that the gclient file should be specified without a relative path, even though it's located in the directory above src/.

As the branch gets further away from the trunk, the sync will start to see conflicts in src/third_party. Usually these can just be deleted and the sync restarted. As things diverge, re-syncing will get slower and slower (it may be worth using a symlinked .git repo).

To get back to the trunk (may also require deletions in third_party):

cd .../src/
git checkout my_trunk_branch
gclient sync --jobs 8

Also, if you need to merge changes to DEPS, see the internal ChromeReleaseBranches wiki page for some notes.

Checking out trunk/tools

git svn clone svn://svn.chromium.org/chrome -T trunk/tools

We don't recommend fetching tools directly into your src/ checkout as the two are unrelated. You may want to fetch a subdirectory instead since tools/ is fairly large.

Checking out LKGR

In the git commit message, the SVN revision id is present in the git-svn-id field. You can check this information when pulling or rebasing to use LKGR. Or you can use the below script to maintain a local branch named lkgr that points to the correct revision (you still need to do a git fetch first to pull the latest changes from the git repository):

#!/bin/bash

function die() {
  echo "Error: $1" 1>&2
  exit
}

SVN_REV=`wget --quiet -O- 'http://chromium-status.appspot.com/lkgr'`
test "$?" == '0' -a -n "$SVN_REV" || die 'Cannot determine LKGR svn revision'

GIT_REV=`git log --grep="^git-svn-id:.*@$SVN_REV " --pretty=format:%H origin/trunk`
test "$?" == '0' -a -n "$GIT_REV" || die "No git commit for SVN rev $SVN_REV"

BRANCH_NAME=lkgr
git branch -f "$BRANCH_NAME" "$GIT_REV"
test "$?" == '0' || die "Cannot create $BRANCH_NAME branch for $GIT_REV"

echo "Git branch $BRANCH_NAME is now at $(git log -1 $BRANCH_NAME --pretty=oneline)"