My favorites | Sign in
Google
             
Search
for
Updated Jun 18, 2009 by mwhdev
PostCommitWebHooks  
How to use Post-Commit Web Hooks for your project.


Overview

Post-Commit Web Hooks allow projects to setup web services that receive project commit notifications from Google Code. Such services could be used to integrate external tools including continuous build systems, bug trackers, project metrics, and social networks.

Details

Project owners may enable this feature by specifying a target URL in the Administer/Source tab. If the URL contains the special patterns "%p" and "%r", those will be automatically replaced for each commit with the project name and comma-separated list of revisions, respectively.

The POST request payload describes the commit using the Web Hooks model, and consists of a UTF8-encoded JSON dictionary in the following format:

 {
   "project_name": "atlas-build-tool",
   "repository_path": "http://atlas-build-tool.googlecode.com/svn/",
   "revision_count": 1,
   "revisions": [
     { "revision": 33,
       "url": "http://atlas-build-tool.googlecode.com/svn-history/r33/",
       "author": "mparent61",
       "timestamp":   1229470699,
       "message": "working on easy_install",
       "path_count": 4,
       "added": ["/trunk/atlas_main.py"],
       "modified": ["/trunk/Makefile", "/trunk/constants.py"],
       "removed": ["/trunk/atlas.py"]
     }
   ]
 }
 

While we will make a best effort to promptly deliver all Post-Commit Web Hook notifications, messages are not guaranteed to be delivered, may arrive multiple times, and may not arrive in order of commit. All requests have a 15 second timeout. If we fail to reach the specified URL, we will retry several times over a 24 hour period. This allows your services to be down for short maintenance windows and still receive all messages.

Web services should respond to the POST request with a 2XX response code to indicate successful delivery. Redirects (3XX response codes) are not followed, and no further delivery attempts will be made. All other response codes, as well as request timeouts, are treated as failures and will be retried.

Note: Notifications for commits via the 'svnsync' command are not yet supported.

Notification Format

The payload's JSON dictionary contains the following items:

Field Type Description
project_name String The name of the project.
repository_path String The project's repository URL.
revision_count Number Number of revisions contained in the 'revisions' list.
revisions List A list of dictionaries describing 1 or more repository commits (see table below).

Each revision contained in the 'revisions' list is a dictionary with the following items:

Field Type Description
revision Number(SVN) / String(Hg) Repository identifier for this commit.
urlStringURL to browse repository history for this revision.
author String Username responsible for commit.
timestamp Number Repository commit timestamp.
messageStringCommit log message.
path_countNumberTotal number of paths modified in this revision. Only a fixed number of paths will be included per revision, so this number is used to determine whether a partial list was sent.
addedListA list of String paths added by this revision.
modifiedListA list of String paths modified by this revision.
removedListA list of String paths removed by this revision.

It is important to note that a revision's list of changed paths will be truncated for large commits in order to limit message sizes. Full commit information can be obtained using standard repository tools.

Example: Processing a notification using a Python AppEngine service

 import logging
 from django.utils import simplejson
 from google.appengine import webapp
 class Listener(webapp.RequestHandler):
   def post(self):
     payload = simplejson.loads(self.request.body)
     for revision in payload["revisions"]:
       logging.info("Project %s, revision %s contains %s paths",
                    payload["project_name"],
                    revision["revision"],
                    revision["path_count"])
 

Authentication

Post-Commit Web Hooks use HMAC-MD5 to authenticate requests. Every project has a unique post-commit 'secret key', visible to project owners in the Administer/Source tab. This key is used to seed the HMAC-MD5 algorithm. Each POST request header contains a HMAC used to authenticate the payload. This value is a 32-character hexadecimal string contained in the 'Google-Code-Project-Hosting-Hook-Hmac' header. By combining your project's secret key and the POST request's HMAC value, you can authenticate the request.

Example: Authentication using a Python AppEngine service

 import hmac
 import logging
 from google.appengine import webapp
 class Listener(webapp.RequestHandler):
   def post(self):
     project_secret_key = "0123456789abcdef"  # From Administer/Source tab
     m = hmac.new(project_secret_key)
     m.update(self.request.body)
     digest = m.hexdigest()
     if digest == self.request.headers["Google-Code-Project-Hosting-Hook-Hmac"]:
       print "Authenticated"
     else:
       print "Authentication failed!"
 

Continuous Integration with Hudson

Hudson, a continuous integration system, can be used to automate the build for Java projects (among others such as .NET projects). Projects built with Hudson can be triggered using Google Code's Post-Commit Web Hooks.

After Hudson has been setup, builds can be triggered on every commit by using the following url on your Hudson host as the Post-Commit URL:

 http://YOURHOST/hudson/job/PROJECTNAME/build 

For more information, refer to the Hudson documentation.


Comment by de...@battams.ca, Mar 02, 2009

Can a java authentication example be provided? I'm trying to authenticate hooks received in a servlet.

Comment by de...@battams.ca, Mar 02, 2009

After a timeout, I managed to roll my own auth implementation for GC's web hook messages. An example and the jar file are available at:

http://webhooks.googlecode.com

Comment by josh.reisner, Mar 04, 2009

In case you're wondering how to catch these web hooks with PHP, I posted my solution here. I just cover catching the POST data, which wasn't readily apparent to me. I haven't covered the authentication yet.

Comment by josh.reisner, Mar 04, 2009

Here's a pretty full script for catching and authenticating these hooks using PHP. I use my own library a little, but it would be easy to extrapolate the key details you'll need.

http://code.google.com/p/joshlib/wiki/CatchingWebHooks

Comment by aleritty, Mar 06, 2009

I realized a php script that reads the web-hook "modified", "added", and "removed" fields and syncronizes your "non-working copy" on the webserver (not running svn!). You can find the script at: http://www.aleritty.net/progetti/post-commitphp/

Comment by bearper...@gmx.net, Mar 10, 2009

Thanks for coming up with this! I've implemented an endpoint of this for the CIA.vc commit tracker, so people with google code can usefully have their commits reported. Latency looks pretty good so far, I'm getting around 5 seconds, most of which might be due to our side. Two things: It may be useful to allow people to set several addresses, "chaining" hook endpoints or setting up one that bounces on to all targets is clumsy. Also, it seems the current implementation treats only HTTP 200 as success: My original code returned a 202 (Accepted) since that seemed like the appropriate code, but this seems to have caused the hook system to retry a few times. You may want to change that.

Comment by challgren, Jul 18, 2009

For those that use Brightkite and Google code. I wrote a script that posts the commit to Brightkite http://pastebin.com/f7ac7cc00

Comment by hufeng1987, Jul 23, 2009

You can use php to proccess the PostCommitWebHooks

<?php
//google code secret key
$google_code_secret_key = 'RD5234999kc51l';
//get revision commit data
$revision_data=file_get_contents('php://input');
//build secret verify info;
$secret_verify=hash_hmac("md5",$revision_data,$google_code_secret_key);
//get google secret info
$google_secret_info=$_SERVER['HTTP_GOOGLE_CODE_PROJECT_HOSTING_HOOK_HMAC'];

//prase revision commit data
$revision_info=json_decode($revision_data);
//var_export $revision info
$revision_var_export=var_export($revision_info,true);

$fc="\n google code secret key : ". $google_code_secret_key;
$fc.="\n secret verify info : ". $secret_verify;
$fc.="\n google secret info : ". $google_secret_info;
$fc.="\n project name : ". $revision_info->project_name;
$fc.="\n revision count : ". $revision_info->revision_count;
foreach($revision_info->revisions as $revision){
$fc.="\n author:" .$revision->author;
$fc.="\n time: " .date('Y/m/d H:i:s',$revision->timestamp);
$fc.="\n url: ".$revision->url;
$fc.="\n message: ".$revision->message;
$fc.="\n revision: ".$revision->revision;
}


file_put_contents("log.txt",$fc);
Comment by nicolas.alvarez, Sep 03, 2009

For those doing continuous integration with Buildbot, I'm working on a ChangeSource? for Google Code web hooks.


Sign in to add a comment