IntroductionThere are several ways to generate a video: - if you want to access a subversion repository, simply run runrepositoryfetch.bat on windows systems or runrepository.sh on linux (linux support for this way has not been tested yet).
A dialog will show up, asking you for the repository url, your username and password. The repository gets fetched, converted and codeswarm will be started. This will create the frames of the video for you. If you want to create a real video, see "Creating the video" (below). - if you want to access a non-svn repository, read on:
This guide explains how to generate a video under a Linux install (it is based on Linux tools) but might be applied to a Windows or OSX workstation as well (some other tools might need to be used) The prerequisite, of course, is to download code_swarm from SVN (check out the Source Checkout page for that). You will also need java and ant to build it. Generating your video is done in 4 steps : - get the activity log out of your versioning system
- convert the activity log to something code_swarm can use
- executing code_swarm
- making a video out of the exported images
Step 1: Getting the activity log | SVN | CVS | darcs | MediaWiki | Mercurial
| To get the activity log from your SVN repository, just get a local copy of your repository, enter the main directory and execute svn log -v > activity.log This will create a file called activity.log. | To get the activity log from your CVS repository, just get a local copy of your repository, enter the main directory and execute cvs log > activity.log This will create a file called activity.log. | darcs changes --summary --reverse > activity.log | To get a wikiswarm report from the wiki use the maintenance script inside of the SwarmExport extension | Nothing to do here, read next step. | Step 2: Convert the logcode_swarm has a python converter in its convert_logs directory. For a list of all the supported systems, use: python convert_logs.py --help . Note: Before you start using it, you might want to read this issue report. | - Move to the convert_logs directory
- Copy the activity.log file generated above into the current directory
- Launch the convert_logs.py script:
| Mercurial$ cd $CODESWARM_HOME/convert_logs/
$ python hg_log.py /Path/To/Your/Mercurial/Repository this produces the file $CODESWARM_HOME/data/hglog.xml for your given repository. | SVNpython convert_logs.py -s activity.log -o activity.xml | CVSpython convert_logs.py -c activity.log -o activity.xml | darcsTo document. | MediaWikipython convert_logs.py --wikiswarm-log=activity.log -o activity.xml | | This will generate a file called activity.xml. At this point, you will not need activity.log anymore, so you can delete it. | Step 3: Executing code_swarm | | Before running, you will need to configure code_swarm for your project. In the data folder you'll find an example of a configuration file (sample.config). Minimally, just edit your configuration file to reflect the correct path to InputFile, which is the previously generated xml (activity.xml or hglog.xml). To execute code_swarm, you need just need ant and javac. Execute run.sh script in the code_swarm main folder. For more details refer to the [HowtoBuild] Wiki page. | Step 4: Creating the video | The previous step created a series of screenshots in the PNG format under the code_swarm/frames/ directory. Now you want to use them to build a video. MEncoder is one tool to do that. Just get into that frames/ directory and type: mencoder mf://*.png -mf fps=24:type=png -ovc lavc -oac copy -o movie.avi For a 15000 commits video, this might take about 40MB, so you want to reduce the size a little bit. You can do that by converting to DivX (MPG4) and reducing the frame rate: mencoder movie.avi -ovc xvid -oac mp3lame -xvidencopts bitrate=200 -o project-activity.avi For a 40MB video, this reduces the size to 10MB. You can also try using the h264 format if your MEncoder has the right libraries to do so: mencoder movie.avi -ovc x264 -oac mp3lame -xvidencopts bitrate=200 -o project-activity2.avi To make a high quality .mov (no loss from the frames) suitable for iMovie or Final Cut: ffmpeg -f image2 -r 24 -i ./frames/swarm-%05d.png -sameq ./out.mov -pass 2 (Or you can use Quick Time Player to open a image sequence and then save a resulting high quality .mov file.) That's it, you should have a beautiful video of your project's activity by now! |
|
Sign in to add a comment
please add a line (for noobs like me) to say what python version is required (2.5?) to convert logs ;)
You can also use ffmpeg for generating the video (from your frames directory): ffmpeg -f image2 -i swarm-%05d.png /tmp/foobar.mpg
maybe plan a option to convert apache/httpd access log into xml event would be a good idea ?
"To execute code_swarm, you need the Processing application (which uses Java)." I'm newbie, so can you please explain me what exactly I have to type in my command line to execute code_swarm (I use Ubuntu 8.04)?
"Processing" is an application you down load from
http://processing.org/
String
must be changed to
Regards, Vyacheslav
Wher do I store activity.xml, I am using version 0.139, and it seems to execute but nothing happens. I have put activity.xml in the data folder, should it be somwhere else?
Just an FYI, if you are trying to generate an activity.xml for CVS (it's on this project's trunk as of this morning), you will need to do the following: 1) Line 142 should be "...strip()", not "...strip('\n')". I had some issues where there was still a carriage-return at the end of the line which caused it not to equate properly. 2) After your output, you WILL need to sort it. In order to do this, you can change line 182 (where the "xml_handle" writes out the event) to write the date out first. Once you've done this, run the converter. Edit the XML output, and remove the first two lines and the very last line. Run sort on it, then add the three lines back in. Now you're ready to go. If anyone has alternatives to make this easier, let me know!
Actually, you will need to write out the date, then author, then filename for 'sort' to work correctly.
What command necessary use with CVS Reposotory for generate correct log file?
I can only speak for Ubuntu 8.04 (although I wouldn't be surprised if this applies to all nix systems). After you've got the activity.xml for CVS and you've removed the appropriate lines do something like the following:
Here's my java n00b question: Is this geared to a specific JDK? I'm using JDK 1.6.0_10-beta-b25 and am getting stack traces using code_swarm.pde with Processing. Also, was this written with a certain version of Processing? (I'm going to see if I can answer my own question and user a previous version of Processing.) Thanks for any help. If I wasn't such a Java dork, I'd help out with this program.
There appears to be a limit, at least on my rinky system (only 1G RAM on Windows Vista), to the size allowed in the generically named activity.xml file. Had a 20 MB SVN dump from a very busy /trunk. Broke this down to ~5MB and everything works fine. I did (finally) read the FAQ. My processing version was too old (I grabbed the 0135 beta, not the newer 0142 beta) and figured out my jdk was just fine. Thanks guys/gals for the cool program.
In order to process a lot of data it does require a lot of memory. Each file node that ever enters CVS/SVN is kept internally for book keeping, so it does add up.
I generally run at about 512m with large files, which seems good enough, but if you don't have that free the only way is to split up the file indeed.
I created a converter to generate the activity xml from Perforce. Let me know if there is interest in this and I can try to upload it somewhere.
ajbanck - email it to me and i'll commit: acgourley (gmail)
For users of ffmpeg, here's the command I used: ffmpeg -f image2 -r 24 -i ./frames/swarm-%05d.png -sameq ./out.mov -pass 2
That means input as images, 24 frames per second, input filename format, high quality, output file, 2 passes.
ajbanck: could you mail the converter to "oregu at yandex dot ru"?
convert_logs.py script now supports perforce: python convert_logs.py "--perforce-path=//depot/..." This command will generate depot.xml including all file activities in specified perforce folder. Command "p4" should be installed and configured.
there is no sound, use the "-nosound" switch for mencoder and remove the -oac copy ...
If you want to create movies from Mercurial repositories, you can use the code from Baptiste Lepilleur:
http://www.nabble.com/forum/ViewPost.jtp?post=18430641&framed=y
With it he created this beautiful Mercurial codeswarm video:
http://www.vimeo.com/1334189
To answer some of the above questions:
To actually run this follow the instr. above and:
-Sebastian
I just created a script which can pull in and combine commits from many different projects and creates codeswarms from only the changes since the last created codeswarm.
It isn't yet polished, but I think it might be quite useful.
http://rakjar.de/shared_codeswarm/readme.html
I use it to gather commit data from many different repositories, but I plan to use it to track code evolution of bigger projects in near-realtime (means: checking once a day or so :) ).
It automatically creates a video, and names it shared_movie_DATE-TIME.avi
For using it, you need to write a "simple.config" file like the following: - http://freehg.org/u/ArneBab/shared_codeswarm/raw-file/tip/simple.config
Then put it into a directory and start the community_codeswarm.py script in that directory.
- python community_codeswarm.py --codeswarm-path "absolute path to the codeswarm dir"
For it to work, you need pyyaml, Mercurial and naturally codeswarm installed. - http://pyyaml.org/ - http://selenic.com/mercurial
Please tell me how the script works for you! - http://draketo.de/contact
License: GPLv3 - like codeswarm. - Arne Babenhauserheide
For everyone's info:
I was able to get code_swarm running beautifully on an Intel MacBook? Pro running 10.5+. I had to use MacPorts? to install a number of applications, but it works just like the instructions for a GNU/Linux setup.
I would also like to request that the steps in Sebastian's comment (Sept 04, 2008) be put in the general instructions. I was lost without the tip to create the frames directory and update the data/sample.config. One could also edit the run.sh file and point the "default_config" variable to a different named file.
-Ammon Shepherd
ajbanck, i see you've worked with perforce. anyone else? if you have the way to create the perforce activity xml - can you tell us where it's posted or how to get it?
cliff
@latestlinux thanks, that got me started quickly
Did the Perforce script get committed?
recent mencoders need the following for h264 output:
mencoder mf://code_swarm-0.png -mf fps=25:type=png -ovc x264 -x264encopts bitrate=200 -o output.avi
Hello @all, Generating den PNGs from SVN Activity was pretty easy.
Now i try to use mencoder for Windows to generate an avi. But all i get is: MEncoder Sherpya-SVN-r28311-4.2.5 (C) 2000-2009 MPlayer Team CPU: Intel(R) Core(TM)2 CPU T7600 @ 2.33GHz (Family: 6, Model: 15, Stepping: 6) CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1 Compiled for x86 CPU with extensions: MMX MMX2 SSE SSE2
success: format: 16 data: 0x0 - 0x0 MF file format detected. mf? search expr: D:\Projekte\CodeSwarm?\frames\code_swarm-00001.png mf? number of files: 1 (4) VIDEO: MPNG? 800x600 24bpp 25.000 fps 0.0 kbps ( 0.0 kbyte/s) V? filefmt:16 fourcc:0x474E504D size:800x600 fps:25.000 ftime:=0.0400 Opening video filter: osd=1? Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
Opening video decoder: ffmpeg? FFmpeg's libavcodec codec family Selected video codec: ffpng? vfm: ffmpeg (FFmpeg PNG decoder)
Flushing video frames. Filters have not been configured! Empty file? Writing index... Writing header... ODML: Aspect information not (yet?) available or unspecified, not writing vprp header.
Video stream: nan kbit/s (-2147483648 B/s) size: 0 bytes 0.000 secs 0 frames
Does anyone can help me, or tell me wich tool is easyer to use under Windows?
Regards
I had some real problems with mencoder
Trying to get an alreayd compiled version for Windows took for ages - for anyone who is interested, if you download "Media Coder" mencoder and ffmpeg are bundled with it in the codecs folder.
For some reason it didn;t like PNGs, came out with what looked muvh like static on a TV. So I converted all the files to bitmaps, that solved that issue.
Then I had trouble getting mencoder to create a file that windows media player could read without any extra codecs. Here's the command line I used in the end that allowed me to do this:
mencoder "mf://code_swarm-.bmp" -mf fps=25 -o swarm.mpeg -ovc lavc -lavcopts vcodec=msmpeg4v2
Hey, everyone I am using Windows XP Mencoder was throwing exepctions on my machine, I didn't manage to get it up. So I just used a tool called VideoMach?... and I got the pretty decent results.
Cheers.
Mercurial support
Assuming codeswarm is located in $CODESWARM_HOME, we have a python script in convert_logs that may be used for Mercurial repositories:
this produces the file $CODESWARM_HOME/data/hglog.xml
Afterwards, just edit your configuration file ($CODESWARM_HOME/sample.config is fine) to reflect that InputFile=data/hglog.xml .
The final step is just to go into the main folder ($CODESWARM_HOME) and type ./run.sh (or run.bat).
Note: Do not use ant run, it will fail for mercurial repositories. But you might need to run ant to compile the code.
Would be nice to get some kind of feedback from the app on what's the status of png generation (like do a sys out when next percent of the work is done)