Create a command line program to read Proctor (http://www.doughellmann.com/projects/Proctor/) test results and create trac (http://trac.edgewall.org) tickets from them.

Proctor uses the unittest framework to run tests, and produces a parseable output report. It also includes a library for parsing those results so they can be processed. Your task is to use that library to create a program to load the test results into trac to open tickets for failing tests.

All of the tests from the same class which FAIL for any reason or have the same ERROR (determined by the name of the exception causing the error) should be included in the same ticket. The summary of the ticket should include the test name, as reported by Proctor. The body of the ticket should include all of the information available from the parsed test report (for each test, the name, exit status, and traceback, if any).

Your program must search for existing tickets for the same failures and avoid submitting duplicate tickets if there is already an open ticket for those failures. One simple way to accomplish that is to use a well-defined formula for setting the summary so you can search for the value.

For extra credit, include a command line option to your program to set the component and keywords of the new tickets created.

Completion:

You may start your own project for this program, or contribute it to Proctor (you will be credited when a new release of Proctor comes out if you contribute there). If you start your own project, link to the home page for the project via a comment on this ticket. In either case, attach a copy of the source for your program to this ticket. If you need to make changes or enhancements to Proctor, attach a diff file suitable for patching the source.

Task duration: please complete this task within 5 days (120 hours) of claiming it.

I claim this task.

Okay, I'm working on this in a separate Google Code project but eventually I'll create an svn diff against Proctor with this included. It's not easy to find documentation for Trac though...

Yes, I thought finding docs for trac might be the hard part. :-)

Somewhere there exists a tool to import bugzilla data into trac. That source may give you ideas for how to implement this. One difference is that the bugzilla importer does not generate email for new tasks, but I would really like for this tool to do so.

I should qualify that: If you can get it working without sending email, that's acceptable. If you manage to make it send email, I'll count that as extra credit.

Generate email? What does that entail? As in it specifies a CC Address for the tasks? Because the application can already do that. If, however, you would like me to make it a command-line option to send an email digest of new tasks, then sure, no problem. In fact, I think it should be quite easy to integrate this as part of Proctor, but I don't know what to do about the dependency on Trac being installed.

Okay, email aside (although I will add that when you get back to me), I've uploaded a python source distribution (using setuptools) to the google code homepage. Check out http://code.google.com/p/proctorticket/, it's a featured download and the whole source is in SVN anyway. Enjoy! I did this in 24 hours. :)

For "generating email" I meant to have trac do that, just like with Google Code. The bugzilla importer does not do that, so I wasn't sure if it disabled email or just didn't call the function in the trac library to cause email to be sent.

I'm reviewing what you have already posted right now.

Ah, I see. The bugzilla importer doesn't generate email because it uses SQL directly in the same way you do.

A few comments:

1. I had to make a small code change to get it to work with python 2.4 and sqlite (I'm not sure which actually required the change). I've attached a diff.

2. You're doing a good job of eliminating duplicate tickets, but you missed the requirement that says: "All of the tests from the same class which FAIL for any reason or have the same ERROR (determined by the name of the exception causing the error) should be included in the same ticket." Some of the failures from example.log are part of the same test class, and should be combined into a single ticket instead of having 30 separate tickets.

So, would I be correct in assuming that "test: proctorlib.tests.ExampleTestCase.test03Failure" and "test: proctorlib.tests.ExampleTestCase.test14Error", for example, are part of the same test class? In addition, I've noticed that all of the tests are duplicated because proctor goes into the build/ directory in the root of source distributions, which is going to be the same anyway, so I shall remove duplicates as well. Thanks for the info.

Yes, Proctor scans the entire directory so if you have a build dir with copies of your code the tests are run twice.

The tests are named "test: proctorlib." + module + class + method.

I've fixed the whole summary thing; now it only adds three tickets for all of the proctor built in tests and it uses wiki markup for the descriptions. It also removes the 'build' tests. It's packaged as a setuptools sdist.

I was wondering if the email problem can be fixed by adding a record to ticket_change. I'm working on the program at the moment so that it will update fields instead of miss them out when tests have already been carried out.

I think that in 0.4 I've pretty much hit the nail on the head with the updating thing. It seems to work really well, and only updates issues if the descriptions (i.e. test results) have changed. Because the individual tests within a test class are separated out, it allows for splitting up of descriptions into individual results. This is very cool because you can then use python 'set' types to find out what's changed. Tell me what you think. About the email, I've got it adding records to ticket_change but I don't know what else to do. It can set the "cc" field in the database though, so changes to an ticket can be tracked that way. Have fun with this release.

tiny change in documentation, not big enough for a new version.

Tell me if there's anything wrong with this version, I've tested it a lot and there seems to be nothing wrong with it.

Sorry, I was out of commission for most of the day yesterday. I'll look at your latest version today.

There's another bugfix here: I copied some code from one function to another and forgot to rename variables.

Comment deleted

Bugfix for updates and a few other things.

This does almost everything I need, so I'm going to mark it completed. I'll open bug reports on your google code project, but I'd also appreciate it if you add me as a project member so I can submit fixes directly.

Good work!

Done, thanks. =]