Introducing Traceback Tracker 29 Mar, 2009
I think that identifying errors, in an automated way, is important for lots of software projects to reduce the effort required to find duplicate tickets or respond to users when nobody else is around. Launchpad uses something for identifying similar bugs when reporting a bug, but it appears that it’s just Bayesian, since it has to work with lots of different errors — some in the gui, some from different runtimes, etc. When your problem space goes down to just Python errors, which have a well-defined format, you can do smarter, more exact things. This is a project that Eli Carter and I are working on, feel free to join in on the Testing in Python list.
I’ve seen entirely too many IRC “conversations” that go like this:
[02:30 am] * steve__ joined
[02:31 am] <steve__> Anyone seen this, can you help me? http://somepastebin.com/blah-be-de-blah
[03:00 am] * steve__ quit
[07:30 am] <otherguy> Morning.
And a lot that go like this:
[10:40 am] <steve__> I'm getting http://somepastebin.com/blah-be-de-blah when I florb the blots.
[10:50 am] <otherguy> What database server are you using?
[11:00 am] <steve__> Red Hat
[11:05 am] <otherguy> No, I mean your sql database server
[11:15 am] <steve__> mysql.
[11:20 am] <otherguy> Are you using Trac >=0.11? We had a known issue with certain versions and mysql.
[11:25 am] <steve__> I don't know, how can I check?
[11:30 am] <otherguy> Look in the footer or the about page.
What I’d like to add to the first one is a short delay followed by helpful solutions:
[02:33 am] * identibot suggests checking tickets #123, #234, #345 while waiting
for a response, sometimes people are asleep. Please stay around,
or file a ticket if it's not listed above.
And for the second, to summarize the traceback, both obvious information and latent facts that we’re able to derive from comparing against open source releases:
[10:40 am] * identibot thinks steve__ is using Trac 0.10 with MySQL backend x.y.z,
and got UnicodeDecodeError, on POSIX with Python from rpms.
So, at the bare minimum, we need a bot that identifies those facts so followup questions for facts that can be determined don’t waste time. This helps both us and our users, because they get a faster response. What would be really great is to index problems/solutions, say, something like tickets from a ticket system, and identify solutions based on this for when users aren’t around (most of the activity in #trac is during the US daytime).
The goals as they exist now are
- Parse some tracebacks (DONE)
- Make educated guesses at what’s in use (DONE)
- Support most alternate formatting (right now it’s pretty targeted at what we get in Trac reports, but can be extended)
- Fingerprint tracebacks, so you can find similar bugs based on code path (first stab, use
filename:1,2,5;filename:20,31;TypeError
and compute edit distance) - Respond to users in IRC and when reporting tickets in Trac
but please, join the discussion and let’s find out if there are other ways to use it. Here’s the source code and the release announcement