iamcal.com

10

Issue Tracking - The Good & The Awful

By Cal Henderson, March 22nd 2010.

(In which I expand on this tweet with some details).

Issue tracking (or bug tracking, or whatever you want to call it) is an all too often overlooked piece of the development toolset. Having a bad issue tracking system will make your software worse. You will have fewer bugs filed. You are more likely to get duplicate bugs. People will hate you. Please stop doing it.

A bad example

WebSVN is a nice piece of software written in PHP to browse local and remote SVN repositories. It's much easier to use than its Perl, Python or Ruby friends because it doesn't require SVN language bindings, which are close to impossible to compile from scratch. It's quite a pretty piece of software too; looks good, easy to use.

When I had an issue with it, I visited the project website, and then found the issue tracker under "Contribute". The link on this page for the issue tracker then takes you to an entirely different site with this awesome page:

How do a report an issue? No idea at all. Following each of these links does not help. After giving up and coming back to it later, I wonder if maybe I have to log in first? I give that a try, and then come back to the same page:

Bingo! Good job for letting me know, guys. What am I entering? I figure it's probably a defect, but what's the difference between a defect and a task? What's a feature versus an enhancement? And to report a bug I need to be a project member (what does that mean? how do I become one?) and know the component I want to report on (what does that mean?). I guess I'll click on defect and see how it goes.

Holy mother of god, I hate this form. I don't understand the sub components. Platform and OS seem hugely redundant. Why does it default to the oldest version? How do I know what priority this bug is? Who should I assign it to? Or CC? What is the URL field for? Why do forms ever have a fucking reset button? Just in case I wanted to accidentally wipe everything I entered? Why does it tell me that it used my user agent to initialize fields, when it clearly didn't? Hateful.

I understand that some projects like to have a hugely complicated bug tracking system, but that doesn't mean you need to push all of those settings onto every person who reports a bug. Even Mozilla now have a much more streamlined bug filing process. There is no excuse for this in 2010.

A good example

So if that's bad, what is good? Github is all the rage with the kids these days, but it turns out it's actually pretty awesome. I found an issue with Facebook's Scribe project, so I visited the project web page. There's an "Issues" link right in the main navigation, so I click that:

Seems simple enough - on the top left there's a "Create Issue" button, not hidden away. Clicking on it takes me right to a login page:

While it doesn't explicitly tell me that I need to log in to file a bug, it seems pretty obvious. So I go ahead and do that and am presented with the pinnacle of bug reporting forms:

A title and description. And a quick search for previous issues. That's it. Just report the bug. I don't need to understand how their development team is organized, how they handle priorities, or what they call individual parts of the software.

Behind the scenes (from my point of view) these tools can be as powerful and complex as the developers like. Just don't make me learn your particular system to let you know about bugs.

Post-Script

I write things too fast and don't spend enough time checking my work - If you spot any glaring mistakes or omissions, drop me an email and make me look dumb: cal [at] iamcal.com

I spend too much time filing bugs at my day job to tolerate crappy tools. I moan about that extensively in my book too.

Copyright © 2010 Cal Henderson.

The text of this article is all rights reserved. No part of these publications shall be reproduced, stored in a retrieval system, or transmitted by any means - electronic, mechanical, photocopying, recording or otherwise - without written permission from the publisher, except for the inclusion of brief quotations in a review or academic work.

10 people have commented

Andy
# March 22, 2010 - 4:03 pm PST
I agree. Keeping the internal organizational bureaucracy out of the external/customer facing side is a good lesson in design in general.
Michal Migurski
# March 22, 2010 - 10:52 pm PST
Agree, on some fronts. The inclusion of platforms and so on in the first example seems obviously connected to writing software that behaves differently on different platforms, duh. The Github example is short and sweet, but I can't say it's helped me much as a project owner and issue recipient on Github.

The one thing I've ever found truly important in issue trackers is that people who submit tickets start the names with verbs. I'm looking at your screenshots and seeing "better variable naming" and crying just a little bit inside. We've got a rule of thumb inside Stamen that issue names must read like imperatives: "improve variable names", "delete blah functionality", "fix broken jimmy-jammers", etc. Nothing focuses the mind of the reporter like being asked to specify what exactly they'd like to see done, and it's much easier for a developer to scan a list with actual tasks right in the sentence construction.
Cal
# March 22, 2010 - 11:00 pm PST
The platforms part in this example would make sense if it wasn't for two things:

1) the OS and Platform drops downs were equivalent and only one is needed. If I pick "Mac System 7" in OS (yes, it's a choice) then I shouldn't need to choose 'Macintosh' for Platform

2) This is a web app. It doesn't ask about your web server or PHP version.

My focus is generally on the reporting of bugs. Once they've been reported, it's up to engineers or QA to triage them into actual tasks. In an open source project, a bug reporter should be doing just that - reporting bugs. Coming up with specific tasks for engineers is not their goal. I've always worked the same way on engineering teams - I'd rather have more bugs reported and have to manage them than have fewer bugs reported with more details.

Besides, do you think the more feature-rich forms get better reports? They don't seem to, looking through open source projects' bug trackers. People who are good at reporting bugs will report them well, even if they just have a single text box. People who are bad at it will either submit bad bugs or just not do it at all.

That said, I quite like the new Mozilla flow.
Michal Migurski
# March 22, 2010 - 11:15 pm PST
Audience is critical. In my day-to-day, issue trackers are where the developers look to see what to fix or work on. From the outside world (clients, etc.) we get email which must to be broken down into actual problems, feature requests, and the like. I think this is closer in spirit to the Github form and the way you describe the pre-triage step.

I like there to be division between the reporting and the assignment, which is something I don't see in the various tools like Google Code, Trac, Github, etc. - they all basically conflate issues, reports, problems, and tasks into an undifferentiated stew of trouble. It often means that the issue tracking feature simply gets turned off on a project, because the reported problems bear little relationship to the things the developers are working on.

There's an important role for both examples above. The first is developer focused, and steeped in the details important for programmers to organize their work. Take this away and it will be replaced by paper, spreadsheets, or whiteboards. The second is user focused and tuned for easy, rapid reporting from people outside the project. Take this away, and it will be replaced by email or simple disuse.
Adam | Web Developer
# March 24, 2010 - 7:15 pm PST
The two different forms of issue tracking should be used for different things.

The 2nd form is good when you have users reporting bugs. It gives the users quick access to post whats wrong and they won't have to fill out unnecessary details. This should be your front-end issue tracker.

The first form should be used as an internal issue tracker. Our QA issue tracker looks much like this, and while ugly as anything it's extremely useful at finding out what exactly is wrong. Sure some of the info is irrelevant and could be taken out but a lot of the stuff helps us developers triage bugs much more quickly (and also can be used to keep track of features).

Both have their place in the world for different reasons (though I always wish the first one would look nicer, but they never do).
Marty
# March 25, 2010 - 11:13 am PST
We have an IT issue form that is based on an out-of-the-box e-commerce platform. There is no "Submit a Ticket" button anywhere on the front page - you have to go into "Service Catalog" to find a menu of:

-- Generic Service Request
-- Report an Incident
-- Reset a Password

Not sure really what the difference is between the first two - they seem like the same form.

But it gets better -- when you submit your ticket, you can choose a quantity and your confirmation shows that each one cost you $0.00! Even better, the confirmation isn't really a confirmation - it's a "review" that you have to click a green button that is supposed to mean "Submit". If you don't - no ticket.

Gotta love systems that haven't had a UX person within a mile of them.
Peter M.
# April 2, 2010 - 3:29 pm PST
I agree, filling out those forms for bug tracking can easily take up a large portion of your day because of how complex it is to fill out all the fields.
Somehow I think whoever designed them threw in everything including the kitchen sink just so it is was all covered.
Mike
# May 4, 2012 - 12:28 pm PST
You should try Snowy Evening - designed for clients and developers to use together. snowy-evening.com. Even has GitHub integration.
Leon
# August 20, 2013 - 2:32 pm PST
If you're looking for an even better example, try AssiTrack: www.assitrack.com

I just started using it and it is awesome. Granted, it's pretty basic. But it seems to be launched recently so maybe they will add more features.
cal
# August 20, 2013 - 4:42 pm PST
Alas, windows/linux only. Something web based is pretty much essential.

Leave your own comment

Your name: Required
Email address: Optional, will be hidden
URL: Optional
Prove you're human: Enter the answer to 3 + 4