Bug reporting sucks, and how to fix it

3 Responses · April 23, 2011

Bug report­ing sucks. Every time I’m forced to do it, it’s labo­ri­ous and soul-draining. It makes me give up hope for free soft­ware hav­ing a viable future. But it’s nec­es­sary, (a) as the way I “pay” for free soft­ware, and (b) to see if maybe I can slightly has­ten a hypo­thet­i­cal future where the soft­ware I rely on daily might work rea­son­ably well (it’s coming!!).

Here are the stages of bug report­ing, and why each of them sucks:

  1. Tol­er­at­ing the flaws in the soft­ware because it’s bet­ter than fil­ing bugs about them (time frame: 2 weeks to a life­time)
    This stage is prob­a­bly the worst, in the most insid­i­ous way, because although it is less tedious and frus­trat­ing than the fol­low­ing stages, it’s the one that inspires the most hope­less­ness in you. Unless you are com­pletely enshrouded in ide­ol­ogy, you will begin to resent your com­puter more than Win­dows users do.
  2. Fig­ur­ing out against which pack­age to file a bug (1 minute to 2 hours)
    We’re only in the sec­ond stage, and already you have to have a pretty deep famil­iar­ity with Linux and bug report­ing in gen­eral to move for­ward. Say, for instance, that Flash is crash­ing Fire­fox. Holy hell do you have a lot of work to do! There are prob­a­bly at least ten inde­pen­dent vari­ables at play here, includ­ing the obvi­ous cul­prits Fire­fox and Flash, but also your video card dri­vers, your Fire­fox plu­g­ins, and your win­dow man­ager. If you don’t spend enough time exper­i­ment­ing with dif­fer­ent com­bi­na­tions of these vari­ables (remem­ber­ing that this bug almost cer­tainly isn’t repro­ducible on com­mand), then you’ll be wast­ing everybody’s time by fil­ing against the wrong package.
  3. Fig­ur­ing out where the cul­prit pack­age tracks its bugs (2 to 10 min­utes)
    Ooh, this is fun. First go to Launch­pad, nat­u­rally. You might encounter sev­eral pages that look sus­pi­ciously sim­i­lar; if that hap­pens, just close your eyes and pick one. If your pack­age isn’t on Launch­pad, you might get lucky and find a link to some Bugzilla site or the project’s home page (“I hope the devs still own that domain!”), oth­er­wise you are at the mercy of Google.
  4. Remem­ber­ing your password/creating an account at the cor­rect bug tracker (1 sec­ond to 10 min­utes)
    Oh right, there is no uni­ver­sal login for *all* Bugzilla sites, of course, so I’m going to have to cre­ate a new account. Oh, it says there already is an account asso­ci­ated with my email address. Huh. Ok, send me the link to cre­ate a new pass­word. Did that go into my spam folder? Oh, ok, finally, here it is. Alright, click. Now to file a new bug. Oh wait, I’m not logged in? I thought that would have done it. Ok, “Log In.” Ok.
  5. Look­ing for an exist­ing report con­cern­ing your bug (0 sec­onds to 15 min­utes)
    I have seen bugs with dozens of dupli­cates attached to them. Devel­op­ers must love that. Launch­pad does a pretty ok job here, search­ing for exist­ing bugs based on the title you give. It’s not often that I see a match, and when I don’t, I’m not com­pletely con­fi­dent that there isn’t one some­where in there. I usu­ally resort to Google and use my own human brain to fig­ure out other ways the symp­toms of the bug might have been worded. I don’t want to be one of those dunces who files dupli­cates, after all.
  6. Ver­bally artic­u­lat­ing the symp­toms of your bug in a way that is suc­cinct, unam­bigu­ous, and thor­ough (2 read­ings of Strunk & White)
    Ok, I hope you’ve got a lib­eral arts degree. Oh wait, you don’t, because you are using Linux. Hell, even if you do, some­times the sub­tle con­di­tions that lead to the expo­sure of a bug are so dif­fi­cult to describe that you end up with bloated sen­tences in which you’re mostly try­ing to con­vey some­thing visual or with a far too com­plex lin­guis­tic parse tree:

    “So, then if I click on the but­ton again, this time *after* clear­ing the .con­fig direc­tory, but with­out hav­ing restarted the pro­gram, the drop-down list (usu­ally just the “Pro­files” one, but some­times all of them) will lose any entries that I cre­ated (except for the ones made before I upgraded to 0.0.2.11).“

    What you find your­self want­ing to do is take a video of the prob­lem occur­ring, but that would require sub­ject­ing your­self to the hell that is desk­top video record­ing soft­ware for Linux, a path that will only leave you spi­ral­ing fur­ther and inescapably down into the toi­let that is the bug report­ing process.

  7. You’re done! Oh oops, you’re not. Col­lect sys­tem logs and info (2 min­utes to 30 min­utes)
    One of my least favorite mem­o­ries in life is the hours I spent diag­nos­ing a prob­lem with hiber­na­tion on my lap­top. Or was it hiber­na­tion and sleep? Who the hell knows, the shit didn’t work and I banged my head against a brick wall of diag­nos­tics and bug report­ing and research­ing so furi­ously that by the end of it my room­mate found me naked on the cold tile floor of our bath­room, shiv­er­ing and blan­keted in sweat. Oh, did the bug ever get fixed? Is there a dif­fer­ence between a bug get­ting fixed and your brain cop­ing with the stress of it by mak­ing you numb to it? I don’t know. I don’t know. Oh any­way, yeah, be sure to grab some logs and shit, they’ll prob­a­bly need that.

That’s it. Oh my god I hate it every time, it’s like pulling god damn teeth, and it’s a won­der any­body chooses to do this with their free time. What in the hell is wrong with me? And more impor­tantly, how can we make this bet­ter? I have an idea, but I’m too emo­tion­ally exhausted right now to get into it.

Update: Part 2

Gnome’s bug tracker is espe­cially tedious, I don’t like bugzilla per se, but Gnome’s one is not even search­able with search engines for pri­vacy con­cerns (I think?).
Ubuntu + Launch­pad is a good com­bi­na­tion, mostly because Ubuntu has apport that retrieves logs for you.
What I also like of launch­pad is that you can clone the devel­op­ment branch of the soft­ware and pro­vide quite eas­ily a link to your branch with the fix in it, which is much bet­ter than for­mat­ting a patch, and then send it by email etc.
I think many projects would gain a ben­e­fit by switch­ing to some­thing like that or even github (or a github clone).
Oh, and I hate that I have to file bugs two times! One for the dis­tro, one for upstream. And then the thread of com­ments is split in two; it would be awe­some to merge them in some way.

Anon · 23 Apr 2011

Oh of course, how could I for­get GNOME’s bug tracker? The worst. Trac too!

Jay · 23 Apr 2011

Great post­ing! You sum­ma­rized my thoughts about why I hate fil­ing bugs bet­ter than I could have done myself.

jnns · 25 Apr 2011

Leave a Comment or Subscribe