Bug reporting sucks, and how to fix it: Part 2

6 Responses · May 4, 2011

Bug report­ing sucks, it’s clear. How can it be better?

One idea that could alle­vi­ate a lot of the headaches of bug report­ing while pro­vid­ing increased func­tion­al­ity is a desk­top bug-reporting appli­ca­tion. I won’t pre­tend to be famil­iar with any APIs that are made avail­able by Launch­pad, GNOME, Bugzilla, Trac, etc., but I have to imag­ine that some exist. I also won’t pre­tend that I’m the first to have this idea — it’s entirely pos­si­ble that it’s been attempted and that those attempts have failed many times. But from where I’m sit­ting it seems like a near-perfect solu­tion. Here are the ben­e­fits we’d get from a desk­top bug-reporting application:

  1. It can man­age your bug tracker logins.
    Imag­ine hav­ing all your bug tracker logins stored in a lit­tle data­base for you on your desk­top. More than that, imag­ine being able to reg­is­ter at a new bug tracker with just a few clicks using the per­sonal infor­ma­tion kept by the application.
  2. It can iden­tify the bug tracker used by a given appli­ca­tion.
    This may actu­ally be a bit dif­fi­cult. It’s pos­si­ble that this could be done by hav­ing appli­ca­tions make this info avail­able in some way, although that puts a bur­den on appli­ca­tion devel­op­ers, and I assume there is no sys­tem cur­rently in place for some­thing like this. It’d require some creativity.
  3. It can iden­tify the pack­age in the appro­pri­ate bug tracker against which to file a bug.
    Launch­pad is prob­a­bly the bug tracker with which I’m most famil­iar, and there in par­tic­u­lar it seems to me to be plau­si­ble for a bug-reporting appli­ca­tion to eas­ily find the pack­age against which a bug should be filed. How many peo­ple even know that Ubuntu’s default PDF viewer is called “Evince,” or that its default image viewer is called “Eye of GNOME,” or espe­cially that the Eye of GNOME pack­age is called “eog”? Imag­ine instead being able to choose from a list of run­ning appli­ca­tions and hav­ing the bug-reporting appli­ca­tion do the rest.
  4. It can pro­vide bet­ter search for dupli­cates.
    Launch­pad only presents you with pos­si­ble dupli­cates after you’ve entered the title of your bug report. A desk­top appli­ca­tion, on the other hand, could con­tinue to search for dupli­cates as you’re elab­o­rat­ing on the bug in your descrip­tion, updat­ing on-the-fly and pre­sent­ing you with rel­e­vant excerpts from the poten­tial dupli­cates’ descrip­tions. And if it has a high enough con­fi­dence in a match, it can ask “Are you sure?” before allow­ing you to submit.
  5. It can col­lect and upload sys­tem logs.
    Not only could a desk­top appli­ca­tion attach basic sys­tem logs and hard­ware info, but it could cat­a­log which version(s) of your application’s depencenc(y/ies) you have installed, poten­tially sav­ing hours of work per bug.
  6. It can open an impromptu chat with oth­ers suf­fer­ing from the bug, or with pack­age main­tain­ers.
    By check­ing the online avail­abil­ity of other bug suf­fer­ers or pack­age main­tain­ers, this appli­ca­tion could inte­grate with Empa­thy, Pid­gin, or IRC to put you in con­tact with them, if they choose to make them­selves available.
  7. It can search exist­ing help avenues for poten­tial solu­tions to your prob­lem.
    Stack Exchange (Ask Ubuntu, Super User), Quora, Ubuntu Forums — search­ing all of these places takes time, and might not even occur to a large num­ber of users. Search­ing these places on-the-fly as bugs are being described might pre­vent unnec­es­sary bugs from being filed in the first place, or direct bug suf­fer­ers toward tem­po­rary workarounds.
  8. It can look for PPAs that con­tain updated pack­ages that might fix the bug you’re report­ing.
    Many times the appli­ca­tions in offi­cial repos­i­to­ries are out­dated, and many times the most com­mon bugs have been fixed in more recent releases. Given ade­quate safe­guards, this could be invalu­able for peo­ple des­per­ate for a solution.
  9. It could inte­grate with the mes­sag­ing menu.
    “Some­one just marked one of your book­marked bugs as ‘Affects me too.’ They are avail­able for chat. Would you like to con­tact them?“
    “The bug ‘[bug descrip­tion here]’ has a new comment.“

I’m sure with my lim­ited imag­i­na­tion I’ve not even scratched the sur­face of what a desk­top bug-reporting appli­ca­tion is capa­ble of. What else would you like to see in one? What would some of the pit­falls be? Is this prac­ti­cal or imprac­ti­cal, and why?

∞

I do agree that bug track­ers suck, but I don’t think your idea makes a lot of sense:

> It can col­lect and upload sys­tem logs.
> It can iden­tify the bug tracker used by a given appli­ca­tion.
> It can iden­tify the pack­age in the appro­pri­ate bug tracker against which to file a bug
Ubuntu has ‘apport‘ and a lot of appli­ca­tions have “Report a Prob­lem” entry in the Help menu, that should take care of bug report­ing (at least for GUI appli­ca­tions). It also col­lects logs.

> It can man­age your bug tracker logins.
this is already done by the browser

> It can pro­vide bet­ter search for dupli­cates.
this is already done by bug track­ers (or at least should), what I’d like to see is inte­gra­tion between bug track­ers (upstream/downstream)

> It can open an impromptu chat with oth­ers suf­fer­ing from the bug, or with pack­age main­tain­ers.
I don’t think it would be fea­si­ble, nor it would make a lot of sense

> It can search exist­ing help avenues for poten­tial solu­tions to your prob­lem.
> It can look for PPAs that con­tain updated pack­ages that might fix the bug you’re report­ing.
This infor­ma­tion should stay in the bug web­page as a reference.

What I would like to see is inte­gra­tion between dif­fer­ent bug track­ers and the abil­ity to for­ward a bug with­out hav­ing to login again, copy­ing the bug report etc. Also, merge bug com­ments in a mean­ing­ful way to min­i­mize dupli­ca­tion of effort. If you have a bug in EOG you will prob­a­bly have three dif­fer­ent bug reports about it, one upstream, one in Ubuntu’s bug tracker and one in Debian’s. It is not unusual that each of them have dif­fer­ent info about a fix or a patch attached, this is plain stu­pid.
I would push for strict DVCS inte­gra­tion with bug trackers.

I like launch­pad because you can link a branch with the fix in the bug page, AFAIK bugzilla can’t do that.

Anon · 4 May 2011
∞

I worked on the design of Launchpad’s bug tracker (along with other parts of Launch­pad) from 2005 to 2007. We got some things wrong, but I think at least we improved on Bugzilla. ;-)

The pur­pose of a bug tracker is to improve soft­ware. It does this by help­ing devel­op­ers make the best use of their time, giv­ing answers to the ques­tion, “What should I do next?”.

It is easy to con­fuse this pur­pose with other pos­si­ble pur­poses. For exam­ple, it is not the pur­pose of a bug tracker to act as a tech sup­port forum. A bug report used as a sup­port forum rapidly becomes harder and harder to use as a forge for actu­ally fix­ing the bug. So, it may be inter­est­ing to put reporters in touch with other peo­ple suf­fer­ing the same prob­lem, but when design­ing a fea­ture like that you would need to be care­ful not to encour­age chit-chat in the bug report itself.

As another exam­ple, it is not the pur­pose of a bug tracker to make it easy to com­ment on bugs. Bugzilla got this wrong, with large com­ment fields and com­ment num­ber­ing and reply quot­ing and so on, which made it eas­ier to chat in a bug report and there­fore harder to pay atten­tion to bug noti­fi­ca­tions or to under­stand what a bug report is actu­ally about.

Most impor­tantly, it is not nec­es­sar­ily the pur­pose of a bug tracker to make it easy to report bugs. As the num­ber of open bug reports increases lin­early, the time for triagers and engi­neers to deal with those reports also increases nearly lin­early, but the prob­a­bil­ity that any of those bugs will be the most impor­tant thing for a devel­oper to work on next *decreases* expo­nen­tially (because new reports are more often dupli­cates, or tiny edge cases, than old ones are). So, some­times more bug reports are actu­ally a bad thing.¹

For exam­ple, right now Ubuntu has 96180 open bug reports, of which 56241 — more than half — are “New”, i.e. Unconfirmed.So what would help Ubuntu improve most are things like bet­ter dupli­cate detec­tion, more effi­cient triage, and pos­si­bly even *fewer* bug reports to begin with.² None of that would be helped by a GUI appli­ca­tion for bug reporting.

That isn’t true for every project, though. A project may have few users, but an active main­tainer, who is happy to give care­ful atten­tion to every bug report. In that case, bug report­ing should be easy, and some of the other fea­tures you describe would make sense.

One awk­ward thing about Launchpad’s bug tracker is that it doesn’t cater for this vari­ety of projects. It is exactly as easy to report a bug on a project that is over­flow­ing with bug reports, than on a project that really would like more of them .

______
¹ In a pub­lic open-source project, this isn’t the whole story. For exam­ple, a new con­trib­u­tor may start out by brows­ing the bug list, and pick a low-importance bug first because it’s easy to fix.
² For example,anyone used to be able to report a bug on Ubuntu with­out spec­i­fy­ing the pack­age first. Nowa­days, you can do that only if you’re a mem­ber of a par­tic­u­lar team, or if you know the secret URL. This is a rough way of simul­ta­ne­ously mak­ing bug-reporting a bit harder, and mak­ing triag­ing, on aver­age, a bit faster.

mpt · 4 May 2011
∞

Thanks for your feed­back mpt. I always enjoy read­ing your take on things.

I under­stand what you’re say­ing about the risk of turn­ing bug reports into sup­port forums. That’s a pet peeve of mine as well, both as some­one who sub­scribes fre­quently to bug activ­ity and as some­one who sym­pa­thizes with the plight of those respon­si­ble for bug reports. I under­stand that putting peo­ple in touch with pack­age devs and main­tain­ers could be a major annoy­ance, and I don’t think I ade­quately expressed my trep­i­da­tion at the idea. Of course there’d have to be many safe­guards. It’s not a mature idea.

I do think how­ever that it could be valu­able to have IRC chan­nels devoted to a bug. I can imag­ine a sce­nario where two or more peo­ple who are affected by a bug would find it pro­duc­tive to talk about their symp­toms and poten­tially arrive at a bet­ter diag­no­sis or a workaround them­selves. If any­thing I think some­thing like that could help alle­vi­ate the bug-report-as-forum prob­lem. Not to men­tion a simul­ta­ne­ous search on actual sup­port forums. Again, these ideas are not fully fleshed-out.

Launch­pad is in my opin­ion one of the bet­ter bug track­ers there is, so I appre­ci­ate your involve­ment there. I’m not sure I agree with you how­ever that it shouldn’t be easy to report bugs. Surely it shouldn’t be easy to report dupli­cate bugs, or detail-starved ones. But pro­vided that all bug reports are new and well-articulated, how can that be a bad thing?

A GUI wouldn’t help as much, of course, if Launch­pad were the only bug tracker (or if any sin­gle bug tracker were, in fact), but it’s not. A uni­ver­sal inter­face to all vari­eties of bug track­ers would, I think, be an enor­mous boon to peo­ple who would like to help out by fil­ing bugs.

Another thing you didn’t address — some­thing that I think is one of the most valu­able parts of the idea — is that a desk­top GUI can know far more about your OS than a web­site. It’s not uncom­mon that I see a con­ver­sa­tion tak­ing place within a bug report wherein devs are ask­ing users for more info, info that could be more effort­lessly pro­vided through a desk­top appli­ca­tion. I remem­ber in par­tic­u­lar a bug I was report­ing with Miro — (a) whose pack­age is at lp.net/democracy (I remem­ber Miro from the days when it was called “Democ­racy Player,” but who else does?), and (b) whose bug tracker is at pculture.org (???) — and this bug actu­ally came down to a bug with libtor­rent. Surely it would have been eas­ier to iden­tify that the prob­lem was with a depen­dency rather than with the Miro pack­age itself if peo­ple report­ing on the bug had thought to include their ver­sion of libtor­rent — but of course most don’t.

Also if you’ve never read my blog before, please excuse the cranky tone. I exaggerate.

Jay · 4 May 2011
∞

I think it’s a great idea. I like launch­pad, but when I have to go to any other project and enter a report, It’s always a pain. Even cooler, this kind of app could help you keep bet­ter track of all bugs you’ve com­mit­ted, not just the ones on a par­tic­u­lar site.

Daniel · 5 May 2011
∞

Project Plan­ning and Sup­port Forums may work bet­ter as sep­a­rate appli­ca­tions. See the var­i­ous web appli­ca­tions that use get­sat­is­fac­tion for com­mu­nity man­age­ment and piv­otal tracker for project plan­ning. These are just a pair of the more pop­u­lar ones, and FOSS may want to exper­i­ment with a sim­i­lar model.

Jonathan Dahan · 5 May 2011
∞

I can’t prac­ti­cally tell which bug report about Ubuntu gets com­ments most fre­quently. But the most-commented ones I’ve seen are con­tro­ver­sial, not severe, and even they get a com­ment only every few hours. Com­bine that with the pro­por­tion of bug reporters who would be inter­ested in chat­ting at all, and the chance that any­one else would be in a bug report’s IRC chan­nel at the same time as you are is tiny.

I shall try again to explain why more bug reports isn’t always bet­ter. The more bug reports there are right now, the more likely the next one — no mat­ter how well artic­u­lated —will be a dupli­cate that, for what­ever rea­son, isn’t inter­cepted auto­mat­i­cally. And the more likely it will be an edge case that, while unfor­tu­nate, isn’t impor­tant enough ever to come up as the thing that one of the exist­ing devel­op­ers should work on next. But the time required to triage that bug report is pretty much exactly the same as the time required to triage any other bug report. There­fore, after a point, bug reports have dimin­ish­ing returns. If the num­ber of devel­op­ers and triagers (and the effi­ciency of the bug tracker) increases to keep pace with the num­ber of bug reporters, this is fine. But except in the very geeki­est projects, it does not.

Many Ubuntu appli­ca­tions already have a “Report a Prob­lem” menu item that nav­i­gates to the cor­rect Ubuntu pack­age. I don’t know whether it also trans­mits other data (like depen­dency ver­sions) appro­pri­ate to that pack­age. Unfor­tu­nately, the menu item appears even in Ubuntu releases, not just in alphas and betas — while crashes, which could eas­ily be tracked in a sys­tem that doesn’t require a Launch­pad account or Web browser at all, are tracked *only* in alphas and betas. It is rather back­wards. Also unfor­tu­nately, the menu item is AWOL from many appli­ca­tions, includ­ing Miro — though Miro has its own “Report a Bug” menu item going to its own bug tracker.

mpt · 5 May 2011

Leave a Comment or Subscribe