Bridging The Bug Gap
Bugs are part and parcel of software: human beings make software, and human beings make mistakes. Handling this influx of defects, errors and oopses is a complex job in itself, but the real challenge is how we bridge the gap between developers and users. Jono Bacon and Stuart ‘Aq’ Langridge explore the nuances of how we bridge this gap, whether it is possible and what opportunities we face if we get it right.
Ladies and gentlemen, boys and girls, we are the very start of the conversation, so let’s dial up the heat and make this a rocking conversation by you good folks sharing your thoughts on how we bridge the bug gap. What do you think? What is your experience with bug reporting, how do you think it can be improved? What are your observations on the cultural differences between developers and users? Join the conversation in the shot comments below…
23 Comments to “Bridging The Bug Gap”
Leave a Reply
@sil I don’t have permission to see bug #338160
:p
Me Neither
What Stuart said about duplicates in large projects is a good point, it’s easy to miss them.
Perhaps if bugs could be “tagged” to organize them into some kind of grouping, e.g. “crash”, “ldap”, “resolution” etc. and you could just view those tags, it might make it easier to see duplicates in projects with a large number of bugs. Even users who don’t know much about a bug could probably tag them with a common term…
(Note: I know some bug trackers have “keywords” which is similar (well, the same) but I can’t see that functionality in LP (unless I’m being dumb))
LP has tags; in between the bug description and the comments. They do help.
So it does! Perhaps they should be more prominent
hmmm … I’m thinking about a semantic bug tracking tool right now
I have tried to get into triaging from time to time, but the largest barrier for me is the fear that I may be making things worse by triaging incorrectly. My own idea on fixing this problem is a bug triaging simulator built into the bug-tracker. The idea is someone, probably a developer, finds a bug that has been successfully triaged/fixed, then takes note of all the tasks that had to be completed by the triager to get the bug in a state that the bug could be fixed. Using the admin side of the simulator the developer sets tasks to be completed by the triage trainee on a pseudo bug, (look for duplicates, asking for additional information, adding appropriate tags, finding the right package, etc). The trainee could get points or at least feedback on the number of tasks they completed correctly, and what they should have done instead, all automatically from the bug tracker based on what the Dev originally selected as the correct way to triage the bug. Its like a Dev creates a lesson from a correctly triaged bug, for the trainee to learn the correct process of triaging that bug by triaging a simulated bug based on an actual bug. Once its done it can be reused by many. It provides a way for developers to say “this bug was done correctly now you go work through it correctly”. The process can teach a triager best practice and gain confidence that they are doing it correctly.
Ideally it would use the same type of interface as the bug tracker so when a trainee “graduates” from the simulator every thing is very natural.
Not every thing could be learned in the simulator but I think there are many things that could be learned, and the bug ecosystem would be better off if more people were doing those things which could be learned.
My two cents, great shot.
I think that triaging could be done in a similar way that Aardvark (http://vark.com) works.
It would have software (launchpad) to keep track of triaging-voluntaries and what software and hardware they are using. Then when a bug is filed launchpad would get in touch, through e-mail, IM or something, with other users/voluntaries who uses similar software/hardware and ask them to confirm the bug and perhaps expand on the information.
Aardvark tries to be clever about contacting those people who are online/have time right now, rather than spamming everyone with the same stuff at the same time. I think that approach is rather genial and makes it rather fun instead of a burden.
Same principles can of course also be used for triaging.
By the way Aardvark was “just” bought by google so I’m sure they must have done at least something right
https://bugzilla.mozilla.org/show_bug.cgi?id=338160 – Add dates of replying/forwarding messages
https://bugzilla.mozilla.org/show_bug.cgi?id=85793 – link turnes into description. (sic)
Not very special…?
Gerv
Touché
Launchpad has half a million bugs? Hey, bugzilla.mozilla.org has over half a million for the Mozilla project alone. We have more bugs than you, ner ner ner etc.
Kudos to jono for saying that bug trackers are hard to build. He’s quite right. Everyone has an opinion on how it should work, they are all strong (because people use them every day), and most of them are contradictory. And, at the moment, the Bugzilla development community (a bit of software with over 10,000 installs, including probably the majority of high profile free software projects) is pretty small.
Gerv
On the Xibo project, we encourage our users not to file bugs, but instead to use Launchpad Answers to ask questions when they encounter problems.
It’s then really simple for us to keep track of what issues are outstanding, and if there is a problem that we think is a bug we can help them to gather the appropriate information and then file a complete bug report outright.
I really think that’s working alot better for us than people filing “it doesn’t work” bugs.
Alex
That’s an interesting approach. So people use Answers in a way like getsatisfaction, and then you can convert an Answer into a bug. Clever! This is one of the use cases for Answers, but people aren’t doing it much
Pretty much. The flow goes that any question/support request goes to Answers. The user then gets a response within a few hours normally either saying that the issue they have is already fixed (and given an ETA for a landing in a stable release and a link to the appropriate bug), is a new issue and what new information is required (at which point someone will attempt to duplicate and confirm and then convert to a bug as appropriate), or a general answer if it’s something non-support related.
Feature requests go straight in to Blueprints.
It works really well for us, and most of our users seem to like the support service they get from us.
Alex
hey that sounds like a really good idea
Speaking as a Launchpad Bugs dev, I can vouch for how hard it is to build a good bug tracker.
We’ve had a number of different strategies over the years for picking how to proceed with development, but I think we’ve finally found a really good one now that we have a Technical Architect (Bjorn Tillenius) and a Product Strategist (Jono Lange). We’re now doing a lot of consulting with out major stakeholders (which is mostly Ubuntu, true, but there are several other users who need us to pay attention to their problems) before we decide what to implement next.
In fact it’s just been decided this week that for the next LP development cycle we’ll be focussing on making Launchpad mail notifications not suck, which they have done for a while. There’s a lot of work to be done there, and it’s perhaps 80% bug fixing and 20% feature development, but I think that it will finally make a lot of people a lot comfier with using LP to track bugs.
Erk. Should have been “with our,” not “with out.”
The mail notifications are a double-edged sword. It’s really great as it is now to know exactly what’s going on but releases generate literally 50 or 60 emails for us (and we’re only a small project).
In general though we’re really happy with the services Launchpad provide and LP Bugs is easily the best of the bunch that I’ve used to date.
Alex
Yeah, we’re aiming to make your LP mail notifications configurable, so you should be able to filter what you want to receive at the source.
And thanks. We try.
This is one of those areas that is hard to fix because of the divide between the different levels of users. Basic users: Dont know anything about the inner workings of the system. If they see a bug they tend to report it even if they dont know how it was caused. Then they trust that the bug will be fixed soonish but in many cases it wont because they didnt give enough info. This group is the most likely to file a bug against the wrong package or to use the term “I dont know” or fill in anything in the additional info section. Advanced users/developers: Know what went wrong a lot of the time will report the bug or fix it themselves if its something small. This group also know who to ask if there is a major bug that needs attention. The basic users have a bit of a problem getting attention I think. If they find some big bug it takes a while for the devs to see it and give it proper attention. So then we get a sometimes large time gap before a fix is released. The sheer volume of bug reports causes its own problem. Devs and bug huggers the world over spend a lot of time (way too much time) looking at sometimes already fixed bugs or old bugs or bugs without enough info. There are some serious problems with the bug process but still its better than no process at all…
I think a significant part of this problem comes down to the attitude of developers who are working on ‘free’ software relative to those who’s users have paid licensing or support fees.
In the later case, often more courtesy is extended because the reporter of the bug is someone who is literally putting food on the developer’s table. In contrast, when dealing with bug reports for free software the developer can often view it as a favor that they are extending to the user.
I work on both free and commercial codebases and this often runs through my head when I receive bug reports. As a result it is very easy to be terse or ignore bug reports from users of the free software. For example, if a duplicate bug report is posted, the free software user has in effect wasted some of my time, whereas paying customers ‘are always right’.
I have also found that some users of free software can be more demanding with their support requests. I think this probably stems from the fact those receiving commercial support have a better comprehension of what your time is worth, and understand the limits of what is a reasonable request for help.
Maybe it is a case that when it comes to free software and bug reporting we need to establish a social currency that allows both the reporter of the bug and the developer to better understand the value of the process and what each other’s time is worth. For example the points system used by StackOverflow could be a useful guide if it was to find its way into the process of bug reporting.
I know this shot is a week old but I have only just listened to it (and it has touched a bit of a raw nerve).
My favourite bug is number 173332 (on Launchpad). I raised this bug in December 2007 after trying to use Ubuntu as a file server but this became impossible as NFS was causing serious network problems. I know the details were pretty sketchy but I expected a developer to come along and ask me to try this, that and the other and report the results. In fact, I didn’t hear anything until nearly two years later, when someone noted that I was using an old version of Ubuntu, and could I please upgrade and try again. Of course I was using an old version – three versions had been released since I raised this bug! I had since given up trying to use Ubuntu and I was not inclined to re-design my network to test this again. It is entirely possible that this was not just a feature of the way I had my network set up, rather than a genuine bug, but now we will never know – the bug is now marked ‘invalid’ and, perhaps, there may be scores of other people who have also tried and given up with an Ubuntu file server. All I know is that a CentOS server with Ubuntu clients has no such problem.
OK, rant over.
I work as a software tester for a large IT company, so perhaps I see all this differently. Defects like the one I reported, which appears to be a regression from Feisty to Gutsy, would need to have either been fixed or adequately explained in a release report before the software went out the door. One problem we have in the “Open Source” world is that testing is generally done after release and so there is no opportunity to hold up a release when a serious problem is found. Another problem is that there often isn’t someone whose sole task is to be responsible for defects and to push back on the developers when he or she things that the software is not good enough.
(I know “Open Source” is not really correct here. What should I say? Community Driven Software Development?)
By the way, as a bug report for your site, it would be useful to have a ‘Preview’ button as well as ‘Submit’. I like to read through what I have written before submitting and this is not so easy in a 50×8 text box.
My experience of filing bugs in Ubuntu has been horrific. For example, in the current Lucid Beta 1, I found a bug and took a screenshot of it. I looked for menu items relating to filing bugs. Couldn’t find any. I went to launchpad.net and eventually found the tiny Ubuntu icon at the bottom of the page. This linked to a page with another link saying “how to file bugs in Ubuntu”. This in turn took me to a page on wiki.ubuntu.com which told me to… use the “Report a bug” option on the Help menu. This still didn’t exist when I looked. Other options on that page didn’t help either: apport just wanted to file bugs about kernels and X and other low level stuff. I only wanted to report a theme bug. This would be the experience of any new user wanting to contribute to the development process by trying out a beta. The problem isn’t that there are bugs, it’s that we make it too hard for people to report them.
So I have experienced two bugs in Lucid which I haven’t reported. Similarly, where I have found (via Google!) bugs in Launchpad which affect me, I have seen few been pro-actively fixed. At best they get reviewed 6 months down the line when someone spots this is no longer a problem in the latest release.
Whilst developers need quality information in bug reports, users need to feel they can file bugs without pulling teeth out.