Triaging Bugs for Firefox

The document is to help anyone that's interested in QA at Mozilla to learn techniques on how to help bugs go along their way. It will help important bugs get fixed faster and optimize our community's Bugzilla efficiency. Before we get into the details, it's important to know what the Triage does as well as why its important.

What is Triaging?

Bug Triaging is the process of moving bugs logically from state to state, so they get resolved in an efficient and easy-to-comprehend manner. In Mozilla, QA must not only test and then file their own bugs, but must also help harness the incoming flow of community bugs for valid problems.

Why would you want to help?

Bugzilla is the heart of the Mozilla project, as every Mozilla product tracks its software defects and enhancements in BMO. There is nothing in the project quite as central or as important as BMO. Triaging is required to ensure that all the defects reported on Bugzilla get responded to. A backlog of bugs in any state, especially the UNCONFIRMED state, affects our ability to react to your important issues quickly and efficiently. Thus, our bug triagers are incredibly vital to our product development.

Requirements

Get Set Up

• Download the newest nightly build (recommended)
• Register an account with Bugzilla

Bugzilla Permissions

• In order to modify bugs, you need to have at least "canconfirm" permissions.
• If you've been using Bugzilla for quite some time, you may already have these privileges. To double check, look at your Bugzilla Preferences. If your permissions say "Can confirm a bug", then you have them!
• Otherwise, you'll need to get them. To find a good place to learn about gaining permissions, please read What to do and Not to do on Bugzilla.

Creating Search Queries on Bugzilla

Bugzilla offers a couple of ways for a user to find bugs on its database. There is a simple search option and an advanced search option. Each one offers a unique way of searching that allows the user some flexibility in how they want to perform their search. A simple search will only restrict search parameters to any currently not-fixed bug that match the string input by the user. Meanwhile, the advanced search form allows the user the ability to really cut down on any glut found through a simple search. But it can be a bit confusing. So, Bugzilla offers a "Give me Some Help" hyperlink located in the top left portion of the form that reloads the page with hover-enabled tool-tips to guide you through each module. This link also give you more information about searching in Bugzilla.

Persistent Queries and how to Manage them

• After creating a query, you might want to keep that query for future use. If that's the case, it can be transitioned into a Saved Search, which will appear in the page footer. To save a search, scroll to the bottom of the page, fill out the text field at the bottom of the page and click on the "Remember Search" button.
• It's very useful if you only plan to go through the bugs every so often, or if you need to track something based on keywords or fields other than product/component, or when you need to look through older bugs.

Tracking Components

If you want something more immediate, you can choose to track a component, such as Disability Access. This is possible via your Bugzilla preferences using the "Component Watching" option. Make sure to track the relevant component in both Core->Disability Access APIs and in the products you care about, such as Firefox (Firefox->Disability Access) or Thunderbird.

Clean incoming Bugs

Why is it Important?

Ensuring bugs are properly marked helps make sure they don't get ignored. For example, if a bug that really affects all platforms is marked "Solaris" because the original bug author was on a Solaris machine, most developers will ignore it. Other common issues include major bugs that are marked normal, bugs with a confusing summary, etc. So, ensuring that the bug is properly marked is very helpful in getting the attention it deserves.

What would Speed Up the Process?

Each time you submit changes to the bug it creates extra Bugmail, so it's better to do the following steps in batches, if possible.

• Find the Inherent Issue: Read very carefully to try and understand what they're talking about. For a guide on how to simplify test cases, go here.
• Look for duplicates: A lot of bug reporters mistakenly assume their bug haven't been filed yet. Search for previously filed bugs by searching for keywords in the summary in the bug. For example, if the bug is about the control key, try spelling it ctrl as well. If the user includes a crash report, you can easily clink on the link to see if the crash has already been reported.
• Use the "needinfo" flag to ask for more information from the reporter: If you don't understand the bug or can't reproduce the bug, ask the reporter for more information by selecting "Set Flags" (on the right hand side of the bug entry page) until you have enough information to try it yourself. If they did not provide build information, get that right away, because many bug reports are for older versions that have long been fixed. Some other information you could ask for: - Does the bug occur in Safe Mode? With a clean profile? - If it's a crash, ask for Crash Reporter Data found via the "about:crashes" page. - If the reporter still can reproduce but you can't, ask him/her nicely if he/she can test the latest nightly build.
• Ensure appropriate hardware/platform tests: most bug filers will only have tested on one platform. If bug was filed for a particular platform, see if it's a bug on another platform. If it's been reproduced on at least 2 platforms that aren't both Unix/Linux, then go ahead and mark it All/All ((Hardware/OS). If you don't have the capability to test across multiple platforms, going to the #qa channel on irc.mozilla.org is an excellent way to find people that can verify your bug on other platforms.
• Ensure a concise and precise summary: The summary should be 60 characters or less and allow the bug to be reachable via an accurate simple search.
• Ensure Proper Severity: Go to this link and scroll down to the "Severity" section for a handy guide. If you feel like the listed severity on the bug report is not accurate, make sure to comment why the bug should be changed and change the state.
• Change Product, Component and owner: Many bugs get marked "Firefox" when they are really bugs in the Core engine. Core shared components used by Firefox and other Mozilla software include :
  • Handling of Web content
  • Gecko
  • HTML
  • CSS
  • layout
  • DOM
  • scripts
  • images
  • networking
  • Issues with web page layout probably go in Core
  • Firefox user interface issues belong in the Firefox product.  After you submit a Product change to the bug, Bugzilla will allow you to change the Component as well. When changing Product and/or Component you usually want to select the radio button "Reassign bug to default assignee and QA contact of selected component ".
• Add Keywords: here are just a few of the important ones (the full list of keywords is located here).
  •  "qawanted": bugs which need more info, or it needs reproducing, or testcasing, or it's a dupe but you can't find what it's a dupe of. Remove this keyword when the wanted QA work has been completed -
  • "regressionwindow-wanted": needs someone to narrow down the time period where it happened, ideally to a specific checkin - "testcase-wanted": needs a testcase or its current testcase needs to be simplified in order for the bug to be fixed
  • "access": mark accessibility bugs with the keyword "access". This allows accessibility bugs to not be tied to the "Disability Access" or "Keyboard navigation" components, but to the actual component with the problem. For example, the developers for Places should really be responsible for accessibility bugs in Places, but adding the keyword "access" makes sure.
  • "regression": mark regressions with "regression". Was this ever not a bug? For example, did it work in Firefox 37? See below for more important information on how to deal with regressions.
  • "crash": pretty obvious :)
  • "dataloss": this bug will cause users to lose data
  • "hang": freezes or locks up the application
  • "testcase": indicates that a simplified testcase is included (see below)

Steps to Reproduce

Sometimes, a bug report contains no steps to reproduce or steps that only work for very few people. In that case, it is extremely difficult for a developer to debug and fix the reported issue. Most of the time, these bug reports (and crash reports, where applicable) will contain multiple clues or variations of steps and it is needed for someone to work through them and narrow down reliable steps to reproduce. Here are a few guidelines for narrowing down those steps:

• Read everything in the bug report and any other information linked there (crash reports, support entries etc). This information will tell you where to start.
• Use your instincts. Your experience with using the browser can help you realize where you should be looking.
• Ask for additional information from the reporter using the needinfo flag if there are no helpful clues in the bug report.
• Working with the same OS and Firefox version as the issue was reported on will give more chances of reproducing it and finding the right steps.
• Even if you don't manage to find reliable steps to reproduce, update the bug with the details of your work. This will help anyone else willing to give it a try later.

Minimal Test Cases

Reporters usually add to their bug reports the links for the web pages they saw issues on. Some of these pages are very complex, making it quite difficult to realize where or why the bug happens. To help investigate and debug a issue with a complex web page, you first need to reduce it to a minimal test case (i.e. a test case that contains the minimum set of elements that reproduces the issue). Here are some steps for using the process of elimination to create a minimal test case:

• Save the web page on your local hard drive. Make a second copy.
• From the first copy, cut out big pieces of CSS, JS or HTML -- start with the pieces that are most likely to not have the bug. For example, at first just try to strip out plain paragraphs as these are relatively simple. Try to make as much as possible inline. Try to strip out all images, frames. If any image is required to get the bug, try changing the src to something that isn't likely to expire, such as: the Google Logo.
• If the new version of the file still has the bug, save it as your second copy and repeat step 2.
• If the new version does not have the bug, then your second copy still contains the bug. In this case, repeat step 2 but avoid cutting out the same piece that you did last time.
• When you can no longer cut out any HTML without destroying the ability of the file to show the bug, you have a minimal testcase.

Hopefully you'll be left with a file that has only 1-5 HTML elements in it, but still reproduces the bug. At this point you can actually try to start stripping out irrelevant attributes as well You may end up with something like: <label for="myfile">Enter file name:</label><input id="myfile" disabled="disabled" type="file"> ...instead of 50k worth of HTML. When finished, attach the minimal testcase, mark the bug with keyword testcase, and change the bug summary if necessary. For example, "No borders in tables at http://www.sillystatsandstuff.com" can become something much more precise such as "tables with rowspans do not get borders". If it doesn't affect the test - add a title so it can be referred to, e.g <title>Testcase #1 for bug 123456. Also, add the bug number in the filename of the testcase. You have to understand that people from different parts of the world will be reading this, so making concise and direct sentences are much appreciated. For example, if the bug involves line wrapping, then the width of the window and the font size may impact the results. In this case, you can ensure that your reduced testcase will work for everyone by using "font-family: monospace;" and a width specified in "ch" units.

What do I take away from this?

This can all be rather intimidating for people who haven't been playing around with different web technologies such as AJAX, JS, or XML for a long time, but the most important thing to take away from this page is that there's a backend to the bugs you find when interacting with any software, hardware, or general product. Make sure to take that into account when writing your next bug or triaging a host of incoming bugs on Bugzilla! Continue reading all about Finding a Regression Window.