Helping the Graphics Team

This document is a work in progress. Please contact Anthony Hughes if you have any questions or feedback.

The Graphics team (aka GFX) works on all things rendering in Firefox: acceleration, compositing, fonts, games, images, scrolling, and everything in between.

Getting traction on Bugs

The best way to get attention of the GFX team on a bug report is to use the needinfo? flag and setting that to either milan (the team manager) or ashughes (the team QA engineer). You'll also find us on irc.mozilla.org in the #gfx channel. Please refer to wiki.mozilla.org to learn more about the team and the bugzilla components in our perview.

Incoming Bugs

Graphics is one of the most complex and active areas of code resulting in numerous bug reports each and every day. To manage the volume of bug reports the GFX team utilizes a rotating triage duty. Each week a different member of the team takes turn triaging the incoming bugs, making sure every bug report is prioritized and brought to the attention of the relevant developer. Unfortunately sometimes the volume is too great resulting in some bugs slipping through the cracks. QA acts as a safety net to help mitigate these occurrences. You can find out more about this process on wiki.mozilla.org.

Pending Investigation

As mentioned above the GFX team sees a high volume of bugs every day. We do our best to prioritize and investigate these bugs but sometimes a bug gets lost in the noise, sometimes an investigation runs cold. QA works to help the GFX team minimize the impact of these bugs by retriaging and retesting them. If a bug report has gone for a long time without any confirmation or inclusion of actionable information we take one last crack at finding a new lead. If that fails we close the bug report but make sure to leave the door open should some new information come to light. You can find out more about this process on wiki.mozilla.org.

Sanity Testing

The GFX team takes a three-prong approach to sanity testing, ensuring that pre-release Firefox builds do not regress core graphics functionality.

Crowd Testing

We have enabled broader testing utilizing volunteers as part of the strategy to ensure pre-release Firefox builds do not regress core graphics functionality. Over the summer of 2015 we developed a sanity test via third-party and in-house testing (see below). This sanity test is now available to volunteers through One & Done with results being triaged as they come in.

Developer Edition

Every six weeks a new version of Firefox is elevated from development (Nightly) to Developer Edition (Aurora). Within the first two weeks of this cycle we arrange a new sanity testrun via a third-party lab. This lab gives us access to a wide range of systems, many of which represent legacy hardware we could not test otherwise. Typically we opt for broad coverage by running a sanity test focused on image rendering, performance, scrolling, and video playback on some top sites. The exception is when a Firefox version comes with a fairly complex, bleeding edge graphics feature in which case opt to run a series of tests focused on that feature. Testing is tracked on wiki.mozilla.org.

Toronto Lab

The office in Toronto has a "lab" of computers for graphics testing and debugging. Lab is put in quotes since this is currently little more than a random gathering of computers on a table. Starting in the summer of 2015 we began running a graphics sanity test on these machines with the help of some interns. Unfortunately, due to resource constraints, we have been unable to continue this testing. We are currently investigating opportunities to make the lab more official and to automate some of the tasks. In the meantime sanity testing is being covered through crowd testing and third-party outsourcing as detailed above. Please refer to wiki.mozilla.org to learn more about this work.

Improving Stability

Documentation on stability projects will be coming in Q4 2015.

Reporting new crashes

This page lists the various stability reports, maintained by Robert Kaiser (:KaiRo). Typically we're most interested in checking the Firefox Nightly, Aurora, Beta, and Release explosiveness reports. More information on these reports and KaiRo's methodology can be found on this wiki. We'll use the following as an illustrative example.

Explaining the Report

The above table is an example of the Firefox Nightly Explosiveness Report for October 20, 2015.

  • TC# is the top crash rank - in the above example the crashes are ranked #1, #12, #10, and so on.
  • Signature is the crash signature or where in the code Firefox crashed.
  • Bugs links to any reports in bugzilla. A strike-through indicates a closed report whereas no number indicates an unreported crash.
  • Explosiveness is the rate of change over a 1-day and 3-day period. A rating of 2.0 or higher is considered explosive and should be treated with utmost urgency.
  • Data is the number of crashes reported per one million users for the day indicated, this is sometimes referred to as "volume".

How to Use the Report

When looking at the explosiveness report we are basically looking for two things:

  1. Any graphics crashes which are explosive (ie. rated 2.0 or higher).
  2. Any graphics crashes which are new (ie. show up at 0.000 volume until more recently in the table).

The process for reviewing the report is the same whether you're looking for an explosive crash or a new crash:

  1. Check to see if the signature indicates a graphics crash. Typically anything in gfx, layers, canvas, D3D, D2D, Cairo, skia, or a driver DLL file is inidicative of a graphics crash.
  2. Check to see if the signature has an associated bug number. If not, click the signature link and report a bug from within Socorro (more on this later).
  3. Check any associated bug reports that have been resolved. If the bug was resolved more than two weeks ago the signature should not show up in the report. You'll need to follow up in the associated bug report and provide the data from the explosiveness report as the bug may need follow-up work.
  4. Check any associated bug reports that have not been resolved. If the bug has not been updated in more than two weeks check the bug report to see if there's any new information you can provide to move it forward. Keep in mind that the information needed may be in Socorro and not in the explosiveness report itself.

If you have any questions about this process or the explosiveness reports, please contact Anthony Hughes (ashughes on IRC).

Reporting a bug from Socorro

...more instructions to come.

Measuring Quality

Documentation on metrics projects will be coming in Q4 2015.

Getting Involved

The following is a list of the various ways you can get involved with the GFX team's quality efforts.

Other Resources

Document Tags and Contributors

 Contributors to this page: Ashughes
 Last updated by: Ashughes,