QA/Platform/Graphics/Guides

Finding a Regression Window

Installing Mozregression

Instructions for Windows:

• Download and install Python 2.7
• Open a Windows command window (Run cmd.exe)
• Execute this command:

  pip install -U mozregression

• Confirm mozregression is installed by executing this command:

  mozregression --version

• If you get stuck, see the full instructions here

How Mozregression Works

Mozregression works by downloading Firefox Nightly builds from a range of dates allowing you to easily narrow down the first Nightly which introduced a bug. Once a Nightly is downloaded and installed, Mozregression will start Firefox with a temporary profile and allow you to test your bug. After testing the bug, you give Mozregression a good or bad command and it will split the range in half, downloading the next build. Once the range has been narrowed down to a single day it will give you a pushlog for the build that will identify the changes which may have caused the bug. This should be attached to the bug report.

As mentioned above, the range will first be narrowed down to a particular Nightly build, then continue to an Inbound build, which will often get us down to a single patch. After the Nightly build search is done, mozregression will note the range for it, then continue to bisect the Inbound builds. When recording the results of mozregression, it is useful to record the Nightly range as well, even if the Inbound range is found.

Using Mozregression

Note: We recommend narrowing down the range manually before using mozregression.

If you don't know the dates, simply run mozregression from a terminal/command window. This will download Firefox Nightly builds starting from January 1, 2009. However, you can save yourself some time and hassle if you give mozregression a narrower range. You may find some of the really old builds will just fail to work, especially on newer platform versions.

If you know the dates, simply run mozgression --good YYYY-MM-DD --bad YYYY-MM-DD where good is the date of the last build you know is good and bad is the date of the first build you know is bad.

Narrowing the Range

Note: We recommend narrowing down the range to six weeks between builds before using mozregression.

• Start from the date of the build you reproduced the bug with
• Open the about:support page and make note of the Build ID from the Application Basics section (the build ID follows a YYYYMMDDhhmmss format).
• Install a build from six weeks before that date from ftp.mozilla.org
• If that build reproduces the bug, go back another six weeks
• Repeat this process until you find a build that does not reproduce the bug and make note of its build date
• Use these dates to run mozregression

For example, if the build ID of a good build is 20150401123456 and the build ID of a bad build is 20150410123456:

mozregression --good 2015-04-01 --bad 2015-04-10

If Mozregression Fails

If mozregression fails for whatever reason, you'll need to narrow the range manually to a single day:

• Get the date of the build where you first saw the bug
• Grab a build that is six weeks older from ftp.mozilla.org
• Test that build and see if the bug reproduces
• Repeat this process until you find a build that does not reproduce the bug

Once you've found a range, begin to narrow the range further:

• Begin by testing the build which represents the midway point.
• Eg. if the good build is from 2015-04-01 and my bad build is from 2015-05-15, I might test the build from 2015-04-21. The range will be reduced to either 2015-04-21:2015-05-15 or 2015-04-01:2015-04-21, depending on whether or not the bug reproduced.
• Split the difference again and retest
• Keep repeating until you've narrowed it down to a single day

Once you find the single day range, get the changeset for each build:

• Start the Firefox build
• Open the about:buildconfig page
• Look at the Source section of this page for an hg.mozilla.org URL
• Copy the 12-digit ID from the end of that URL (this is the changeset ID for the build)
• Eg. the changeset of a build built from https://hg.mozilla.org/mozilla-central/rev/4fe763cbe196 is 4fe763cbe196

Once you have both changesets, manually put together the pushlog URL:

• https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=GOOD&tochange=BAD
• Replace GOOD with the changeset of the last known good build
• Replace BAD with the changeset of the first bad build
• Add this URL to the bug and ask for someone to check it for changes which may have introduced the regression

Once an offending bug has been identified, that bug should be flagged:

• mark your bug as blocking the offending bug
• flag the developer of the offending bug using the needinfo? flag

It may be necessary to bisect the range further but that's outside the scope of this document.

Testing a Try Build

TO BE DETERMINED