Dev-sanity

This page summarizes the work that the A-Team has put into addressing developer workflow/efficiency concerns in 2015Q2, as presented at the Whistler all-hands, with some additional projects that we won't have time to review. We also describe some cool new work coming up in the latter half of 2015.

MozReview

[mcote to fill in]

Intermittent Oranges

[jgriffin to fill in]

Tree Closures

Tree Closures for a number of the integration branches are currently on the rise. We can see this on Futurama. Some of the current tree closures can be attributed to the length that tests take to run (Normalising test run length would solve this).

For this year, the amount of tree closures have been attributed to the following

Tree Closed for a Test or Bustage	Count
Failing Test - No Try run done	7
Try breadth of run not sufficient	3
Bustage - No Try Run	5
Bustage - Try breadth of run not sufficient	3
Perf Regression	1

Developer Experience

Running tests from tests.zip

[ahal to fill in]

Log summaries in mach

[ahal to fill in]

Test selection on try

Select tests within a test job to run on try. This is achieved by extending try syntax to support paths and tags as arguments. This means shorter turn around times for getting feedback on particular tests and less redundant machine time running tests that probably aren't going to fail as a result of a particular change.

This is primarily exposed through a new mach command, |mach try|, that takes paths and/or tags as arguments, calculates an appropriate try syntax based on the tests present in the tree, and pushes the result to try. The implementation was discussed in 1149670. Details can be found by running |mach help try|.

Automatic retrigger of oranges on try

Retrigger failed jobs on try automatically. This is a pulse service that listens for failed test jobs and triggers each one two additional times. The idea is to reduce time spent determining whether a particular failure on try is an intermittent issue or a persistent failure introduced by a specific change. A common response to orange jobs on try is to retrigger them to determine if the failure persists, and although it results in more test jobs on try, the service is intended to save developer time by replacing this manual process with an automated one.

A related feature implemented by this service is the ability to automatically build every test job on a push a certain number of times. This is a long-requested feature intended to be used by those investigating or proving a fix for an intermittent failures as well as those establishing a baseline for comparing Talos results across revisions.

The rollout of the service is ongoing and tracked by 1163698 (details on the implementation can be found there as well). As of this writing it's enabled for try pushes initiated by the autoland user.

Others?

run-by-dir? chunk normalization?

e10s support...