Auto-tools/Meetings/2012-08-20

Goals Progress

Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.

• [ON TRACK] Support the B2G and platform team by moving existing automation to that platform as well as expanding automation on B2G and B2G devices for basecamp deliverables.
  • [ahal] Fixed three bugs related to reftests on b2g (bug 782655, bug 778725, bug 778072)
• [ON TRACK] Expand Mobile testing platforms for B2G and Fennec Native to include user devices, NEON Chipsets (panda boards), and ARMv6 (Beagle boards).
  • [wlach] / [ctalbert]: More work on panda smoketest (bug 781341), will be used to verify that all the panda hardware we have is good for use.
  • [mcote] / [mihneadb]: C++ SUTAgent basic implementation (most commands except push, pull, exec, and reboot) done today. Other commands should be done in a couple days.
• [ON TRACK] Establish new Datazilla system as a useful tool for capturing and diagnosing talos performance regressions on a per-push basis, as well as capturing data for network and B2G performance tests.
  • NOTE: This is largely a change to the data harvesting and analysis side of the Talos system (even though some changes are required to the test harness and tests themselves). We will not be replacing existing Talos data harvesting system in Q3. By the end of Q3, we want to have the new performance data harvesting and analysis system Datazilla to a point at which it can shadow the old system and we can use Q4 to verify that the new approach captures just as many regressions as the old system does. Potential timeframe for conversion of the old system to the new Datazilla system is in Q1 2013 (earliest "thumb in the wind" estimate).
  • [jeads] New metrics branch implementing welch ttests and false discovery rate control for consecutive hg pushes should land this week.
• [ON TRACK] Upgrade BMO to 4.2, deploy tracking flags optimizations, UX improvements (comment tagging, mid-air resolution)
  • Some initial work on midair collision issues
• [ON TRACK] Make TBPL More Orange aware to make it easier to file intermittent orange bugs, and so that we can more adequately track (and therefore address) issues around android-related and memory leak intermittent failures
• [ON TRACK] Enable QA and contributors to increase their skills and participation by crafting automated tests for B2G and desktop smoketests by providing examples, mentoring, and documentation
  • [henrik] Automation Development meeting has been rescheduled for a more European friendly timeslot
  • [henrik] Public Announcement for weekly office hours (Q&A sessions) will go out today. Those will happen each Thursday at 12:00pm UTC
  • [henrik] About to finalize the new ESX cluster for Mozmill CI in MV. Will probably go live tomorrow with one node per platform. Additional two nodes will be added throughout the week.

Check out Pivotal Tracker

Highlights

This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.

• [edmorley] Native Android mochitest-3 now finally green and unhidden (after: https://bugzil.la/782254,782328,782431,778954,783509,783513 !!). This means all native and XUL Android testsuites are now unhidden :-D
• [edmorley] https://treestatus.mozilla.org is ready to go live (to replace tinderbox for closing the trees) - just waiting on IT for a push to production.

In-Depth Project Discussion

Pick one project/presenter per week who will give a detail presentation on their project.

The Overview

Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!

AutoPhone/Noah's Ark

• [bc] Wrote patch to download and install tests in cache directory for use by unit tests
• [bc] Tested running mochitest on phones. Nexus One and Droid Pro were not able to run the tests but Samsung Galaxy S2 was able to do so.
• [mcote] Found out that on some bad SD cards, we can create directories but not files. The SD card check was only doing the former, so it wasn't detecting them. Fixed that and a few other small things.

Bugzilla

• pushes: http://globau.wordpress.com/2012/08/17/happy-bmo-push-day-9/
• [glob] some unplanned PTO
• [glob] more work on bmo 4.2 search functionality
• [glob+dkl] BMO administrative duties
• [dkl] small amount of work on new dashboards
• [dkl] worked on midair collision in upstream bugzilla
• [dkl+glob] some upstream activity (reviews and patches)

Eideticker

• [mdas] got a few b2g eideticker tests fully running, need to get the rest going
• [mdas] need to hook up results to an accessible webserver

Jetperf

• [jhammel] jetperf finally unblocked on datazilla client
• [jhammel] will assess where the project is this week

Marionette

• Good First Bugs have landed from contributors.

Mobile Automation

• [mihneadb] Continued work on the new SUTAgent - improved rmdr and implemented exec
• [mihneadb] Tried to figure a way to have the agent start at boot time
  • tried using this: https://github.com/android/platform_system_core/blob/master/init/readme.txt
  • cannot test on otoro, does not work on the emulator
  • need to try on a pandaboard
• [ahal] Fixed three bugs related to reftests on b2g (bug 782655, bug 778725, bug 778072)
• [ahal] Stabilized the reftest desktop machine a bit (no results recently as nightly builds haven't been uploaded).
• [ahal] Did some triaging of the larger chunks of failing tests, disabled and filed bugs.
• [wlach] / [ctalbert]: More work on panda smoketest (bug 781341), will be used to verify that all the panda hardware we have is good for use.
• [mcote] / [mihneadb]: Most commands on C++ agent working and will be pushed today. Need a few more days for bigger commands (push, pull, exec, reboot).

MozBase

• [mihneadb] add autolog support and improve moztest bug 782848

New Graphserver (Datazilla)

• [jeads] Finishing unit tests for the new metrics branch for datazilla. Should land this week.
• [jeads] Helped cdawson with a new datazilla branch that collects basic stats on all of the projects and database schemas in datazilla.
• [jeads] Working on a new collection of web service methods that covers the new metrics/stats data.

Talos

• [jhammel] talos now uses datazilla client
• [jmaher] new deployment this week for outstanding changes
• (WIP) counters and xperf are getting better

Sheriffing & Trees

• [edmorley] Day-to-day sheriffing of the trees.
• [edmorley] Sheriffing/tree closure issues of note:
  • https://bugzil.la/782627,782665,782901,782981,783019,782196,783505,783518,783588,782309
• [edmorley] Updated the list of active hg hooks on each repo (bug 758994) with the bug 663585 & bug 780815 changes.
• [edmorley] Patch for bug 774062 - Make configure error out if the DirectX SDK redistributables cannot be found (was hitting this when building locally).

Treestatus

• [edmorley] Testing the prevent browser caching fix and rollout of it to production.
• [edmorley] Populated https://treestatus.mozilla.org with the current tree status messages from Tinderbox.
• [edmorley] Filed more TreeStatus followup bugs: #20, #21.
• [edmorley] Finalised the POA for making TreeStatus live, with testing plan & rollback steps. Filed bug 782239 for push to production. Have been ready to go since 13th Aug, but blocked on IT :-(

Tinderboxpushlog

• [edmorley] Reviewed:
  • bug 704006 - Add a separate job status/colour for 'Cancelled by buildapi'.
  • bug 686263 - Switch tbpl to loading pending and running over https.
• [edmorley] Tested the patch for getting TBPL to work in Internet Explorer (bug 771055).
• [edmorley] Updated the TBPL section on the TryServer wiki page.
• [edmorley] Fixed:
  • bug 782826 - Make TBPL recognise OS X 10.8.
  • bug 758957 - Clarify the "Open tree admin panel" & "Tree Status Admin" links on the tree info menu.
  • bug 764894 - Improve TBPL's docs for setting up a dev environment.

War on Orange

• [DBurns] Created a simpler way to handle dependencies when setting up an OrangeFactor environment. Waiting Review in bug 782953.

• [edmorley] OrangeFactor work:
  • Patch to improve the readme for setting up a local testing instance (bug 782939).
  • Filed bugs for a number of dev+user UX improvements: https://bugzil.la/782941,782949,783803,783804,783805
  • More discussion about orangeseed's implementation (bug 669316).

• [edmorley] Intermittent orange work:
  • Made the final push to get the last hidden Android test suite (native mochitest-3) green. Frustratingly, every time a test was disabled, we'd fail on a later one (https://bugzil.la/782254,782328,782431,778954,783509,783513 !!). I eventually identified dom/indexedDB/ipc/test_ipc.html as the real cause - so after disabling that, was able to get M3 green even with most of the recently disabled M3 tests re-enabled :-D
  • Backports of fixes to aurora/beta/esr: https://bugzil.la/623242,555392,733848,718260,707733,625273,656943,682141,694248,567367,661996,763621,771979
  • Searched recently filed [orange] to make sure there is always someone CCed & for newly landed tests ensuring we back out rather than letting the orange stick until it's too late. As a result, backed out: https://bugzil.la/773732,782903,783723,767636
  • Disabled devicestorage tests on Android at dougt's request (bug 782275).
  • Disabled various reftests on Android XUL at khuey's request, for failing after he landed bug 685516.
  • Disabled some tests for excessive orange & apparent lack of developer interest even after 18-24 months of failures: https://bugzil.la/608634,552424
  • Patch to fix the suspected cause of intermittent tinderbox 'Breakpad tools do not support compiling on WINNT' configure failures (bug 776405).

Automation Development

• [rwood] Developing more tests for B2G WebTelephony WebAPI
• [henrik] Automation Development meeting has been rescheduled for a more European friendly timeslot
• [henrik] Public Announcement for weekly office hours (Q&A sessions) will go out today. Those will happen each Thursday at 12:00pm UTC
• [henrik] About to finalize the new ESX cluster for Mozmill CI in MV. Will probably go live tomorrow with one node per platform. Additional two nodes will be added throughout the week.

Upcoming Events

Standing weekly meeting related to our projects should go here

All times Pacific Time, click on link to find your timezone

• Platform Meeting (all platform engineers) Tuesday at 11 AM, Conf # 8605.
• Mobile Test Automation Wednesday at 10:30 AM, Vidyo Warp Core
• Mozharness Wednesday at 1 PM, Vidyo Warp Core
• Marionette/Eideticker Meeting Thursday at 10 AM Conf # 304
• Signal from Noise Thursday at 11 AM #343

Round Table

Put your questions and things to raise with the entire team here

• Last week some ateam members (ctalbert, mcote, bc, jmaher) participated in the mobile workweek in Boston.
  • Any new automation priorities / ideas that came out of discussions during the week?
  • Other highlights?

Holidays and Trips

• [davehunt] Web QA work week (24-31 August)
• [mcote] PTO 27-31 August.
• [UK people] Public holiday Monday 27th August

Next week MC

Was going to be edmorley, but we remembered afterwards it was a UK public holiday, oops!

Misc

Find something you think is cool, interesting, funny, or exciting and put it here

• Irrational tipping \o/