Auto-tools/Meetings/2012-08-20: Difference between revisions
< Auto-tools | Meetings
Jump to navigation
Jump to search
Line 52: | Line 52: | ||
** cannot test on otoro, does not work on the emulator | ** cannot test on otoro, does not work on the emulator | ||
** need to try on a pandaboard | ** need to try on a pandaboard | ||
* [ahal] Fixed three bugs related to reftests on b2g ({{bug|782655}}, {{bug|778725}}, {{bug|778072}}) | |||
* [ahal] Stabilized the reftest desktop machine a bit (no results recently as nightly builds haven't been uploaded). | |||
* [ahal] Did some triaging of the larger chunks of failing tests, disabled and filed bugs. | |||
=== [[Auto-tools/Projects/Mobile_Evangelism_Site_Scraper|Mobile Evangelism Site-Scraping Tool]] === | === [[Auto-tools/Projects/Mobile_Evangelism_Site_Scraper|Mobile Evangelism Site-Scraping Tool]] === |
Revision as of 14:59, 20 August 2012
Goals Progress
Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.
- [ON TRACK] Support the B2G and platform team by moving existing automation to that platform as well as expanding automation on B2G and B2G devices for basecamp deliverables.
- [ON TRACK] Expand Mobile testing platforms for B2G and Fennec Native to include user devices, NEON Chipsets (panda boards), and ARMv6 (Beagle boards).
- [ON TRACK] Establish new Datazilla system as a useful tool for capturing and diagnosing talos performance regressions on a per-push basis, as well as capturing data for network and B2G performance tests.
- NOTE: This is largely a change to the data harvesting and analysis side of the Talos system (even though some changes are required to the test harness and tests themselves). We will not be replacing existing Talos data harvesting system in Q3. By the end of Q3, we want to have the new performance data harvesting and analysis system Datazilla to a point at which it can shadow the old system and we can use Q4 to verify that the new approach captures just as many regressions as the old system does. Potential timeframe for conversion of the old system to the new Datazilla system is in Q1 2013 (earliest "thumb in the wind" estimate).
- [ON TRACK] Upgrade BMO to 4.2, deploy tracking flags optimizations, UX improvements (comment tagging, mid-air resolution)
- [ON TRACK] Make TBPL More Orange aware to make it easier to file intermittent orange bugs, and so that we can more adequately track (and therefore address) issues around android-related and memory leak intermittent failures
- [ON TRACK] Enable QA and contributors to increase their skills and participation by crafting automated tests for B2G and desktop smoketests by providing examples, mentoring, and documentation
Highlights
This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.
- [edmorley] Native Android mochitest-3 now finally green and unhidden (after: https://bugzil.la/782254,782328,782431,778954,783509,783513 !!). This means all native and XUL Android testsuites are now unhidden :-D
- [edmorley] https://treestatus.mozilla.org is ready to go live (to replace tinderbox for closing the trees) - just waiting on IT for a push to production.
In-Depth Project Discussion
Pick one project/presenter per week who will give a detail presentation on their project.
The Overview
Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!
AutoPhone/Noah's Ark
- [bc] Wrote patch to download and install tests in cache directory for use by unit tests
- [bc] Tested running mochitest on phones. Nexus One and Droid Pro were not able to run the tests but Samsung Galaxy S2 was able to do so.
Bughunter
Bugzilla
Eideticker
- [mdas] got a few b2g eideticker tests fully running, need to get the rest going
- [mdas] need to hook up results to an accessible webserver
Firebug Testing
Jetperf
Marionette
- Good First Bugs have landed from contributors.
Mobile Automation
- [mihneadb] Continued work on the new SUTAgent - improved rmdr and implemented exec
- [mihneadb] Tried to figure a way to have the agent start at boot time
- tried using this: https://github.com/android/platform_system_core/blob/master/init/readme.txt
- cannot test on otoro, does not work on the emulator
- need to try on a pandaboard
- [ahal] Fixed three bugs related to reftests on b2g (bug 782655, bug 778725, bug 778072)
- [ahal] Stabilized the reftest desktop machine a bit (no results recently as nightly builds haven't been uploaded).
- [ahal] Did some triaging of the larger chunks of failing tests, disabled and filed bugs.
Mobile Evangelism Site-Scraping Tool
Mochitest
MozBase
- [mihneadb] add autolog support and improve moztest bug 782848
Mozmill
MozTrap
New Graphserver (Datazilla)
Open Web Apps
Peptest
Powerball
Pulse
Signal From Noise
Speedtests
Systems
Talos
Sheriffing & Trees
- [edmorley] Day-to-day sheriffing of the trees.
- [edmorley] Sheriffing/tree closure issues of note:
- [edmorley] Updated the list of active hg hooks on each repo (bug 758994) with the bug 663585 & bug 780815 changes.
- [edmorley] Patch for bug 774062 - Make configure error out if the DirectX SDK redistributables cannot be found (was hitting this when building locally).
Treestatus
- [edmorley] Testing the prevent browser caching fix and rollout of it to production.
- [edmorley] Populated https://treestatus.mozilla.org with the current tree status messages from Tinderbox.
- [edmorley] Filed more TreeStatus followup bugs: #20, #21.
- [edmorley] Finalised the POA for making TreeStatus live, with testing plan & rollback steps. Filed bug 782239 for push to production. Have been ready to go since 13th Aug, but blocked on IT :-(
Tinderboxpushlog
- [edmorley] Reviewed:
- bug 704006 - Add a separate job status/colour for 'Cancelled by buildapi'.
- bug 686263 - Switch tbpl to loading pending and running over https.
- [edmorley] Tested the patch for getting TBPL to work in Internet Explorer (bug 771055).
- [edmorley] Updated the TBPL section on the TryServer wiki page.
- [edmorley] Fixed:
- bug 782826 - Make TBPL recognise OS X 10.8.
- bug 758957 - Clarify the "Open tree admin panel" & "Tree Status Admin" links on the tree info menu.
- bug 764894 - Improve TBPL's docs for setting up a dev environment.
War on Orange
- [DBurns] Created a simpler way to handle dependencies when setting up an OrangeFactor environment. Waiting Review in bug 782953.
- [edmorley] OrangeFactor work:
- Patch to improve the readme for setting up a local testing instance (bug 782939).
- Filed bugs for a number of dev+user UX improvements: https://bugzil.la/782941,782949,783803,783804,783805
- More discussion about orangeseed's implementation (bug 669316).
- [edmorley] Intermittent orange work:
- Made the final push to get the last hidden Android test suite (native mochitest-3) green. Frustratingly, every time a test was disabled, we'd fail on a later one (https://bugzil.la/782254,782328,782431,778954,783509,783513 !!). I eventually identified dom/indexedDB/ipc/test_ipc.html as the real cause - so after disabling that, was able to get M3 green even with most of the recently disabled M3 tests re-enabled :-D
- Backports of fixes to aurora/beta/esr: https://bugzil.la/623242,555392,733848,718260,707733,625273,656943,682141,694248,567367,661996,763621,771979
- Searched recently filed [orange] to make sure there is always someone CCed & for newly landed tests ensuring we back out rather than letting the orange stick until it's too late. As a result, backed out: https://bugzil.la/773732,782903,783723,767636
- Disabled devicestorage tests on Android at dougt's request (bug 782275).
- Disabled various reftests on Android XUL at khuey's request, for failing after he landed bug 685516.
- Disabled some tests for excessive orange & apparent lack of developer interest even after 18-24 months of failures: https://bugzil.la/608634,552424
- Patch to fix the suspected cause of intermittent tinderbox 'Breakpad tools do not support compiling on WINNT' configure failures (bug 776405).
Web QA
Automation Development
[rwood] Developing more tests for B2G WebTelephony WebAPI
Upcoming Events
Standing weekly meeting related to our projects should go here
Round Table
Put your questions and things to raise with the entire team here
Take notes here and copy and paste them to this section afterwards.
Notes
Holidays and Trips
Next week MC
Misc
Find something you think is cool, interesting, funny, or exciting and put it here