Auto-tools/Meetings/2012-11-19: Difference between revisions
< Auto-tools | Meetings
Jump to navigation
Jump to search
(→Tinderboxpushlog: Add edmorley) |
(→Misc) |
||
(12 intermediate revisions by 7 users not shown) | |||
Line 18: | Line 18: | ||
** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}} | ** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}} | ||
** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing}} | ** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing}} | ||
*** Should be done first version early this week. | |||
** {{ok|Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.}} | ** {{ok|Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.}} | ||
* '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks''' | * '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks''' | ||
Line 25: | Line 26: | ||
* '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.''' | * '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.''' | ||
** {{ok|Deploy 4.2 version of Bugzilla}} | ** {{ok|Deploy 4.2 version of Bugzilla}} | ||
*** Waiting on IT... | |||
** {{ok|Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.}} | ** {{ok|Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.}} | ||
** {{ok|Complete the 'Sandstone' CSS update}} | ** {{ok|Complete the 'Sandstone' CSS update}} | ||
*** Updated skin on http://bugzilla.allizom.org | |||
** {{ok|Rework browserID <--> BMO integration}} | ** {{ok|Rework browserID <--> BMO integration}} | ||
** {{ok|Deliver user and product dashboards}} | ** {{ok|Deliver user and product dashboards}} | ||
Line 33: | Line 36: | ||
** {{ok|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL}} | ** {{ok|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL}} | ||
** {{ok|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}} | ** {{ok|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}} | ||
*** [edmorley] Created v2 of the bookmarklet used to simplify the TBPL -> 'viewing job on the buildbot master' workflow ({{bug|809908}}). | |||
*** [edmorley] Fixing the test harness issues on Android that cause us to miss all crashes on robocop and all shutdown crashes on mochitest/crashtest/reftest/jsreftest ({{bug|808410}}). Also fixed similar issues with checking for hung processes ({{bug|812135}}). | |||
*** [edmorley] Parser improvements: https://bugzil.la/812205,812207,812214 | |||
*** [edmorley] Adding an easier way to look up the slave history for the machine that performed a job ({{bug|736085}}). | |||
** {{done|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}} | ** {{done|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}} | ||
** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}} | ** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}} | ||
Line 58: | Line 65: | ||
=== [[BMO|Bugzilla]] === | === [[BMO|Bugzilla]] === | ||
* http://globau.wordpress.com/2012/11/15/268/ | |||
* [glob] significant performance related investigations, resulting in much discussion and a few patches | |||
* [glob] bugzilla push extension deployed to production | |||
* [dkl] significant progress on the mozilla skin for bmo | |||
* [glob+dkl] lots of upstream patches and reviews | |||
=== [[Project_Eideticker|Eideticker]] === | === [[Project_Eideticker|Eideticker]] === | ||
Line 69: | Line 81: | ||
=== Mobile Automation === | === Mobile Automation === | ||
* [mcote] mozpool back end just about complete (~1-2 days). | * [mcote] mozpool back end just about complete (~1-2 days). | ||
* [ahal] wasted most of the week on {{bug|809437}} | |||
** working on getting b2g tests unhidden on tbpl | |||
** also on getting unittests to dump logcat so we can debug these issues more easily in the future | |||
=== [[Auto-tools/Projects/Mobile_Evangelism_Site_Scraper|Mobile Evangelism Site-Scraping Tool]] === | === [[Auto-tools/Projects/Mobile_Evangelism_Site_Scraper|Mobile Evangelism Site-Scraping Tool]] === | ||
Line 79: | Line 94: | ||
=== [[Auto-tools/Projects/MozTrap|MozTrap]] === | === [[Auto-tools/Projects/MozTrap|MozTrap]] === | ||
* [camd] - Still working on MozTrap perf issues. My fixes help, but may need more attention. | |||
* [camd] - In talks with Clint / Mark Cote / Matt Brandt about continuing working on MT into January and what features need completion prior to mid-December. | |||
=== [[Auto-tools/Projects/Datazilla|New Graphserver (Datazilla)]] === | === [[Auto-tools/Projects/Datazilla|New Graphserver (Datazilla)]] === | ||
Line 116: | Line 133: | ||
=== [[Auto-tools/Projects/WarOnOrange|War on Orange]] === | === [[Auto-tools/Projects/WarOnOrange|War on Orange]] === | ||
* [edmorley] Another big attack on our OrangeFactor this week. Last 7 days are at 1.35 :-D | |||
** Though OF currently doesn't recognise B2G/emulator failures ({{bug|812551}}) & our test harnesses miss some android crashes ({{bug|808410}}) - so once those are fixed we'll see a slight increase again. (They aren't new issues - we've just not been counting them until now). | |||
=== Web QA === | === Web QA === | ||
=== Automation Development === | === Automation Development === | ||
* [rwood] WebAPI emulator tests for Proximity Sensor | |||
== Upcoming Events == | == Upcoming Events == | ||
Line 141: | Line 161: | ||
* PyCon Canada had some interesting talks! | * PyCon Canada had some interesting talks! | ||
** Science and Python: Retrospective of a (mostly) successful decade ([http://pyvideo.org/video/1605/science-and-python-retrospective-of-a-mostly-s video]) | ** Science and Python: Retrospective of a (mostly) successful decade ([http://pyvideo.org/video/1605/science-and-python-retrospective-of-a-mostly-s video], [http://fperez.org/talks/1203_ipython_pycon.pdf slides] (seems to be a simpler version of his actual PyCon.Ca talk)) | ||
*** This was ''amazing''. He got a standing ovation. :) | *** This was ''amazing''. He got a standing ovation. :) | ||
** | ** Twisted History of Python Packaging ([http://pyvideo.org/video/1601/twisted-history-of-python-packaging video], [https://speakerdeck.com/myusuf3/twisted-history-of-python-packaging slides]) | ||
** Python for Humans (by the author of the [http://docs.python-requests.org/en/latest/ Requests] package) ([http://pyvideo.org/video/1591/python-for-humans-0 video]) | ** Python for Humans (by the author of the [http://docs.python-requests.org/en/latest/ Requests] package) ([http://pyvideo.org/video/1591/python-for-humans-0 video]) | ||
** How you can contribute to Python ([http://pyvideo.org/video/1578/how-you-can-contribute-to-python video]) | ** How you can contribute to Python ([http://pyvideo.org/video/1578/how-you-can-contribute-to-python video]) | ||
Line 149: | Line 169: | ||
** Wilderness Labs ([http://pyvideo.org/video/1603/wilderness-labs video]) | ** Wilderness Labs ([http://pyvideo.org/video/1603/wilderness-labs video]) | ||
** Network Programming for Automation Developers :) ([http://pyvideo.org/video/1584/network-programming-for-automation-developers video], [http://mrcote.info/presentations/network-programming-for-automation/ slides]) | ** Network Programming for Automation Developers :) ([http://pyvideo.org/video/1584/network-programming-for-automation-developers video], [http://mrcote.info/presentations/network-programming-for-automation/ slides]) | ||
* Mobile Test Summit talks are up! | |||
** [http://www.youtube.com/watch?v=u9gWNnxkgg4&feature=plcp wlach's talk on eideticker] | |||
** [http://www.youtube.com/watch?v=3cH_nXnKMb8&feature=plcp mdas's talk on marionette] |
Latest revision as of 05:32, 20 November 2012
Goals Progress
Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.
- Ensure B2G has all test automation available to prepare for launch
- [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
- [DONE] Mochitest-plain (emulator first, panda second)
- [SKIPPED] Mochitest-chome (emulator first, panda second)
- [DONE] Reftest (emulator first, panda second
- [ON TRACK] xpcshell (panda)
- [DONE] webAPI (emulator only
- [ON TRACK] webQA - gaia front end tests emulator and panda
- [ON TRACK] gaia dev integration/unit tests emulator and panda
- [ON TRACK] Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng
- [ON TRACK] Ensure all webAPIs testable through the emulator have comprehensive automated tests
- [ON TRACK] Continue to aid QA in creating automated tests for B2G
- [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
- Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.
- [ON TRACK] Have APC.io (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone
- [ON TRACK] Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations
- [ON TRACK] Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing
- Should be done first version early this week.
- [ON TRACK] Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.
- Create and deploy performance automation for desktop, fennec, and B2G product benchmarks
- [ON TRACK] Create Eideticker tests for B2G on panda boards and run them once a day
- [ON TRACK] Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis
- [ON TRACK] Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions
- Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.
- [ON TRACK] Deploy 4.2 version of Bugzilla
- Waiting on IT...
- [ON TRACK] Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.
- [ON TRACK] Complete the 'Sandstone' CSS update
- Updated skin on http://bugzilla.allizom.org
- [ON TRACK] Rework browserID <--> BMO integration
- [ON TRACK] Deliver user and product dashboards
- [ON TRACK] Deploy tracking flags extension database optimization
- [ON TRACK] Deploy 4.2 version of Bugzilla
- Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem
- [ON TRACK] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2g tests show up in TBPL
- [ON TRACK] Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures
- [edmorley] Created v2 of the bookmarklet used to simplify the TBPL -> 'viewing job on the buildbot master' workflow (bug 809908).
- [edmorley] Fixing the test harness issues on Android that cause us to miss all crashes on robocop and all shutdown crashes on mochitest/crashtest/reftest/jsreftest (bug 808410). Also fixed similar issues with checking for hung processes (bug 812135).
- [edmorley] Parser improvements: https://bugzil.la/812205,812207,812214
- [edmorley] Adding an easier way to look up the slave history for the machine that performed a job (bug 736085).
- [DONE] Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.
- [ON TRACK] Make orange factor update more than once a day to help make it easier to see changes over time
- [ON TRACK] Assist with getting Datazilla's UI integrated with TBPL, phase 1
- Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.
- [ON TRACK] Re-organize and improve the automated testing documentation at MDN
- [ON TRACK] Develop first round of unit tests for WebRTC
- [ON TRACK] Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage
Highlights
This is a place for people to put announcements or to describe activities that they'd like to highlight. It's not the responsibility of the MC to find things to put here.
In-Depth Project Discussion
Pick one project/presenter per week who will give a detail presentation on their project.
The Overview
Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!
AutoPhone/Noah's Ark
Bughunter
Bugzilla
- http://globau.wordpress.com/2012/11/15/268/
- [glob] significant performance related investigations, resulting in much discussion and a few patches
- [glob] bugzilla push extension deployed to production
- [dkl] significant progress on the mozilla skin for bmo
- [glob+dkl] lots of upstream patches and reviews
Eideticker
Firebug Testing
Jetperf
Marionette
Mobile Automation
- [mcote] mozpool back end just about complete (~1-2 days).
- [ahal] wasted most of the week on bug 809437
- working on getting b2g tests unhidden on tbpl
- also on getting unittests to dump logcat so we can debug these issues more easily in the future
Mobile Evangelism Site-Scraping Tool
Mochitest
MozBase
Mozmill
MozTrap
- [camd] - Still working on MozTrap perf issues. My fixes help, but may need more attention.
- [camd] - In talks with Clint / Mark Cote / Matt Brandt about continuing working on MT into January and what features need completion prior to mid-December.
New Graphserver (Datazilla)
Open Web Apps
Peptest
Powerball
Pulse
Signal From Noise
Speedtests
Systems
Talos
Sheriffing & Trees
- [edmorley] Day-to-day sheriffing of the trees.
- [edmorley] Collating the various new Panda/B2G failure modes, ensuring all filed, hiding emulator tests due to failure rate. Will hopefully soon be able to be unhidden (bug 812149 and dependants).
- [edmorley] Created v2 of the bookmarklet used to simplify the TBPL -> 'viewing job on the buildbot master' workflow (bug 809908).
- [edmorley] Patches:
- Fixing the test harness issues on Android that cause us to miss all crashes on robocop and all shutdown crashes on mochitest/crashtest/reftest/jsreftest (bug 808410). Also fixed similar issues with checking for hung processes (bug 812135).
- Other: https://bugzil.la/810995,812513,811286
Treestatus
Tinderboxpushlog
- [edmorley] Patches:
- Parser improvements: https://bugzil.la/812205,812207,812214
- Adding an easier way to look up the slave history for the machine that performed a job (bug 736085).
- Other: https://bugzil.la/812472,812558
- [edmorley] Reviewed bug 812103.
War on Orange
- [edmorley] Another big attack on our OrangeFactor this week. Last 7 days are at 1.35 :-D
- Though OF currently doesn't recognise B2G/emulator failures (bug 812551) & our test harnesses miss some android crashes (bug 808410) - so once those are fixed we'll see a slight increase again. (They aren't new issues - we've just not been counting them until now).
Web QA
Automation Development
- [rwood] WebAPI emulator tests for Proximity Sensor
Upcoming Events
Standing weekly meeting related to our projects should go here
Round Table
- B2G Deadline December 10.
Take notes here and copy and paste them to this section afterwards.
Notes
Holidays and Trips
- US out Thursday and Friday this week
Next week MC
Misc
- PyCon Canada had some interesting talks!
- Science and Python: Retrospective of a (mostly) successful decade (video, slides (seems to be a simpler version of his actual PyCon.Ca talk))
- This was amazing. He got a standing ovation. :)
- Twisted History of Python Packaging (video, slides)
- Python for Humans (by the author of the Requests package) (video)
- How you can contribute to Python (video)
- Beyond Passwords: Secure Authentication with Mozilla Persona :) (video)
- Wilderness Labs (video)
- Network Programming for Automation Developers :) (video, slides)
- Science and Python: Retrospective of a (mostly) successful decade (video, slides (seems to be a simpler version of his actual PyCon.Ca talk))
- Mobile Test Summit talks are up!