QA/Test Automation/2011-06-29

From MozillaWiki
Jump to navigation Jump to search

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Attendees

Last action items

  • David: Work with Geo and Anthony to get some first ideas for changes of the review process; we will schedule a meeting once Henrik is back (should happen in June)
    • We should have a deeper discussion during our upcoming work week (Henrik)

Goals Overview

Risky Goals

  • Complete the refactoring of the API
  • Create the top 5 tests for the discovery pane
    • 4 tests now in SR, remainder need revisions

Project Status

Redesign Shared Modules (Geo/Henrik)

  • Window object model changing from a mixin-based model to an inheritance based model. Is in debug now, code will be delivered today.
  • Known state implemented for test start/end:
    • fresh default bookmarks
    • no tab history
    • all tabs closed except about:blank.
  • Default tabs can be left for test start if desired (probably only for the test that verifies the default tab set).
  • Still should add "close all open windows but the original and focus the browser" to known state; will be settled once model change is in.
  • Windowing is the last fundamental being worked on. In-content windows (addons manager, notably) still unaddressed, bumped to next quarter.
  • Implicit waitFor added on element finding (and is there for window finding as well).
  • Test areas and top tests have been triaged and prioritized and added to Pivotal.

On-demand release tests (Geo/Henrik)

  • Henrik is working on the final version of the staging script, which will also cover candidate builds and daily builds
  • Geo is working on adding BFT kickoff to the daemons, as well as code cleanup per Henrik's code review.

Endurance Tests (Dave/Anthony)

  • Flash videos landed on mozqa.com and litmus-data, will target test page and test case writing next quarter
  • Waverley to assist next quarter by making a portion of their test development Endurance tests related to the focus area
  • Looking to align with MemShrink and Telemetry

Others

Mozmill Crowd

  • We are about to release Mozmill Crowd 0.1.3 by latest tomorrow (bug 667902)
  • Waiting for preliminary review on addons.mozilla.org
  • Testday for Mozmill Crowd (and test development) this Friday, July 1st

Dashboard

Current dashboard installation (Henrik)
  • no updates
New dashboard driven by ES (Owen)

Sync Server Load Testing (Owen)

  • Project on hold until we have agreed upon goals with the services team.

On Demand Provisioning (Cloud Automation) (Al/Henrik)

  • Vsphere/ESX box is fully operational
  • Al is creating reference virtual machines on it from scratch (porting existing VMs is impossible...mostly)
    • First post-reference VM will be a squid proxy
  • What specific needs do people have here for on demand machines?

Tests / Shared Modules

General (Henrik)
Broken Tests
  • A couple of broken tests across brances in the last days. It has been caused by Henrik's patch for the Mozmill change in handling 'documentLoaded'. As is turned out today it's a Mozmill bug which blocks the thread execution if the window passed to the controller is null. See 668202 bug 668202.
Softvision (Anthony)
  • 4 Discovery Pane tests in SR, remainder need revisions
  • Will focus on Add-ons Manager restart tests next quarter, tacking on Web Apps or a Firefox 8 feature pending feature readiness

TCM (Cameron/Rebecca B.)

  • .4 progress hampered a bit by platform instability. We will not make all features by 7/15.
  • Negotiating with uTest on process and staffing improvements on their side.
  • Hashing out task assignments for Rebecca and Matt B.
  • Work on Environment management continuing.

Shadow web server (Al/Henrik)

  • SSL will require more IPs for VPS
    • Cannot reliably use multiple certs on one IP
      • Could use SNI to do this but it requires recompile of both OpenSSL and Apache, which carries its own maintenance and other headaches.
    • Getting three more IPs from IT (and Rackspace)
    • We are taking over last year's summit book cert and address (https://summitbook.mozilla.org) as its SSL cert expires soon and IT was going to retire it.
    • We have the OV and DV certs we asked for (as of this last Friday). These two plus the summitbook and self-signed certs will fill four total IP addresses.
  • Flash test files have been deployed.
  • Mozmill Crowd files are being maintained by Henrik.
  • What other needs do people have for mozqa.com?

Automation Scripts / Infrastructure (Henrik)

  • Download script has been enhanced to also download daily builds of Nightly or Aurora
  • We can now even download Nightly builds from project branches - anyone interested in?

Personal Status

For the personal status please check the weekly status updates:

Roundtable

General
  • Changes to our bi-weekly meeting regarding the org changes (Henrik)
    • We should setup preliminary pages which describe our new team next week
    • We should move the meeting pages below the new team page and call it "QA Automation Services Meeting"
  • Review process (Henrik)
    • A more detailed discussion should happen during our work week in Cambridge
    • We have to find an interim solution to get not stuck in review requests
      • If you are not able to react to a review request from Softvision or contributors within one day, you should pass the review to another person
      • If there are issues you are running into, spread this information across the team. It takes lesser time to solve it together
  • Discovery Pane tests
    • Do we want to allow tests to auto-skip themselves? If not we should mark those tests as skipped on default and aurora as long as the dependencies with WebDev are not solved.

Meeting Notes

Action items

Retrieved from "https://dev.wikimo.nonprod.webservices.mozgcp.net/index.php?title=QA/Test_Automation/2011-06-29&oldid=323971"