QA/Test Automation/2011-06-01: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 78: Line 78:
*FC for .4 slated for July 15th
*FC for .4 slated for July 15th
*I'm looking for a second on the project.  Hopefully Matt Brandt
*I'm looking for a second on the project.  Hopefully Matt Brandt
*Currently deployed on Oddsites.  But I want to get a deployment at Mozilla on mozqa as soon as I can.


== [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Shadow_Web_Server Shadow web server] (Al/Henrik)==
== [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Shadow_Web_Server Shadow web server] (Al/Henrik)==

Revision as of 20:42, 1 June 2011

previous meeting | Meetings | next meeting »

Dial in

 # 650-903-0800 or 650-215-1282 x92 Conf# 315 (US/INTL)
 # 1-800-707-2533 (pin 369) Conf# 315 (US)
 # irc.mozilla.org #mozmill for backchannel

Attendees

Last action items

  • Geo: Work with Al and Anthony to get the on-demand update testing running on qa-set (April 27th)
  • [DONE] Henrik: Talk to David for a meeting about handling patch reviews with Softvision (include Web Automation)
    • It will not happen before Henrik comes back from vacation
  • [DONE] Henrik: Create automation scripts for handing new test-run called "remote" (May 19th)
  • [DONE] Anthony: Get in contact with Vlad by end of today to kick of Discovery Pane tests (May 18th)
  • Dave: Please reference last meeting notes from endurance meeting if you can't attend
  • Al: Collect possible use cases and goals on-demand provisioning
  • Al: Create remaining invalid certificates (May 20th)

Goals Overview

Risky Goals

Project Status

Redesign Shared Modules (Geo/Henrik)

  • No further API updates since the last meeting
  • Geo and Owen are working on porting browser tests so we can start creating the ui map soon
  • Henrik is working on the documentation for window handling

On-demand release tests (Geo/Henrik)

  • Anthony has discovered a "Connection Refused" issue with the daemon yesterday during release testing which prevents us from relaunching the daemon. We had to fallback to the old way of running update tests

Endurance Tests (Dave/Anthony)

  • Fixed the failing Panorama test
  • Stats are now precalculated in the automation script
  • Test results now displayed on the dashboard
  • Support for Firefox 6 is in review, dashboard patch will require updating
  • We're back to not restarting by default due to the outstanding issue with disconnects

Others

Mozmill Crowd

  • Patch for further launch points (add-on bar, Firefox button) has been landed
  • Working on the documentation and the version checker for test environments

Dashboard

  • Pushed some updates to the dashboard to make the display of failures more visually

Dashboard 4.0 (Owen)

Sync Server Load Testing (Owen)

  • Developing Functional Grinder Scripts
  • Creating a back end for generalized weave server intereaction

On Demand Provisioning (Cloud Automation) (Al/Henrik)

  • Machine is set up in PHX Colo. We're waiting for VSphere access from Phong.
  • Al will begin uploading and testing VMs as soon as access is available.

Tests / Shared Modules

General (Henrik)
Broken Tests
  • All broken functional tests have been fixed for branches down to 4.0
  • Endurance test for Tab Switching in Panorama is still failing - we really have to ensure to get those kind of failures fixed faster
    • Update - this is now fixed.
Softvision (Anthony)
  • Vlad and Alex working away on tests now that they are unblocked by Shared Modules
  • Investigating unnecessary tests based on what is already covered by Selenium

TCM (Cameron/?)

  • FC on .3 features
  • Prioritized tasks for .4 features
  • FC for .4 slated for July 15th
  • I'm looking for a second on the project. Hopefully Matt Brandt
  • Currently deployed on Oddsites. But I want to get a deployment at Mozilla on mozqa as soon as I can.

Shadow web server (Al/Henrik)

  • Mozqa.com domain should be transferred to Mozilla ownership on 6/2 according to updates yesterday.
    • OV and DV certs will be issued when Mozilla owns domain and deployed to server.
    • Self-signed expired cert has been issued and will be deployed today.

Automation Scripts / Infrastructure (Henrik)

Personal Status

For the personal status please check the weekly status updates:

Roundtable

General
  • To ensure a fast progress please work on action items right after the meeting, or at least get the process triggered. Doing that 1 or 2 days before the next meeting will push us back by nearly 2 weeks. (Henrik)
  • We have to get better in fixing broken tests. We can't leave broken ones failing for over two weeks! (Henrik)
    • Triage the results daily (Free for all, Assigned "sheriff", or Component "owners") and file bugs for failures not already on file. Initially assign to component "owner", if they can't fix in 24 hours, put call out to qa-auto@ and waverley@ to pick up. Should give us at least a week turn around time factoring in the review process (ashughes)

Meeting Notes

Action items

  • Geo: Send pivotaltracker details for API refactoring project to team members