QA/Mozmill Test Automation: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
 
(32 intermediate revisions by 6 users not shown)
Line 1: Line 1:
=Overview=
__NOTOC__
[https://developer.mozilla.org/en/Mozmill Mozmill] is a relatively new test tool to run functional tests in an automated manner. It can be used by applications which are based on the Gecko platform or [https://developer.mozilla.org/en/XULRunner XulRunner].


[http://quality.mozilla.org/ Mozilla QA] is using Mozmill to automate existing manual tests which were added to [https://litmus.mozilla.org/ Litmus] and folded into different testgroups, like Smoketests, Basic Functional Tests (BFT), and Full Funcional Tests (FFT). The primary goal is to lower the active testing time for any of our QA engineers and contributors. As a result we will be able to cover more than only one platform and locale for BFT testing. To get a better impression of the enhancements just compare those raw numbers: To run all the ~250 BFT tests manually it takes one man day while running all the BFT tests on every platform in parallel will eventually take about 40 minutes with Mozmill!
= OUT OF DATE WARNING =
This page is outdated since beginning of July 2011 when the [[QA/Automation|Automation Services]] team has been formed. Check its team page for further details.
 
= Overview =
 
[https://developer.mozilla.org/en/Mozmill Mozmill] is a test tool to run functional tests in an automated manner. It can be used by applications which are based on the Gecko platform or [https://developer.mozilla.org/en/XULRunner XulRunner].
 
[http://quality.mozilla.org/ Mozilla QA] is using Mozmill to automate existing manual tests which were added to [https://litmus.mozilla.org/ Litmus] and folded into different testgroups, like Smoketests, Basic Functional Tests (BFT), and Full Funcional Tests (FFT). The primary goal is to lower the active testing time for any of our QA engineers and contributors. As a result we will be able to cover more than only one platform and locale for BFT testing. To get a better impression of the enhancements just compare those raw numbers: To run all the BFT tests manually it takes about one man day while running all the tests on every platform in parallel will eventually take about 40 minutes with Mozmill!  
 
= Team Details  =


=Team Details=
Given the number of new features and regression fixes for any new Firefox major or security release, we will continuously enhance our functional testing framework by adding more tests of any type, increasing the number of platforms and locale builds to run the tests on, and by making the analysis of test reports more user friendly.  
Given the number of new features and regression fixes for any new Firefox major or security release, we will continuously enhance our functional testing framework by adding more tests of any type, increasing the number of platforms and locale builds to run the tests on, and by making the analysis of test reports more user friendly.  


[[File:Mozmill_Test_Automation_Map.png|650px|thumb|Mindmap of All Related Areas of Work]]
[[Image:Mozmill Test Automation Map.png|thumb|center|800px|Mozmill Test Automation Map.png]]  


== Team Members and Assignments  ==
== Team Members and Assignments  ==
Line 26: Line 33:
| [mailto:ahughes@mozilla.com ahughes@mozilla.com]  
| [mailto:ahughes@mozilla.com ahughes@mozilla.com]  
| Employee  
| Employee  
| Fixing failures and writing new tests
| Fixing failures and writing new tests, Endurance Tests
|-
| Al Billings
| [mailto:abillings@mozilla.com abillings@mozilla.com]
| Employee aka Team Punching Bag
| Writing new tests
|-
|-
| Geo Mealer  
| Geo Mealer  
| [mailto:gmealer@mozilla.com gmealer@mozilla.com]  
| [mailto:gmealer@mozilla.com gmealer@mozilla.com]  
| Employee
| Employee  
| MozMill testing, updating Shared APIs, Update Tests
| MozMill testing, updating Shared APIs, Update Tests
|-
|-
| Dave Hunt
| Dave Hunt  
| [mailto:dhunt@mozilla.com dhunt@mozilla.com]
| [mailto:dhunt@mozilla.com dhunt@mozilla.com]  
| Employee
| Employee  
|
| Endurance Tests
|-
|-
| Aaron Train  
| Aaron Train  
| [mailto:atrain@mozilla.com atrain@mozilla.com]  
| [mailto:atrain@mozilla.com atrain@mozilla.com]  
| Employee
| Fixing test failures, writing new tests, testing frameworks
|-
| Owen Coutts
| [mailto:ocoutts@mozilla.com ocoutts@mozilla.com]
| Employee (Intern)
| Revising Mozmill Dashboard
|-
| Vlad Maniac
| [mailto:vmaniac@mozilla.com vmaniac@mozilla.com]
| Contractor
| Mozmill tests
|-
| Alex Lakatos
| [mailto:alex.lakatos@softvision.ro alex.lakatos@softvision.ro]
| Contractor
| Fixing test failures, writing new tests, testing frameworks
|-
| Remus Pop
| [mailto:remus.pop@softvision.ro remus.pop@softvision.ro]
| Contractor  
| Contractor  
| Fixing test failures, writing new tests, testing frameworks
| Fixing test failures, writing new tests, testing frameworks
|}
|}


You can also get in contact with us by subscribing to the [http://groups.google.com/group/mozmill-dev?pli=1 mozmill-dev list] on Google or by joining the [http://www.mibbit.com/chat/?server=irc.mozilla.org&channel=%23mozmill #mozmill channel] on IRC.
<br> You can also get in contact with us by subscribing to the [http://groups.google.com/group/mozmill-dev?pli=1 mozmill-dev list] on Google or by joining the [http://www.mibbit.com/chat/?server=irc.mozilla.org&channel=%23mozmill #mozmill channel] on IRC.


=Calendar=
= Calendar =
You can subscribe to the [http://www.google.com/calendar/embed?src=f9liigdtlqadpc1f9pfisf24n0%40group.calendar.google.com&ctz=America/Los_Angeles Google calendar] ([http://www.google.com/calendar/ical/f9liigdtlqadpc1f9pfisf24n0%40group.calendar.google.com/public/basic.ics ical]) to see our events, milestones, and other deliverable dates.


=Meetings, Work Weeks, and Dates=
You can subscribe to the [http://www.google.com/calendar/embed?src=f9liigdtlqadpc1f9pfisf24n0%40group.calendar.google.com&ctz=America/Los_Angeles Google calendar] ([http://www.google.com/calendar/ical/f9liigdtlqadpc1f9pfisf24n0%40group.calendar.google.com/public/basic.ics ical]) to see our events, milestones, and other deliverable dates.  
We have bi-monthly public meetings about ongoing and upcoming projects. If you want to join the meetings, please check the [[QA/Test_Automation|meeting page]] for details. The same applies for work weeks which will be held once or twice within a year.


=Roadmap=
= Meetings, Work Weeks, and Dates =
While we wanna see as much as possible tests from all the Litmus testgroups automated, we do not have the needed man power to call it done in the next weeks or months. To get an idea how long it will take to reach the final goal some milestones have to be set for each of the upcoming quarters.


Lets summarize some facts about existing Litmus tests:
We have a bi-weekly public team project meeting. If you want to join the meetings, please check the [[QA/Test Automation|meeting page]] for details.  
* Around 250 tests in the BFT testgroup
* Around 710 remaining tests in the FFT testgroup
* Other testgroups with important tests, e.g. Software Updates


Beside the test creation work we will also focus other areas of work. In general the following goals have been set for the next quarters:
= Roadmap =


'''10/Q4'''
The QA Test Automation team has a couple of ongoing projects. Priorities for each of those will be re-defined at the beginning of each quarter.
* Release the Mozmill crowd-source extension to support localizers and add-on authors for Firefox 4.
 
* Implement Mozmill Dashboard views for software update and add-on tests 
== Q2 2011 ==
* Integrate local data-based Firefox 4 mozmill tests under Buildbot execution control and reporting
 
* Refactoring the shared module system based on the specification from Q3
*Our high-level goal is to complete the work on the [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Shared_Modules_Refactor API refactoring project]. It's necessary for a more streamlined and consistent way of creating Mozmill tests for Firefox. In combination with the basic API changes, new shared modules and ui maps have to be created for each single component our Mozmill tests cover. For this quarter we want to focus on the browser interface and the new Add-ons Manager.
* Collaborate with development team to expand Mozmill Panorama support to include an automated test suite for functional and performance testing of the Firefox 4 Panorama feature.
*With the new Add-ons Manager in-place for Firefox 4 there are needs for automating the existing manual tests on Litmus. We want to start with the tests for the Discovery Pane (Get Add-ons).
* Enlarge the amount of tests with local test data (Phase II)
*To follow-up on the great feedback we got for [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Endurance_Tests Endurance Tests] last quarter, we will work on ways to retrieve even more performance and system related data. That's necessary to have a broader range of Endurance Tests (not only memory usage) and data for our own analysis.
*With more and more parallel releases coming up in the future our internal testing strategy has to be improved by relying on the Pulse system for distributed messages. Those will be used to broadcast testing tasks across machines. It's useful for [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/On_Demand_Update_Testing update testing] but also for our functional tests.
 
= Areas of Work =


=Areas of Work=
<onlyinclude>
<onlyinclude>
As you can imagine a project like Mozmill automation has a couple of areas where work is happening or at least planned for the future. The following chapter will give you a good overview about all of our projects.
As you can imagine a project like Mozmill automation has a couple of areas where work is happening or at least planned for the future. The following chapter will give you a good overview about all of our projects.
Line 86: Line 104:
| style="background:#EFEFEF; width:66%" | '''Description'''
| style="background:#EFEFEF; width:66%" | '''Description'''
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Fixing Broken Tests|Broken Tests]]  
| [[QA/Mozmill Test Automation/Test Writing|New Mozmill Tests]]  
| Making sure to always have a green test-run for Mozmill tests.
| Create automated tests to minimize our manual testing efforts.
|- valign="top"
| [[QA/Mozmill Test Automation/Shared Modules|New Shared Modules]]
| <strike>Simplify Mozmill test scripting by using shared modules.</strike>
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Test Writing|Mozmill Tests]]  
| [[QA/Mozmill Test Automation/Endurance Tests|Endurance Tests]]  
| Create automated tests to minimize our manual testing efforts.
| <strike>Create Mozmill tests to ensure Firefox endurance and detection of behaviours under sustained use.</strike>
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Shared Modules|Shared Modules]]  
| [[QA/Mozmill Test Automation/Fixing Broken Tests|Handling of Broken Tests]]  
| Simplify Mozmill test scripting by using shared modules.
| Making sure to always have a green test-run for Mozmill tests.
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Shared Modules Refactor|Shared Modules Refactor]]  
| [[QA/Mozmill Test Automation/Shared Modules Refactor|Shared Modules Refactor]]  
| Exploring new ways to present the Shared API to simplify test scripts even further.
| <strike>Exploring new ways to present the Shared API to simplify test scripts even further.</strike>
|- valign="top"
| [[QA/Mozmill Test Automation/Test Modules Refactor|Test Modules Refactor]]
| Work in parallel with the Shared Modules Refactor project to simplify the test modules.
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Update Tests|Software Update Tests]]  
| [[QA/Mozmill Test Automation/Update Tests|Software Update Tests]]  
| Create Mozmill tests to ensure lesser problems during an upgrade of Firefox.
| Create Mozmill tests to ensure lesser problems during an upgrade of Firefox.
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Accessibility Tests|Accessibility Tests]]  
| [[QA/Mozmill Test Automation/Addon Tests|Add-ons Tests]]  
| Create Mozmill tests to ensure Firefox is accessible by everyone.
| Create Mozmill tests to run functional tests against add-ons.
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/L10n Tests|L10n Tests]]  
| [[QA/Mozmill Test Automation/L10n Tests|L10n Tests]]  
| Create Mozmill tests to ensure that localized builds are not broken.
| Create Mozmill tests to ensure that localized builds are not broken.
|}
=== Automated Testing ===
{| class="fullwidth-table"
|-
| style="background:#EFEFEF; width:33%" | '''Project'''
| style="background:#EFEFEF; width:66%" | '''Description'''
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Addon Tests|Add-ons Tests]]  
| [[QA/Mozmill Test Automation/Crowd Testing|Crowd-sourced Testing]]  
| Create Mozmill tests to run functional tests against add-ons.
| <strike>Make it totally easy for everyone to run Mozmill tests from within Firefox.</strike>
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Panorama Testing|Panorama Testing]]  
| [[QA/Mozmill Test Automation/Automated Testruns|Automated Test-runs]]  
| Create a high-quality suite of performance and functional tests to support the Panorama feature of Firefox 4
| <strike>Create scripts for a collection of automated test-runs.</strike>
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Test Modules Refactor|Test Modules Refactor]]  
| [[QA/Mozmill Test Automation/Triggered Update Testing|Triggered Update Testing]]  
| Work in parallel with the Shared Modules Refactor project to simplify the test modules.
| <strike>Perform nightly update testing triggered by landing of the build</strike>
|- valign="top"
|- valign="top"
| [[QA/Mozmill Test Automation/Endurance Tests|Endurance Tests]]  
| [[QA/Mozmill Test Automation/On Demand Update Testing|On-Demand Update Testing]]  
| Create Mozmill tests to ensure Firefox endurance and detection of behaviours under sustained use.
| <strike>Streamline how update testing is performed on releases</strike>
|}
|}


===Automated Testing===
=== Reporting ===
{| class="fullwidth-table"
| style="background:#EFEFEF; width:33%" | '''Project'''
| style="background:#EFEFEF; width:66%" | '''Description'''
|- valign="top"
|[[QA/Mozmill_Test_Automation/Crowd_Testing|Crowd Testing]]
|Make it totally easy for everyone to run Mozmill tests from within Firefox.
|- valign="top"
|[[QA/Mozmill_Test_Automation/Automated_Testruns|Automated Test-runs]]
|Create scripts for a collection of automated test-runs.
|- valign="top"
|[[QA/Mozmill_Test_Automation/Triggered_Update_Testing|Triggered Update Testing]]
|Perform nightly update testing triggered by landing of the build
|- valign="top"
|[[QA/Mozmill_Test_Automation/On_Demand_Update_Testing|On-Demand Update Testing]]
|Streamline how update testing is performed on releases
|}


===Reporting===
{| class="fullwidth-table"
{| class="fullwidth-table"
| style="background:#EFEFEF; width:25%" | '''Project'''
|-
| style="background:#EFEFEF; width:25%" | '''Leads'''
| style="background:#EFEFEF; width:25%" | '''Project'''  
| style="background:#EFEFEF; width:50%" | '''Description'''
| style="background:#EFEFEF; width:25%" | '''Leads'''  
|- valign="top"
| style="background:#EFEFEF; width:50%" | '''Description'''
|[[QA/Mozmill_Test_Automation/Dashboard|Web Dashboard]]
|- valign="top"
|Henrik Skupin<br/>Anthony Hughes
| [[QA/Mozmill Test Automation/Dashboard|Web Dashboard]]  
|Dashboard to visualize collected Mozmill test results from the automated test-runs and the Mozmill Crowd extension
| Henrik Skupin<br>Anthony Hughes  
|}
| <strike>Dashboard to visualize collected Mozmill test results from the automated test-runs and the Mozmill Crowd extension</strike>
|}


===Documentation===
===Documentation===
Line 158: Line 174:
  |}
  |}


===Miscellaneous===
</onlyinclude>
 
=== On hold ===
 
{| class="fullwidth-table"
{| class="fullwidth-table"
| style="background:#EFEFEF; width:33%" | '''Project'''
|-
| style="background:#EFEFEF; width:66%" | '''Description'''
| style="background:#EFEFEF; width:33%" | '''Project'''  
|- valign="top"
| style="background:#EFEFEF; width:66%" | '''Description'''
|[[QA/Mozmill_Test_Automation/On_Demand_Provisioning|On Demand Provisioning]]
|- valign="top"
|On Demand virtual machines for use in automation or other testing.
| [[QA/Mozmill Test Automation/Accessibility Tests|Accessibility Tests]]  
|}
| Create Mozmill tests to ensure Firefox is accessible by everyone.
</onlyinclude>
|- valign="top"
| [[QA/Mozmill Test Automation/Panorama Testing|Panorama Testing]]
| Create a high-quality suite of performance and functional tests to support the Panorama feature of Firefox 4
|}
 
= Finished Projects =


=Finished Projects=
{| class="fullwidth-table"
{| class="fullwidth-table"
| style="background:#EFEFEF; width:33%" | '''Project'''
|-
| style="background:#EFEFEF; width:66%" | '''Description'''
| style="background:#EFEFEF; width:33%" | '''Project'''  
|- valign="top"
| style="background:#EFEFEF; width:66%" | '''Description'''
|[[QA/Mozmill_Test_Automation/Results_Data|Result Data]]
|- valign="top"
|Collecting necessary result data of test-runs for each test type.
| [[QA/Mozmill Test Automation/Results Data|Result Data]]  
|}
| Collecting necessary result data of test-runs for each test type.
|}
 
= Contribution =
 
Each of the above mentioned sub-projects needs love from someone with passion to work on such a challenging task. There will be a project lead who will offer help in various ways.
 
If you are interested in helping us to push Mozmill a big step forward, please get in contact with [mailto:hskupin@mozilla.com Henrik Skupin]. We will figure out what's the best way for your contribution.


=Contribution=
== First Steps ==
Each of the above mentioned sub-projects needs love from someone with passion to work on such a challenging task. There will be a project lead who will offer help in various ways.


If you are interested in helping us to push Mozmill a big step forward, please get in contact with [mailto:hskupin@mozilla.com Henrik Skupin]. We will figure out what's the best way for your contribution.
Joining an existing project can be hard especially when you see all the stuff in the mind map. To give you a handy guide for the first steps a [https://developer.mozilla.org/en/Mozmill_Tests rich documentation] has been created. It covers the following areas:


==First Steps==
*[https://developer.mozilla.org/en/Mozmill_Tests#Installation_of_Mozmill Installation of Mozmill]
Joining an existing project can be hard especially when you see all the stuff in the mind map. To give you a handy guide for the first steps a [https://developer.mozilla.org/en/Mozmill_Tests rich documentation] has been created. It covers the following areas:
*Getting in touch with the [https://developer.mozilla.org/en/Mozmill_Tests#The_Test_Repository mozmill-test repository]
*Executing existent [https://developer.mozilla.org/en/Mozmill_Tests#Running_Mozmill_Tests Mozmill tests]


* [https://developer.mozilla.org/en/Mozmill_Tests#Installation_of_Mozmill Installation of Mozmill]
All those documents should bring you into a state to be able to play around with Mozmill. At least by running those tests against nightly or release builds of Firefox and reporting failed tests would be a great help for us.  
* Getting in touch with the [https://developer.mozilla.org/en/Mozmill_Tests#The_Test_Repository mozmill-test repository]
* Executing existent [https://developer.mozilla.org/en/Mozmill_Tests#Running_Mozmill_Tests Mozmill tests]


All those documents should bring you into a state to be able to play around with Mozmill. At least by running those tests against nightly or release builds of Firefox and reporting failed tests would be a great help for us.
= Resources =


=Resources=
There are a couple of resources which are related to Mozmill. A nearly complete list you will find below:  
There are a couple of resources which are related to Mozmill. A nearly complete list you will find below:


* [http://quality.mozilla.org/docs/mozmill/ Overview] about the Mozmill project on QMO
*[http://quality.mozilla.org/docs/mozmill/ Overview] about the Mozmill project on QMO  
* Complete documentation for [https://developer.mozilla.org/en/Mozmill Mozmill]
*Complete documentation for [https://developer.mozilla.org/en/Mozmill Mozmill]  
* Documentation for [https://developer.mozilla.org/en/Mozmill_Tests/Shared_Modules Shared Modules]
*Documentation for [https://developer.mozilla.org/en/Mozmill_Tests/Shared_Modules Shared Modules]  
* How to use Mozmill and [https://developer.mozilla.org/en/Mozmill_Tests create tests] for Firefox
*How to use Mozmill and [https://developer.mozilla.org/en/Mozmill_Tests create tests] for Firefox

Latest revision as of 09:45, 14 January 2014


OUT OF DATE WARNING

This page is outdated since beginning of July 2011 when the Automation Services team has been formed. Check its team page for further details.

Overview

Mozmill is a test tool to run functional tests in an automated manner. It can be used by applications which are based on the Gecko platform or XulRunner.

Mozilla QA is using Mozmill to automate existing manual tests which were added to Litmus and folded into different testgroups, like Smoketests, Basic Functional Tests (BFT), and Full Funcional Tests (FFT). The primary goal is to lower the active testing time for any of our QA engineers and contributors. As a result we will be able to cover more than only one platform and locale for BFT testing. To get a better impression of the enhancements just compare those raw numbers: To run all the BFT tests manually it takes about one man day while running all the tests on every platform in parallel will eventually take about 40 minutes with Mozmill!

Team Details

Given the number of new features and regression fixes for any new Firefox major or security release, we will continuously enhance our functional testing framework by adding more tests of any type, increasing the number of platforms and locale builds to run the tests on, and by making the analysis of test reports more user friendly.

Mozmill Test Automation Map.png

Team Members and Assignments

Name Contact Availability Project Assignments
Henrik Skupin hskupin@mozilla.com Employee Team Lead
(Details see Areas of Work)
Anthony Hughes ahughes@mozilla.com Employee Fixing failures and writing new tests, Endurance Tests
Geo Mealer gmealer@mozilla.com Employee MozMill testing, updating Shared APIs, Update Tests
Dave Hunt dhunt@mozilla.com Employee Endurance Tests
Aaron Train atrain@mozilla.com Employee Fixing test failures, writing new tests, testing frameworks
Owen Coutts ocoutts@mozilla.com Employee (Intern) Revising Mozmill Dashboard
Vlad Maniac vmaniac@mozilla.com Contractor Mozmill tests
Alex Lakatos alex.lakatos@softvision.ro Contractor Fixing test failures, writing new tests, testing frameworks
Remus Pop remus.pop@softvision.ro Contractor Fixing test failures, writing new tests, testing frameworks


You can also get in contact with us by subscribing to the mozmill-dev list on Google or by joining the #mozmill channel on IRC.

Calendar

You can subscribe to the Google calendar (ical) to see our events, milestones, and other deliverable dates.

Meetings, Work Weeks, and Dates

We have a bi-weekly public team project meeting. If you want to join the meetings, please check the meeting page for details.

Roadmap

The QA Test Automation team has a couple of ongoing projects. Priorities for each of those will be re-defined at the beginning of each quarter.

Q2 2011

  • Our high-level goal is to complete the work on the API refactoring project. It's necessary for a more streamlined and consistent way of creating Mozmill tests for Firefox. In combination with the basic API changes, new shared modules and ui maps have to be created for each single component our Mozmill tests cover. For this quarter we want to focus on the browser interface and the new Add-ons Manager.
  • With the new Add-ons Manager in-place for Firefox 4 there are needs for automating the existing manual tests on Litmus. We want to start with the tests for the Discovery Pane (Get Add-ons).
  • To follow-up on the great feedback we got for Endurance Tests last quarter, we will work on ways to retrieve even more performance and system related data. That's necessary to have a broader range of Endurance Tests (not only memory usage) and data for our own analysis.
  • With more and more parallel releases coming up in the future our internal testing strategy has to be improved by relying on the Pulse system for distributed messages. Those will be used to broadcast testing tasks across machines. It's useful for update testing but also for our functional tests.

Areas of Work

As you can imagine a project like Mozmill automation has a couple of areas where work is happening or at least planned for the future. The following chapter will give you a good overview about all of our projects.

Mozmill Tests

Project Description
New Mozmill Tests Create automated tests to minimize our manual testing efforts.
New Shared Modules Simplify Mozmill test scripting by using shared modules.
Endurance Tests Create Mozmill tests to ensure Firefox endurance and detection of behaviours under sustained use.
Handling of Broken Tests Making sure to always have a green test-run for Mozmill tests.
Shared Modules Refactor Exploring new ways to present the Shared API to simplify test scripts even further.
Test Modules Refactor Work in parallel with the Shared Modules Refactor project to simplify the test modules.
Software Update Tests Create Mozmill tests to ensure lesser problems during an upgrade of Firefox.
Add-ons Tests Create Mozmill tests to run functional tests against add-ons.
L10n Tests Create Mozmill tests to ensure that localized builds are not broken.

Automated Testing

Project Description
Crowd-sourced Testing Make it totally easy for everyone to run Mozmill tests from within Firefox.
Automated Test-runs Create scripts for a collection of automated test-runs.
Triggered Update Testing Perform nightly update testing triggered by landing of the build
On-Demand Update Testing Streamline how update testing is performed on releases

Reporting

Project Leads Description
Web Dashboard Henrik Skupin
Anthony Hughes 		Dashboard to visualize collected Mozmill test results from the automated test-runs and the Mozmill Crowd extension

Documentation

Project Description
Shared Modules Documentation on how to use shared modules and their feature set.


On hold

Project Description
Accessibility Tests Create Mozmill tests to ensure Firefox is accessible by everyone.
Panorama Testing Create a high-quality suite of performance and functional tests to support the Panorama feature of Firefox 4

Finished Projects

Project Description
Result Data Collecting necessary result data of test-runs for each test type.

Contribution

Each of the above mentioned sub-projects needs love from someone with passion to work on such a challenging task. There will be a project lead who will offer help in various ways.

If you are interested in helping us to push Mozmill a big step forward, please get in contact with Henrik Skupin. We will figure out what's the best way for your contribution.

First Steps

Joining an existing project can be hard especially when you see all the stuff in the mind map. To give you a handy guide for the first steps a rich documentation has been created. It covers the following areas:

All those documents should bring you into a state to be able to play around with Mozmill. At least by running those tests against nightly or release builds of Firefox and reporting failed tests would be a great help for us.

Resources

There are a couple of resources which are related to Mozmill. A nearly complete list you will find below:

Retrieved from "https://dev.wikimo.nonprod.webservices.mozgcp.net/index.php?title=QA/Mozmill_Test_Automation&oldid=891051"