Releases/Firefox 65/Test Plan: Difference between revisions
(Created page with "= Summary = This page is to track testing of Firefox 65 from mozilla-central (Nightly) through to mozilla-release. This is intended to be a minimal set of information to help...") |
No edit summary |
||
(60 intermediate revisions by 10 users not shown) | |||
Line 111: | Line 111: | ||
''This section tracks bugs representing the most urgent stability issues. [[QA/Desktop_Firefox/Walkthroughs/Stability|Consult this wiki]] for more information.'' | ''This section tracks bugs representing the most urgent stability issues. [[QA/Desktop_Firefox/Walkthroughs/Stability|Consult this wiki]] for more information.'' | ||
* Ensure new and explosive crashes have actionable bug reports which are in progress | * Ensure new and explosive crashes have actionable bug reports which are in progress | ||
* [https://crash-stats.mozilla.com/topcrashers/?product=Firefox&version= | * [https://crash-stats.mozilla.com/topcrashers/?product=Firefox&version=65.0a1&days=7 Top Crashes] - ensure the highest ranked crashes have actionable bug reports which are in progress | ||
* [https://bugzilla.mozilla.org/buglist.cgi?f1=keywords&o1=anywordssubstr&query_format=advanced&bug_status=UNCONFIRMED&v1=crash&component=Untriaged Untriaged Bugs] - ensure bugs are actionable and in the correct component | * [https://bugzilla.mozilla.org/buglist.cgi?f1=keywords&o1=anywordssubstr&query_format=advanced&bug_status=UNCONFIRMED&v1=crash&component=Untriaged Untriaged Bugs] - ensure bugs are actionable and in the correct component | ||
* [https://bugzilla.mozilla.org/buglist.cgi?j_top=OR&keywords=topcrash-win%2C%20topcrash-mac%2C%20topcrash-linux%2C%20topcrash-plugin%2C%20&keywords_type=anywords&list_id= | * [https://bugzilla.mozilla.org/buglist.cgi?j_top=OR&keywords=topcrash-win%2C%20topcrash-mac%2C%20topcrash-linux%2C%20topcrash-plugin%2C%20&keywords_type=anywords&f1=cf_status_firefox65&list_id=14453385&o1=nowordssubstr&o2=anyexact&query_format=advanced&f2=target_milestone&v1=---%20unaffected%20verified%20wontfix&v2=Firefox%2062%2Cmozilla65 topcrash bugs] | ||
=== Unconfirmed Bug Triage === | === Unconfirmed Bug Triage === | ||
Line 151: | Line 151: | ||
Queries: | Queries: | ||
* Dashboard showing current active iteration: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/ | * Dashboard showing current active iteration: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/ | ||
* Iteration | * Iteration 65.1, all bugs: https://bugzilla.mozilla.org/buglist.cgi?f1=OP&j1=OR&f2=cf_fx_iteration&o2=substring&v2=65.1&f3=status_whiteboard&o3=substring&v3=s%3D65.1&f4=CP | ||
* Iteration | * Iteration 65.2, all bugs: https://bugzilla.mozilla.org/buglist.cgi?f1=OP&j1=OR&f2=cf_fx_iteration&o2=substring&v2=65.2&f3=status_whiteboard&o3=substring&v3=s%3D65.2&f4=CP | ||
* Iteration | * Iteration 65.3, all bugs: https://bugzilla.mozilla.org/buglist.cgi?f1=OP&j1=OR&f2=cf_fx_iteration&o2=substring&v2=65.3&f3=status_whiteboard&o3=substring&v3=s%3D65.3&f4=CP | ||
= Feature status = | = Feature status = | ||
Line 172: | Line 172: | ||
! width="200" | TEST REPORT | ! width="200" | TEST REPORT | ||
|- | |- | ||
| style="text-align:center;" | 2018- | | style="text-align:center;" | 2018-11-01 | ||
| style="background:green; color: white;" | GREEN | | style="background:green; color: white;" | GREEN | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* | * [https://bugzil.la/1503603 1 new bug] uncovered, low-impact. | ||
* Known (old) bugs are low-impact. | * Known (old) bugs are low-impact. | ||
| style="text-align: center;" | --- | | style="text-align: center;" | --- | ||
| style="text-align: center;" | [https://bugzilla.mozilla.org/buglist.cgi? | | style="text-align: center;" | [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1481823%2C%201282075%2C%201393460%2C%201476569%2C%201476585%2C%20558178%2C%201416721%2C%201396732%2C%201443511%2C%201476569%2C%201194214%2C%201441881%2C%201493930%2C1194214%2C%201441881%2C%201476569%2C%201481755%2C%201282075%2C%201490459%2C%201416721%2C%20558178%2C%201476585%2C%201493930%2C%201465445%2C%201393460%2C%201481823%2C%201481496%2C%201493930%2C%201360545%2C1378105%2C%201449956%2C%201494258%2C%201445644&list_id=14453367 Bugzilla link] | ||
| style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/ | | style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/plans/view/13228 TestRail link] | ||
|- | |- | ||
| style="text-align:center;" | 2018- | | style="text-align:center;" | 2018-11-15 | ||
| style="background:green; color: white;" | GREEN | | style="background:green; color: white;" | GREEN | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* [https://bugzil.la/ | * [https://bugzil.la/1506831 1 new bug] uncovered, low-impact. | ||
* Known (old) bugs are low-impact. | * Known (old) bugs are low-impact. | ||
| style="text-align: center;" | --- | | style="text-align: center;" | --- | ||
| style="text-align: center;" | [https://bugzilla.mozilla.org/buglist.cgi? | | style="text-align: center;" | [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1481823%2C%201282075%2C%201393460%2C%201476569%2C%201476585%2C%20558178%2C%201416721%2C%201396732%2C%201443511%2C%201476569%2C%201194214%2C%201441881%2C%201493930%2C1194214%2C%201441881%2C%201476569%2C%201481755%2C%201282075%2C%201490459%2C%201416721%2C%20558178%2C%201476585%2C%201493930%2C%201465445%2C%201393460%2C%201481823%2C%201481496%2C%201493930%2C%201360545%2C1378105%2C%201449956%2C%201494258%2C%201445644&list_id=14453367 Bugzilla link] | ||
| style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/plans/view/13594 TestRail link] | |||
| style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/ | |||
|- | |- | ||
|} | |} | ||
Line 230: | Line 212: | ||
! style="text-align: center;" width=400 | Known (old) | ! style="text-align: center;" width=400 | Known (old) | ||
|- | |- | ||
| style="text-align: center;" | | | style="text-align: center;" | 65.0b1-build1 | ||
| style="text-align: center; background: green; color:white" | GREEN | | style="text-align: center; background: green; color:white" | GREEN | ||
| | | | ||
* No new issues uncovered during testing. | * No new issues uncovered during testing. | ||
* Known issues are low-impact. | * Known issues are low-impact. | ||
| style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/ | | style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/plans/view/14059 TestRail link] | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* Regressions: none | * Regressions: none | ||
Line 241: | Line 223: | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* High-impact: none | * High-impact: none | ||
* Low-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch= | * Low-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=837985%201057858%201154015%201194214%201358102%201393460%201441128%201441881%201446411%201459211%201459905%201470409%201485292%201498567&list_id=14463925 list] | ||
|- | |- | ||
| style="text-align: center;" | | | style="text-align: center;" | 65.0b2-build1 | ||
| style="text-align: center; background:green; color:white" | GREEN | | style="text-align: center; background: green; color:white" | GREEN | ||
| | | | ||
* | * Two new issues uncovered during testing {{bug|1512599}} and {{bug|1512611}}. | ||
* Known issues are low-impact. | * Known issues are low-impact. | ||
| style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/ | | style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/plans/view/14135 TestRail link] | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* Regressions: none | * Regressions: none | ||
* Non-regressions: | * Non-regressions: {{bug|1512599}} and {{bug|1512611}}. | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* High-impact: none | * High-impact: none | ||
* Low-impact: | * Low-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1194214%201233359%201360545%201378105%201443511%201446411%201481400%201490459%201504197&list_id=14463800 list] | ||
|- | |- | ||
| style="text-align: center;" | | | style="text-align: center;" | 65.0b6-build1 | ||
| style="text-align: center; background:green; color:white" | GREEN | | style="text-align: center; background: green; color:white" | GREEN | ||
| | | | ||
* | * Two new issue uncovered during testing {{bug|1516254}} {{bug|1516256}} | ||
* Known issues are low | * Known issues are low impact. | ||
| style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/ | | style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/plans/view/14426 TestRail link] | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* Regressions: | * Regressions: {{bug|1516254}} and {{bug|1516256}} | ||
* Non-regressions: none | * Non-regressions: none | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* High-impact: none | * High-impact: none | ||
* Low-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=428378%20558178%20761651%201194214 | * Low-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=428378%20558178%20761651%201009132%201194214%201306561%201309534%201407718%201410009%201411533%201414833%201429065%201430276%201441128%201463062%201490614%201497897%201504721%201512599%201515624&list_id=14486812 list] | ||
|- | |- | ||
| style="text-align: center;" | | | style="text-align: center;" | 65.0b9-build1 | ||
| style="text-align: center; background:green; color:white" | GREEN | | style="text-align: center; background: green; color:white" | GREEN | ||
| | | | ||
* | * Two new issue uncovered during testing {{bug|1518825}} {{bug|1518816}} | ||
* Known issues are low | * Known issues are low impact. | ||
| style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/ | | style="text-align: center;" | [https://testrail.stage.mozaws.net/index.php?/plans/view/14580 TestRail link] | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* Regressions: | * Regressions: {{bug|1518825}} and {{bug|1518816}} | ||
* Non-regressions: none | * Non-regressions: none | ||
| style="text-align: left;" | | | style="text-align: left;" | | ||
* High-impact: none | * High-impact: none | ||
* Low-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=428378%20558178% | * Low-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=428378%20558178%20940733%201009132%201056056%201194214%201306561%201309534%201370743%201407718%201411533%201414833%201429065%201463062%201490614%201512599%201516254%201518788&list_id=14504122 list] | ||
|- | |- | ||
|} | |} | ||
Line 289: | Line 271: | ||
=== Beta 3 === | === Beta 3 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b3-candidates/build2/ 65.0b3-build2] | [https://hg.mozilla.org/releases/mozilla-beta/rev/0cf938ef3c98 0cf938ef3c98] | (ID: 20181210164452) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/3 Firefox 65 Beta 3] | ||
** | ** 65.0b3-build2 | ||
*** {{mdone|}} Smoke and Exploratory Testing with focus on: | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Plugin compatibility & support | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/reports/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/reports/view/# TestRail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/1bQQ_NrV9aOAf2NI9kq7aiJRNnRXBNoQ6TCqw5vXtHbs/edit report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/1bQQ_NrV9aOAf2NI9kq7aiJRNnRXBNoQ6TCqw5vXtHbs/edit report] | ||
=== Beta 4 === | === Beta 4 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b4-candidates/build1/ 65.0b4-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/b1216967cdd0 b1216967cdd0] | (ID: 20181211223337) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/4 Firefox 65 Beta 4] | ||
** | ** 65.0b4-build1 | ||
*** {{mdone|}} Smoke and Exploratory Testing with focus on: | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Crash Reporting, Safe Browsing, WebExtensions compatibility & support. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/reports/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/reports/view/1333 Testrail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/14xetP6WH8terDDuyQWEvZ1syMGqsAkVvSSp2xxJTmOc/edit report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/14xetP6WH8terDDuyQWEvZ1syMGqsAkVvSSp2xxJTmOc/edit report] | ||
=== Beta 5 === | === Beta 5 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b5-candidates/build1/ 65.0b5-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/dfe78db5e4ae dfe78db5e4ae] | (ID: 20181217180946) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/5 Firefox 65 Beta 5] | ||
** | ** 65.0b5-build1 | ||
*** {{mdone|}} Smoke and Exploratory Testing with focus on: | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Toolbars & window controls and Content scroll, zoom & drag support. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14335 Testrail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/1z2uttTcj-m0TAnEprDW0PpV35Z7Ymfa7ga4j83vOMjw/edit report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/1z2uttTcj-m0TAnEprDW0PpV35Z7Ymfa7ga4j83vOMjw/edit report] | ||
=== Beta 6 === | === Beta 6 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b6-candidates/build1/ 65.0b6-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/4f9b811b0f2a 4f9b811b0f2a] | (ID: 20181220174318) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/6 Firefox 65 Beta 6] | ||
** | ** 65.0b6-build1 | ||
*** {{mdone}} Smoke and Exploratory Testing with focus on: Media playback & support and | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Graphics compatibility & support (skia), Media playback & support (audio) and Download manager. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14401 Testrail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/1sybnyQFA3zFzO2aWXASM9LbQn_Vl_i7dImd9TFcclKo/edit report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/1sybnyQFA3zFzO2aWXASM9LbQn_Vl_i7dImd9TFcclKo/edit report] | ||
=== Beta 7 === | === Beta 7 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b7-candidates/build1/ 65.0b7-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/22da6211feb6 22da6211feb6] | (ID: 20181227144402) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/7 Firefox 65 Beta 7] | ||
** | ** 65.0b7-build1 | ||
*** {{mdone}} Smoke and Exploratory Testing with focus on: | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Profile support & compatibility, Private Browsing & Geolocation. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14401 Testrail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https:// | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://drive.google.com/open?id=1mQi2VMsuCs9ge8hCEEhPi0IzN_VLmXgNlX4WXhnAX1A report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https:// | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://drive.google.com/open?id=1mQi2VMsuCs9ge8hCEEhPi0IzN_VLmXgNlX4WXhnAX1A report] | ||
=== Beta 8 === | === Beta 8 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b8-candidates/build1/ 65.0b8-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/35ba72605a9a 35ba72605a9a] | (ID: 20190103150357) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/8 Firefox 65 Beta 8] | ||
** | ** 65.0b8-build1 | ||
*** {{mdone}} Smoke and Exploratory Testing with focus on: | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Search, In-browser PDF support and Image compatibility & support. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14495 Testrail]. | ||
*** {{ | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/1XBVdGMDLQqP0YMa04EHGWg03b1FAl7mCFEjuQ4Ba_9g/edit report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/1XBVdGMDLQqP0YMa04EHGWg03b1FAl7mCFEjuQ4Ba_9g/edit report] | ||
=== Beta 9 === | === Beta 9 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b9-candidates/build1/ 65.0b9-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/df848230cb30 df848230cb30] | (ID: 20190107180200) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/9 Firefox 65 Beta 9] | ||
** | ** 65.0b9-build1 | ||
*** {{mdone}} Smoke and Exploratory Testing with focus on: | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Printing, Default & custom theme support, Bookmarks and Smart Card. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14537 Testrail]. | ||
*** {{ | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/1tAxSlFAADqaAEbFIJDOZl-E24h46nQb2GTX4ww3o5_w/edit report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/1tAxSlFAADqaAEbFIJDOZl-E24h46nQb2GTX4ww3o5_w/edit report] | ||
=== Beta 10 === | === Beta 10 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b10-candidates/build1/ 65.0b10-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/f0bb91ca0015 f0bb91ca0015] | (ID: 20190107180200) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/10 Firefox 65 Beta 10] | ||
** | ** 65.0b10-build1 | ||
*** {{mdone}} Smoke and Exploratory Testing with focus on: File system interaction | *** {{mdone|}} Smoke and Exploratory Testing with focus on: Geolocation, Privacy, File system interaction and Data Migration. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14663 Testrail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/1x4gxSDuP8LwQoZwuctfMAbmas2MQzTmsygPj-5D0rNQ/edit# report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/1x4gxSDuP8LwQoZwuctfMAbmas2MQzTmsygPj-5D0rNQ/edit# report] | ||
=== Beta 11 === | === Beta 11 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b11-candidates/build1/ 65.0b11-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/8574d2808aaa 8574d2808aaa] | (ID: 20190114172331) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/11 Firefox 65 Beta 11] | ||
** | ** 65.0b11-build1 | ||
*** {{mdone}} Smoke and Exploratory Testing with focus on: History & | *** {{mdone|}} Smoke and Exploratory Testing with focus on: History and Content scroll, zoom & drag support. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14727 Testrail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/14iTyl9q8yq3UYedpnOo04sU0NTEeBMdzoGuB5SxkhaI/edit report] | ||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels , [https://docs.google.com/document/d/ | ** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/14iTyl9q8yq3UYedpnOo04sU0NTEeBMdzoGuB5SxkhaI/edit report] | ||
=== Beta 12 === | |||
==== Build Information ==== | |||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0b12-candidates/build1/ 65.0b12-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/666a44dfa6eb 666a44dfa6eb] | (ID: 20190117232427) | |||
=== Beta 12 === | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/12 Firefox 65 Beta 12] | |||
** 65.0b12-build1 | |||
*** {{mdone|}} Smoke and Exploratory Testing with focus on: Plugin compatibility & support, Session Restore and Password Manager & Form Autofill. | |||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14809 Testrail]. | |||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | |||
* Updates testing | |||
** {{mdone|}} Updates on ''beta-cdntest'' & ''aurora-cdntest'' channels, [https://docs.google.com/document/d/1WlwZ0xlkrDw8ewn1pqHswf0pux65YOQMGDFyaM0uNtU/edit report] | |||
** {{mdone|}} Updates on ''beta'' & ''aurora'' channels, [https://docs.google.com/document/d/1WlwZ0xlkrDw8ewn1pqHswf0pux65YOQMGDFyaM0uNtU/edit report] | |||
== Release Candidates == | |||
=== 65.0-build1 === | |||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0-candidates/build1/ 65.0-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/be4781c53713 be4781c53713] | (ID: 20190121133710) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/RC1 Firefox 65 RC1] | ||
** | ** 65.0-build1 | ||
*** {{mdone}} Smoke | *** {{mdone|}} Smoke Testing on: Web Compatibility. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14871 TestRail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on ''beta-cdntest'' | ** {{mdone|}} Updates on ''beta-cdntest'' channel, [https://docs.google.com/document/d/16oFDVlAV6iLM6ex6JamWHEnSHRI1fvQWYB0laJA_iDk/edit report] | ||
** {{ | ** {{mnotstarted|}} Updates on ''beta'' channel, [https://docs.google.com/document/d/16oFDVlAV6iLM6ex6JamWHEnSHRI1fvQWYB0laJA_iDk/edit report] | ||
** {{mnotstarted|}} Updates on ''release-localtest'' channel, [https://docs.google.com/document/d/16uqxeX1zJJQ47Ea16SPH8mIdFCSi4rwcijL2X3b62FE/edit report] | |||
** {{mnotstarted|}} Updates on ''release-cdntest'' channel, [https://docs.google.com/document/d/16uqxeX1zJJQ47Ea16SPH8mIdFCSi4rwcijL2X3b62FE/edit report] | |||
** {{mnotstarted|}} Updates on ''release'' channel, [https://docs.google.com/document/d/16uqxeX1zJJQ47Ea16SPH8mIdFCSi4rwcijL2X3b62FE/edit report] | |||
=== 65.0-build2 === | |||
==== Build Information ==== | |||
* Build 2: [https://archive.mozilla.org/pub/firefox/candidates/65.0-candidates/build2/ 65.0-build2] | [https://hg.mozilla.org/releases/mozilla-release/rev/be4781c53713 be4781c53713] | (ID: 20190121133710) | |||
==== Testing ==== | |||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/RC2 Firefox 65 RC2] | |||
** 65.0-build2 | |||
*** {{mdone|}} Smoke Testing on: Web Compatibility. | |||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/14923 TestRail]. | |||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | |||
* Updates testing | |||
** {{mdone|}} Updates on ''beta-cdntest'' channel, [https://docs.google.com/document/d/1ZNUE-V9dXqgNF70S9uEaP9agqmGOV1pSXNuudklw9Ss/edit report] | |||
** {{mdone|}} Updates on ''beta'' channel, [https://docs.google.com/document/d/1ZNUE-V9dXqgNF70S9uEaP9agqmGOV1pSXNuudklw9Ss/edit report] | |||
** {{mdone|}} Updates on ''release-localtest'' channel, [https://docs.google.com/document/d/1ZXhpbYcuahkw_wRYGQyaEHS42gYBj52ZaXUZCI6Qd40/edit report] | |||
** {{mdone|}} Updates on ''release-cdntest'' channel, [https://docs.google.com/document/d/1ZXhpbYcuahkw_wRYGQyaEHS42gYBj52ZaXUZCI6Qd40/edit report] | |||
** {{mdone|}} Updates on ''release'' channel, [https://docs.google.com/document/d/1ZXhpbYcuahkw_wRYGQyaEHS42gYBj52ZaXUZCI6Qd40/edit report] | |||
=== | === 65.0.1-build2 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build | * Build 2: [https://archive.mozilla.org/pub/firefox/candidates/65.0.1-candidates/build2/ 65.0.1-build2] | [https://hg.mozilla.org/releases/mozilla-release/rev/1ea7b51ef5bb 1ea7b51ef5bb] | (ID: 20190211233335) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/65.0.1 Firefox 65.0.1] | ||
** | ** 65.0.1-build2 | ||
*** {{mdone}} Smoke | *** {{mdone|}} Smoke Testing on: Web Compatibility. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/15299 TestRail]. | ||
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid= | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=898339968 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on '' | ** {{mdone|}} Updates on ''release-localtest'' channel, [https://docs.google.com/document/d/1AGjKgoXD_m4g6gzKeEzwQuwBnXtUTHcEMXaGmRAbA4o/edit report] | ||
** {{mdone|}} Updates on '' | ** {{mdone|}} Updates on ''release-cdntest'' channel, [https://docs.google.com/document/d/1AGjKgoXD_m4g6gzKeEzwQuwBnXtUTHcEMXaGmRAbA4o/edit report] | ||
** {{mdone|}} Updates on ''release'' channel, [https://docs.google.com/document/d/1AGjKgoXD_m4g6gzKeEzwQuwBnXtUTHcEMXaGmRAbA4o/edit report] | |||
=== | === 65.0.2-build1 === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/ | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/65.0.2-candidates/build1/ 65.0.2-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/2541c8f9cbf1 2541c8f9cbf1] | (ID: 20190225143501) | ||
==== Testing ==== | ==== Testing ==== | ||
* Manual - [https://wiki.mozilla.org/ | * Manual - [https://wiki.mozilla.org/Releases/Firefox_65/Test_Plan/Beta/65.0.2 Firefox 65.0.2] | ||
** | ** 65.0.2-build1 | ||
*** {{ | *** {{mdone|}} Smoke Testing on: Web Compatibility & Geolocation. | ||
**** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/ | **** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/15792 TestRail]. | ||
*** {{ | *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=166794611 Bug verification] with focus on the latest fixes to land in this build. | ||
* Updates testing | * Updates testing | ||
** {{mdone|}} Updates on '' | ** {{mdone|}} Updates on ''release-localtest'' channel, [https://docs.google.com/document/d/1WZdCJh6J3-U3t_dTlDyyVTl1Rr807TiKNENlSeJbAFw/edit report] | ||
** {{mdone|}} Updates on '' | ** {{mdone|}} Updates on ''release-cdntest'' channel, [https://docs.google.com/document/d/1WZdCJh6J3-U3t_dTlDyyVTl1Rr807TiKNENlSeJbAFw/editt report] | ||
** {{mdone|}} Updates on ''release'' channel, [https://docs.google.com/document/d/1WZdCJh6J3-U3t_dTlDyyVTl1Rr807TiKNENlSeJbAFw/edit report] |
Latest revision as of 21:39, 28 February 2019
Summary
This page is to track testing of Firefox 65 from mozilla-central (Nightly) through to mozilla-release. This is intended to be a minimal set of information to help Firefox QA understand and communicate the quality of upcoming releases. Our current walkthrough of the release and QA process might be changing due to recent reorgs.
The QA efforts are mostly split up between two teams:
- Engineering QA
- team reports to Tom Grabowski
- the Project Manager (and main contact) for the team is Rares Bologa
- team is based in Romania (most part) and Las Vegas (5 people)
- the team mainly tests the Desktop app, but covers the Mobile (Android, iOS) side as well
- main responsibilities:
- Feature Testing - new Firefox features, prior to landing and during Nightly
- Bug Triage - older as well as incoming Unconfirmed bugs
- Bug Investigation - various investigation as requested by other people (e.g. finding steps, setting component, finding regression windows)
- Misc - other miscellaneous requests
- Release QA Desktop and Release QA Mobile
- team reports to Tom Grabowski
- the Project Manager (and main contact) for the team is Andrei Vaida
- team is based in Romania
- main responsibilities:
- Build Validation - mostly Beta, Release, and ESR builds (including automated ondemand update, and throttling tests)
- Feature Testing - new features not covered by the Engineering QA team, and those covered by the Engineering QA team after they enter Beta
- Bug Verification - verifying the most important / high risk fixes, with focus on those closer to release
- Bug Investigation - various investigation as requested by other people (e.g. finding steps, setting component, finding regression windows)
- Misc - other miscellaneous requests
Schedule
Milestone | Date | Checks |
---|---|---|
Nightly | - | Conduct daily bug triage, regular build validation, and feature testing |
Beta | - | Feature testing and sign off every beta release, regular testing of DevEdition builds |
Release | - | Review tests and sign off release |
Please consult the Rapid Release Calendar for more information.
Meetings
Meeting | Purpose | When | Vidyo Room | IRC Backchannel | Notes |
---|---|---|---|---|---|
Channel Status | Raise quality concerns with the Release Management team | 10:00 AM Pacific on Tuesday & 08:00 AM Pacific on Thursday | ReleaseCoordination | #planning | Notes archive |
Nightly
While Firefox 65 is on the Nightly channel, QA will mainly focus on:
- Feature Testing - EngQA team (for Firefox features assigned to them) + RelQA team (for other new features - Platform or Firefox)
- Regular Nightly validation - 3 rounds per cycle - EngQA team + RelQA team, alternately
- Triaging newly reported bugs - EngQA team
- Bug Verification - both teams - with main focus on: fixes related to new features, fixes tracked as Sprint work, fixes for major bugs
Reference: https://wiki.mozilla.org/QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Nightly
- End of cycle status
- Total bugs filed affecting 65: tbd
- Bugs fixed / verified on 65: tbd / tbd
- Conditional signoff for merge: <date>, <person> (email sent to release-drivers at
- QE signoff: N/A
Beta
During the Beta cycle, there are several beta releases. Communication with the release coordination, release engineering, and automation teams is key.
While Firefox 65 is on the Beta channel, QA will mainly focus on:
- Signing off each Beta build prior to being pushed to the beta channel - RelQA
- Feature Testing - RelQA team
- Focus is on fixes that land in Beta, and Regression and Smoke testing
- Regular validation of DevEdition builds - about once every two weeks
- Triaging newly reported bugs - EngQA team
- Bug Verification - mostly RelQA - with main focus on: fixes related to new features, fixes for major bugs, fixes assessed as high risk
QA also analyzes the results of update testing which is done manually at this stage. We run them on at least two channels for each beta release cycle.
Pivotal bugs may need verification during each beta cycle.
- Link to release notes. review & update.
- Crash evaluation/escalation.
- Feature evaluation.
QA Activities
QA wanted
The following tracks bugs which have requests for urgent QA assistance.
- Choose a bug to test from the list below
- Read it and make sure you understand what information is needed (this may be steps or URLs to reproduce, a regression range, or something else altogether)
- Use the latest build of this Firefox version and test until you've discovered the information necessary. Ask for help if you need it.
- Report this information to the bug and remove the qe-verify+ flag
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Stability
This section tracks bugs representing the most urgent stability issues. Consult this wiki for more information.
- Ensure new and explosive crashes have actionable bug reports which are in progress
- Top Crashes - ensure the highest ranked crashes have actionable bug reports which are in progress
- Untriaged Bugs - ensure bugs are actionable and in the correct component
- topcrash bugs
Unconfirmed Bug Triage
The following tracks triage and testing of incoming, unconfirmed bug reports.
- Choose a bug to test from the list below
- Use the latest Nightly build to confirm the bug reproduces
- Flag with reporter with needinfo if you cannot reproduce and need more information
- Set the status to NEW and ensure the bug is in the appropriate component if you can reproduce the bug (provide as much detail about your testing as possible)
- If it's a Firefox bug you may want to add the firefox-backlog=? flag to it.
Queries:
- All Unconfirmed Bugs
- Unchanged Since One Week
- These bugs may need extra attention.
- https://bugzilla.mozilla.org/buglist.cgi?list_id=14453327&chfieldto=-1w&query_format=advanced&bug_status=UNCONFIRMED&version=65%20Branch
- Unchanged Since One Month
- These are likely to be confusing or non-reproducible issues. We should make decisions to either close these or tag these non-actionable, and filter them out
- https://bugzilla.mozilla.org/buglist.cgi?list_id=14453331&chfieldto=-1m&query_format=advanced&bug_status=UNCONFIRMED&version=65%20Branch
Verification Needed Bugs
- Review the bugs below in the Pending Triage section and select a bug you want to test
- Make sure you understand the bug before you begin testing it
- Test on a previously known broken build to make sure you can reproduce the bug
- Test on the latest build from the branches which are fixed to confirm the bug no longer reproduces
- Mark the bug VERIFIED FIXED if the bug appears to be fixed and set the appropriate status-firefox flag to verified
- Reopen the bug if it still reproduces
Iteration Development
Firefox development during Nightly works on two-week iterations as of 2014. This section tracks the bugs selected for development in these iterations. Please consult the Iteration Dev Walkthrough and the Firefox/IterativeDevelopment page for more information on this process.
- Ensure all bugs are marked qe-verify:+ or -
- Ensure all qe-verify:+ bugs have a QA Contact (The Feature Owner page may help here)
- Ensure all selected bugs are VERIFIED before the end of the iteration
- Anything incomplete at the end of the iteration will be automatically carried over to the next iteration
Queries:
- Dashboard showing current active iteration: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/
- Iteration 65.1, all bugs: https://bugzilla.mozilla.org/buglist.cgi?f1=OP&j1=OR&f2=cf_fx_iteration&o2=substring&v2=65.1&f3=status_whiteboard&o3=substring&v3=s%3D65.1&f4=CP
- Iteration 65.2, all bugs: https://bugzilla.mozilla.org/buglist.cgi?f1=OP&j1=OR&f2=cf_fx_iteration&o2=substring&v2=65.2&f3=status_whiteboard&o3=substring&v3=s%3D65.2&f4=CP
- Iteration 65.3, all bugs: https://bugzilla.mozilla.org/buglist.cgi?f1=OP&j1=OR&f2=cf_fx_iteration&o2=substring&v2=65.3&f3=status_whiteboard&o3=substring&v3=s%3D65.3&f4=CP
Feature status
Milestones
Regular Nightly Validation
SIGN-OFF DATE | SIGN-OFF STATUS | REASON(S) | MAJOR ISSUES | MINOR ISSUES | TEST REPORT |
---|---|---|---|---|---|
2018-11-01 | GREEN |
|
--- | Bugzilla link | TestRail link |
2018-11-15 | GREEN |
|
--- | Bugzilla link | TestRail link |
Nightly Migration
Pre-sign-off Checks:
- [NOT STARTED] Features - all scoped features are signed off as ready for Beta by owners. (If you haven't heard anything to the contrary, that is probably true, but make sure we don't know of blocking issues.)
Post-sign-off Checks (as soon as updates are enabled again):
- None yet for migration of Nightly to Beta
Developer Edition
Note: Developer Edition builds are usually signed off once every two weeks.
BUILD | SIGN OFF | ENCOUNTERED BUGS | |||
---|---|---|---|---|---|
Status | Reason(s) | Test Report | New | Known (old) | |
65.0b1-build1 | GREEN |
|
TestRail link |
|
|
65.0b2-build1 | GREEN |
|
TestRail link |
|
|
65.0b6-build1 | GREEN |
|
TestRail link |
|
|
65.0b9-build1 | GREEN |
|
TestRail link |
|
|
Beta
Note: Beta builds are usually signed off twice a week.
Beta 3
Build Information
- Build 1: 65.0b3-build2 | 0cf938ef3c98 | (ID: 20181210164452)
Testing
- Manual - Firefox 65 Beta 3
- 65.0b3-build2
- [DONE] Smoke and Exploratory Testing with focus on: Plugin compatibility & support
- Detailed test results: TestRail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Plugin compatibility & support
- 65.0b3-build2
- Updates testing
Beta 4
Build Information
- Build 1: 65.0b4-build1 | b1216967cdd0 | (ID: 20181211223337)
Testing
- Manual - Firefox 65 Beta 4
- 65.0b4-build1
- [DONE] Smoke and Exploratory Testing with focus on: Crash Reporting, Safe Browsing, WebExtensions compatibility & support.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Crash Reporting, Safe Browsing, WebExtensions compatibility & support.
- 65.0b4-build1
- Updates testing
Beta 5
Build Information
- Build 1: 65.0b5-build1 | dfe78db5e4ae | (ID: 20181217180946)
Testing
- Manual - Firefox 65 Beta 5
- 65.0b5-build1
- [DONE] Smoke and Exploratory Testing with focus on: Toolbars & window controls and Content scroll, zoom & drag support.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Toolbars & window controls and Content scroll, zoom & drag support.
- 65.0b5-build1
- Updates testing
Beta 6
Build Information
- Build 1: 65.0b6-build1 | 4f9b811b0f2a | (ID: 20181220174318)
Testing
- Manual - Firefox 65 Beta 6
- 65.0b6-build1
- [DONE] Smoke and Exploratory Testing with focus on: Graphics compatibility & support (skia), Media playback & support (audio) and Download manager.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Graphics compatibility & support (skia), Media playback & support (audio) and Download manager.
- 65.0b6-build1
- Updates testing
Beta 7
Build Information
- Build 1: 65.0b7-build1 | 22da6211feb6 | (ID: 20181227144402)
Testing
- Manual - Firefox 65 Beta 7
- 65.0b7-build1
- [DONE] Smoke and Exploratory Testing with focus on: Profile support & compatibility, Private Browsing & Geolocation.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Profile support & compatibility, Private Browsing & Geolocation.
- 65.0b7-build1
- Updates testing
Beta 8
Build Information
- Build 1: 65.0b8-build1 | 35ba72605a9a | (ID: 20190103150357)
Testing
- Manual - Firefox 65 Beta 8
- 65.0b8-build1
- [DONE] Smoke and Exploratory Testing with focus on: Search, In-browser PDF support and Image compatibility & support.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Search, In-browser PDF support and Image compatibility & support.
- 65.0b8-build1
- Updates testing
Beta 9
Build Information
- Build 1: 65.0b9-build1 | df848230cb30 | (ID: 20190107180200)
Testing
- Manual - Firefox 65 Beta 9
- 65.0b9-build1
- [DONE] Smoke and Exploratory Testing with focus on: Printing, Default & custom theme support, Bookmarks and Smart Card.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Printing, Default & custom theme support, Bookmarks and Smart Card.
- 65.0b9-build1
- Updates testing
Beta 10
Build Information
- Build 1: 65.0b10-build1 | f0bb91ca0015 | (ID: 20190107180200)
Testing
- Manual - Firefox 65 Beta 10
- 65.0b10-build1
- [DONE] Smoke and Exploratory Testing with focus on: Geolocation, Privacy, File system interaction and Data Migration.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Geolocation, Privacy, File system interaction and Data Migration.
- 65.0b10-build1
- Updates testing
Beta 11
Build Information
- Build 1: 65.0b11-build1 | 8574d2808aaa | (ID: 20190114172331)
Testing
- Manual - Firefox 65 Beta 11
- 65.0b11-build1
- [DONE] Smoke and Exploratory Testing with focus on: History and Content scroll, zoom & drag support.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: History and Content scroll, zoom & drag support.
- 65.0b11-build1
- Updates testing
Beta 12
Build Information
- Build 1: 65.0b12-build1 | 666a44dfa6eb | (ID: 20190117232427)
Testing
- Manual - Firefox 65 Beta 12
- 65.0b12-build1
- [DONE] Smoke and Exploratory Testing with focus on: Plugin compatibility & support, Session Restore and Password Manager & Form Autofill.
- Detailed test results: Testrail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke and Exploratory Testing with focus on: Plugin compatibility & support, Session Restore and Password Manager & Form Autofill.
- 65.0b12-build1
- Updates testing
Release Candidates
65.0-build1
Build Information
- Build 1: 65.0-build1 | be4781c53713 | (ID: 20190121133710)
Testing
- Manual - Firefox 65 RC1
- 65.0-build1
- [DONE] Smoke Testing on: Web Compatibility.
- Detailed test results: TestRail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke Testing on: Web Compatibility.
- 65.0-build1
- Updates testing
65.0-build2
Build Information
- Build 2: 65.0-build2 | be4781c53713 | (ID: 20190121133710)
Testing
- Manual - Firefox 65 RC2
- 65.0-build2
- [DONE] Smoke Testing on: Web Compatibility.
- Detailed test results: TestRail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke Testing on: Web Compatibility.
- 65.0-build2
- Updates testing
65.0.1-build2
Build Information
- Build 2: 65.0.1-build2 | 1ea7b51ef5bb | (ID: 20190211233335)
Testing
- Manual - Firefox 65.0.1
- 65.0.1-build2
- [DONE] Smoke Testing on: Web Compatibility.
- Detailed test results: TestRail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke Testing on: Web Compatibility.
- 65.0.1-build2
- Updates testing
65.0.2-build1
Build Information
- Build 1: 65.0.2-build1 | 2541c8f9cbf1 | (ID: 20190225143501)
Testing
- Manual - Firefox 65.0.2
- 65.0.2-build1
- [DONE] Smoke Testing on: Web Compatibility & Geolocation.
- Detailed test results: TestRail.
- [DONE] Bug verification with focus on the latest fixes to land in this build.
- [DONE] Smoke Testing on: Web Compatibility & Geolocation.
- 65.0.2-build1
- Updates testing