QA/Firefox3/TestPlan: Difference between revisions
Line 28: | Line 28: | ||
** {{Bug|412008}} – Can't access report/list for 1.9 branch topcrashes | ** {{Bug|412008}} – Can't access report/list for 1.9 branch topcrashes | ||
** {{Bug|411438}} – Crash reporter app should offer restart Firefox button when it throws an error dialog | ** {{Bug|411438}} – Crash reporter app should offer restart Firefox button when it throws an error dialog | ||
* {{bug|402335}} - Meta bug for memory leaks. Tracked by tomcat. Ongoing testing and bugfixes. | * {{bug|402335}} - Meta bug for memory leaks. Tracked by tomcat. Ongoing testing and bugfixes. '''-Tomcat'''- Triage of this bugs will start this week | ||
* [http://tinyurl.com/2mvcdj Critical Printing Bugs] - These are printing crash bugs from printing. Bugs listed in this query are marked in Trunk, and have been confirmed by QA members. | * [http://tinyurl.com/2mvcdj Critical Printing Bugs] - These are printing crash bugs from printing. Bugs listed in this query are marked in Trunk, and have been confirmed by QA members. | ||
* {{Bug|391598}} and {{Bug|407780}} SW update anti-fatigue. Need to improve modal dialogs for updates. Need to add to Fx3 testplan. | * {{Bug|391598}} and {{Bug|407780}} SW update anti-fatigue. Need to improve modal dialogs for updates. Need to add to Fx3 testplan. |
Revision as of 18:46, 15 January 2008
Top Critical issues for QA
This is a working list that QA is using to request be fixed or completed for the Upcoming Fx3 Milestones.
If you have untracked issues in these areas, please enter them here:
- Design still incomplete?
- Major bugs blocking your feature that should be addressed?
- Major bugs blocking overall regression of the trunk that should be fixed before shipping?
QA Beta 1 Relnotes Tracking
QA Beta 2 Relnotes Tracking
- Must fix by Beta 3
- Top crashes
- bug 411249 – Top crash [@ jsds_ScriptHookProc[@jsds_ScriptHookProc(JSDContext*, JSDScript*, int, void*)] on shutdown with Firebug 1.1.0b10 installed
- bug 411369 - Crash while trying to print [@ @0x0 @0x1000 _cairo_meta_surface_create] [@ _moz_cairo_surface_finish ] - This is a major regression that is blocking any printing on Vista and XP.
- Breakpad issues
- bug 412375 - Older crashreports no longer visible ?. Unable to retrieve reports prior to Jan 10th. Webdev investigating. Update - data has been lost due to running out of space on db. unrecoverable
- bug 390568 - vista fails to submit breakpad reports. This has been broken since alpha8, and we are losing more crash data as long as this remains unfixed.
- bug 378528 – crash reporter should attempt to resubmit pending reports. Needs to be rolled up into a queue for later if breakpad is down. This causes backlog of missing crash data that takes days later to resurface. Takes away time from researching critical issues.
- bug 405198 Cannot drag and drop folders / items in Places Organizer - this cause the problem that its not possible to move bookmarks from one folder to another folder oder to merge folder etc .... - Tomcat Why is this assigned to nobody?
- bug 324857 – MathML all screwed up in Cairo builds. Blocking MathML usage. Lots of action in the bug, but no resolution yet?
- Must Fix by Release Candidate
- Breakpad issues
- bug 394247 – check that nsITransfer::Init succeeded (crash [@nsDownloadProxy::OnStateChange])
- bug 412008 – Can't access report/list for 1.9 branch topcrashes
- bug 411438 – Crash reporter app should offer restart Firefox button when it throws an error dialog
- bug 402335 - Meta bug for memory leaks. Tracked by tomcat. Ongoing testing and bugfixes. -Tomcat- Triage of this bugs will start this week
- Critical Printing Bugs - These are printing crash bugs from printing. Bugs listed in this query are marked in Trunk, and have been confirmed by QA members.
- bug 391598 and bug 407780 SW update anti-fatigue. Need to improve modal dialogs for updates. Need to add to Fx3 testplan.
- bug 407459 - Hook up new allocator to our build. QA will probably not test this. noting it down in case we need to look at it.
- bug 404024 - Add AMO integration pane. Testplan completed, but no date set yet. Tracking spec is at: http://wiki.mozilla.org/Update:Remora_Meeting_Notepad. For Addons Manager UI, last update was 4 checkins were pending UX review for b3.
- bug 401301 - firefox.exe process still around after all windows closed] Still investigating firefox hang process. Hard to repro, reported by accessibility team and important for the community to resolve.
- bug 404645 – memory usage spikes dramatically while doing nothing with fresh profile. High memory utilization reported by many blogs.
- bug 403232 - Focused textfield loses focus when switching to another tab and back. UPDATE - fix should land by b3
- Accessibility issues
- bug 393398 Add Bookmark Dialog Inaccessible: Needs to be classified as a dialog or a window in order to work properly with most AT. Needs more work and can be post bedta 2. One screen reader has severe problems with the panel being classified as a menu, the others we test don't. This is also true for the new AutoComplete panel, which is currently being classified as a menu with alist within. (MarcoZ)
- Bookmark/Places Panel not Accessible, need menu items to duplicate toolbar entries.
- There are equivalence in the Bookmarks menu to the toolbar buttons--for some reason they weren't accessible from my profile. However, there are sec508 concerns with the Places Organizer contained in bug 400703 and bug 402104 which will require more extensive testing and should be resolved by beta 2, if at all possible. -TimK
- Since neither of these bugs have patches yet, this should slip past beta 2, but is a sec508 issue that needs to be resolved before shipping. (MarcoZ)
- RESOLVED FIXED bugs, needing more information or testcases
- Bug 375750 - PDFs generated by the trunk are huge. The patch seems to make the size smaller, but the size is still not in line with what the branch size is
- Bug 402000 - Cannot be verified according to dev
Overview
The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.
Major QA Activities:
- Litmus Cleanup
- Litmus Triage Team
- New feature testing - see below
- L10n testing - TBD
- Milestone testing - TBD
Requirements and Ownership
Archives:
Test Areas
Feature Focused Areas
Key: L = Large (3-6 weeks) M = Medium (2-4 weeks) S = Small (1-2 weeks)
Feature Area and Spec | Test Plan Location | QA Contact | Size | Description |
Addons | Addons Test Plan | tchung (new Addons Manager UI, Extensions), stephend (Unified Plugin Manager) | M | Add-on management system, plugin management system, localized user-facing addon text, configuration, updates, Installation, Discovery, FUEL 0.2 |
Content Handling | Microformat Test Plan Web Application Test Plan Download Manager Test Plan |
ctalbert, stephend (Download Manager) | L | microformats, Content Type Processing, download manager, MIME type support, User Interface, Media Plugins (needs definition), firebug (at risk) |
Partner Distribution | TBD | tracy, ctalbert | M | High-Touch/Low-Touch distros, regression, updates, test automation |
OS Platform Integration | Vista Test Plan | Tomcat, marcia | M | Vista Support (At Risk), MSI bundling, Airbag / Breakpad, parental controls (At Risk), Notification, Vista Installer |
Gecko Platform Integration | Gecko Platform Test Plan | martijn, marcia | M | Cairo graphics, nxTextFrame migration to Thebes, Reflow Branch, Caret painting rewrite, Cross-domain XMLHttpRequest (Cocoa widgets covered under Mac Test Plan) |
Mac OS Platform Integration (includes Gecko) | Mac OS Integration Test Plan |
Tomcat, marcia | M | Leopard support, Cocoa implementation, widgets, form controls, Growl Integration on Mac, Mac Installer |
Places | Places Test Plan | Tracy, ABillings | L | Bookmarks, History Indexing, Data Management, User Interface, Extensibility, Performance |
Location Bar | Location Bar Test Plan | Stephend | S | new Location Bar design. Auto-complete |
Search / Tabbed Browsing / Visual Refresh | Search, Tabs, Visual Refresh Test Plan Tab Switching Test Mac Theme Test Plan |
Tomcat (search), Marcia (tabbed browsing) | S | User Interfacing, Animations, Data management |
Security, Privacy / Password, Identity | Security / Privacy Test Plan Password Manager Testplan |
Tomcat, juanb | L | password management, Phishing protection, Private web browsing, Blacklisting (At Risk), password Manager, Password Security |
| ||||
L10N Testing | L10N Test Plan | ctalbert, tchung | S | test automation, new locales, string lengths, bookmarks, rss, search plugins |
UI Accessibility | UI Accessibility Test Plan | MarcoZ | S | UI accessibility convention testing |
Major Update Tests (2.0.x -> FF3) | Major Update Test Plan | tchung, juanb, tracy | M | update verification from 2.0.x builds to 3.0b1+. includes upgrades/downgrades/sidegrade cases |
Memory Leaks | Memory Leaks Test Plan | tomcat | M | Checking for memory leaks and asserts on debug builds |
Regression Areas
- Top Web accessibility Tests
- Plugins (eg. Flash, java, AJAX)
- Extensions (eg. Firebug, NoScript, Fotofox)
- Themes (eg. Vista, Joga, Simpsons)
- Top Site Tests (eg. Alexa top 50)
- Financial Sites (eg. banks, online trading)
- Accessibility Tests
- User Performance Testing
- Major / Minor Update Testing
- L10N Testing (may coincide with Feature area)
- Distribution Testing (may coincide with Feature area)
Ownership Needed Litmus Test Case Areas
These are Litmus test case areas that are currently unowned.
- RSS->marcia
- Printing -> Tomcat, tchung
- Location Bar -> Stephend
- Find -> TBD
- Customize Toolbar -> TBD
- Mac Leopard -> marcia
Tiger Team Testing
Daily smoketests are ran against nightly trunk builds. Please see marcia for more information.
Link to Schedule and Assignments
Extensions
For Firefox 2 we had Bug 353206 as extension Tracking bug.
The following extensions were never fixed from their extension compatibly authors to be able to install in Firefox 2.
- Slim Extension List [1]
- Bookmarks Synchronizer 3 [2]
- AutoMarks [3]
- Extended Link Properties [4]
- Snapper [5]
- Cacheout[6]
- Resize Search Box [7]
- superT 0.7.9.1 https://addons.mozilla.org/en-US/firefox/addon/2224
- Reveal extension [8]
Top Crash reports
The main report is the easiest to view.
Note that of those in that list, only a few seem to be active lately. I filed bug 402661 for the crash in RtlpCoalesceFreeBlocks / XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode). I still need to file a bug for the crash in CallGetService(nsID const&, nsID const&, void**). Neither of these should be blocking beta given their (relative) infrequency.
Banking and Financial Sites
For Firefox 3, we’re asking community members and MoCo QA to state which banks and financial institutions they're willing to test. Please add your name to the list.
Review Feedback
In this section, we will roll up all common feedback pulled from external sources that appears common and should be tracked by QA. Feedback sources can include:
- Hendrix Feedback system (mozilla.feedback newsgroups)
- Bugzilla
- Breakpad Crash Reporter
- Mozillazine Forums
Feedback summary and review notes are posted here.
Flash Beta plugin Testing
- http://weblogs.macromedia.com/labs/archives/2007/11/flash_player_9_6.cfm
- Team members actively testing betas
- marcia
- abillings
- stephend
Additional References
New FF3 Feature list - An incomplete list of what Firefox 3 developers are working on.