QA/Firefox3/TestPlan: Difference between revisions
Line 13: | Line 13: | ||
;Must fix by Beta 3 | ;Must fix by Beta 3 | ||
* {{Bug|390568}} - vista fails to submit breakpad reports | * {{Bug|390568}} - vista fails to submit breakpad reports | ||
* {{Bug|378528}} – crash reporter should attempt to resubmit pending reports. Needs to be rolled up into a queue for later if breakpad is down. | * {{Bug|378528}} – crash reporter should attempt to resubmit pending reports. Needs to be rolled up into a queue for later if breakpad is down. |
Revision as of 03:11, 15 January 2008
Top Critical issues for QA
This is a working list that QA is using to request be fixed or completed for the Upcoming Fx3 Milestones.
If you have untracked issues in these areas, please enter them here:
- Design still incomplete?
- Major bugs blocking your feature that should be addressed?
- Major bugs blocking overall regression of the trunk that should be fixed before shipping?
QA Beta 1 Relnotes Tracking
QA Beta 2 Relnotes Tracking
- Must fix by Beta 3
- bug 390568 - vista fails to submit breakpad reports
- bug 378528 – crash reporter should attempt to resubmit pending reports. Needs to be rolled up into a queue for later if breakpad is down.
- bug 405198 Cannot drag and drop folders / items in Places Organizer - this cause the problem that its not possible to move bookmarks from one folder to another folder oder to merge folder etc .... - Tomcat
- bug 324857 – MathML all screwed up in Cairo builds. Blocking MathML usage
- bug 411369 - Crash while trying to print [@ @0x0 @0x1000 _cairo_meta_surface_create] [@ _moz_cairo_surface_finish ] - This is a major regression that is blocking any printing on Vista and XP.
- Must Fix by Release Candidate
- bug 402335 - Meta bug for memory leaks. Tracked by tomcat. Ongoing testing and bugfixes.
- bug 391598 and bug 407780 SW update anti-fatigue. Need to improve modal dialogs for updates. Need to add to Fx3 testplan.
- bug 407459 - Hook up new allocator to our build. QA will probably not test this. noting it down in case we need to look at it.
- Critical Printing Bugs - These are printing crash bugs from printing. Bugs listed in this query are marked in Trunk, and have been confirmed by QA members.
- In general, QA doesn't yet have an idea what "improve printing" means for this release; is it a metric-based, top-site notion, or something that includes some subset of prior-known, broken-printing pages, or...?
- - stephend
- In general, QA doesn't yet have an idea what "improve printing" means for this release; is it a metric-based, top-site notion, or something that includes some subset of prior-known, broken-printing pages, or...?
- bug 404024 - Add AMO integration pane - Testplan completed, but no date set yet. Tracking spec is at: http://wiki.mozilla.org/Update:Remora_Meeting_Notepad.
- Addons Manager UI has a spec, but no dev resource or timeline to work on.
- UPDATE - addons manager UI in the works with Remora project.
- Bug 401301 - firefox.exe process still around after all windows closed Still investigating firefox hang process. Hard to repro, but working with accessibility.
- bug 404619 - Firefox memory usage zooms when Norton Antivirus is running. This is creating a high memory usage reported by many in blogs. Needs more investigation. Same as bug 404645
- UPDATE - narrowed to one applet. P4 priority Bug 401064 – SnowFX applet requires a click to display. Affects some java plugin sites, but not all. Needs to be fixed, but doesnt block java testing.
- Mac Focus issues still a problem; bug 354768.
- Accessibility: Download Manager, being redesigned; would like the UI solidified earlier rather than later.
- bug 393248 The info button currently brings up a popup that is classed as a menu, which confuses AT. Also the search box has no accessible label. -TimK
- In latest Trunk, the Info button is still there, and the search box still has no label. The changes have aparently not landed yet. (MarcoZ)
- I'd like to know in what form of autocomplete the final Firefox 3 location bar will/should take; again, I understand there are ongoing experiments in order to get the best fit--across the board in the application--I just want everyone to be aware that churn really takes a toll on solidifying testcase-writing/execution. (Boy, I really sound whiny, no? I honestly just want us all to be on the same page, QA feels really, really behind in most areas, testing-wise, but perhaps that's just my perception...)
- - stephend
- Accessibility: bug 393398 Add Bookmark Dialog Inaccessible: Needs to be classified as a dialog or a window in order to work properly with most AT. Needs more work and can be post bedta 2. One screen reader has severe problems with the panel being classified as a menu, the others we test don't. This is also true for the new AutoComplete panel, which is currently being classified as a menu with alist within. (MarcoZ)
- Accessibility: Bookmark/Places Panel not Accessible, need menu items to duplicate toolbar entries.
- DISCUSSION - beltzner mentions there is work being done already. TimK, please comment on what specifics are blocking.
- There are equivalence in the Bookmarks menu to the toolbar buttons--for some reason they weren't accessible from my profile. However, there are sec508 concerns with the Places Organizer contained in bug 400703 and bug 402104 which will require more extensive testing and should be resolved by beta 2, if at all possible. -TimK
- Since neither of these bugs have patches yet, this should slip past beta 2, but is a sec508 issue that needs to be resolved before shipping. (MarcoZ)
- DISCUSSION - beltzner mentions there is work being done already. TimK, please comment on what specifics are blocking.
- RESOLVED FIXED bugs, needing more information or testcases
- Bug 375750 - PDFs generated by the trunk are huge. The patch seems to make the size smaller, but the size is still not in line with what the branch size is
- Bug 402000 - Cannot be verified according to dev
Overview
The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.
Major QA Activities:
- Litmus Cleanup
- Litmus Triage Team
- New feature testing - see below
- L10n testing - TBD
- Milestone testing - TBD
Requirements and Ownership
Archives:
Test Areas
Feature Focused Areas
Key: L = Large (3-6 weeks) M = Medium (2-4 weeks) S = Small (1-2 weeks)
Feature Area and Spec | Test Plan Location | QA Contact | Size | Description |
Addons | Addons Test Plan | tchung (new Addons Manager UI, Extensions), stephend (Unified Plugin Manager) | M | Add-on management system, plugin management system, localized user-facing addon text, configuration, updates, Installation, Discovery, FUEL 0.2 |
Content Handling | Microformat Test Plan Web Application Test Plan Download Manager Test Plan |
ctalbert, stephend (Download Manager) | L | microformats, Content Type Processing, download manager, MIME type support, User Interface, Media Plugins (needs definition), firebug (at risk) |
Partner Distribution | TBD | tracy, ctalbert | M | High-Touch/Low-Touch distros, regression, updates, test automation |
OS Platform Integration | Vista Test Plan | Tomcat, marcia | M | Vista Support (At Risk), MSI bundling, Airbag / Breakpad, parental controls (At Risk), Notification, Vista Installer |
Gecko Platform Integration | Gecko Platform Test Plan | martijn, marcia | M | Cairo graphics, nxTextFrame migration to Thebes, Reflow Branch, Caret painting rewrite, Cross-domain XMLHttpRequest (Cocoa widgets covered under Mac Test Plan) |
Mac OS Platform Integration (includes Gecko) | Mac OS Integration Test Plan |
Tomcat, marcia | M | Leopard support, Cocoa implementation, widgets, form controls, Growl Integration on Mac, Mac Installer |
Places | Places Test Plan | Tracy, ABillings | L | Bookmarks, History Indexing, Data Management, User Interface, Extensibility, Performance |
Location Bar | Location Bar Test Plan | Stephend | S | new Location Bar design. Auto-complete |
Search / Tabbed Browsing / Visual Refresh | Search, Tabs, Visual Refresh Test Plan Tab Switching Test Mac Theme Test Plan |
Tomcat (search), Marcia (tabbed browsing) | S | User Interfacing, Animations, Data management |
Security, Privacy / Password, Identity | Security / Privacy Test Plan Password Manager Testplan |
Tomcat, juanb | L | password management, Phishing protection, Private web browsing, Blacklisting (At Risk), password Manager, Password Security |
| ||||
L10N Testing | L10N Test Plan | ctalbert, tchung | S | test automation, new locales, string lengths, bookmarks, rss, search plugins |
UI Accessibility | UI Accessibility Test Plan | MarcoZ | S | UI accessibility convention testing |
Major Update Tests (2.0.x -> FF3) | Major Update Test Plan | tchung, juanb, tracy | M | update verification from 2.0.x builds to 3.0b1+. includes upgrades/downgrades/sidegrade cases |
Memory Leaks | Memory Leaks Test Plan | tomcat | M | Checking for memory leaks and asserts on debug builds |
Regression Areas
- Top Web accessibility Tests
- Plugins (eg. Flash, java, AJAX)
- Extensions (eg. Firebug, NoScript, Fotofox)
- Themes (eg. Vista, Joga, Simpsons)
- Top Site Tests (eg. Alexa top 50)
- Financial Sites (eg. banks, online trading)
- Accessibility Tests
- User Performance Testing
- Major / Minor Update Testing
- L10N Testing (may coincide with Feature area)
- Distribution Testing (may coincide with Feature area)
Ownership Needed Litmus Test Case Areas
These are Litmus test case areas that are currently unowned.
- RSS->marcia
- Printing -> Tomcat, tchung
- Location Bar -> Stephend
- Find -> TBD
- Customize Toolbar -> TBD
- Mac Leopard -> marcia
Tiger Team Testing
Daily smoketests are ran against nightly trunk builds. Please see marcia for more information.
Link to Schedule and Assignments
Extensions
For Firefox 2 we had Bug 353206 as extension Tracking bug.
The following extensions were never fixed from their extension compatibly authors to be able to install in Firefox 2.
- Slim Extension List [1]
- Bookmarks Synchronizer 3 [2]
- AutoMarks [3]
- Extended Link Properties [4]
- Snapper [5]
- Cacheout[6]
- Resize Search Box [7]
- superT 0.7.9.1 https://addons.mozilla.org/en-US/firefox/addon/2224
- Reveal extension [8]
Top Crash reports
The main report is the easiest to view.
Note that of those in that list, only a few seem to be active lately. I filed bug 402661 for the crash in RtlpCoalesceFreeBlocks / XPCWrappedNative::CallMethod(XPCCallContext&, XPCWrappedNative::CallMode). I still need to file a bug for the crash in CallGetService(nsID const&, nsID const&, void**). Neither of these should be blocking beta given their (relative) infrequency.
Banking and Financial Sites
For Firefox 3, we’re asking community members and MoCo QA to state which banks and financial institutions they're willing to test. Please add your name to the list.
Review Feedback
In this section, we will roll up all common feedback pulled from external sources that appears common and should be tracked by QA. Feedback sources can include:
- Hendrix Feedback system (mozilla.feedback newsgroups)
- Bugzilla
- Breakpad Crash Reporter
- Mozillazine Forums
Feedback summary and review notes are posted here.
Flash Beta plugin Testing
- http://weblogs.macromedia.com/labs/archives/2007/11/flash_player_9_6.cfm
- Team members actively testing betas
- marcia
- abillings
- stephend
Additional References
New FF3 Feature list - An incomplete list of what Firefox 3 developers are working on.