QA/Send Tab

Revision History

This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.

Date	Version	Author	Description
7/31/2017	1.0	Abe Masresha	Created first draft
8/15/2017	1.1	Abe Masresha	updated links and updated some sections

Overview

Purpose

The purpose of this work is to make the send tab discoverable to non-Sync/FxA users.

Scope

• Testing will focus on:

Stability of the browser with this work.

If send tab is discoverable to non-Sync/FxA users (Desktop only - at this time)

If user able to send links to other devices when signed-in to sync

• How testing will be performed:

Manual

• Automation is covered by the developer

[Link to automation test script will be provided here]

Ownership

Mozilla Development

Edouard Oger

QA Eng. Softvision Las Vegas
Stefan Georgiev - Team Lead
Abe Masresha - Test Lead
Justin Williams - Tester
Grover Wimberly IV - Tester
Kanchan Kumari - Tester

Testing summary

Scope of Testing

In Scope

This feature will be tested manually on Windows, Mac, and Linux operating systems and on iOS and Android devices.

Out of Scope

Automation will be handled by the developer

Risk Analysis

• The send tab feature may not send tab
• The send tab may not be discoverable to non-Sync/FxA users (Desktop only - at this time)

Requirements for testing

Test Environment Combinations

Windows 10 with iOS device
Windows 7 32bits with Linux OS
Linux with Android device
MacOSX 10.12 with Android device
Android with another Android device

Channel dependent settings (configs) and environment setups

Nightly

Firefox Nightly 56.0a1, no special configuration is required

Beta

Beta 56.0bx, no special configuration is required

Test Strategy

Risk Assessment and Coverage

ID	Description / Threat Description	Covered by Test Objective	Magnitude	Probability	Priority	Impact Score
RAC-1	Update may not be applied to the browser	TO-1	3-High	2-Possible	3-High	18
RAC-2	Update may not be applied successfully if the browser has add-on	TO-2	3-High	2-Possible	3-High	18
RAC-3	SHA384 support may not be applied	TO-3	3-High	1-Unlikely	3-High	9
RAC-4	SHA384 key size of 4096 bits may not be applied	TO-4	2-Moderate	1-Unlikely	3-High	6
RAC-5	May behave differently if update download is interrupted by no internet connection	TO-5	2-Moderate	1-Unlikely	2-Medium	4
RAC-6	May behave differently in different operating systems	TO-6	2-Moderate	1-Unlikely	2-Medium	4

Values:

• Magnitude: 1- Low , 2-Moderate, 3-High

• Probability: 1-Unlikely, 2-Possible, 3-Almost Certain

• Priority: 1 - Low, 2-Medium, 3-High

Impact Score Breakdown:

• An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
• An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
• An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.

Test Objectives

Ref	Function	Test Objective	Evaluation Criteria	Test Type	RAC	Owners
1	Update the browser	Browser should update to its latest build	update the browser	Manual	RAC-1,RAC-2,RAC-5,RAC-6	Eng Team
2	Check if SHA384 support is applied	SHA384 support should be applied to update	Check if SHA384 support is applied. steps are here	Manual	RAC-3	Eng Team
3	Check if SHA384 support has 4096 key size	Key size of 4096 bits should be used for the SHA384 support	Check if key size is 4096 bits. steps are here	Manual	RAC-4	Eng Team

Builds

This section should contain links for builds with the feature -

• Links for Nightly build: Nightly build
• Links for Beta build: use beta build later than 56b3

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	5/30/2017
Study documentation/specs received from developers	7/20/2017	8/14/2017
QA - Test plan creation	7/20/2017	8/14/2017
QA - Test cases/Env preparation	7/20/2017	8/1/2017
QA - Nightly Testing	8/1/2017	8/1/2017
QA - Beta Testing
Release Date

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	TestRail / Google docs
Test case execution	TestRail
Bugs management	Bugzilla

Builds

This section should contain links for builds with the feature -

• Links for Nightly build: use the latest official Nightly build
• Links for Beta build: use the latest official beta build

Test Execution Schedule

The following table identifies the anticipated testing period available for test execution.

Project phase	Start Date	End Date
Start project	07/25/2017
Study documentation/specs received from developers	07/25/2017	07/27/2017
QA - Test plan creation	7/31/2017	7/25/2017
QA - Test cases/Env preparation	7/25/2017	7/31/2017
QA - Nightly Testing	7/27/2017	7/31/2017
QA - Beta Testing
Release Date

Testing Tools

Detail the tools to be used for testing, for example see the following table:

Process	Tool
Test plan creation	Mozilla wiki
Test case creation	TestRail
Test case execution	TestRail
Bugs management	Bugzilla

Status

Overview

Landed on Nightly on: 7/19/2017
Merged to Beta on: [date]

References

• Main bug:1359894
  
• bug:1370574
  

Testcases

Test Areas

Test Areas	Covered	Details
Multi-Process Enabled	Yes	The feature should work proprely in e10s browser
Install/Upgrade
Feature upgrades /Browser update	n/a
Add-ons
Testing with existing/popular addons	yes	Feature functionality will be tested with popular addons
Web Compatibility
Survey of popular sites for compatibility	n/a
Interoperability
Interaction of this feature with other browser features	n/a

Test suite

Full Test suite -Link to test rail
Smoke Test suite -Full test suite is used for pre-beta test cycle
Regression Test suite - Full test suite is used for pre-beta test cycle

Bug Work

Bug No	Summary	Status	Firefox Version
1385440	Only one device receives the link when "Send to All Devices" is clicked	NEW	56
1385442	Android device is not listed in sync'd device list on mac	NEW	56
1385451	in Android "send to other devices" is not displayed	NEW	56
1385454	on Android, "verify your account" option is not available when the user changes password from other device	NEW	56
1385930	Frozen spinner is shown when there are no devices to send tab to (iOS)	NEW	56
1385442	In Android, Send Tab is not discoverable to non Sync/FxA users	RESOLVED INVALID	56
1385447	All previously logged in devices are displayed in sync'd device list	RESOLVED WORKSFORME	56

Sign off

Criteria

Checklist

• All test cases should be executed
• Has sufficient automated test coverage (as measured by code coverage tools)
• All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

Nightly testing
List of OSes that will be covered by testing- listed in environments section

• • Link for the tests run
  • Daily Smoke, not avialable (n/a)
  • Regression Test suite: not avialable (n/a)
    

Merge to Beta Sign-off
List of OSes that will be covered by testing

• Link for the tests run
  • Full Test suite

Checklist

Exit Criteria	Status	Notes/Details
Testing Prerequisites (specs, use cases)	[DONE]
Testing Infrastructure setup	[DONE]
Test Plan Creation	[DONE]
Test Cases Creation	[DONE]
Automation Coverage
Performance Testing
All Defects Logged	[DONE]	All found bugs are logged
Critical/Blockers Fixed and Verified	IN PROGRESS	Devs are fixing bugs
QA mid-Nightly Signoff	[MISSED]	Missed (test build and feature document were not avialable to QE)
QA pre-Beta Signoff	[DONE]	YELLOW (conditional sign-off) sent on 7/31/2017
QA Beta - Full Testing	IN PROGRESS
QA pre-Release Signoff	IN PROGRESS	Email to be sent