Identity/BrowserID/KPI Dashboard: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Moving data glossary out of Privacy Review)
(→‎Codebase: Codebases under mozilla now)
 
(14 intermediate revisions by 5 users not shown)
Line 16: Line 16:
* '''number_emails:''' When an interaction proceeds to the point where the user authenticates to the personaid service we include the number of emails that this user has verified with BrowserID
* '''number_emails:''' When an interaction proceeds to the point where the user authenticates to the personaid service we include the number of emails that this user has verified with BrowserID
* '''new_account:''' If during an interaction a new browserid account is created, this property is true (as opposed to an interaction which represents sign in using an existing account)
* '''new_account:''' If during an interaction a new browserid account is created, this property is true (as opposed to an interaction which represents sign in using an existing account)
* '''language:''' the language that was displayed to this user during the interaction
* '''lang:''' The language that was displayed to this user during the interaction
* '''number_sites_logged_in:''' If the user is authenticated to the Persona servers at any point during the interaction, we include the number of distinct sites that the user has logged into recently using browserid
* '''number_sites_remembered:''' If the user is authenticated to the Persona servers at any point during the interaction, we include the number of distinct sites that the user has logged into recently using browserid.
* '''number_sites_signed_in:''' A subset of "number_sites_remembered" is "number_sites_signed_in". If a site is remembered and the user is signed in at that site, when the user visits the site an assertion will be generated automatically. If a site is remembered but the user is not signed in at that site, an assertion will not be generated.
* '''screen_size:''' The screen dimensions of the device used by the user, determined programatically with javascript
* '''screen_size:''' The screen dimensions of the device used by the user, determined programatically with javascript
* '''orphaned:''' If the dialog is opened but the user never logs into a site, the dialog is considered orphaned.
* '''sample_rate:''' Rate at which the server is sampling clients data for KPI messages. 0.1 would be a 10% sample rate. We plan on shipping with 100% traffic, so 1.0
* '''sample_rate:''' Rate at which the server is sampling clients data for KPI messages. 0.1 would be a 10% sample rate. We plan on shipping with 100% traffic, so 1.0
* '''user_agent''' A generalized version of user agent which includes coarse grained details for Operating System, Browser, and Browser version. Does not contain original user agent string. Implementation will prefer non-fingerprintable values over technical accuracy (okay to encode 'unknown' for Operating System or 'Windows' if reg doesn't match a given version of Windows NT). No Operating system version is collected. Browser version will be as course grained as possible, while remaining useful. Examples: Firefox 13, Chrome 13, Safari 5.1 - no build number or other fine grained value is recorded.
* '''user_agent''' A generalized version of user agent which includes coarse grained details for Operating System, Browser, and Browser version. Does not contain original user agent string. Implementation will prefer non-fingerprintable values over technical accuracy (okay to encode 'unknown' for Operating System or 'Windows' if reg doesn't match a given version of Windows NT). No Operating system version is collected. Browser version will be as course grained as possible, while remaining useful. Examples: Firefox 13, Chrome 13, Safari 5.1 - no build number or other fine grained value is recorded.
Line 24: Line 26:
== Codebase ==
== Codebase ==
* [https://github.com/mozilla/browserid/blob/dev/lib/configuration.js#L136 BrowserID] collects the data
* [https://github.com/mozilla/browserid/blob/dev/lib/configuration.js#L136 BrowserID] collects the data
* [https://github.com/jedp/kpiggybank KPIggyBank] is the data store
* [https://github.com/mozilla/kpiggybank KPIggyBank] is the data store
* [https://github.com/nmalkin/kpi-dashboard KPI Dashboard] is the frontend which will be behind VPN
* [https://github.com/mozilla/kpi-dashboard KPI Dashboard] is the frontend which will be behind VPN
 
== Environments ==
 
{| border="1" cellspacing="0" cellpadding="2"
|-
! Environment
! Test Link
! Persona Login
! Backend
! Dashboard
|-
| Development
| http://dev.123done.org
| https://dev.diresworb.org
| https://kpiggybank.hacksign.in/wsapi/interaction_data
| http://dashboard.personatest.org/
|-
| Stage
| http://beta.123done.org/
| https://diresworb.org
| TBD
| TBD
|-
| Production
| http://123done.org/
| https://browserid.org
| TBD
| TBD
|}


== Domain Name ==
== Domain Name ==
This lives on the BrowserID server.
This lives on the BrowserID server, so no new public domain names.
 
New internal sub-domains for the backend and Dashboard will be created.


== Project Management ==
== Project Management ==
=== Team ===
=== Team ===
* skinny - UX
* Crystal Beasley - UX
* dmills - Product
* Dan Mills - Product
* aelliot - Metrics
* Annie Elliot - Metrics
* rforbes - Security
* Raymond Forbes - Security
* Alina - Privacy
* Alina Hua - Privacy
* nathan, lloyd, jparsons, ozten, stomlinson - Development
* Katie Parlante, Shane Tomlinson - Development
* Services QA
* Gene Wood - Services
* John Morrison - QA


=== Schedule ===
=== Schedule ===
==== Phase I Backend ====
* 4/23 - Security Review KPI Backend
* 4/23 - Security Review KPI Backend
* <strike>4/25</strike> TBD - Data Safety Meeting
* <strike>4/25</strike> TBD - Data Safety Meeting
Line 46: Line 81:
* 4/25 - 5/9 Dev sprint for KPI Backend
* 4/25 - 5/9 Dev sprint for KPI Backend
* 5/9 - 5/23 Document all KPIs
* 5/9 - 5/23 Document all KPIs
* 5/9 - 5/23 Dev sprint for "implement first cut visualization of one KPI"
 
* 5/23 - 6/6 Dev sprint for "implement visualization of all KPIs"
==== Phase II Dashboard ====
* ? - Stage Env for Backend
* 5/9 - 6/30 Dev sprint for "implement first cut visualization of one KPI"
* 5/23 - 6/5 - Stage Env for Backend
* 5/23 - 7/6 Dev sprint for "implement visualization of all KPIs"
* 7/7 - 7/20 Dev sprint for Dashboard polish
* ? - [https://github.com/jedp/kpiggybank/issues/2 Collect test data]
* ? - Prod Env for Backend
* ? - Prod Env for Backend
* ? - Security Review KPI Dashboard
* ? - Legal & Privacy Review KPI Dashboard
* ? - Stage Env for Frontend
* ? - Stage Env for Frontend
* ? - Prod Env for Frontend
* ? - Prod Env for Frontend
* ? - Deploy Backend and Collect data
* ? - Deploy Dashboard to display data
* 8/24 - nmalkin's Internship ends


=== Bugs or Forms ===
=== Bugs or Forms ===
==== Backend ====
* {{Bug|742796}} - Security Review for Backend
* {{Bug|742796}} - Security Review for Backend
* {{Bug|746245}} - Privacy/Legal Review for Backend
* {{Bug|746245}} - Privacy/Legal Review for Backend
* [https://etherpad.mozilla.org/identity-kpi-backend KPI Data Saftey Review Etherpad]
* [https://privacy.etherpad.mozilla.org/66 Privacy Review Questions Etherpad]
* [https://wiki.mozilla.org/Privacy/Reviews/KPI_Backend Privacy Review]
==== Frontend ====
* {{Bug|773400}} - Tracking bug
* {{Bug|773404}} - Security review KPI Dashboard
* {{Bug|773399}} - Legal review KPI Dashboard
* {{Bug|746231}} - First KPI Visualization
* {{Bug|746231}} - First KPI Visualization
* {{Bug|746233}} - All Remaining KPI Visualizations
* {{Bug|746233}} - All Remaining KPI Visualizations
* [https://github.com/mozilla/browserid/issues/1060 instrument site in preparation for a/b testing of UX experiments]
* [https://github.com/mozilla/browserid/issues/1060 instrument site in preparation for a/b testing of UX experiments]
* [https://etherpad.mozilla.org/identity-kpi-backend KPI Data Saftey Review Etherpad]
* [https://privacy.etherpad.mozilla.org/66 Privacy Review Questions Etherpad]
* [https://wiki.mozilla.org/Privacy/Reviews/KPI_Backend Privacy Review]


== Get Involved ==
== Get Involved ==

Latest revision as of 19:00, 14 February 2013

KPI Dashboard

A set of visualizations for UX and Product to evaluate how well we're doing with improving user's experience of BrowserID.

Architecture

KPI Dashboard architecure.jpg

There are two main components, KPI Backend and KPI Frontend.

See https://wiki.mozilla.org/Privacy/Reviews/KPI_Backend

Data Glossary

During the users interaction with the dialog we capture various information:

  • timestamp: the time that the interaction started, rounded off to 10 minute intervals
  • event_stream: interesting events that occurred during the user's interaction, including both events initiated by the user (mouse clicks) as well as events originating from running javascript code (keypair generation). Each event is uniquely named and includes a time offset for when it occurred measured from when the dialog was displayed
  • email_type: In the event that the interaction results in the user selecting an email address to use to sign in, we include the type of email used: "primary" is an email address from a domain that has browserid support, "secondary" is an email address from a domain that does not directly support browserid
  • number_emails: When an interaction proceeds to the point where the user authenticates to the personaid service we include the number of emails that this user has verified with BrowserID
  • new_account: If during an interaction a new browserid account is created, this property is true (as opposed to an interaction which represents sign in using an existing account)
  • lang: The language that was displayed to this user during the interaction
  • number_sites_remembered: If the user is authenticated to the Persona servers at any point during the interaction, we include the number of distinct sites that the user has logged into recently using browserid.
  • number_sites_signed_in: A subset of "number_sites_remembered" is "number_sites_signed_in". If a site is remembered and the user is signed in at that site, when the user visits the site an assertion will be generated automatically. If a site is remembered but the user is not signed in at that site, an assertion will not be generated.
  • screen_size: The screen dimensions of the device used by the user, determined programatically with javascript
  • orphaned: If the dialog is opened but the user never logs into a site, the dialog is considered orphaned.
  • sample_rate: Rate at which the server is sampling clients data for KPI messages. 0.1 would be a 10% sample rate. We plan on shipping with 100% traffic, so 1.0
  • user_agent A generalized version of user agent which includes coarse grained details for Operating System, Browser, and Browser version. Does not contain original user agent string. Implementation will prefer non-fingerprintable values over technical accuracy (okay to encode 'unknown' for Operating System or 'Windows' if reg doesn't match a given version of Windows NT). No Operating system version is collected. Browser version will be as course grained as possible, while remaining useful. Examples: Firefox 13, Chrome 13, Safari 5.1 - no build number or other fine grained value is recorded.

Codebase

Environments

Environment Test Link Persona Login Backend Dashboard
Development http://dev.123done.org https://dev.diresworb.org https://kpiggybank.hacksign.in/wsapi/interaction_data http://dashboard.personatest.org/
Stage http://beta.123done.org/ https://diresworb.org TBD TBD
Production http://123done.org/ https://browserid.org TBD TBD

Domain Name

This lives on the BrowserID server, so no new public domain names.

New internal sub-domains for the backend and Dashboard will be created.

Project Management

Team

  • Crystal Beasley - UX
  • Dan Mills - Product
  • Annie Elliot - Metrics
  • Raymond Forbes - Security
  • Alina Hua - Privacy
  • Katie Parlante, Shane Tomlinson - Development
  • Gene Wood - Services
  • John Morrison - QA

Schedule

Phase I Backend

  • 4/23 - Security Review KPI Backend
  • 4/25 TBD - Data Safety Meeting
  • 4/25 - 5/9 Document one KPI
  • 4/25 - 5/9 Dev sprint for KPI Backend
  • 5/9 - 5/23 Document all KPIs

Phase II Dashboard

  • 5/9 - 6/30 Dev sprint for "implement first cut visualization of one KPI"
  • 5/23 - 6/5 - Stage Env for Backend
  • 5/23 - 7/6 Dev sprint for "implement visualization of all KPIs"
  • 7/7 - 7/20 Dev sprint for Dashboard polish
  • ? - Collect test data
  • ? - Prod Env for Backend
  • ? - Security Review KPI Dashboard
  • ? - Legal & Privacy Review KPI Dashboard
  • ? - Stage Env for Frontend
  • ? - Prod Env for Frontend
  • ? - Deploy Backend and Collect data
  • ? - Deploy Dashboard to display data
  • 8/24 - nmalkin's Internship ends

Bugs or Forms

Backend

Frontend

Get Involved

Retrieved from "https://dev.wikimo.nonprod.webservices.mozgcp.net/index.php?title=Identity/BrowserID/KPI_Dashboard&oldid=519932"