QA/Mozmill Test Automation/Dashboard/Roadmap/0.4

Version 0.4

With version 0.4 we are planning to replace the CouchDB backend of the Dashboard with probably ElasticSearch. Therefore more time is necessary, so we do not expect to see the release before end of Q2 in 2011. Owen Coutts will mainly work on that release during his internship.

Functionality

Front End

• Filter by:
  • Locale
  • OS
  • Build/Version (Ideally select a range)
  • Date
• Tag Bugs
• Linking test modules to the repository for easier investigation of failures

Application Server

• Deal with users
  • Ideally through LDAP so we don't have to roll a registration system
• Tag Bugs
  • Add/Change/Remove of bug numbers to specific failures
  • Add Comments

Elastic Search

• Connect River to backend storage

Backend Storage

• Map-Reduce Reports
  • failures/passes per function
  • Probably will require a different db/index

Tasks

Q1

• Define specs and requirements for the replacement dashboard
• Create an Elastic Search pilot application to confirm the expected functionality

Q2

• Update specs and requirements based on pilot application
• Implement features based on specs in multiple milestones
• Deploy application to production
• Hold a brownbag to present the new dashboard

Technical Background

Architecture

• Backend
  • Storage should be done with hadoop. Hadoop infrastructure is available and proven to scale. The decisions to move away from couch db was made because of the inability to scale and the concerns about the maturity of the software.
• Search/filtering dealing with data.
  • Elastic search has the capability to search/filter and use natural language to do so.
  • We will need to use some form of map reduce to deal with converting the data from lots of reports to aggregated functions.
• CRUD Server
  • Owen is using Django as a CRUD server (Create, Register, Update, Delete). This will allow us to give registered users the option to tag things/comment on them. Mozilla uses playdoh on the web-dev team. This is an added layer that gives some extra functionality that has been useful in the past. At a first glance, many things will be ignored. I imagine that localization of the app and some other functionality like that will be out of the scope of this project.
  • This will also be used as a proxy into the firewall for elastic search and hadoop (which are behind the firewall)
• Front End
  • I will try to use something that has been used before. Likely, sammy.js views that allow for all the functionality listed in the functionality section.

Unknowns

• Map Reduce then filter is the wrong order and will not allow us to aggregate a filtered data set. Right now results are stored based on the report they were submitted with but we want to show the user information aggregated by test function. We need to do this aggregation and we also want to filter by data in the report (local, language, operating system). This needs to happen in real time.
  • Ideally we would filter and search the data with elastic search and then run a map reduce in order to aggregate this data. The systems are not usually designed for this.
  • Another option is to duplicate all the data per function. We wouldn't have to map the data of reports to functions in that case but we would have a much larger database with a lot of extra data in it.

Progress

Prototype

• Proof of concept to show that at least the same functionality can be rolled with elastic search.
• Technical Knowledge learned can be found here: Proof of Concept Prototype