Sunday, June 29, 2014

Like all magnificent things, it's very simple.

While working on the Dashboard I learned about the analysis workflow. This is a diagram stolen from Jonas:




Two weeks ago I started working on Telemetry Analysis Framework.
We are simplifying the MapReduce workflow to be as flexible yet as easy to use and debug as possible. Jonas has been developing TaskCluster for a while and he came up with the idea of porting the analysis to it.


What is TaskCluster

TaskCluster is a set of components that manages task queuing, scheduling, execution and provisioning of resources. It is designed to run automated builds and tests at Mozilla. You can imagine it like the diagram below:



 

TaskCluster for our MapReduce Analysis

When you talk about MapReduce it is usually referring to a directed graph workflow like the one below. Taskcluster provides task graphs that can easily be used as a MapReduce workflow.

     
As mentioned above we want this framework to be:
  • simple to use
  • flexible
  • easy to debug


Simple to use 
Simplicity in this case means the programmer has to specify as little as possible:

 

Docker Image
For the purpose of this application you can consider a Docker image as a lightweight virtual machine. In it you store the setup for the analysis: needed utilities and custom code.

Because we know that starting with something new can be annoying we provide a base image where you only need to write your custom code. Also we provide dead easy documentation for each step. More information about Docker can be found here.

 

Filter.json 

Filter.json is the file given so we can extract from S3 bucket the files needed for the analysis.


By providing these two elements the framework will proceed as follows:


The Analysis Framework will parse the Filter.json, make a request to the index database, split the response in sublists of files and start a map task for each batch of files.

A map task would look as follows:


 
 For a mapper task we would need to specify the Docker image (line 25), a command (line 27) and the files that need to be processed (lines 28 and 29).
The output of this task will be an artifact  /bin/mapper/result.txt and will be uploaded to an intermediate bucket on Amazon.

Flexible

We call this framework flexible because you can customize all the layers provided.
The framework comes with a downloader utility, several mapper helpers (python, javascript) that can decompress the files and feed the result to the loaded custom map function.
If you would like a different Docker image you can customize that too. The framework includes a default image and also the Docker file out of which you can build the image. By modifying that file you can easily get another Docker image that suits your needs.

If you are on MacOS you probably need some way to work with Docker. By installing Vagrant and adding the Vagrantfile to your directory you can work easily with Docker on your machine.

Easy to debug

Until now the analysis that we performed had close to zero logging. This had to change so we could get a robust and easy way to debug the workflow. With the approach we are trying to develop right now we have logging at each step. The developer has the option to see how many rows were processed, what were the files downloaded with success, what were the ones decompressed successfully and if an error occurred. With this information some retry policies could be implemented. 

Last but not least
I will be blogging about this next week and also provide you with the link to the official repo. Right now a lot of changes happen in my playground :) ..














Tuesday, June 10, 2014

“Alice:How long is forever? White Rabbit:Sometimes, just one second.”

Last week I finally pushed the new dashboard to production.

The problem with bugs is that ...the later you find them the harder they bite your ass. I won't talk about bugs because you don't talk bad about the dead :p

Telemetry Dashboards measure how Firefox behaves on desktop and mobile in real world use.This application provides the aggregate view of the data collected.

One can hit http://telemetry.mozilla.org/ in three ways:

  • request for default page (no cookie set)
  • request with cookie set (cookie expiration 3 days)
  • request for specific state (aka url encoding a particular state)


The page itself has two states:

Single filter (when you look for a "one at a time version plot") you get histogram/table plots and summary details.







Several filters



By adding multiple filters on your plot you switch from one mode to another. If you want to get back into single mode state you just have to remove all the filters you added.

Locked/Unlocked state

There are two ways you can use the filters:

Locked State

You want all the measures to be in sync with the first filter. Whatever change made in your first filter will be propagated to the ones below. If the ones below don't have data for this particular measure they will be set to a default measure.





Unlocked State

Use the unlock button to compare between different measures. 




Legend

The filters can get very long. In some cases it doesn't make sense to show the full state because we only have one measure across multiple versions in the locked state. That's why we show the title of the measure and the details for it and simplify the legend by making it show only the versions.





In the case of mixed filters (aka unlocked state) we show only the part of the filter that fits one line, for more information you can look in the tooltip.







Caching

When you add another filter we instantiate a new selector but we don't make a request. Requests are done only if we don't have that data in the cache.

Url and TinyUrl

All the page state: filters, lock/unlock, selector state (percentiles), table/histogram are preserved in the url. This way the state gets saved for a return with state or cookie session.





The end... :)