The Rust Project needs much better visibility into important metrics

[brson]

Hi @dikaiosune!

@dikaiosune

I have some other questions about the discussion here, but I just want to quickly throw this one out there: should the dashboard care about the duration of failed CI builds? I would assume that the times would vary heavily depending on the stage of the failure, and so wouldn't be useful, but I wanted to check before I throw away times for failed builds.

That number doesn't interest me right now.

@dikaiosune

Should I expect this to grow into a larger interactive application which needs something like Angular or Ember? Should I worry about that later and just build a dashboard MVP with jQuery?

I don't know enough about Ember to say. Personally I'd first figure out how to display the data graphically to prove that you can get the data all the way to the screen. If Ember helps with that then use Ember.

@dikaiosune

A scraper daemon which scrapes those three data sources at configurable intervals

Do you have a sense for how much data is being transferred in the scrapes? Do subsequent scrapes retransfer all the same data or are they incrementalized somehow?

@dikaiosune

A JSON API server with one working endpoint (/summary/) which serves a summary of the last 120 days of activity

How large is the summary data now?

@dikaiosune

A bunch of queries written mostly in Diesel (many many many thanks to @sgrif for his help), yielding pretty good performance. After parallelization, the full summary endpoint (15-20 queries + JSON marshalling) takes ~80ms per request on my desktop with an SSD, down from ~150ms when done serially (which is still pretty good though).

That's so cool you are using Diesel!

@dikaiosune

Write a simple JavaScript page/app/thing to grab the JSON, parse it, and graph it (still need to decide how to do this -- the KISS in me says jQuery + Highcharts/Highstock)

SGTM.

The data you've collected already has been interesting and useful!