In a previous post, I showed you the performance of our node.js application servers for the Webmaker suite.
As my boss has noted, though, app server performance is not the same thing as user experience or browser performance.
Though we recently added monitoring of AWS resources in New Relic and now had information about load balancer utilization, RDS(Amazon Web Services database service) utilization, disk volume health, and even SES bounce rates, he was right…
We still had no idea how users actually experienced our applications.
My last DevOps gig got me used to having RUM(real user measurement), and I’ve been telling coworkers and boss “Soon, soon we’ll have browser performance data like these other languages”. While I’ve been telling them that, I’ve been experimenting with other solutions like Stackdriver, Pingdom RUM, bucky, and Google Analytics, despairing that node.js was a second class citizen in New Relic.
New Relic just announced v1.4.0 of its node.js agent, which finally includes browser performance. You can read about it here.
Today, we put this into production on webmaker.org
I have wondered how our location in Virginia (AWS datacenter) and lack of CDN impacted our international. After all, the webmaker team just spent a ton of effort localizing webmaker pages. Side note: We’re ranked in the top five or ten on transifex for localization projects. (HIGH FIVE AALI/TEAM)
So then, how do we do internationally on webmaker.org pages?
Highlight countries to see data
Well, those results are pretty expected. A lot of overseas traffic, including places with underserved internet, is far slower than North American traffic.
So, overall then, how much time is being spent in the browser? Is it all just network, as the previous map might imply? Let us check the overall time spent.
This tells me we’re spending a lot of time rendering, which we can resolve with some work on a CDN and more diversity geographically with our infrastructure (Context: We’re in us-east-1 on AWS)
One other thought…what kind of data can I find for client type (mobile vs. desktop, etc)?
Here’s what the last 24h look like for webmaker.org
Another interesting thing to look at in the new views of data is browser prevalence and performance.
Being able to have all of these views for all languages and frameworks we use, combined with the monitoring AWS resources (Load balancer stats, DB server stats, disk volume stats, SES stats) the New Relic plugins make it a pretty complete package for monitoring and alerting. Even better, plugins for Mongo, Elasticsearch, and tons of other apps and utilities fill yet another gap.
Overall, I’m digging the new RUM in New Relic as well as the wide array of plugins on the marketplace, but here are a few things I still yearn for.
Separation of app server activities
Below is an example of first a python (django, specifically)app perf chart in production and a then one for a node.js app.
I, as an avid fan of color charts, am a bigger fan the more colors it has.
Needs Moar Dashboard
I make heavy use of custom dashboards in New Relic, giving every app a detail page and every project an overview page.
With the advent of all of these plugins, such as those for Elasticsearch or Mongo, and the RUM and browser data, the limit of 15 charts per dashboard just isn’t gonna cut it.
Dashboards showing application server performance, database response and throughput, error rates, top transactions, scalability analyses, load balancer latency/throughput/response stats, RDS and Elasticache stats(CPU %, Free space, Mem), Elasticsearch index performance, and Mongo performance can really help speed up correlation and problem resolution.
We need AWS plugins to be able to associate names with tags, not just hostnames
One awesome thing in Stackdriver was the ease with which I could make a group.
“OK, um, all “webmakerorg-production”
Magic! There were my autoscale groups with instances in them, aggregating CPU and Memory by autoscale group. This is one place New Relic falls short of Stackdriver.
All said, we’ve had amazing progress for us node.js production shops from New Relic and the community developing plug-ins. From engineering here at Mozilla Foundation, thanks!