New login app being load tested

Today I took a little bit of a hammer to our new testing logging app, and the results were quite nice.
Now, there are caveats….

  • I used a sampling of 10 URL’s I had seen used in testing.  I am not certain yet what real life traffic will look like.  In fact, I am pretty certain that this will not be the real traffic pattern.
  • I’m not amazing at MongoDB yet, so there could be some caching going on making the results skew.
  • This was pretty fast and dirty.

Tools used:

  • New Relic and Cloudwatch for graphs
  • Siege(along with a list of a few URL’s I’d seen used in testing) on two AWS nodes for generating load
  • AWS Kit: m1.small web tier node, single AZ ELB, m1.medium MongoDB server.
Scalability Analysis

Scalability Analysis

Screen Shot 2013-05-04 at 1.43.05 PMThroughput during the test (requests per min)

(Requests per minute)

 

CPU Utilization on web node (m1.small 64bit Ubuntu)

CPU Utilization on web node (m1.small 64bit Ubuntu)

 

ELB Latency

ELB Latency  

 

Mozilla Foundation Metrics – Starting the Conversation

Today, Ross Bruniges(@rossbruniges, fellow Mozillian) and I gave a presentation to our fellow Mozilla Foundation engineers about metrics and how to get the data for a data driven operation.

The purpose of our presentation was not to dictate or announce how we would do metrics, but to really begin a process and conversation.

Integrating metrics into your organization can be done in a painfully large number of ways, so part of the challenge we have and you may have is that there are so many options.  I think the trick here is to pick something, start small, lay out a basic framework(feel free to borrow ours :) ), and then have your developers play with that.  I have a feeling that in a week or two I’ll have engineers measuring their intake of coffee like the Etsy team. (Awesome write up and tool Etsy, thanks!)

Our conversation laid out a framework and a beginning toolset.  It defined the naming structure of statsd metrics, for example, and demonstrated a few streams we can tap into for metrics(for now). We used four layers of metrics to define what kind of metrics we would look to measure:  Organizational, Team, Product/App, and Operational

That will likely be very adaptable to many organizations, possibly replacing Product/App with Service or Department.  I’d be curious to hear what you are using for determining how you tie stats together to create metrics.

 

AWS Re:Invent 2012 – Obama for America Panel

In November 2012, me and this group of guys (and another giant group of guys and gals not in the video) reelected the President of the United States.  Later that month, we were at AWS Re:Invent 2012 to talk about it.

In this video(from the top left):

Me, Leo Zhadanovsky, Jay Edwards, Scott VanDenplas

Ben Hagen, Brian Holcomb, Harper Reed

Moderator: Miles Ward