June 11, 2012

The Grocery Store Analogy


So imagine you're the manager of a brand new grocery store that's opened up on the block. 

You just opened up and hired your first cashier, Bob, and your store isn't too busy so he's able to handle the customers that need to be served pretty easily and no one has to wait that long to checkout. The name of the game is to get as many visitors checked out in as little time as possible.


Suddenly Ashton Kutcher tweets about your awesome new grocery store and you get a spike of new customers coming in. Poor old Bob-the-cashier has a line backed up all the way to toiletries in aisle 10. What's the most obvious thing you could do? Well, hire more cashiers of course.

 So you hire 9 more cashiers, now you can serve your huge spike of customers much faster.

### OR CAN YOU???

Only wait — how are the cashiers looking up grocery prices? It turns out that in our store every cashier has their own terminal into a central service that lets you look up grocery prices, after which the cashier can ring up and charge the customer. The problem is that this service is really slow and get's even slower whenever you have more cashiers connected to it.  Surprisingly, you'd make things even slower if you just went with the shotgun approach of just adding more cashiers.


I know, I feel ya :/  We're in this position now where we have 10 cashiers, but they spend a lot of time not being productive because they have to wait so long to look up their customers grocery items. What if every cashier could do other kinds of work (e.g. bag their groceries or even start ringing up the next customer) while they were waiting for their turn to look up groceries?


In this scenario, a customer going through the checkout lane is like a visitor coming to your site. When you get a large influx of customers, you get really backed up checkout lanes just like you would with a  web application. The terminal/grocery takes the role of the database which, just like in our scenario, is the main bottleneck for many apps operating at a large scale.

Allowing cashiers to perform work while they wait for the lookup to finish is what evented systems like Node.js allow you to do in Web Architecture Land. Cashiers can tell their terminal that they want to look up a grocery item, and then while we wait for the result they can start bagging items. Our terminal will send us a nice notification once the lookup is done so we can finish ringing up and charging the customer.  


There's really a lot more to this that we could improve. That whole one-lookup-at-a-time thing is awful, and there's a few things you could do:

– **Split up the database.** We can take the grocery store items and put the items that start with letters A-M in one database and N-Z in another. Now, cashiers who are looking up the price of "Bananas" they only block up one database and other cashiers are still free to use the N-Z database.  We could even go to the extreme of having one database for each letter of the alphabet. In Web Architecture Land, this is called "database sharding".

– **Database caching.** If 90% of our customers are just buying bacon, then maybe it'd be quicker to just give each cashier a piece of paper with the price of bacon on it. Now they don't have to look it up in the database every time which saves us *a lot* of time. But you can imagine how this would quickly become inefficient if we just gave each cashier a giant book of every price and instead of looking things up in the database they started flipping through pages for prices.


Maybe we don't split up the database and just have multiple copies. Maybe we can train the cashiers to be just a little bit faster at bagging. Or maybe we don't even solve the problem with technology but somehow find a way to get people to not come to our grocery store all at the same time — if our customers are spread out throughout the day than even a couple cashiers could probably handle it.

Admittedly, we've stretched this analogy a bit in places, but it can be a useful mental model for web architecture if you're a beginner. The important thing to note is that there are hundreds of ways to architect your application for performance but it all starts with identifying specific problems and not just using the latest tools.

Tweet about this on TwitterShare on LinkedInShare on FacebookShare on RedditEmail this to someone