Taking scalability to the next level. What methods can help me make sure I do the right thing?

Question

Myself and a fellow PHP dev are starting to research a new web app that we think (hope) will have a global reach.

The trouble is that, whilst we both consider ourselves capable, we are concerned with the sheer size of the potential traffic (neither of us has done anything with a reach beyond a few thousand. This site will hopefully serve millions) and wonder what we can do along the way to ensure we have tested things for scale.

The site will be dependant on mass simultaneous log ins, uploads of relatively large files (3-10mb audio, >4mb images) and also streaming of audio.

Answer 1

Testing For Scalability

Testing for scalability across multiple servers can be quite hard - load/stress testing can be quite simple though, and tools are available for that (I've heard a few good things about JMeter). Once you have some figures on how your app performs on one server, that should give you some idea of what sort of infrastructure you might need. However, having a certain performance on 1 server doesn't mean you'll get double performance with 2 servers, as you're accepting user uploads. User uploads means merging uploads across multiple servers. So you'll need to test performance across at least 2 servers, then compare how much performance improves when adding a 3rd server. If you're continously merging uploads across multiple servers you may end up getting diminishing returns with each new server added (so 10 times more servers won't mean 10 times better performance).

Designing for scalability

There are a number of things you can do to improve the scalability of you app. These include:

1. Use a Content Delivery Network. If you're using JQuery for example, make sure you use google's CDN when you link the files. You can put static images on a CDN as well. Basically anything you can do to stop your servers getting hit is a good thing.
2. Use proper headers like content-expires - and set the date a long time in the future so visiters use cached versions of files.
3. Make use of AJAX for lazy-loading of your pages - speed is what people perceive it to be. If your main content comes across quickly, people don't notice that other parts take longer to load.

Actually Scaling

In my opinion, the most important thing is how you can scale up quickly when needed. Adding a server can take ages, and can be very expensive. One option is Windows Azure. I know you're working with PHP, but libraries are available to bridge the gap. The idea is, if you need more capacity, you can log in to your Azure admin panel, and scale up to use additional servers straight away. Then in quieter times you can scale back down to save yourself the cost. It doesn't make sense (to me, and others from the other answers i've seen) to invest in a massive infrastructure at this stage. So having an infrastructure (such as Azure) which you can take advantage of is something to consider.

I appreciate this might not answer your question completely in terms of how you test these sort of things, but maybe it'll give you some other things to think about.

Answer 2

Read this article on scalability by Ted Dzibua or just skip to the tl;dr: "Shut up about scalability, no one is using your app anyway."

Answer 3

I'm completely with what Eric and Bartek said. The reality is its not a problem you'll have to face any time soon, and its best to focus attention on those problems you actually have. Doing so will make it more likely you'll have the 'problem' of having to scale to millions of users. Plus, its hard to plan for 'scaling' when you dont know what it is you have to actually scale yet.

A single, good server can handle far more than most people think. Before throwing hardware at the problem, instead make sure you've squeaked out what you can from the app itself. As your traffic starts ramping up, you'll want to start ensuring your database queries are efficient and tables are properly indexed (you should be doing that anyways, but a lot of sites simply dont), and the db is properly tuned. You'll want to make sure the images you use are as small as possible (in file size) to help with bandwidth. PHP caching, obviously. Gzipping text content. CSS and .js are broken into separate files so the content is retrieved with each page load. All that can greatly increase a server's capacity.

And then, once you've done all that and whatever other optimizations you can think of, you'll know where the bottleneck is if things start getting slow. Is it the db? then its time for replication. Is it bandwidth? Then you need to increase the pipe - adding servers isnt going to help with that (and then you'll be greatful you didnt waste money buying them).

Answer 4

First, worry about building a good site that actually has users. That is an incredibly tough task in and of itself. Follow decent design principles and use technologies that make sense for the problem you are solving and you'll scale fine for a long time with almost any reasonable architecture. If you get to the point where performance is a problem because of your site's popularity, realize that this is a good problem to have and deal with it then. Trying to optimize too much ahead of time - long before you know where your bottlenecks will actually be - just slows down your ability to ship quickly. And on the Internet, being slow to iterate & ship is far more likely to kill your site than being slow due to overwhelming load.

Answer 5

Jeff Dean has a pretty famous quote which goes something like (paraphrased):

"There's no need to burden yourself with making a system scale more than 10x further than it needs to, as long as you're confident that you'll be able to scale it as you grow... it's also nearly impossible to scale a system 100x: you simply have no idea what the traffic patterns and bottlenecks will be at 100x the volume."

Get thousands of users with this new application before you worry about getting millions.

Make sure that your design for thousands will still work if you have tens of thousands; but this design will not work for millions. Shooting for millions off the bat is foolish and bound to fail - you have no idea how these millions of hypothetical users will use your site, where the heavy access points will be, what the bottlenecks might be, etc.

Instead of wasting time speculating... build a product that people would want to use. Then scale it.

If you have a beautifully scalable app that has crummy features and poor UX design, then guess what - no one will ever use it to start with! Users do not care how beautifully your app may potentially scale one day. They only care that the product is something they like to use.

Answer 6

Programming Axiom number one: "Premature optimization is the root of (nearly) all evil."

This is premature optimization. You haven't even written the site yet, and you're already worried about how your code is going to scale? I hate to be this guy, but you're probably not going to have millions of users. You're probably not even going to have a single million. Not because I don't believe that you've got the talent, or the idea, it's just that the percentages aren't with you.

Even if you do get to millions of users, worry about crossing that river when you get to it. Running a site that big is usually a full time job, which means that you'll have eight hours per day (or more) to work on it when you get to it.

Answer 7

This is a topic that I have a great interest in...there are many parameters to consider when scaling your network, backend/database or the application. Its a vast topic with a multitude of scaling techniques to consider depending on the nuances of your application. You can access my slides of a presentation I did on why and how of highly scalable websites recently: http://www.slideshare.net/faizanj/why-and-how-of-highly-scalable-web-sites

Answer 8

You can have a look at this sample application from Microsoft which showcases some of the data-structures relevant to your problem domain.

http://code.msdn.microsoft.com/BidNowSample

It's runnable out of the box on Azure, but even if you want another platform, it's good code.

Answer 9

scalability in the end is about resource allocation. you want a system architecture that allows you to deliver on a resource allocation guarantee as the system grows.

i find these real techniques are useful for scaling systems:

1. statelessness. a system that minimizes complex shared states can be grown more easily than those that have substantial shared states

2. asynchronicity. get expensive work out of the way of the main flow of control as soon as possible

3. exploit locality. loosely speaking, this means caching

Answer 10

You must use an architecture that can grow horizontably, i.e. by adding more hardware.

This must be thought of from the beginning as it will be the way you handle growth. Having any "there can be only one" bottleneck in your application will bite you some day. You do not have to actually implement e.g. advanced databases, but you need to know that you CAN replace your simple database with a large disstributed one if needed.