Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Myself and a fellow PHP dev are starting to research a new web app that we think (hope) will have a global reach.

The trouble is that, whilst we both consider ourselves capable, we are concerned with the sheer size of the potential traffic (neither of us has done anything with a reach beyond a few thousand. This site will hopefully serve millions) and wonder what we can do along the way to ensure we have tested things for scale.

The site will be dependant on mass simultaneous log ins, uploads of relatively large files (3-10mb audio, >4mb images) and also streaming of audio.

share|improve this question

closed as too broad by gnat, MichaelT, Ampt, GlenH7, Kilian Foth Sep 15 '14 at 10:09

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

I suspect that most people just don't know. Maybe if you told everyone "Assume that I am guaranteed to get millions of views starting on day 1, and I don't want to crash; what are your suggestions?" you can avoid everyone quoting the same old song. – Steve Evers Oct 29 '10 at 15:25
I would not use PHP in the first place, because imho scalability through code (as opposed to throwing hardware at it) is easier in strongly typed, preferably functional languages. The same goes for reactive programming, asynchronous IO, the tasks-pattern, map/reduce and parallel algorithms. If you choose PHP however, A LOT of gained by merely separating reads and writes. Google CQS and CQRS and the CAP theorem. – Henrik Feb 12 '11 at 16:28
Thanks Henrik, there is a lot in that answer I don't have a clue what you're talking about. Google is going to be taking a hit!! – Mild Fuzz Feb 16 '11 at 11:34
Facebook write their code in PHP and seem to be able to scale .. – jasonk Mar 30 '12 at 9:47
facebook develop in PHP, but deploy to C++ using hiphop – Mild Fuzz Apr 1 '12 at 11:10

10 Answers 10

up vote 8 down vote accepted

Testing For Scalability

Testing for scalability across multiple servers can be quite hard - load/stress testing can be quite simple though, and tools are available for that (I've heard a few good things about JMeter). Once you have some figures on how your app performs on one server, that should give you some idea of what sort of infrastructure you might need. However, having a certain performance on 1 server doesn't mean you'll get double performance with 2 servers, as you're accepting user uploads. User uploads means merging uploads across multiple servers. So you'll need to test performance across at least 2 servers, then compare how much performance improves when adding a 3rd server. If you're continously merging uploads across multiple servers you may end up getting diminishing returns with each new server added (so 10 times more servers won't mean 10 times better performance).

Designing for scalability

There are a number of things you can do to improve the scalability of you app. These include:

  1. Use a Content Delivery Network. If you're using JQuery for example, make sure you use google's CDN when you link the files. You can put static images on a CDN as well. Basically anything you can do to stop your servers getting hit is a good thing.
  2. Use proper headers like content-expires - and set the date a long time in the future so visiters use cached versions of files.
  3. Make use of AJAX for lazy-loading of your pages - speed is what people perceive it to be. If your main content comes across quickly, people don't notice that other parts take longer to load.

Actually Scaling

In my opinion, the most important thing is how you can scale up quickly when needed. Adding a server can take ages, and can be very expensive. One option is Windows Azure. I know you're working with PHP, but libraries are available to bridge the gap. The idea is, if you need more capacity, you can log in to your Azure admin panel, and scale up to use additional servers straight away. Then in quieter times you can scale back down to save yourself the cost. It doesn't make sense (to me, and others from the other answers i've seen) to invest in a massive infrastructure at this stage. So having an infrastructure (such as Azure) which you can take advantage of is something to consider.

I appreciate this might not answer your question completely in terms of how you test these sort of things, but maybe it'll give you some other things to think about.

share|improve this answer
+1 for Azure. It supports PHP. And for good points. – Henrik Feb 12 '11 at 16:49
I'll note that there are other "scalable" farms too, if one does not want to go with Windows. I do like the generic idea. – Matthieu M. Feb 12 '11 at 17:50

Read this article on scalability by Ted Dzibua or just skip to the tl;dr: "Shut up about scalability, no one is using your app anyway."

share|improve this answer
+1 , I haven't laughed this hard in such a long time. I've permanently booked-marked Ted Dzibua's blog. Really love the straight no bullshit writing style. – Darknight Dec 19 '10 at 11:55
Aww, it's just sad and cynical that this is the highest rated answer. – Henrik Feb 12 '11 at 16:30

I'm completely with what Eric and Bartek said. The reality is its not a problem you'll have to face any time soon, and its best to focus attention on those problems you actually have. Doing so will make it more likely you'll have the 'problem' of having to scale to millions of users. Plus, its hard to plan for 'scaling' when you dont know what it is you have to actually scale yet.

A single, good server can handle far more than most people think. Before throwing hardware at the problem, instead make sure you've squeaked out what you can from the app itself. As your traffic starts ramping up, you'll want to start ensuring your database queries are efficient and tables are properly indexed (you should be doing that anyways, but a lot of sites simply dont), and the db is properly tuned. You'll want to make sure the images you use are as small as possible (in file size) to help with bandwidth. PHP caching, obviously. Gzipping text content. CSS and .js are broken into separate files so the content is retrieved with each page load. All that can greatly increase a server's capacity.

And then, once you've done all that and whatever other optimizations you can think of, you'll know where the bottleneck is if things start getting slow. Is it the db? then its time for replication. Is it bandwidth? Then you need to increase the pipe - adding servers isnt going to help with that (and then you'll be greatful you didnt waste money buying them).

share|improve this answer
Another way of saying this is, you have no idea of what part of your set up will have bottlenecks until you actually implement the site and get a few users. When that happens, you will have a good idea what needs speeding up. – Dominic McDonnell Oct 30 '10 at 22:14
Its the old addage adapted to programming... no plan survives first contact with users. – GrandmasterB Oct 30 '10 at 22:23

First, worry about building a good site that actually has users. That is an incredibly tough task in and of itself. Follow decent design principles and use technologies that make sense for the problem you are solving and you'll scale fine for a long time with almost any reasonable architecture. If you get to the point where performance is a problem because of your site's popularity, realize that this is a good problem to have and deal with it then. Trying to optimize too much ahead of time - long before you know where your bottlenecks will actually be - just slows down your ability to ship quickly. And on the Internet, being slow to iterate & ship is far more likely to kill your site than being slow due to overwhelming load.

share|improve this answer

Jeff Dean has a pretty famous quote which goes something like (paraphrased):

"There's no need to burden yourself with making a system scale more than 10x further than it needs to, as long as you're confident that you'll be able to scale it as you grow... it's also nearly impossible to scale a system 100x: you simply have no idea what the traffic patterns and bottlenecks will be at 100x the volume."

Get thousands of users with this new application before you worry about getting millions.

Make sure that your design for thousands will still work if you have tens of thousands; but this design will not work for millions. Shooting for millions off the bat is foolish and bound to fail - you have no idea how these millions of hypothetical users will use your site, where the heavy access points will be, what the bottlenecks might be, etc.

Instead of wasting time speculating... build a product that people would want to use. Then scale it.

If you have a beautifully scalable app that has crummy features and poor UX design, then guess what - no one will ever use it to start with! Users do not care how beautifully your app may potentially scale one day. They only care that the product is something they like to use.

share|improve this answer
I think it's possible to analyse bottle necks ahead of time despite what Jeff says (whoever he is). – Henrik Feb 12 '11 at 16:36

Programming Axiom number one: "Premature optimization is the root of (nearly) all evil."

This is premature optimization. You haven't even written the site yet, and you're already worried about how your code is going to scale? I hate to be this guy, but you're probably not going to have millions of users. You're probably not even going to have a single million. Not because I don't believe that you've got the talent, or the idea, it's just that the percentages aren't with you.

Even if you do get to millions of users, worry about crossing that river when you get to it. Running a site that big is usually a full time job, which means that you'll have eight hours per day (or more) to work on it when you get to it.

share|improve this answer
Strongly disagree. Design for scalibility is important and is hardly premature optimization. yes, they may not get the millions of users they want, but it costs no more to deisgn correctly from the start for scalibility than not and it is silly not to if you havea plan to support that kind fo traffic. – HLGEM Oct 29 '10 at 13:46
@HLGEM: What I said was basically a really short, nicer version of the Ted Dzibua blog post linked above. If the user was good enough to design a linearly scalable application from the start, he/she wouldn't have asked the question they did. QED, they don't need to worry about it, and should focus on more important things, like actually getting users. Hope may get you elected President, but it doesn't get you page views. Shipping does. – EricBoersma Oct 29 '10 at 14:54
There are some things about scalablity and building sites that cater many, many concurrent users that you MUST know even if building only a small site, so that when your application grows to that size, you can easily enable them. – user1249 Oct 31 '10 at 3:50

This is a topic that I have a great interest in...there are many parameters to consider when scaling your network, backend/database or the application. Its a vast topic with a multitude of scaling techniques to consider depending on the nuances of your application. You can access my slides of a presentation I did on why and how of highly scalable websites recently:

share|improve this answer

You can have a look at this sample application from Microsoft which showcases some of the data-structures relevant to your problem domain.

It's runnable out of the box on Azure, but even if you want another platform, it's good code.

share|improve this answer

scalability in the end is about resource allocation. you want a system architecture that allows you to deliver on a resource allocation guarantee as the system grows.

i find these real techniques are useful for scaling systems:

  1. statelessness. a system that minimizes complex shared states can be grown more easily than those that have substantial shared states

  2. asynchronicity. get expensive work out of the way of the main flow of control as soon as possible

  3. exploit locality. loosely speaking, this means caching

share|improve this answer
I tend to think of cache as a cane you want to throw away at the earliest possible time. At least the classic caches that just keep data from a data-store in memory. Treated data, such as read-views are another matter, imho. Generally there are often better options than artificial caches. – Henrik Feb 12 '11 at 16:45

You must use an architecture that can grow horizontably, i.e. by adding more hardware.

This must be thought of from the beginning as it will be the way you handle growth. Having any "there can be only one" bottleneck in your application will bite you some day. You do not have to actually implement e.g. advanced databases, but you need to know that you CAN replace your simple database with a large disstributed one if needed.

share|improve this answer
what do you mean by 'there cab be only one' bottle necks – Mild Fuzz Oct 31 '10 at 19:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.