Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How do sites like Facebook and Twitter optimize their sites for massive traffic. Aside from spending big bucks on getting the best servers, what can be optimized in your code to accommodate massive traffic?

I've read about caching your pages to static HTML, but that's impractical for social networking sites where the pages are constantly updated.

share|improve this question

closed as too broad by gnat, Corbin March, MichaelT, GlenH7, Kilian Foth Sep 9 '13 at 12:03

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

If you get 10000 access per second on an average, even a 1 second caching will save you 9999 script processing. – user2567 Dec 12 '10 at 13:43
You might be interested in this too (I can't find the original post by SO owners):… – user2567 Dec 12 '10 at 14:09
Here is a video and presentation describing some of the stuff the engineers at facebook did to scale up. – aufather Dec 12 '10 at 14:44

Facebook is a particularly extreme example of load, but in general, high-traffic sites:

  1. use a content delivery network (CDN) for distributing static assets (CSS and JavaScript) and media (images, videos, etc.).
  2. use in-memory caches like Memcached whenever possible to reduce load on the database.
  3. have a load balancer that distributes traffic to a pool of web servers
  4. cache generated HTML fragments (e.g. parts of your profile page) and include them when generating a full response
share|improve this answer
+1 Couldn't have said it better myself. For a CDN network, I can vouch for Amazon S3 + Cloudfront which I found to work very well for serving static content. For memory cache, check out Redis as well. – Martin Wickman Dec 12 '10 at 14:54
Why is Facebook a bad example? – peter Dec 12 '10 at 15:33
Oh, it's not bad; I'm just saying that most websites will never experience load on the level Facebook or Google experience. – Bill Dec 12 '10 at 16:09
facebook now has 60,000 servers – Steven A. Lowe Dec 12 '10 at 18:34

Content Delivery Network

Facebook uses static content delivery network for serving photos and videos. Check the source of any image in facebook and you will see something like

share|improve this answer
Can you atleast explain what is CDN, in context of Facebook? For instance, like the link – Kanini Dec 12 '10 at 14:02
@Kanini You can – Gulshan Dec 12 '10 at 14:24
What's the difference between Cloud Computing and CDN? – peter Dec 12 '10 at 23:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.