Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 8 down vote favorite
4

How do sites like Facebook and Twitter optimize their sites for massive traffic. Aside from spending big bucks on getting the best servers, what can be optimized in your code to accommodate massive traffic?

I've read about caching your pages to static HTML, but that's impractical for social networking sites where the pages are constantly updated.

share|improve this question
If you get 10000 access per second on an average, even a 1 second caching will save you 9999 script processing. – user2567 Dec 12 '10 at 13:43
You might be interested in this too (I can't find the original post by SO owners): highscalability.com/blog/2009/8/5/… – user2567 Dec 12 '10 at 14:09

3 Answers

up vote 13 down vote

Facebook is a particularly extreme example of load, but in general, high-traffic sites:

  1. use a content delivery network (CDN) for distributing static assets (CSS and JavaScript) and media (images, videos, etc.).
  2. use in-memory caches like Memcached whenever possible to reduce load on the database.
  3. have a load balancer that distributes traffic to a pool of web servers
  4. cache generated HTML fragments (e.g. parts of your profile page) and include them when generating a full response
share|improve this answer
1  
+1 Couldn't have said it better myself. For a CDN network, I can vouch for Amazon S3 + Cloudfront which I found to work very well for serving static content. For memory cache, check out Redis as well. – Martin Wickman Dec 12 '10 at 14:54
Why is Facebook a bad example? – petercpwong Dec 12 '10 at 15:33
Oh, it's not bad; I'm just saying that most websites will never experience load on the level Facebook or Google experience. – Bill Dec 12 '10 at 16:09
facebook now has 60,000 servers datacenterknowledge.com/the-facebook-data-center-faq-page-2 – Steven A. Lowe Dec 12 '10 at 18:34
up vote 2 down vote

Here is a video and presentation describing some of the stuff the engineers at facebook did to scale up.

share|improve this answer
up vote 1 down vote

Content Delivery Network

Facebook uses static content delivery network for serving photos and videos. Check the source of any image in facebook and you will see something like fbcdn.net.

share|improve this answer
Can you atleast explain what is CDN, in context of Facebook? For instance, like the link blog.facebook.com/blog.php?post=2406207130 – Kanini Dec 12 '10 at 14:02
@Kanini You can check-en.wikipedia.org/wiki/Content_delivery_network – Gulshan Dec 12 '10 at 14:24
What's the difference between Cloud Computing and CDN? – petercpwong Dec 12 '10 at 23:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.