Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 4 down vote favorite
8

I bury a good deal of my ideas for fear that I don't know enough about scaling web applications and high-traffic websites. That said, I'd like to know of any general topics to research in order to ensure that your web app doesn't break / slow down when you start getting to Twitter-level traffic.

I'm looking for research topics with, preferably, additional resources.

For example: Make sure you optimize SQL queries (see High Performance MySQL Optimization)

share|improve this question

1 Answer

up vote 7 down vote accepted

Likely Suspects

Most likely, the things you want to pay attention to are:

  • your caching tier,
  • your database tier,
  • your general architecture's design (if it's not designed to scale, it just won't).

Further Reading

Words of caution about the resources below:

  • Some are generic resources, others are specific to one stack or language but still provide insight on the types of issues to deal with and how to (try to) address or mitigate them.
  • Some provide general information of scalability.
  • Some give only hints on basic pipeline and programming optimizations.
  • Some are sponsored or hosted by industry-actors, in which case conclusions and comparisons may or may not be accurate and/or impartial.

Research papers

Some papers' links may be unaccessible for you if you don't have a subscription to some publication networks.

Books

Articles, Blogs and Web Resources

Famous Last Words

While I can understand your original intuition that you should bury some projects out of fear that they won't adapt well and survive, I think that's the wrong approach. While the "build it and they will come" approach can be wrong, the "build it only once it's perfect" counter-part can be just as wrong.

You don't know if your projects will reach the critical mass where they actually need to care about these issues, so why not build them anyways?

Maybe you'll fail at first, but then you'll have learned and your next product will fare better. It's better to try it this way than to one day really have to dive in the deep end of the pool without a net, for a real product. So go ahead and give these ideas a chance.

While it's true that it's hard to make an application scalable when it hasn't been originalyl designed to be, at least you'll have an application that needs to be modified for scalability. In my book it's better than no application at all.

share|improve this answer
A very good answer. Informative and inspirational to boot. Thank you. – user745434 Jul 1 '12 at 23:09
1  
@user745434: You're welcome. Glad you like it. The right way to show recognition for merit on StackExchange is by voting (on questions and answers, or even comments) and by accepting answers. If my question is the one you find the most helpful, and if you find it answers your original question correctly, you can upvote and accept it. Though for now I'd recommend you wait a bit longer, as others might come up with additional answers in the course of the next day or even week. (Note that you can vote on multiple answers, but accept only one). – haylem Jul 1 '12 at 23:15
1  
Your answer is very good, but i might add that making benchmarks and small prototypes will help a lot, looking at the cold numbers can send you into a nice quest of understanding. – alfa64 Jul 1 '12 at 23:17
@alfa64: indeed. There's no point in reading all of the above and then thinking it will necessarily solve your problems. You're going to work on specific hardware, with specific platforms, specific software, specific environmental issues, and specific usage patterns. You need to identify your bottlenecks and know where they are. Also, the sad thing is that sometimes optimizing a section that IS NOT a bottleneck will make another bottleneck worth (think of traffic lights, for instance: they may look like they slow you down in one spot, but it's to optimize a wider design). – haylem Jul 1 '12 at 23:33
@haylem I will give it a couple of days. I just wanted to thank you for a great answer. – user745434 Jul 1 '12 at 23:35
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.