Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've read that Facebook uses this technique DB sharding to manage its data volume and that the idea of cloud computing is elastic resources. So I'm wondering does the cloud instance take care of sharding for you automatically? Or do you still have to do that part manually? And if you do have to do it manually - then the DB doesn't seem very elastic to me.

share|improve this question

closed as off-topic by iterationx, GlenH7, gnat, thorsten müller, Dynamic Dec 20 '13 at 19:41

  • This question does not appear to be about software development within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

Doesn't Cassandra auto shard? – JeffO Jul 24 '11 at 21:42
This question appears to be off-topic because it fails to demonstrate a minimal understanding of the techniques or technologies underlying the question. – GlenH7 Dec 20 '13 at 13:58
up vote 1 down vote accepted

You sure can have automatic sharding - try out ScaleBase (disclosure - I work there). You can install it on any cloud environment (or physical environment). BTW - Sharding is not something that any RDBMS provides - Oracle and MS-SQL solutions are not sharding - and at least Oracle's RAC solution doesn't really scale, nor does it run on cloud environments. MySQL has MySQL cluster - which does scale, but of course it's not free...

share|improve this answer
had a look at ScaleBase -- looks good -- is it open source? – treecoder Jul 25 '11 at 7:54
Sounds awesome! – iterationx Jul 25 '11 at 18:53

Sharding in practice is complicated because its rarely possible to automatically (and thus elastically in the cloud) partition your DB to load balance queries effectively. In practice you always land up with hot zones and one of the few ways around is to architect the db schema anticipating such hot zones (which is something cloud elasticity won't give you).

The other problem with sharding is the loss of availability of a shard on failure of the node carrying the shard. Typically, you have to give up either strict consistency or availability when you are sharding. You may want to look at Brewer's CAP theorem which points out the trade-offs you have to make when building a distributed system. In practice, distributed systems tend to get complicated with intertwined systems with different availability and consistency properties.

Bottomline... there's little automatic here. Proper design of distributed components can get you very far.

share|improve this answer

This feature is not of the cloud instance but of the DBMS. Oracle & SQL Server do it by free MySQL uses plugins.

You can use this on any kind of app not just cloud based ones.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.