Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

This question already has an answer here:

Recently I read a lot about noSQL DBMSs. I understand CAP theorem, ACID rules, BASE rules and the basic theory. But didn't find any resources on why is noSQL scalable more easily than RDBMS (e.g. in case of a system that requires lots of DB servers)?

I guess that keeping constraints and foreign keys cost resources and when a DBMS is distributed, it is a lot more complicated. But I expect there's a lot more than this.

Can someone please explain how noSQL/SQL affects scalability?

share|improve this question

marked as duplicate by gnat, Martijn Pieters, Kilian Foth, Joachim Sauer, thorsten müller Apr 9 '13 at 11:28

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

4  
"I guess that keeping constraints and foreign keys cost resources and when a DBMS is distributed, it is a lot more complicated. But I expect there's a lot more than this." - Actually, that's it. More accurately, that's the one common characteristic that makes most NoSQL solutions more scalable than their SQL cousins (for certain data models). But NoSQL is an extremely vague term, different families of NoSQL databases have different characteristics that make them more scalable. –  Yannis Rizos Apr 8 '13 at 21:49
5  
Of course SQL databases scale perfectly well into trillions of records, they just need some expertise to design and set them up that application developers don't have. And generally a fairly expensive set of hardware and licenses. –  HLGEM Apr 8 '13 at 22:19
    
+1 for Nice Question. –  Md. Mahbubur R. Aaman Apr 9 '13 at 6:06
    
3  
In my opinion this question is not a duplicate of either of those. The mongodb question is (besides a bad title making it seem more specific) asking something else which is in fact more general. Voted to reopen. –  Joeri Sebrechts Apr 15 '13 at 14:31

4 Answers 4

up vote 14 down vote accepted

noSQL databases give up a massive amount of functionality that a SQL database gives you by it's very nature.

Things like automatic enforcement of referential integrity, transactions, etc. These are all things that are very handy to have for some problems, and which require some interesting techniques to scale outside of a single server (think about what happens if you need to lock two tables for an atomic transaction, and they are on different servers!).

noSQL databases don't have all that. If you need that stuff, you need to do it yourself, but if you DON'T need it (and there are a lot of applications that don't), then boy howdy are you in luck. The DB doesn't have to do all of these complex operations and locking across much of the dataset, so it's really easy to partition the thing across many servers/disks/whatever and have it work really fast.

share|improve this answer

It's not about NoSQL vs SQL, it's about BASE vs ACID.

Scalable has to be broken down into its constituents:

  • Read scaling = handle higher volumes of read operations
  • Write scaling = handle higher volumes of write operations

ACID-compliant databases (like traditional RDBMS's) can scale reads. They are not inherently less efficient than NoSQL databases because the (possible) performance bottlenecks are introduced by things NoSQL (sometimes) lacks (like joins and where restrictions) which you can opt not to use. Clustered SQL RDBMS's can scale reads by introducing additional nodes in the cluster. There are constraints to how far read operations can be scaled, but these are imposed by the difficulty of scaling up writes as you introduce more nodes into the cluster.

Write scaling is where things get hairy. There are various constraints imposed by the ACID principle which you do not see in eventually-consistent (BASE) architectures:

  • Atomicity means that transactions must complete or fail as a whole, so a lot of bookkeeping must be done behind the scenes to guarantee this.
  • Consistency constraints mean that all nodes in the cluster must be identical. If you write to one node, this write must be copied to all other nodes before returning a response to the client. This makes a traditional RDBMS cluster hard to scale.
  • Durability constraints mean that in order to never lose a write you must ensure that before a response is returned to the client the write is flushed to disk.

To scale up write operations or the number of nodes in a cluster beyond a certain point you have to be able to relax some of the ACID requirements:

  • Dropping Atomicity lets you shorten the time tables (sets of data) are locked. Example: MongoDB, CouchDB.
  • Dropping Consistency lets you scale up writes across cluster nodes. Examples: riak, cassandra.
  • Dropping Durability lets you respond to write commands without flushing to disk. Examples: memcache, redis.

NoSQL databases typically follow the BASE model instead of the ACID model. They give up the A, C and/or D requirements, and in return they improve scalability. Some, like Cassandra, let you opt into ACID's guarantees when you need them. However, not all NoSQL databases are more scalable all the time.

The SQL API lacks a mechanism to describe queries where ACID's requirements are relaxed. This is why the BASE databases are all NoSQL.

Personal note: one final point I'd like to make is that most cases where NoSQL is currently being used to improve performance, a solution would be possible on a proper RDBMS (mysql doesn't count) by using a correctly normalized schema with proper indexes. As proven by this very site (powered by MS SQL Server) RDBMS's can scale to high workloads, if you use them properly. People who don't understand how to optimize RDBMS's should stay away from NoSQL, because they don't understand what risks they are taking with their data.

share|improve this answer
    
While I agree that some NoSQL stores replace ACID with BASE, that still isn't a common feature for all stores that fall under the NoSQL "category", which is an ill-defined in the first place. After a while, the interpretation of the term switched from "No SQL" to "Not Only SQL", but as many such databases still do JOINs or have started implementing SQLesque dialects, Mark Madsen has re-coined the term to mean something else in his history of databases in no-tation: "No, SQL" ;-) –  Lukas Eder Dec 15 '13 at 8:41

It's true that NoSQL databases (MongoDB, Redis, Riak, Memcached, etc.) don't maintain foreign key constraints, and atomic operations must be more explicitly specified. It's also true that SQL databases (SQL Server, Oracle, PostgreSQL, etc.) can be scaled to handle very large performance requirements by seasoned DBAs.

NoSQL databases allow seasoned programmers, who are well aware of race-conditions and atomic operations, to forego a large amount of processing only required in a small percentage of today's web application code. NoSQL databases certainly have atomic operations and most all transactional requirements present in SQL databases can also be obtained NoSQL databases. The difference is the level of abstraction. NoSQL databases remove the higher levels of abstraction and hand that capability to the application programmer, thereby resulting is faster code overall with the increased probability of data corruption by unseasoned programmers.

As a result we are much more likely to see NoSQL databases being used more and more heavily in the web application space, where development time and performance are very important. Financial and corporate software is likely to retain it's SQL heritage because hardware performance is relatively cheap, they have seasoned DBAs on-hand, and the increased risk caused by unseasoned programmers is not palatable.

share|improve this answer
    
I'm not sure I agree with the part about atomic transactions, in the ACID sense (although it's difficult to comment on "NoSQL", since it's up for debate what exactly we mean). Most of the performance gains in "typical" NoSQL DBs are achieved through loosening of consistency guarantees (see: eventual consistency, ACID vs. BASE). If eventual consistency is good enough for an application (and it often is), then this allows for much more efficient horizontal scaling. –  Daniel B Apr 9 '13 at 6:19

From http://www.ibm.com/developerworks/cloud/library/cl-nosqldatabase/index.html?ca=drs-

Scalability is the system that should be able to support very large databases with very high request rates at very low latency.

NoSQL systems have a number of design features in common:

  • The ability to horizontally scale out throughput over many servers.
  • A simple call level interface or protocol (in contrast to a SQL binding).
  • Support for weaker consistency models than the ACID transactions in most traditional RDBMS.
  • Efficient use of distributed indexes and RAM for data storage.
  • The ability to dynamically define new attributes or data schema.

Why relational databases may not be optimal for Scaling

In general, relational database management systems have been considered as a "one-size-fits-all solution for data persistence and retrieval" for decades. They have matured after extensive research and development efforts and very successfully created a large market and solutions in different business domains.

The ever-increasing need for scalability and new application requirements have created new challenges for traditional RDBMS, including some dissatisfaction with this one-size-fits-all approach in some web-scale applications. The answer to this has been a new generation of low-cost, high-performance database software designed to challenge dominance of relational database management systems. A big reason for the NoSQL movement is that different implementations of web, enterprise, and cloud computing applications have different requirements of their databases — not every application requires rigid data consistency, for example.

Another example: For high-volume websites like eBay, Amazon, Twitter, or Facebook, scalability and high availability are essential requirements that cannot be compromised. For these applications, even the slightest outage can have significant financial consequences and impacts customer trust.

From http://dba.stackexchange.com/questions/4508/what-does-horizontal-scaling-mean

Horizontal Scaling is essentially building out instead of up. You don't go and buy a bigger beefier server and move all of your load onto it, instead you buy 1+ additional servers and distribute your load across them.

Horizontal scaling is used when you have the ability to run multiple instances on servers simultaneously. Typically it is much harder to go from 1 server to 2 servers then it is to go from 2 to 5, 10, 50, etc.

Once you've addressed the issues of running parallel instances, you can take great advantage of environments like Amazon EC2, Rackspace's Cloud Service, GoGrid, etc as you can bring instances up and down based on demand, reducing the need to pay for server power you aren't using just to cover those peak loads.

Relational Databases are one of the more difficult items to run full read/write in parallel.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.