Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am working on the application which in near future (hopefully) will have to process tens or hundreds of thousands of items (item is currently one row in relational database table) per second.

If I understood it right NoSQL databases are superheroes of data retrieval thanks to map-reduce magic. But will NoSQL databases provide extra performance in comperatison to relational ones in my case?

share|improve this question

2 Answers 2

up vote 1 down vote accepted

100,000's of rows (of reasonable size) is a small enough that it can probably fit in main memory of one or possibly a few compute nodes. It's probably going to be in your best interest to design your application to operate on large chunks of the dataset at a time (shard your dataset by primary key), and use a database to persist the information periodically.

If you really need a higher level of consistency (say, you are doing financial transactions), then you don't have much choice but to use a regular database (relational or document) which provides the necessary integrity and shard your dataset at the application level.

share|improve this answer

I remember there is a feature in RavenDb that indexes are updated in separate threads. The advantage of this is higher performance of modification operations and disadvantage is that index is not always up-to-date. But I am not sure whether some relational databases have this feature.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.