Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am doing an internship on Big Data technologies so I am new to this area. My question is about the use of NoSQL in the Big Data architecture. Do we need always to use a distributed storage (like HDFS in the case of Hadoop) then to put on top a NoSQL databases (like Hbase )?

I find it difficult to understand the typical BIG Data architecture specially for unstructured data .

If you can help me to see better I will be so thankful.

share|improve this question
17  
There is no such thing as NoSQL. There is just a whole bunch of new database technologies with completely different design philosophies and use cases, and all they have in common are things they also have in common with SQL. You can't evaluate "NoSQL" as a whole. You need to evaluate each database system individually. –  Philipp Mar 25 '13 at 9:34
    
@Philipp Thank you for your answer , but to manipulate large volume of data do they share the fact of using a cluster distribution ? If it is not the case what other technics they can use to proceed ? Riak use the distibuted storage but what is the case for MangoDB and Oracle Nosql for example ? –  soufiane.989 Mar 25 '13 at 10:40
1  
NoSQL is shorthand for non-relational database. Some have suggested that "no" should in this case stand for "not only." –  Md. Mahbubur R. Aaman Mar 28 '13 at 12:56

4 Answers 4

up vote 0 down vote accepted

Most big data technologies are distributed by design, because the idea of big data is that your database is too big and too frequented to be handled by one server alone.

But most distributed new database technologies (some of which might be called NoSQL) do not use a separate plattform like HDFS as distributed storage backend but come with their own. MongoDB, to name just one example, runs as different processes on multiple servers which communicate with each other to form clusters and shards.

share|improve this answer

From http://en.wikipedia.org/wiki/NoSQL

NoSQL database systems are often highly optimized for retrieval and appending operations and often offer little functionality beyond record storage (e.g. key–value stores). The reduced run-time flexibility compared to full SQL systems is compensated by marked gains in scalability and performance for certain data models.

In short, NoSQL database management systems are useful when working with a huge quantity of data when the data's nature does not require a relational model. The data can be structured, but NoSQL is used when what really matters is the ability to store and retrieve great quantities of data, not the relationships between the elements.


  • Do we need always to use a distibuted storage (like HDFS in the case of Hadoop) then to put on top a NoSQL databases (like Hbase )?

For large scale it is better to use distributed storage with NoSQL. But for small scale you can use ordinary storage system.


  • I find it difficult to understand the typical BIG Data architecture specially for unstructured data .

From http://en.wikipedia.org/wiki/Unstructured_data

Unstructured Data (or unstructured information) refers to information that either does not have a pre-defined data model and/or does not fit well into relational tables. Unstructured information is typically text-heavy, but may contain data such as dates, numbers, and facts as well. This results in irregularities and ambiguities that make it difficult to understand using traditional computer programs as compared to data stored in fielded form in databases or annotated (semantically tagged) in documents.

From http://en.wikipedia.org/wiki/Big_data

In information technology, big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications.

Big data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data in a single data set. With this difficulty, new platforms of "big data" tools are being developed to handle various aspects of large quantities of data.

To understand BIG Data architecture specially for unstructured data, have a look how Giants work with Big Data.

For example, Google

For example, IBM

enter image description here

For example, Facebook

enter image description here

share|improve this answer

Distributed storage is an implementation detail more than a necessary thing to understand to handle BIG data. I would focus on grasping unstructured data first, which is more of a feature and more important to grok.

share|improve this answer

Using Distributed storage is helping to get the proper utilization of NoSQl technologies, if you need to utilize or take the full advantage of these technologies, with big data you must have to use distributed storage.

share|improve this answer
    
without an explanation, this answer may become useless in case if someone else posts an opposite opinion. For example, if someone posts a claim like "with big data you must not use distributed storage", how would this answer help reader to pick of two opposing opinions? Consider editing it into a better shape –  gnat Mar 28 '13 at 14:27

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.