The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models.

learn more… | top users | synonyms

1
vote
0answers
141 views

Best practices for dashboard of near real-time analytics

I’m currently building a dashboard to view some analytics about the data generated by my company's product. We use MySQL as our database. The SQL queries to generate the analytics from the raw live ...
1
vote
0answers
78 views

Improve communication between controller and trackers in a Twitter fetcher tool using RabbitMQ or Apache Flume

I've been working for a time with some researches developing a tool to fetch tweets from Twitter and process them in some way. The first prototype "worked" but became a pain as we used sockets to ...
0
votes
0answers
12 views

Is there MPI_Reduce in MapReduce?

Is there a tree/logarithmic reduction function in Map Reduce similar to MPI_Reduce? From diagrams it appears the default MR reducer uses the key assigned to data during the mapper operation to direct ...
0
votes
0answers
25 views

How to use Hadoop HBase with Spring Boot without knowing the schema of the database ahead of time

I have created a basic application with spring boot and HSQL which connects an in-memory HSQL database with an angularjs front end using spring-boot and spring JPA with Hibernate. I am now trying to ...
0
votes
0answers
60 views

Execute shell script as one of the steps on EMR

We are thinking to migrate our Hadoop infrastructure from Data Center to AWS EMR. As some of the tasks / stages in ETL process are dependent e.g. flow is like Map Reduce job will generate data Shell ...
-3
votes
0answers
34 views

Optimizing cartesian product using keys in spark

To avoid computing all possible combinations, I'm trying to group values according to a certain key, and then compute the cartesian product of the values for each key, i.e.: Input [(k1, v1), (k1, ...