4
votes
3answers
1k views

Why Big Data Needs To Be Functional?

I started working on a new project lately related to Big Data for my internship. My managers recommended to start learning functional programming (They highly recommended Scala). I had a humbled ...
0
votes
2answers
102 views

finding patterns into logs

I am trying to put together a way to figure out if incidents occurred based on log content. Typically, on log (or db table) would contain a list of transactions composed as follow: {Timestamp} ...
0
votes
2answers
412 views

Algorithms for data structures in distributed system

The hash table data structure can be easily spread across multiple machines with a simple algorithm to distribute the keys: machine_to_query = item_key % machine_count When you want to read and ...
1
vote
2answers
142 views

Using replacement to get possible outcomes to then search through HUGE amount of data

I have a database table holding 40 million records (table A). Each record has a string a user can search for. I also have a table with a list of character replacements (table B) i.e. i = Y, I = 1 ...