Organization Map Reduce?

Question

big data

Hello.
At our startup, we have several tasks that come down to the classic Map Reduce paradigm. We want to spread tasks across multiple servers. What solutions are worth looking at? What you should pay attention to?
Now I'm thinking about two solutions: 1. the famous Hadoop - they talk a lot about it and it is heard everywhere, but do we need it - the tasks seem to be not difficult. 2. it's easy to write map reduce by hand with the help of akka and scatter it over the servers and not take a steam bath.
I must say that everyone in the team has experience with akka, but no one has experience with Hadoop - but the team leader “itches” to try Hadoop. I would like to know if it is worth it and dissuade with arguments and alternatives.
PS: I forgot to add, we use java + scala.

Answer 1

Hadoop is designed for large clusters of medium and weak machines. If you have several powerful servers, then with Hadup you will find a lot of meaningless gestures (code), a few unpleasant restrictions and, perhaps, 1-2 extra data copying per task. I don’t know anything about Akka and other solutions, but in my opinion Hadoop does not fit very well in this case.

Answer 2

You can also look at MongoDB Map-Reduce , here kost_bebix shared his experience Experience using mongodb to calculate statistics , but since then, probably a lot has changed.