M
M
MicleI2018-05-17 16:21:50
Hadoop
MicleI, 2018-05-17 16:21:50

Auto-add nodes in hadoop, is it possible?

I tested the fault tolerance of Hadup and the following question arose:
I have a master and 2 nodes, everything works well
https://ibb.co/fe3zFd
But, for example, one of the nodes has fallen (i.e. the number of replications will decrease in theory) , I understand correctly?
Is it possible to make it so that after the death of one of the nodes, another one is automatically connected / created (with pre-entered parameters) and all information from other replicas (datanode) would be copied to it.
And the same with the master, if it dies, is it possible to implement auto-connection of another, with subsequent copying of information to it from the secondarynamenode or, for example, from some backups?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
mossat, 2018-08-14
@mossat

"But, for example, one of our nodes has crashed (that is, the number of replications will decrease in theory), do I understand correctly?" - not really. The number of replications for only those blocks that were stored on this node. In addition, replication for the lost node starts almost immediately. Therefore, "... after the death of one of the nodes, another one was automatically connected / created (with pre-entered parameters) and all information from other replicas (datanode) would be copied to it." - happens and so, only for working nodes.
In summary, this is not necessary.
Also, the number of StandBy NameNode and HBase Master is set in advance and can be more than one, two, three.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question