Do I need a professional graphics card with ECC to train neural networks?

Question

Video cards

As I understand it, a micro-glitch in the memory of a video card can affect the result of training a neural network. Is this true and how critical is it? What video cards are used in reality to train neural networks?

Answer 1

Yes, failures do happen, with long-term training (a week or more), especially in conjunction with several cards - this quite affects itself. How critical? depends on the software, if you log there every epoch, then you can try to get out and catch the moments of failure, restarting the learning process from the point of failure.