What are the approaches to optimizing cluster computing?

Question

Distributed Computing

Initial data:

1. There is a large (from 1000 cores) deterministically heterogeneous (computing units are different) cluster.
   
2. There are users (from 100 to 500) who set computational tasks.
   
3. Tasks are heavy calculations (without working with a database, etc.), which:
   
   • parallel very well;
     
   • their duration is non-deterministic (from several seconds to hours);
     
   • the job can be interrupted/restarted at any time.
     
   
   
4. Task priorities should be implemented according to one of the approaches:
   
   • each task can have a priority (for example, from 1 - the highest to 10 - the lowest) - in this case, the user sets a priority for his task;
     
   • each user can have a priority - in this case, the administrator determines the priority of the user, and the tasks of one user are executed in the order they are placed in the task queue (i.e., only 1 task is executed from each user at the same time);
     
   • mixed approach - a combination of the previous ones.
     
   
   

It is required to choose an appropriate mathematical model for efficient use of the cluster, minimizing the idle time of the nodes. I'm also interested in the availability of a ready-made implementation.

Answer 1

I think you should look towards Open MPI .