How good is this architecture?

Question

Python

There will be a certain service in which the user can upload a media file, and the service itself must change some properties in it.

Will:

N-master servers. Let's call M*. These servers communicate with the user. Accepting the file from the user and the necessary add-ons on the file.
N-task servers. Let's call W*. These servers will make those changes in files.

The entire implementation will be in python+twisted.

Scheme:

When a user uploads a file to the M server, the file from the M server is sent to the most “free” W server, getting into the queue. The W server reports the successful completion of the operation with the ID of the queue (for communication. Find out the status or when receiving a notification).

After completing the changes to a file on server W, this server notifies M of the success of the task. For all data exchange I want to write my own service.

1. How normal would such a scheme be?
2. Files can be large, 1GB for example. The concern is that you have to "drive" files back and forth, from server M to W (and maybe vice versa, although this step can be bypassed by giving the user the result of the file directly from server W).

Answer 1

1) Normal circuit. Something similar is used by many, for example, Flickr.
2) so what, what will they race? Interserver links are still faster than WAN. And distributing files from workers is a bad idea. Workers should be able to die without affecting anything, only increasing file processing time.
> For all data exchange I want to write own service.
What for? Use a queue server.

Answer 2

I would also single out the Q server, which will contain and serve the queue, raw files will be added to it. Servers W will look, there are still tasks to be executed, and after execution, the result will be added to the same server W, in the queue of finished files.
In my opinion, it will be easier to scale this way, and the logic of the M servers will be a little simpler.