S
S
S1riyS2022-01-09 11:11:37
Python
S1riyS, 2022-01-09 11:11:37

Why is docker exec taking so long and is there a way to speed it up?

I create a container:

docker run -it -d --env TIMEFORMAT=\n%3R --name python_container -v (текущая директория)\temp:/temp docker_python_runner bash


Through the exec command, I run the script inside the container:
docker exec -i python_container bash -c "time python temp/temp_file.py" 2>&1


Let's say it will be a script that simply adds 2 numbers (for example, stdin: 5 10).

Conclusion:
15
0.025


That is, the result of the program: 15 , script execution time: 0.025 , while the total command execution time execis 1.0805566310882568

I run all commands from a python script using the library subprocess:
result = Popen(command, stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=True, encoding='utf-8')
stdout, stderr = result.communicate(input=stdin, timeout=5)


My operating system is Windows

In general, all this is done to test user code => I run the same execcommand 15 - 20 times. How to speed up its execution and maybe there is a way to somehow optimize this particular moment?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Alexander Karabanov, 2022-01-09
@S1riyS

Under Windows, you have to run Linux in a virtual machine to run a Docker container (after all, Docker is built on Linux-based Napespace and Cgroups), and this requires a lot of resources.
Try to enter the container first: docker exec -it python_container bash
And then run the script: time python temp/temp_file.py
Try to use WSL2 - this should change the situation for the better, but it will still be far from native performance.

R
Rodion, 2022-01-09
@rodion4dev

This is probably. does not pull on the answer, but the execution of the command inside the running container, in my opinion, should be heavy.
I haven’t studied this issue in depth, but this is a kind of virtualization: only they communicate with a virtual machine at the level of ordinary protocols, and with a Docker container they communicate with some of their own means and at the application level (as far as I understand it).
As for acceleration - try to make a script that will fall asleep for a while and, waking up, look in volume, pull data from there, and add them up; and write the result to stdout.
Well, by the way, opening a subprocess is also not an easy operation, which is probably why the speed is so high. It is better to start one main process (without children) and perform all necessary operations in it.
Or another option is to assemble your own image, sew ENTRYPOINT there in the form of your script, and transfer data there for calculation.
In general, there are many options.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question