N
N
Nikolai Markov2018-10-03 17:36:47
linux
Nikolai Markov, 2018-10-03 17:36:47

Why do multiple processes start to interfere with each other on a multi-core processor (28 cores)?

I have jenkins on a powerful wheelbarrow (28 cores / 56 logical processors, 128Gb RAM, hdd disks)
It runs rails application tests.
One assembly starts 12 parallel threads to run tests. At the same time, the percentage is loaded by 25%. Memory goes 10Gb. The assembly is completed in ~6 minutes. Disk utilization increases only when building docker images (~30sec of the total build time), and is minimal when running tests.
And here an incomprehensible situation arises. It would seem that there are still enough CPU resources to run at least 3 more builds. But when adding additional assemblies, the build time of all builds increases. So, 4 simultaneous builds are going to be 12! minutes.
Can anyone explain why processes that should not seem to affect each other and that have enough resources behave this way?

Answer the question

In order to leave comments, you need to log in

3 answer(s)
V
vaut, 2018-10-03
@vaut

Why won't it have an effect?
Let's start with the fact that at partial load, dynamic overclocking of the processor frequency can work. And at full drop to the base frequency.
Further, the bandwidth of RAM and disks also has a limit.
Well, in the end, the cache starts to work less efficiently.
I also forgot about the fact that hyperthreads work more slowly and, accordingly, half of them simply start to be considered slower.

D
Dimonchik, 2018-10-03
@dimonchik2013

"should not affect" - is it in your head or in Jenkins code?
and the rails application has the impression that it is Ruby on Rails with its own GIL and that's all

S
Stanislav Bodrov, 2018-10-03
@jenki

And what was expected from single-threaded applications? Moreover, jenkins has never been distinguished by its adequacy in terms of resource consumption.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question