M
M
MrKrot2018-04-09 23:28:29
Python
MrKrot, 2018-04-09 23:28:29

Why, in my implementation of the prefork server, does the RPS drop with an increase in the number of forked processes?

I had to write my own prefork server implementation:

import os
import socket

class WebServer:
    pids = []

    def __init__(self, n_cpu, address, listeners, buffer, handler):
        self.cpu_count = n_cpu
        self.address = address
        self.listeners = listeners
        self.buffer = buffer
        self.handler = handler
        self.server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)

    def exec(self):
        self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.server.bind(self.address)
        self.server.listen(self.listeners)
        for i in range(self.cpu_count):
            pid = os.fork()

            if pid != 0:
                self.pids.append(pid)
            else:
                print("Created worker PID: {}".format(os.getpid()))
                while True:
                    client, client_addr = self.server.accept()
                    request = client.recv(self.buffer)
                    if len(request.strip()) == 0:
                        client.close()
                        continue

                    response = self.handler.handle(request)

                    client.send(response.build())
                    client.close()

        self.server.close()

        for pid in self.pids:
            os.waitpid(pid, 0)

You can also look at the github: https://github.com/kiryanenko/TP-Highload/tree/1.0
During AB testing, I discovered the sad fact that with an increase in the number of forked processes, RPS falls:
n_cpu | RPS
-------------
1     | 3301
2     | 2140
3     | 1762
4     | 1709

Also, the program with 1 process uses it by 100%, and with 2 or more uses it by 60%.
I can't understand why.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexander, 2018-04-10
@alexr64

I do not have at least some knowledge in python, but it seems to me that the threads must also be forcibly spread across the cores.
Also, keep in mind that "virtual" cores, as you put it, are redundant elements of logic on physical ones. That. the performance of logical cores is not always equal to physical cores and is generally (except in rare cases, such as when a thread running on a physical core is waiting for a long time) much lower.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question