Answer the question
In order to leave comments, you need to log in
How to make requests in a multithread with a certain period of time?
Good afternoon. I ran into a problem, googled, but did not find a solution to my particular problem.
You need to write a script so that it makes requests every certain period of time and then, when the answer comes, it adds the request response to the array. Let's say I have 10 sessions. And the required time interval between requests to me: 0.2 seconds. So I need 1 of the sessions to send a request every 0.2 seconds (If no response was received, and 0.2 seconds had already passed, a new request for another session began, but the answer in this thread was still expected). And as a result, the script sends 10 requests in 2 seconds from different sessions and then the cycle starts again
Read it here: Python multithreading Requests, what blocks processing?, but as I understand it, again, here, until the response to the request comes, a new request will not start.
I tried like this:
def multi_parse(self):
self.ticker = 1
self.parse_info = []
with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
executor.map(self.thread_parse, self.multisession)
def thread_parse(self, session):
print(f'Делаю запрос #{self.ticker} {t.time()}')
thread_local.session = session
multi_session = thread_local.session
req = multi_session.get("", timeout=5).json()
self.parse_info.append(req)
t.sleep(0.2)
print(f'Завершил запрос #{self.ticker}')
self.ticker += 1
Answer the question
In order to leave comments, you need to log in
I would do this: I would
create a pool of queues (10 pieces), for each I would launch a thread that would wait for a message from the queue and run the desired request. And in the main thread, once every 0.2 seconds, I would fire 1 request into one queue in a cycle, at each iteration moving on to the next queue. In total, all requests are coordinated through a common "writer", and "readers" process responses each in their own thread.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question