Answer the question
In order to leave comments, you need to log in
Multi-threaded writing to the database. How?
Good afternoon, %username% .
I'm trying to get the script parser to work multi-threaded.
Now it works sequentially:
1. Selects links to sites from the database.
2. Cycle through each link by calling the get_info()
function 3. The get_info() function searches the site for the information I need and at the end makes an SQL query to UPDATE the record with this link.
How to speed it all up?
I tried a bunch of examples using thread, threading, multithreading, multiprocessing modules, everything only aggravates the situation (processing takes longer (!) Than sequentially).
It was not possible to find a suitable and good example, please poke your nose where to look, because it turns out that I'm missing something. Otherwise, why all this bunch of modules that work longer :)
Many thanks in advance.
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question