Answer the question
In order to leave comments, you need to log in
How to correctly and simply implement multithreading in Python?
You need to run 100+ threads in parallel.
Example:
file = open('file.txt')
lines = file.readlines()
def just():
for line in lines:
sleep(1)
print(line)
for i in range(0, 100):
t = threading.Thread(target=just)
t.start()
Answer the question
In order to leave comments, you need to log in
def just(lines):
for line in lines:
sleep(1)
print(line)
file = open('file.txt')
lines = file.readlines()
step = len(lines) // 100
threads = []
for i in range(0, len(lines), step):
t = threading.Thread(target=just, args=(lines[i:i+step],))
t.start()
threads.append(t)
for t in threads:
t.join()
Python uses Global Interpreter Lock and therefore does not have efficient parallel multithreading.
Pythonic multithreading can only be used to organize any controlling threads.
For parallelization, you need to use multiprocessing.
Reading from a file is not parallelized (at least on the same machine, there will be no threads on different machines), since the file is not random access data.
Scattering reading between threads leads to a quadratic algorithm in python, instead of a linear one. Hence the slowdown.
I would recommend picking a smarter task to parallelize. And then start with an example
import multiprocessing
def proc(a):
<…>
return something
pool = multiprocessing.Pool(3) # ну может 4.
result = pool.map(proc, list(range(100)))
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question