Answer the question
In order to leave comments, you need to log in
How to get the sequence number of a process?
Good afternoon.
Question about multiprocessing and tqdm in python on windows.
The code works but there are 2 problems.
1 - tqdm needs to set the position number, otherwise, instead of fixed progressbars, a scrollbar is obtained, new lines are simply added.
2 - for multiprocessing. how to get process sequence number so that it can be inserted into tqdm position number
idea with generator
def gen():
a=[1,2,3,4,5,6,7,8,9,10]
yield a
from multiprocessing import Pool
from multiprocessing.pool import ThreadPool
def GetData(part_hrefs):
df = []
for item in tqdm(part_hrefs, position=2, desc="h", leave=False):
return(df)
def main():
with open('file.txt', 'r') as file:
TotalLinks =file.readlines()
parts = [round(len(TotalLinks)/10)*i for i in range(10)]
parts.append(len(TotalLinks))
names = [TotalLinks[parts[i]:parts[i+1]] for i in range(10)]
pool = ThreadPool(10)
l = pool.map(GetData, names)
itog_data=Reduce_2(l)
pd.DataFrame.to_csv(itog_data,'output.csv')
if __name__=='__main__':
print ("start")
main()
print ("finish")
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question