Answer the question
In order to leave comments, you need to log in
Why does threading stop after starting multiple threads?
I have a block of code:
result = []
def parse(i):
try:
request_phrase = ' '.join(
[words_from_img[x] for x in range(i, i + 10)])
doujin_page = requests.get(f'https://somesite.com/search?search={request_phrase}')
soup = BeautifulSoup(doujin_page.content, 'html.parser')
problem_ids = [i.text.split()[-1] for i in soup.find_all('span', {'class': 'prob_nums'})]
for id in problem_ids:
if id not in result:
result.append(id)
except Exception as E:
pass
thread_pool = []
for i in range(0, len(words_from_img)):
thread = threading.Thread(target=parse, args=(i,))
thread_pool.append(thread)
thread.start()
for thread in thread_pool:
thread.join()
Answer the question
In order to leave comments, you need to log in
First, we need to remove the catching of wide exceptions. Secondly, you need to add more debug output and run under the debugger to understand what exactly "just stops running" means. The obvious assumption is that somesite.com detects parsing, blocks access, an exception is thrown in the parse function, which you silently swallow and the thread exits.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question