D
D
Danil2021-08-11 13:42:21
Python
Danil, 2021-08-11 13:42:21

Processes during parsing remain in the background and are not terminated, thereby leading to a crash in the hosting\multiprocessing.pool system, what is the reason?

There is a site parser that receives the necessary information from a foreign site, the parser itself works correctly, but when introducing the multiprocessing function, the processes do not end on their own, what could be the reason?
Ready to answer questions about what he is responsible for.

def iaai(self):
        try:
            with Pool(40) as p:
                html = Links(self.car, self.from_year)
                links = html.parse_html()
                p.map(self.get_info_car, links)
        except Exception as e:
                exc_type, exc_obj, exc_tb = sys.exc_info()
                fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
                print("IAAI", exc_type, fname, exc_tb.tb_lineno)


I will be glad for any help.

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question