S
S
s_tigran2021-02-20 10:50:22
Python
s_tigran, 2021-02-20 10:50:22

Why doesn't multiprocessing work correctly?

I want to check the proxy for validity + multithreading.
Without multiprocessing, everything works correctly, and with it, not full proxies are issued.
With her: 6030befc1a34b271588374.png

Without her:6030bf044067e676354175.png

def check_prox(proxybase):
    for prox in proxybase:
        user = fake_useragent.UserAgent().random
        proxy = {'http': f'http://{prox}',
                 'https': f'https://{prox}'}
        check_url = 'http://example.com'
        try:
            requests.get(check_url, proxies=proxy, timeout=3)
        except:
            print(f'Proxy: {prox} - UNVALID')
            continue
        else:
            print(f'Proxy: {prox} - VALID')
            with open('valid_proxy.txt', 'a', encoding='utf-8') as file:
                file.write(f'{prox}\n')

 with open('proxy.txt', 'r', encoding='utf-8') as file:
    check_proxy_base = file.read().split('\n')
if __name__ == '__main__':
    pool = Pool(processes=3)
    pool.map(check_prox, check_proxy_base)

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question