U
U
user.2022-02-06 15:56:14
Python
user., 2022-02-06 15:56:14

Is this code multi-threaded?

Good afternoon,
here is a simplified example of a part of my code in which I try to use multithreading, but alas, I don’t notice an increase in speed at all. Actually the question .... Is everything correctly implemented?

#!/usr/bin/env python3

import threading
import requests

start_id = 1
last_id =  99999999
max_thread = 10


def save_progress(f_name, userid):
    f = open(f_name, 'w+')
    f.write(str(userid))
    f.close()


def get_data(userid):
   url = "https://vk.com/id" + userid

   r = requests.get(url, headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"})

   if(r.status_code == 200):
       if((len(r.content)) > 87013):
           print('found vk profile')


class DownloadThread(threading.Thread):
        def __init__(self, name, userid):
            threading.Thread.__init__(self)
            self.name = name
            self.userid = userid

        def run(self):
            get_data(self.userid)


def main():
    global start_id

    if os.path.exists('./start_id.log'):
        f = open('./start_id.log', 'r')
        start_id = int(f.readline())
        f.close()


    for i in range(start_id, last_id):
        try:
            while True:
                if len(threading.enumerate()) <= max_thread:
                    thread = DownloadThread('Thread-' + str(i), i)
                    thread.start()
                    break

        except:
            e = sys.exc_info()[0]
            print('Got exception in main: ', e)
            save_progress('./start_id.log', userid)
            break

if __name__ == "__main__":
    main()

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dr. Bacon, 2022-02-06
@bacon

It's easier to rewrite than to understand. There is an old article https://chriskiehl.com/article/parallelism-in-one-line like there was even a translation on Habré, which tells how to make it much easier and most importantly correctly, it will also be possible to understand from it why your bike is very bad quality.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question