S
S
Sergey Durov2016-05-14 16:35:41
Python
Sergey Durov, 2016-05-14 16:35:41

Real examples of working with multithreading in python?

In books, in articles and attached source codes, everything boils down to them, that you need to create a thread and then make a join. Nowhere are normal examples, at least a little closer to real ones, seen.
Links to articles, books, github where you can see all this.
Thanks

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
Sergey Gornostaev, 2016-05-14
@sergey-gornostaev

I have an online store in Django. He periodically receives from 1C a json file with a catalog of goods and a bunch of jpegs with photos of these goods. The file must be parsed and the data loaded into the database, and for each photo, make 5 thumbnails. Naturally, reading a volumetric image from disk, scaling it 5 times, and writing it back to disk 5 times is much longer than writing data from a dictionary to the database. In addition, there can be up to 200,000 of them. It was reasonable to move the creation of thumbnails into separate threads.
Function executing in a separate thread:

def create_previews(file_path):
    file_name = os.path.basename(file_path)
    gallery_path = os.path.join(settings.MEDIA_ROOT, 'images')
    for size in settings.PREVIEW_SIZES:
        prefix = '%dx%d_' % size
        preview_path = os.path.join(gallery_path, prefix + file_name)
        if not os.path.exists(preview_path):
            try:
                img = Image.open(file_path)
                if (img.size[0] < size[0]) and (img.size[1] < size[1]):
                    continue
                canvas = Image.new("RGB", size, (255,255,255))
                preview = ImageOps.fit(img, size, Image.ANTIALIAS)
                canvas.paste(preview, (0,0))
                canvas.save(preview_path, optimize=True)
            except Exception as e:
                pass

And in the code that parses the json file, for each photo stored in the database:
t = Thread(target=create_previews, args=(image.data.path,))
t.start()

A
Anton Konovalov, 2016-05-18
@akonovalov

I strongly recommend that you watch these reports (there are many more on YouTube - you have to look) and fill in the gaps in theory - why multithreading may be needed at all and how to prepare it in Python:
once or
twice
I myself dealt with multithreading in Python from David Beasley's book "Python. A detailed guide ." 2010.
The book is in electronic form on torrents and other "pirated" book depositories.
You can search for examples of usage in popular Python projects - if you ask Google for "most popular python projects" - a list of links will open with the most popular projects and some of them can use multithreading.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question