How to process sent images in TornadoWeb (Python3)?

Question

Python

Good afternoon,
there is a test site for python3 + tornado(4.3).
Case:
the user uploads a large avatar, I need to shrink it to a standard size and show it to the user for the next step (selecting a section of the image as a photo).
Tell me, please, how to organize the processing of a photo so as not to block other user requests?
PS
For myself, I considered the following options:

1. multiprocessing (move processing into a separate process, but there are difficulties with synchronizing the output to the user)
   
2. asynchronous task in Celery. The question of overhead costs and in general - is it beautiful?
   

Answer 1

Part straight from the project!!!

__author__ = 'vadim7j7'
# ----------------------------------------------------------------------------------------------------------------------
import multiprocessing
from os.path import join, isfile
from os import unlink
from io import BytesIO

import tornado.gen
from concurrent.futures import ThreadPoolExecutor
from tornado.concurrent import run_on_executor
from PIL import Image

from app import BaseHandler
from app.modules.helpers import gen_ran_name, crop_img
from app.sql_raw_methods.User import get_user_info, update_user_photo


# ----------------------------------------------------------------------------------------------------------------------
class UpFile(BaseHandler):
    executor = ThreadPoolExecutor(max_workers=multiprocessing.cpu_count())

    def __init__(self, application, request, **kwargs):
        super().__init__(application, request, **kwargs)
        self.path_constants = {
            '1': self.settings['tmp_path'],
            '2': self.settings['tmp_path'],
            '3': self.settings['user_avatars']}

    @tornado.gen.coroutine
    def post(self):
        self._data['dateType'] = 'json'
        if self.request.files['file']:
            if len(self.request.files['file'][0]['body']) <= 12000000:
                try:
                    thumbnail = yield self.make_thumbnail(self.request.files['file'][0]['body'])
                except IOError as er:
                    self._data['status'] = 500
                else:
                    self._data['body'] = thumbnail
                    if self.get_argument("dataTypeObject") == '3':
                        yield self.rewrite_user_avatar(thumbnail)

        self.render(None)

    def delete(self):
        self._data['dateType'] = 'json'
        name = self.get_argument('file', None)
        from_remove = self.get_argument('from_remove', 'tmp')
        if name is not None:
            patch_file_tmp = join(self.settings['static_path'], from_remove, name)
            if isfile(patch_file_tmp):
                unlink(patch_file_tmp)
                self._data['body'] = 'Ok'
            else:
                self._data['status'] = 404
                self._data['body'] = 'Not file is tmp'
        else:
            self._data['status'] = 301
            self._data['body'] = 'Not correct request'

        self.render(None)

    @run_on_executor
    def make_thumbnail(self, content):
        im = Image.open(BytesIO(content))
        path_out = self.path_constants[self.get_argument('dataTypeObject')]

        with BytesIO() as output:
            new_name = '%s.JPEG' % gen_ran_name()
            patch_file_tmp = join(path_out, new_name)

            if self.get_argument("dataTypeObject") == '3':
                src_width, src_height = im.size
                im = crop_img(im, src_width, src_height, 128, 128)

            im.convert('RGB').save(patch_file_tmp, 'JPEG', quality=100, optimize=True)
            del im

            return new_name

    @tornado.gen.coroutine
    def rewrite_user_avatar(self, photo):
        cursor = yield self.db.execute(get_user_info(self.current_user['id'], 'photo'))
        user = cursor.fetchone()

        if user[0]:
            old_avatar_path = join(self.path_constants.get('3'), user[0])
            if isfile(old_avatar_path):
                unlink(old_avatar_path)
            yield self.db.execute(update_user_photo(self.current_user['id'], photo))


# ----------------------------------------------------------------------------------------------------------------------

def crop_img(original_img, src_width, src_height, max_width, max_height):
    if max_width <= 0:
        max_width = max_height * src_width / src_height

    if max_height <= 0:
        max_height = max_width * src_height / src_width

    src_ratio = float(src_width) / float(src_height)
    dst_width, dst_height = max_width, max_height
    dst_ratio = float(dst_width) / float(dst_height)

    if dst_ratio < src_ratio:
        crop_height = src_height
        crop_width = crop_height * dst_ratio
        x_offset = float(src_width - crop_width) / 2
        y_offset = 0
    else:
        crop_width = src_width
        crop_height = crop_width / dst_ratio
        x_offset = 0
        y_offset = float(src_height - crop_height) / 3

    preview_img = original_img.crop((int(x_offset), int(y_offset),
                                     int(x_offset + int(crop_width)),
                                     int(y_offset) + int(crop_height))).resize((int(dst_width), int(dst_height)), 1)

    return preview_img

Answer 2

Reduce on the client, for example. www.jqueryrain.com/demo/jquery-crop-image-plugin
Well, in general, the tornado is asynchronous, just make the handler asynchronous. To do this, it is enough to wrap the handler in a decorator to make a coroutine out of it. And the decorator from python 3.4 is suitable. And most likely even the new async/await syntax will also work if you configure the tornado on IOLoop from asyncio (which is included in the >= 3.4 python library).
www.tornadoweb.org/en/stable/guide/coroutines.html
There is a code example. Here you need to response = await http_client.fetch(url)replace the line with trimming the file. To make it work asynchronously, you need to trim the file in the form of a coroutine.