B
B
BOSH222022-03-03 00:21:59
Python
BOSH22, 2022-03-03 00:21:59

How to fix concurrent request bug in aiogram?

Good night, the following problem has arisen, which cannot be solved in any way:

There is a bot on Aiogram
The main task is to parse media content, process it and send it back to the user.

The parsing script looks like this:
with open('data/config.json') as json_file:
    config = json.load(json_file)
    VK_TOKEN = config['Bot_Data']['Vk_Token']

session = vk_api.VkApi(token=VK_TOKEN)
vk = session.get_api()

def get_user_data(user_id: str) -> dict:
    data = session.method('users.get', {'user_ids': user_id, 'fields': 'sex, photo_max, followers_count'})
    data_dict = {'id': 0, 'first_name': '', 'last_name': '', 'sex': 0, 'photo': '', 'followers_count': ''}
    data_dict['id'] = data[0]['id']
    data_dict['first_name'] = data[0]['first_name']
    data_dict['last_name'] = data[0]['last_name']
    data_dict['sex'] = data[0]['sex']
    data_dict['photo'] = data[0]['photo_max']
    data_dict['followers_count'] = data[0]['followers_count']
    url = (data[0]['photo_max'])
    img = urllib.request.urlopen(url).read()
    avat = io.BytesIO()
    avat.write(img)
    with open("FIO.txt", "w") as file:
        file.write(data_dict['first_name'] + " " + data_dict['last_name'])
         
    lists = glob('dialogs/*')
    picture = random.choice(lists)
    photo222=open(picture, 'rb')
    im1 = Image.open(photo222)
    rgb_2222 = im1.convert('RGB')
    idrawf = ImageDraw.Draw(rgb_2222)
    tt = open("FIO.txt", 'r')
    txt = tt.read().rstrip()
    text = txt
    font = ImageFont.truetype("tahomabd.ttf", size=23)
    idrawf.text((192, 16), text, font=font, aling=LEFT, )   
    avat.seek(0)
    im3 = Image.open(avat)
    size=(68,68)
    im2 = im3.resize(size)
    bigsize = im2.size[0] * 2, im2.size[1] * 2
    mask = Image.new('L', bigsize, 0)

    draw = ImageDraw.Draw(mask)
    draw.ellipse((0, 0) + bigsize, fill=255)

    mask = mask.resize(im2.size, Image.ANTIALIAS)
    im2.putalpha(mask)

    rgb_2222.paste(im2, (99, 8), mask)
    murkupa = Image.open('murkup.png')
    rgb_2222.paste(murkupa, (150, 280), mask=murkupa)
    global bio
    bio = BytesIO()
    bio.name = 'image1.jpeg'
    rgb_2222.save(bio, 'JPEG')
    bio.seek(0)

    lists1 = glob('dialogs/*')
    picture1 = random.choice(lists1)
    photo111=open(picture1, 'rb')
    im1111 = Image.open(photo111)
    rgb_1111 = im1111.convert('RGB')
    idraw = ImageDraw.Draw(rgb_1111)
    tt111 = open("FIO.txt", 'r')
    txt = tt111.read().rstrip()
    text111 = txt
    font = ImageFont.truetype("tahomabd.ttf", size=23)
    idraw.text((192, 16), text111, font=font, aling=LEFT, )   
    rgb_1111.save('dialogwithname/im2_with_name4.jpg')
    avat.seek(0)
    im311 = Image.open(avat)
    size=(68,68)
    im211 = im311.resize(size)

    bigsize = im211.size[0] * 2, im211.size[1] * 2
    mask111 = Image.new('L', bigsize, 0)

    draw = ImageDraw.Draw(mask111)
    draw.ellipse((0, 0) + bigsize, fill=255)

    mask111 = mask111.resize(im211.size, Image.ANTIALIAS)
    im211.putalpha(mask111)

    rgb_1111.paste(im211, (99, 8), mask111)
    murkup2 = Image.open('murkup.png')
    rgb_1111.paste(murkup2, (150, 280), mask=murkup2)
    global bio2
    bio2 = BytesIO()
    bio2.name = 'image2.jpeg'
    rgb_1111.save(bio2, 'JPEG')
    bio2.seek(0)

    lists_body = glob('by/*')
    pictur_by1 = random.choice(lists_by)
    pictur_by2 = random.choice(lists_by)

    photo111=open(pictur_by1, 'rb')
    im1111 = Image.open(photo111)
    pictur_bo1_blur = im1111.filter(ImageFilter.GaussianBlur(20))
    murkup4 = Image.open('murkup.png')
    w_width = round(pictur_by1_blur.width / 2)
    w_height = round(pictur_by1_blur.height / 2)
    position = (w_width, w_height)
    pictur_by1_blur.paste(murkup4, position, mask=murkup4)
    pictur_by1_blur.save('by_blur/bl1.png', quality=95)

    photo222=open(pictur_by2, 'rb')
    im2222 = Image.open(photo222)
    pictur_by2_blur = im2222.filter(ImageFilter.GaussianBlur(20))
    murkup6 = Image.open('murkup.png')
    w_width = round(pictur_by2_blur.width / 2)
    w_height = round(pictur_by2_blur.height / 2)
    position = (w_width, w_height)

    pictur_by2_blur.paste(murkup6, position, mask=murkup6)
    pictur_by2_blur.save('by_blur/bl2.png', quality=95)
    return data_dict


Result output code:
from tkinter import CENTER, LEFT
from loader import dp
from aiogram import types
from utils import vk_parser
import asyncio
import datetime
import json
import random
from filters.is_vk_link import Is_Vk_Link
from keyboards import search_result_keyboard
from PIL import Image, ImageDraw, ImageFont, ImageFilter
from glob import glob
from random import choice
from io import BytesIO

@dp.message_handler(Is_Vk_Link())
async def search_vk(message: types.Message):
    try:
        user_id = message.text.split('vk.com')[1].replace('/', '')
        user_data = vk_parser.get_user_data(user_id)
            loading = await message.answer('<b>Идёт поиск</b>')
            await loading.delete()
            photo = user_data['photo']
            media = types.MediaGroup()
            vk_parser.bio.seek(0)
            media.attach_photo(types.InputFile(vk_parser.bio), '1')
            vk_parser.bio2.seek(0)
            media.attach_photo(types.InputFile(vk_parser.bio2), '2')
            media.attach_photo(types.InputFile('by_blur/bl1.png'), '3')
            media.attach_photo(types.InputFile('by_blur/bl2.png'), '4')

            text2 = f'''<b>Нажмите, чтобы загрузить информацию</b>'''
            await message.answer_photo(photo)
            await message.reply_media_group(media=media)
            await message.answer(text2, reply_markup=search_result_keyboard.keyboard)
    except:
        await message.answer('<b>На данный момент не работает</b>')


In fact, the problem is that when two different users send a request at the same time, the following happens:
The user who sent the request for 0.3-0.5 seconds. later you receive the issuance - media files at the request of the first user.
The first user does not receive you issuing media files.
Error in logs: ValueError: I/O operation on closed file.

How to fix the moment so that the results of different users are not mixed up and did not understand.
Diligent googling was unsuccessful
Using BytesIO too
I would be grateful if someone could help me figure it out

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
soremix, 2022-03-03
@SoreMix

Name the files differently, for example, adding the user's Id as a postfix….

V
Vindicar, 2022-03-03
@Vindicar

1. The first user sends a message. Runs a copy of search_vk() and executes a synchronous request:
user_data = vk_parser.get_user_data(user_id)
While the request is being executed, the bot is idle because the code is synchronous.
2. get_user_data() does stupid things like

with open("FIO.txt", "w") as file: #открываем один и тот же файл для любого запроса

or
global bio # используем одну и ту же переменную для любого запроса

Well, other media files are also the same for different requests.
3. The message handler of the first user reaches the line
loading = await message.answer('<b>Идёт поиск</b>')

Since this is an await call, the coroutine is scheduled to be called and the asynchronous program moves on to the next available operation while the message is being sent. The collected data is stored in files.
4. The second user sends a message. Another copy of search_vk() is launched and also executes a synchronous get_user_data() request. The second call to get_user_data() overwrites the data in the files.
5. The message handler of the second user reaches the line
loading = await message.answer('<b>Идёт поиск</b>')

and goes to sleep while the message is being sent.
6. In the meantime, the first user's message has been sent. The handler takes control and sends the contents of the files - which have been overwritten - closing them at the same time.
7. The second user's message has been sent. Its copy of search_vk() tries to send files, but the first handler has already closed them.
The matter is further complicated by the network lag, so it is difficult to say who will be the first in the end.
If after reading you still don’t understand what’s the matter, I’ll say it straight: do not use global objects, be it files or variables, in a competitive environment!Make sure each copy of the request handler has its own data stores! Local variables are safe, they are created every time anew. If you can't do without files on disk, either use the tempfile module, or bind their names to the message sender's ID so that at least different users don't clash.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question