How to perform an infinite loop so that it does not block the work of the entire program?

Question

Python

I am writing a telegram bot that parses data from the site (ads). The site is updated quite often, so you need to constantly make requests to the site: "have a new ad arrived"?
Actually, that's why it was decided to wrap these requests in an infinite loop. Having written a couple of decorators and one infinite loop for processing messages, I realized that (of course) the script itself cannot execute decorators for processing incoming messages, because it hangs in a loop.
I write the script in python with the asynchronous aiogram library (initially I thought that it would somehow help me).

The question is : how can these processes be parallelized in order to keep track of updates, and at the same time process incoming messages

PS: I’m rewriting the bot not for the first time, I tried to find the answer myself, I wanted to attach threads or processes here, but I can’t understand what is best in my case for that matter, and how to beat it correctly?
the loop in the code is just for debugging, just like print in this loop

spoiler

import config
import aiogram
from aiogram import Bot, Dispatcher, executor, types

API_TOKEN = config.token

# Инициализация бота и диспетчера
bot = Bot(token=API_TOKEN)
dp = Dispatcher(bot)


#------------------------Цикл ---------------------------------------------------------------
#Будущий парсер
while 1:
    print('цикл')

#------------------------Блок обработки входящих сообщений------------------------------
@dp.message_handler(commands=['start']) #Приветствие
async def echo(message: types.Message):
    await message.answer('''Здравствуйте Вы подключились к боту . Чтобы узнать возможности Бота введите команду /help''')

@dp.message_handler(commands=['help']) #Помощь
async def echo(message: types.Message):
    await message.answer('help')

@dp.message_handler(commands=['add']) #Функция добавления фильтра
async def echo(message: types.Message):
    await message.answer('add')




if __name__ == '__main__':
    executor.start_polling(dp, skip_updates=True)

Answer 1

do something similar. Parallel two tasks in a magnifying glass

import asyncio
import datetime
import random


async def my_sleep_func():
    await asyncio.sleep(random.randint(0, 5))


async def display_date(num, loop):
    end_time = loop.time() + 50.0
    while True:
        print("Loop: {} Time: {}".format(num, datetime.datetime.now()))
        if (loop.time() + 1.0) >= end_time:
            break
        await my_sleep_func()


loop = asyncio.get_event_loop()

asyncio.ensure_future(display_date(1, loop))
asyncio.ensure_future(display_date(2, loop))

loop.run_forever()

Answer 2

I made a similar bot for this site (to get new questions on my tags). The work algorithm was something like this:
1) I created a function that loads and parses the main page of this site. Pulls out of it all links to new questions. The function returns a list of links.
2) In the for loop, I take each link, and check if it is in the file (you can fasten the database). If this link does not exist, then the question is new. I add a link to the file, and send it to myself in telegram. If there is a link in the file, then this question has already been sent to me in telegram, and, therefore, I am not doing anything
3) In the bot, I made a cycle with a delay of 5 minutes, and in the cycle called the parser function.

Answer 3

Create a flow

Answer 4

Try BackgroundScheduler specifically for a function where you have a loop.
documentation: https://apscheduler.readthedocs.io/en/3.x/userguid...
here is an example code: https://github.com/agronholm/apscheduler/blob/3.x/...
here is an example for in your code:
from apscheduler.schedulers.background import BackgroundScheduler
def loop():
while 1: #the loop is optional here as the timer runs print('loop') every 10 seconds
print('loop')
..... processing block incoming messages....
if __name__ == '__main__':
scheduler = BackgroundScheduler()
scheduler.add_job(loop, 'interval', seconds=10)
scheduler.start()
bot.polling(none_stop=True, interval=0)