M
M
Maxim Zubenko2020-09-27 16:58:27
Python
Maxim Zubenko, 2020-09-27 16:58:27

How to make an infinite iteration of a list asynchronous?

There is a code. Very simple. There is a list with values.
In the main function, there is a construction with an infinite loop, the essence of which is the iterative enumeration of values ​​from the list. (Ideally, the list should be populated with different values), but for the sake of example, the value used is removed from the beginning of the list and added again to the end. This creates a queue.
In the process of enumeration, a request is sent to the site through the get_html function.
For convenience, main has a count of iterations through the variable counter_

And all this works great, but I have no idea how to make this design asynchronous. Nowhere is there an example of just such a simple approach with enumeration. I reviewed a bunch of examples, but I really can’t understand how I can implement just such an approach. When there is a list or some kind of queue with replenished values.

Can you tell me how to make my code asynchronous using asyncio and aiohttp?

from datetime import datetime
import requests
import asyncio
import aiohttp

LIST_ = ['Shishel','Mishel','Perdnul','Vishel',
         'Papa','Mama',"Mila",'Ramu',
         'Eniki','Beniki','Eli','Vareniki',
         'Kreks','Feks','Peks','Keks']

def get_html(data):
    url = f'http://httpbin.org/get?acc={data}'
    try:
        r = requests.get(url)
        return r.json()['args']['acc']
    except:
        return 'failed'



def main():
    accs = LIST_
    counter_ = 0
    start = datetime.now()

    while accs:
        counter_ += 1
        one = accs.pop(0)
        print (  counter_ , get_html(one) )
        accs.append( one )
        if counter_ == len(accs):
            counter_ = 0
            end = datetime.now()
            print('*' * 10, end - start, '*' * 10)
            start = end



if __name__ == '__main__':
    main()

Answer the question

In order to leave comments, you need to log in

1 answer(s)
C
cython, 2020-09-27
@JawsIk

You can move the iteration of the loop into a separate asynchronous function that will work as a background task

from datetime import datetime
import requests
import asyncio
import aiohttp

LIST_ = ['Shishel','Mishel','Perdnul','Vishel',
         'Papa','Mama',"Mila",'Ramu',
         'Eniki','Beniki','Eli','Vareniki',
         'Kreks','Feks','Peks','Keks']

accs = LIST_
counter_ = 0
start = datetime.now()


async def get_html(data):
    url = f'http://httpbin.org/get?acc={data}'
    try:
    	async with aiohttp.ClientSession() as session:
        	async with session.get(url) as r:
        		return (await r.json())['args']['acc']
    except:
        return 'failed'


async def acc():
    global counter_, accs, start
    counter_ += 1
    one = accs.pop(0)
    print(counter_, await get_html(one), len(accs))
    accs.append( one )
    if counter_ == len(accs):
        counter_ = 0
        end = datetime.now()
        print('*' * 10, end - start, '*' * 10)
        start = end


async def main():
    tasks = []
    while accs:
        for _ in range(10):  # Ограничение на 10 задач, которые выполняются одновременно
            tasks.append(asyncio.get_event_loop().create_task(acc()))
        await asyncio.wait(tasks)  # Ожидание выполнения всех задач


if __name__ == '__main__':
    asyncio.get_event_loop().run_until_complete(main())

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question