Move your little hand to make a fortune, give it a thumbs up!
Iteration is a fundamental operation in Python. We can iterate over lists, strings, and all other structures.
Asyncio allows us to develop asynchronous iterators. We can create and use asynchronous iterators in asyncio programs by defining an object that implements the aiter() and anext() methods.
1. What is an asynchronous iterator
An asynchronous iterator is an object that implements the aiter() and anext() methods. Before we take a closer look at asynchronous iterators, let’s review classic iterators.
1.1. Iterators
Iterators are Python objects that implement a specific interface. Specifically, the iter() method, which returns an iterator instance, and the next() method, which steps the iterator through a loop and returns a value. Iterators can be stepped using the built-in function next() or traversed using a for loop. Many Python objects are iterable, most notably containers such as lists.
1.2. Asynchronous Iterators
Asynchronous iterators are Python objects that implement a specific interface. Asynchronous iterators must implement the aiter() and anext() methods.
- The aiter() method must return an instance of an iterator.
- The anext() method must return an awaitable that steps the iterator.
Asynchronous iterators can only be stepped or traversed in asyncio programs, such as in coroutines.
An asynchronous iterator can be stepped using the anext() built-in function, which returns an awaitable that executes the iterator one step, such as one call to the anext() method.
An asynchronous iterator can be traversed using an “async for” expression, which will automatically call anext() on each iteration and await the returned awaitable to retrieve the return value.
2. What is an “async for” loop?
The async for expression is used to iterate over an asynchronous iterator. It is an asynchronous for loop statement. Asynchronous iterators are iterators that yield awaitable objects. You may recall that an awaitable is an object that can be awaited, such as a coroutine or task.
An async generator will automatically implement the async iterator method, allowing it to be iterated like an async iterator. The await for expression allows the caller to iterate over the awaitable’s asynchronous iterators and retrieve results from each iterator.
This is not the same as traversing a collection or list of await objects (such as coroutine objects), instead the returned await objects must be provided using the expected asynchronous iterator methods. Internally, the async for loop will automatically resolve or await each awaitable dispatch coroutine as needed.
Because it’s a for loop, it assumes (although it doesn’t require) that each wait object traversed yields a return value. The async for loop must be used inside a coroutine, because it internally uses an await expression that can only be used inside a coroutine. The async for expression can be used to iterate over an asynchronous iterator in a coroutine.
... # traverse an asynchronous iterator async for item in async_iterator: print(item)
This will not execute for loops in parallel. asyncio cannot execute multiple coroutines at once in one Python thread.
Instead, it’s an asynchronous for loop. The difference is that the coroutine executing the for loop pauses and waits internally for each awaitable. Behind the scenes, this may require scheduling and waiting on coroutines, or waiting on tasks. We can also use async for expressions in list comprehensions.
... # build a list of results results = [item async for item async_iterator]
This builds the return value list of the asynchronous iterator.
3. How to use asynchronous iterators
In this section, we’ll take a closer look at how to define, create, step, and traverse asynchronous iterators in asyncio programs. Let’s start with how to define an asynchronous iterator.
- Define an asynchronous iterator
We can define an asynchronous iterator by defining a class that implements the aiter() and anext() methods. These methods are usually defined on Python objects. Importantly, because the anext() function must return an awaitable, it must be defined using an “async def” expression. The anext() method must raise a StopAsyncIteration exception when the iteration is complete.
# define an asynchronous iterator
class AsyncIterator():
# constructor, define some state
def __init__(self):
self.counter = 0
# create an instance of the iterator
def __aiter__(self):
return self
# return the next awaitable
async def __anext__(self):
# check for no further items
if self. counter >= 10:
raise StopAsyncIteration
# increment the counter
self.counter += 1
# return the counter value
return self.counter
Because async iterators are a coroutine, and each iterator returns an await object that is scheduled and executed in the asyncio event loop, we can execute and await await objects within the body of the iterator.
...
# return the next awaitable
async def __anext__(self):
# check for no further items
if self. counter >= 10:
raise StopAsyncIteration
# increment the counter
self.counter += 1
# simulate work
await asyncio. sleep(1)
# return the counter value
return self.counter
- Create an asynchronous iterator
To use asynchronous iterators, we have to create iterators. This involves creating Python objects normally.
... # create the iterator it = AsyncIterator()
This returns an “async iterator”, which is an instance of “async iterator”.
- iterate over an asynchronous iterator
Iterators can be traversed one step using the anext() built-in function, just like classic iterators using the next() function. The result is an awaitable object that is awaited.
... # get an awaitable for one step of the iterator awaitable = anext(it) # execute the one step of the iterator and get the result result = await awaitable
This can be done in one step.
... # step the async iterator result = await anext(it)
- Traversing an asynchronous iterator
Asynchronous iterators can also be traversed in a loop using an “async for” expression, which will automatically await each iteration of the loop.
... # traverse an asynchronous iterator async for result in AsyncIterator(): print(result)
We can also collect the result of an iterator using an asynchronous list comprehension with an “async for” expression.
... # async list comprehension with async iterator results = [item async for item in AsyncIterator()]
4. Asynchronous iterator example
We can explore how to iterate over asynchronous iterators using “async for” expressions. In this example, we’ll update the previous example to use an “async for” loop to iterate over the iterator to completion.
This loop will automatically wait for each awaitable returned from the iterator, retrieve the return value, and make it available within the loop body so that it can be reported in this case. This is probably the most common usage pattern for asynchronous iterators.
#SuperFastPython.com
# example of an asynchronous iterator with async for loop
import asyncio
#define an asynchronous iterator
class AsyncIterator():
# constructor, define some state
def __init__(self):
self.counter = 0
# create an instance of the iterator
def __aiter__(self):
return self
# return the next awaitable
async def __anext__(self):
# check for no further items
if self. counter >= 10:
raise StopAsyncIteration
# increment the counter
self.counter += 1
# simulate work
await asyncio. sleep(1)
# return the counter value
return self.counter
# main coroutine
async def main():
# loop over async iterator with async for loop
async for item in AsyncIterator():
print(item)
# execute the asyncio program
asyncio. run(main())
Running the example starts by creating a main() coroutine and using it as the entry point for the asyncio program. The main() coroutine runs and starts the for loop.
An instance of the asynchronous iterator is created, and the loop automatically steps through it using the anext() function to return an awaitable. The loop then waits on the awaitable and retrieves a value that can be used to report it to the body of the loop. This process is then repeated, suspending the main() coroutine, executing the iterator and suspending one step, then resuming the main() coroutine until the iterator is exhausted.
Once the iterator’s internal counter reaches 10, StopAsyncIteration is raised. This does not terminate the program. Instead, it is expected and handled by the “async for” expression and breaks the loop.
This highlights how to iterate over asynchronous iterators using async for expressions.
1 2 3 4
This article is published by mdnice multi-platform