Introduction to Python Asyncio

Introduction

Are you aiming to make your application work well with a lot of users, all without making them wait too long? Just trying to scale up vertically won't solve everything. You wish to get done more at the same time using concurrency and parallelism. That's where this blog comes in. We'll take a close look at these ideas and explore how Python's asyncio module can make a difference.

Concurrency and Parallelism

Synchronous programming involves executing tasks sequentially, where a function waits for another's output before proceeding. This halts the program until the function completes, essentially permitting only one task at a time.This inefficiency contradicts the potential for multitasking. This is where concurrency and parallelism come into play.

Figure 1. How asynchronous programming helps? [Source]

Concurrency is simply the ability of a program to manage multiple tasks together. Consider a music player: while it's playing a song, it should also be ready to accept your commands to change the volume or skip tracks. Concurrency doesn't demand that everything happens at precisely the same moment. It can be like taking turns – one works for a bit, then steps aside for others. This way, even if a computer has only a few processors, the operating system can handle many tasks simultaneously.

When tasks truly run at exactly the same time, like on a computer with multiple processor cores or in a group of computers working together, that's parallelism. It's a special case of concurrency.

Figure 2. Concurrency and Parallelism [Source]

Choosing the Right Paradigm

While multiprocessing, threading, and asynchronous programming all aid us in achieving concurrent execution, each paradigm is better suited for distinct types of tasks.

If your task revolves around using a lot of CPU power, multiprocessing is the ideal choice. For tasks that are IO bound and have fast IO and limited number of connections, multithreading is the preferred approach. On the other hand, if your task is IO bound with slow responses and large connections, asynchronous programming (async) is the most suitable path to take. ¹

if io_bound:
    if io_very_slow:
        print("Use Asyncio")
    else:
        print("Use Threads")
else:
    print("Multi Processing")

Generators and the event loop

Typically, functions in every programming language follow a subroutine model. This model starts execution from the beginning of a function when it's called. It continues until the function ends or encounters a return statement. Afterward, control returns to the point just after the function call, and subsequent calls restart from the beginning.

There is an alternative mode of code execution, called the coroutine model. In this model, a function can yield control back to the caller without returning. When it yields, execution returns to the point just after the coroutine call. Importantly, future calls to the coroutine don't start from the beginning but pick up where they left off. Python has had the capability to allow this execution model for some time in the form of Generators

Figure 3. Subroutine and Coroutine Pattern [Source]

Asynchronous programming can be thought of as an extension of the coroutine method. When a method is waiting for an IO operation to complete, it can yield the control back to an event loop. The event loop acts as a coordinator, which passes the control to other methods that are ready to run. The event loop leverages the underlying operating system’s select system call to monitor the status of IO tasks.

Vocab of the python asyncio world

Coroutine

Coroutines are the heart of asynchronous programming in Python. They are special functions that can pause their execution and hand over control to another function.

Note that simply calling a coroutine will not execute the function, it will return a coroutine object. To actually run a coroutine, Python's asyncio offers these ways:

• The asyncio.run() function runs the main entry point function.
• Awaiting on a coroutine.
  • Notes:
    • Attempting to await a coroutine more than once will result in an error(RuntimeError: cannot reuse already awaited coroutine).
    • Order of execution is dependent on when the coroutines are awaited.
• Coroutines can be used to create tasks, which we'll discuss further in this blog.

>>> import asyncio
>>> import time

>>> async def say_after(delay, what):
...    # When working with asyncio, `asyncio.sleep` should be
...    # used and not `time.sleep`, otherwise the event loop
...    # will not pause.
...    await asyncio.sleep(delay)
...    print(what)

>>> async def main():
...    print(f"started at {time.strftime('%X')}")
...    say_1 = say_after(1, 'hello')
...    say_2 = say_after(2, 'world')
...    await say_2
...    await say_1
...    print(f"finished at {time.strftime('%X')}")

>>> main()
<coroutine object main at 0x1053bb7c8>

>>> asyncio.run(main())
started at 17:13:52
world
hello
finished at 17:13:55

Tasks

Tasks are used to schedule coroutines concurrently. When a coroutine is wrapped into a Task with functions like asyncio.create_task() the coroutine is automatically scheduled to run and the event loop will soon pick the task for execution.

The current task (running in the event loop) continues its execution just like a regular (synchronous) Python program until it reaches a point where it needs to pause, typically to wait for some i/o operation. Instead of waiting, the Task hands over control to the event loop. The event loop temporarily halts the Task, then resumes it later when the awaited event or condition has occurred. This ensures efficient utilization of resources in asynchronous programming.

Let’s modify the above example and run two say_after coroutines concurrently:

>>> async def main():
...    task1 = asyncio.create_task(
...        say_after(1, 'hello'))
...    task2 = asyncio.create_task(
...        say_after(2, 'world'))

...    print(f"started at {time.strftime('%X')}")
...    # Wait until both tasks are completed (should take
...    # around 2 seconds.)
...    await task1
...    await task2
...    print(f"finished at {time.strftime('%X')}")
>>> asyncio.run(main())
started at 17:14:32
hello
world
finished at 17:14:34

Note that expected output now shows that the snippet runs 1 second faster than before.

A task can be awaited multiple times, but the underlying coroutine will only be executed once, and the same result will be returned for all the await calls.

>>> import asyncio, time

>>> async def foo(delay):
...    await asyncio.sleep(delay)
...    return "bar"

>>> async def main():
...    task1 = asyncio.create_task(foo(3))
...    print(f"started at {time.strftime('%X')}")
...
...    result_once = await task1
...    print(f"task1, value: {result_once}, finished once at {time.strftime('%X')}")
...    result_twice = await task1
...    print(f"task1, value: {result_twice}, finished twice at {time.strftime('%X')}")
    
>>> asyncio.run(main())
started at 14:02:05
task1, value: bar,  finished once at 14:02:08
task1, value: bar, finished twice at 14:02:08

Note that both the await calls return the same value and the program completes in 3 seconds.

Futures

A Future is a special low-level awaitable object that represents an eventual result of an asynchronous operation. (JavaScript developers, think of Promise!)

Unlike a coroutine object, awaiting a Future doesn't trigger the execution of any code block. Instead, a Future serves as a representation of a process happening elsewhere, which might or might not have concluded yet. The process needs to set the eventual result / exception on completion.

Here's what happens when you await a Future:

1. If the process the Future represents has finished:
   • If the process produced a result, the await statement returns that value.
   • If the process encounters an exception, the await statement raises that exception.
2. If the process has not yet finished, the current Task is paused until the process has finished. Once it's done, it behaves as described in the two sub-points described above

Normally there is no need to create Future objects at the application level code. Future objects, sometimes exposed by libraries and some asyncio APIs, can be awaited:

async def main():
    await function_that_returns_a_future_object()

    # this is also valid:
    await asyncio.gather(
        function_that_returns_a_future_object(),
        some_python_coroutine()
    )

If you do need to create your own future directly you can do it with a call to

f = asyncio.get_running_loop().create_future()

Gather

Gather can be used to run a series of awaitable objects directly (as used in the above code).

If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values corresponds to the order of awaitables in gather.

Seeing Asyncio magic in action

Let's explore how asynchronous programming can enhance performance using the classic producer-consumer example from Michal Kennedy's talk. The codes from the example are available on the talk’s GitHub repository.

The producer here is a function that returns numbers with random delays in each iteration, mimicking data arrival from various sources like networks, databases, or files. On the other hand, the consumer is a function that applies a straightforward mathematical operation to the received number, mimicking the data processing step. We run this example with three setups:

1. Synchronous: Traditional sequential execution.
2. Async (2 Producers, 1 Consumer): Parallel execution with asynchronous programming.
3. Async (2 Producers, 3 Consumers): Enhanced parallelism with multiple consumers.

By comparing the results, we can see how asynchronous programming boosts performance by eliminating the IO delay and efficient scheduling. The event-loop is still a single thread paradigm. For compute-intensive tasks, consider using multi processing (see Choosing the Right Paradigm).

Sources

• Demystifying async | Jetbrains | Video
• Demystifying async | Jetbrains | Slides
• How Asyncio works | Stackoverflow
• PBTS: How Asyncio works | TenThousandMeters Blog
• Simpler Codes for asyncio main concepts | Pysheet
• Asyncio Python | RealPython Blog
• Python Asyncio: Complete Guide | SFP Blog
• Good Explanation about generators | BBC Blog | Part 1
• Good Explanation about generators | BBC Blog | Part 2
• I’m too stupid for asyncio
• Understanding Asyncio in Python | Built In
• Concurrent Servers: Part 4 - libuv - Eli Bendersky's website
• asyncio — Asynchronous I/O — Python 3.12.0 documentation

Footnotes

1. Async IO excels when managing multiple tasks burdened by IO waits, like handling numerous database queries. However, async usage depends on libraries supporting async/await. Synchronous calls in coroutines can block other tasks. Complete async integration requires all stack components, even databases, to support async operations. ↩