Mastering Asyncio in Python: Single-Threaded Concurrency Made Simple – P2

In our last blog, we explored threads and processes in Python—great tools for handling multiple tasks. But there’s another powerful option for tackling I/O-bound tasks: Python’s asyncio framework. Today, we’ll dive into what asyncio is, how it offers single-threaded concurrency, and why it’s perfect for things like network requests or file operations. Plus, I’ll share some simple examples to get you started. Let’s jump in!


List blog-related:

  1. Processes, and Concurrency in Python: A Beginner’s Guide – P1
  2. Mastering Asyncio in Python: Single-Threaded Concurrency Made Simple – P2
  3. Deep Dive into Thread Pools and Process Pools in Python: Simplify Concurrency Like a Pro – P3 (final)

What Is Asyncio?

asyncio is Python’s built-in library for writing asynchronous code. Unlike threads or processes, it lets you handle multiple tasks within a single thread. Think of it like a super-efficient waiter taking orders from multiple tables without leaving the dining room—no extra workers (threads) or separate kitchens (processes) needed.

  • Key Idea: With asyncio, you write code that can pause and resume tasks (like waiting for a web response) without blocking the whole program. This is called single-threaded concurrency.
  • Why Single-Threaded? It avoids the complexity of threads (like the GIL or race conditions) while still letting you juggle multiple tasks.

How Does Asyncio Work?

In asyncio, you use two magic keywords: async and await.

  • async def: Defines an asynchronous function (called a coroutine) that can pause itself.
  • await: Pauses the coroutine until a task (like an I/O operation) is done, letting other tasks run in the meantime.

Under the hood, asyncio uses an event loop—a scheduler that keeps track of all your tasks and decides what runs next. It’s like a conductor directing an orchestra, ensuring everything plays in harmony.

Here’s a basic example:

import asyncio

async def say_hello():
    print("Hello")
    await asyncio.sleep(1)  # Simulate waiting (non-blocking)
    print("World")

# Run the coroutine
asyncio.run(say_hello())

Output:

Hello
World

The await asyncio.sleep(1) pauses say_hello for 1 second without blocking the program—perfect for I/O simulation.


Why Use Asyncio for I/O-Bound Tasks?

asyncio shines for I/O-bound tasks—things like fetching web pages, reading files, or querying a database—because it lets your program do other work while waiting. Compare this to threads (which also work for I/O but have GIL limitations) or processes (overkill for I/O due to their overhead).

  • No GIL Issues: Since it’s single-threaded, the Global Interpreter Lock doesn’t slow you down.
  • Lightweight: No need to spawn threads or processes—just one thread handles everything.
  • Scalable: You can manage hundreds or thousands of tasks (like API calls) efficiently.

Let’s see it in action with multiple tasks:

import asyncio

async def fetch_data(task_id):
    print(f"Starting task {task_id}")
    await asyncio.sleep(2)  # Simulate a network delay
    print(f"Finished task {task_id}")

async def main():
    # Run multiple tasks concurrently
    await asyncio.gather(
        fetch_data(1),
        fetch_data(2),
        fetch_data(3)
    )

# Run the event loop
asyncio.run(main())

Output:

Starting task 1
Starting task 2
Starting task 3
Finished task 1
Finished task 2
Finished task 3

Here, all three tasks start almost at once and finish after 2 seconds—not 6 seconds like they would in a sequential program. That’s concurrency in action!


Asyncio vs. Threads vs. Processes

Let’s break it down:

FeatureAsyncioThreadsProcesses
ConcurrencyYes (single-threaded)Yes (multi-threaded)Yes (multi-process)
ParallelismNoLimited by GILYes (multi-core)
OverheadVery lowModerateHigh
Best ForI/O-bound tasksI/O-bound tasksCPU-bound tasks
ComplexityModerate (async/await)Low (simple API)Moderate (separate memory)
  • Asyncio: Ideal for I/O-bound tasks with minimal overhead.
  • Threads: Good for I/O but trickier with shared memory and GIL.
  • Processes: Best for CPU-heavy work needing true parallelism.

Real-World Use Case: Fetching Multiple Web Pages

Let’s simulate fetching data from multiple websites using asyncio. We’ll use the aiohttp library (a popular async HTTP client) to make it realistic.

First, install aiohttp:

pip install aiohttp

Now, the code:

import asyncio
import aiohttp

async def fetch_url(url):
    async with aiohttp.ClientSession() as session:
        print(f"Fetching {url}")
        async with session.get(url) as response:
            await response.text()  # Wait for the response
            print(f"Done with {url}")

async def main():
    urls = ["https://example.com", "https://python.org", "https://github.com"]
    tasks = [fetch_url(url) for url in urls]
    await asyncio.gather(*tasks)

# Run it
asyncio.run(main())

Output (approximate):

Fetching https://example.com
Fetching https://python.org
Fetching https://github.com
Done with https://example.com
Done with https://python.org
Done with https://github.com

All three requests start together and finish as soon as their responses arrive—not one after the other. This is way faster for I/O-bound tasks like web scraping or API calls!


Real-World Use Case: Apply for loop function

def function example(X):
    for x in X:
        x.run()

To apply asyncio to your function and make x.run() asynchronous, you’ll need to ensure a few things:

  1. The x.run() method must itself be an asynchronous function (a coroutine) that you can await. If it’s not already async, you’ll need to modify it or wrap it in an async-compatible way.
  2. Your function a() must be defined as async def so it can use await.
  3. You’ll need to run the async function using an event loop (e.g., with asyncio.run()).

Let’s assume x.run() is a method that performs some I/O-bound task (like a network call or file operation) that can be made asynchronous. I’ll show you how to adapt your code step-by-step, including a practical example.

Step 1: Make x.run() Asynchronous

If x.run() isn’t already an async function, you’ll need to redefine it. For this example, let’s assume X is a class with a run method. Here’s how you might modify it:

import asyncio

class X:
    async def run(self):
        print(f"Running {self}")
        await asyncio.sleep(1)  # Simulate an I/O-bound task (e.g., network request)
        print(f"Finished {self}")

    def __str__(self):
        return f"X instance {id(self)}"

If x.run() is a synchronous function you can’t change (e.g., from a library), you can run it in a thread pool using asyncio.to_thread()—more on that later.

Step 2: Modify a() to Use Asyncio

Your original function loops over a collection X and calls x.run(). To make it async, you’ll use async def and await each x.run() call. Here’s how it looks:

import asyncio

async def a(X):
    for x in X:
        await x.run()  # Wait for each run to complete

This runs x.run() sequentially—each task waits for the previous one to finish. If you want them to run concurrently, you’ll need to collect the coroutines and use asyncio.gather().

Step 3: Run It Concurrently (Optional)

If you want all x.run() calls to happen at the same time (concurrently), you can use asyncio.gather(). Here’s the concurrent version:

import asyncio

async def a(X):
    # Collect all coroutines and run them concurrently
    await asyncio.gather(*(x.run() for x in X))
Full Example: Sequential vs. Concurrent

Let’s put it all together with a complete example, showing both sequential and concurrent approaches.

import asyncio

# Define the class with an async method
class X:
    async def run(self):
        print(f"Running {self}")
        await asyncio.sleep(1)  # Simulate I/O delay
        print(f"Finished {self}")

    def __str__(self):
        return f"X instance {id(self)}"

# Sequential version
async def a_sequential(X):
    for x in X:
        await x.run()

# Concurrent version
async def a_concurrent(X):
    await asyncio.gather(*(x.run() for x in X))

# Create some instances
items = [X(), X(), X()]

# Run the sequential version
print("Sequential:")
asyncio.run(a_sequential(items))

# Run the concurrent version
print("\nConcurrent:")
asyncio.run(a_concurrent(items))

Output:

Sequential:
Running X instance 1407123456789
Finished X instance 1407123456789
Running X instance 1407123456790
Finished X instance 1407123456790
Running X instance 1407123456791
Finished X instance 1407123456791

Concurrent:
Running X instance 1407123456789
Running X instance 1407123456790
Running X instance 1407123456791
Finished X instance 1407123456789
Finished X instance 1407123456790
Finished X instance 1407123456791
  • Sequential: Takes ~3 seconds (1 second per task).
  • Concurrent: Takes ~1 second (all tasks run together).
What If x.run() Is Synchronous?

If x.run() is a synchronous function you can’t modify (e.g., it uses time.sleep() instead of asyncio.sleep()), you can run it in a thread pool using asyncio.to_thread(). Here’s how:

import asyncio
import time

class X:
    def run(self):  # Synchronous method
        print(f"Running {self}")
        time.sleep(1)  # Blocking delay
        print(f"Finished {self}")

    def __str__(self):
        return f"X instance {id(self)}"

async def a(X):
    # Wrap synchronous run() in a thread and await it
    await asyncio.gather(*(asyncio.to_thread(x.run) for x in X))

# Create instances
items = [X(), X(), X()]

# Run it
asyncio.run(a(items))

Output:

Running X instance 1407123456789
Running X instance 1407123456790
Running X instance 1407123456791
Finished X instance 1407123456789
Finished X instance 1407123456790
Finished X instance 1407123456791

Here, asyncio.to_thread() offloads each x.run() to a thread, allowing concurrency without modifying the original synchronous code.


Tips for Using Asyncio

  1. Always Use asyncio.run(): It’s the cleanest way to start your async program.
  2. Avoid Blocking Calls: Don’t use time.sleep() or other blocking functions—use asyncio.sleep() instead.
  3. Combine with Libraries: Use async-compatible libraries like aiohttp or aiomysql for I/O tasks.
  4. Debugging: If something hangs, check if you forgot an await.

Final Thoughts

asyncio is your go-to tool for I/O-bound concurrency in Python. It’s lightweight, powerful, and perfect for tasks where your program spends a lot of time waiting. While it takes a little practice to get used to async and await, the payoff is huge—faster, more scalable programs without the overhead of threads or processes.

Try tweaking the examples above—maybe fetch some real APIs or read files asynchronously. Play around, break things, and learn! Next time you’re building a web scraper or a network app, give asyncio a shot.

2 responses to “Mastering Asyncio in Python: Single-Threaded Concurrency Made Simple – P2”

  1. daniel vu Avatar
    daniel vu

    hay đấy

    1. hahoanglc97 Avatar

      Cảm ơn bạn đã đọc và theo dõi. Hãy donate cho mình ít tiền để mình có thể ra được những bài viết hay hơn

Leave a Reply

Your email address will not be published. Required fields are marked *