Concurrency Model

The GIL, threading, asyncio, and multiprocessing — picking the right tool

Why

The GIL explains why multithreading doesn't speed up CPU-bound Python work — a fact that surprises many developers. Picking the wrong concurrency model can make performance worse, not better, and this comes up in system design rounds when discussing parallel data fetching, serving ML models under load, or writing efficient ETL pipelines.

Intuition

The GIL (Global Interpreter Lock) is a mutex inside CPython that allows only one thread to execute Python bytecode at a time. For IO-bound work (waiting on network/disk), threads release the GIL during the wait — multiple threads genuinely help. For CPU-bound work, threads fight over the GIL and don't help at all.

Explanation

Decision matrix

IO-bound CPU-bound (network, disk) (compute, parsing) threading ✓ works ✗ GIL blocks asyncio ✓ best choice ✗ single-threaded multiprocessing ✓ overkill ✓ bypasses GIL Rule: IO-bound → asyncio (if async libs exist) or threading CPU-bound → multiprocessing (ProcessPoolExecutor) Mixed → asyncio for IO + run_in_executor for CPU tasks Note: NumPy, pandas, and most C extensions release the GIL — threading can help with these even for "CPU" work.

ThreadPoolExecutor and ProcessPoolExecutor

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor # IO-bound — threads, GIL released during network wait with ThreadPoolExecutor(max_workers=10) as pool: results = list(pool.map(fetch, urls)) # CPU-bound — processes, each gets its own GIL def cpu_heavy(n): return sum(i * i for i in range(n)) with ProcessPoolExecutor(max_workers=4) as pool: results = list(pool.map(cpu_heavy, [10**6]*4))

Both executors expose the same map / submit API — switching from threads to processes is often just changing one class name.

← Previous Next →