CS Fundamentals Trainer · Guides

Python performance and concurrency

Profiling first, what the interpreter costs per item, NumPy against loops, the GIL measured with threads and processes, process pool overhead, asyncio and blocking calls, and how to choose.

16 min read

"Python is slow" is true in one specific way and false in several others, and the useful skill is telling them apart. Plain Python code that does a little work per item, in a loop, over millions of items, pays a cost on every step. Python code that hands the same work to a built-in, a C library or the operating system can be as fast as anything. Python threads cannot run two pieces of Python code at once in the standard build, and still make I/O-heavy programs many times faster. This guide measures each of these on CPython 3.13 so you can decide with numbers: where the time goes, how to find it, how to remove it, and which kind of concurrency fits which kind of problem. It builds on how Python works; the experiment scripts are in the appendix.

All timings come from one small machine with two CPU cores, so the parallel speedups below top out at about 2. They show the shape of the results, not numbers to quote.

1. First, find out what is slow

Guessing wins rarely. Use the tools in this order:

  1. Measure the whole thing (wall-clock time of the real workload). If it is fast enough, stop.
  2. Profile to find where the time goes. cProfile counts every function call and the time spent in it; the standard library prints a table with pstats.
  3. Time small alternatives for the hot spot with timeit, which repeats a snippet many times and avoids one-off noise (take the minimum of several repeats).
  4. For memory, use tracemalloc (peak allocation, and which lines allocated it).

I profiled a function that removes duplicates by testing if x not in out on a list, next to one that uses dict.fromkeys, over 30,000 items with 3,000 distinct values:

FunctionCumulative time
slow_unique (membership test on a list)0.221 s
fast_unique (list(dict.fromkeys(items)))0.001 s

The profile pointed at one function out of the whole program, and the fix was a better data structure, not cleverer code: a list membership test scans the list, a dict or set does a hash lookup (see how Python works). That is the usual shape of a Python speedup. Changing the algorithm or data structure gives factors of hundreds; rewriting loops gives tens of percent.

2. What the interpreter costs

CPython runs your code by executing bytecode in a virtual machine, one instruction at a time, and checks types as it goes. I summed the squares of one million integers four ways:

ApproachTime for 1,000,000 items
for x in data: t += x * x44.4 ms
sum([x * x for x in data]) (builds a list first)54.1 ms
sum(x * x for x in data) (generator)37.0 ms
NumPy: (arr * arr).sum()0.9 ms

Three lessons:

Other micro-costs I measured, to calibrate your intuition:

ChangeResult
One million calls of add(i, 1) against the same addition written inline46 ms against 26 ms: a Python function call costs roughly 20 ns on top of the work
math.sqrt(i) against a local alias sq = math.sqrt39 ms against 36 ms: a small gain, about 3 ns per call

Both are real and small. Inlining code or caching attribute lookups is almost never the right first step; if a function is called in the hot loop a million times, the better question is whether the loop can be removed.

NumPy is fast, and different

Vectorised libraries change the rules, so know the differences:

Pandas, SciPy, scikit-learn, PyTorch and similar libraries work the same way: Python describes what to do, compiled code does it in bulk. If you must write the loop yourself, tools such as Cython, Numba, PyO3 or a C extension compile it; and recent CPython releases include an adaptive interpreter that specialises hot bytecode (that is part of why a simple loop is faster on 3.13 than the folklore says), but none of this replaces choosing a good algorithm.

3. The GIL

The standard CPython build has a global interpreter lock (GIL): only one thread executes Python bytecode at any moment. It exists because the interpreter's memory management (reference counts, see how Python works) is not safe to run from two threads at once. A thread releases the lock while it waits for I/O, while it sleeps, and inside many C libraries that do long native work.

The consequence depends on what the threads are doing. I ran the same job on threads and on processes:

TestTime
CPU work, four jobs one after the other1.00 s
CPU work, four jobs on 4 threads1.06 s (no faster; slightly slower)
CPU work, four jobs on 2 processes0.63 s (two cores, plus process start-up)
I/O work (20 waits of 0.2 s), one after the other4.00 s
I/O work, 20 threads0.20 s

(sys._is_gil_enabled() printed True for this standard build. Python 3.13 added an optional free-threaded build with the GIL removed, described in PEP 703; it is experimental, many extension modules are not ready for it, and single-thread speed is somewhat lower. Check what your deployment actually uses before planning around it.)

Two caveats about the GIL: it does not make your data structures thread-safe (x += 1 on a shared variable is several bytecode steps and can still lose updates; use a threading.Lock or a queue, see the concurrency guide), and C code that holds the lock for a long time can block other threads.

4. Processes: real parallelism, with a cost

concurrent.futures.ProcessPoolExecutor (or multiprocessing) starts worker processes and sends them work. Everything crossing the process boundary is pickled, copied and unpickled, so overhead matters:

TestResult
2,000 tiny tasks (x + 1) in a loop0.12 ms
The same 2,000 tasks through 2 worker processes, one task at a time366 ms (three thousand times slower)
The same, but with chunksize=500 (four batches)1 ms
Sending a 2-million-item list to a worker117 ms

Rules that follow:

5. asyncio: many waits on one thread

asyncio runs many tasks on one thread, switching between them whenever a task reaches an await on something that is not ready. It is built for programs that spend their time waiting: network calls, sockets, queues.

I started many coroutines that each wait 0.2 s:

Concurrent waitsTotal time
200.20 s
1,0000.21 s
10,0000.29 s

Ten thousand waits cost about as much as one, with no thread per task. That is why asyncio fits servers with many mostly idle connections.

The rule that keeps it fast is that a coroutine must not block. Five tasks that each called the ordinary time.sleep(0.2) instead of await asyncio.sleep(0.2) took 1.00 s: the blocking call stopped the whole event loop, so the tasks ran one after another. Calling a blocking library (a synchronous HTTP client, a database driver, a big computation) inside a coroutine does the same. Options: use an async library, or run the blocking call in a thread with asyncio.to_thread() (or in a process pool for CPU work) and await its result.

6. Choosing

Your problemReach for
One slow function in a large programprofile first (cProfile), then fix the hot spot
Repeated membership tests, lookups or sortingthe right data structure (set, dict, bisect, heapq)
Numeric loops over big arraysNumPy or another vectorised library
Waiting on many network or disk operationsasyncio (many connections) or threads (simple code or blocking libraries)
CPU-heavy work that splits into big independent piecesprocesses (ProcessPoolExecutor), batched
CPU-heavy code with a tight, un-vectorisable loopcompile it (Cython, Numba, a C or Rust extension) or rewrite that part
Need to share mutable state between workersa queue, a database or a lock; avoid shared mutable globals

7. Common mistakes

8. Questions interviewers like to ask

"What is the GIL and why does it exist?"

A lock that lets only one thread run Python bytecode at a time in standard CPython, which keeps reference counting and the interpreter's internals safe without fine-grained locks. It limits CPU-bound threads but is released during I/O, so threads still help I/O-bound programs.

"Threads, processes or asyncio?"

I/O-bound with many connections: asyncio. I/O-bound with blocking libraries or simpler code: threads. CPU-bound: processes or native code. Mixed workloads combine them, for example an async server that sends heavy work to a process pool.

"How would you speed up a slow Python program?"

Measure, then profile; fix the algorithm or data structure first; move loops into built-ins or vectorised libraries; use concurrency that matches the bottleneck; compile only what remains hot.

"Why can multiprocessing be slower than a plain loop?"

Starting workers and pickling arguments and results costs far more than a tiny task. Batch the work, keep payloads small, and use processes only for tasks that are large enough.

"Does asyncio make code faster?"

It makes waiting concurrent, not computing faster. It uses one thread, so a CPU-heavy or blocking call stalls everything.

"Is x += 1 thread-safe with the GIL?"

No. It compiles to several bytecode steps (load, add, store), and a thread switch between them can lose an update. Use a lock, an atomic structure such as queue.Queue, or avoid shared state.

9. Things to mention in an interview

Quick recap

Appendix: the experiment code

Python 3.13 (CPython), standard library plus NumPy for speed.py. Run with python3 file.py. The machine had two CPU cores; with more cores the process results improve and the thread results for CPU work do not. Timings vary from run to run and between machines.

gil.py: CPU-bound and I/O-bound work on threads and processes:

import time, threading, sys
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def cpu(n):
    s = 0
    for i in range(n):
        s += i * i
    return s

def io(_):
    time.sleep(0.2)

def timed(label, fn):
    t = time.perf_counter(); fn(); print(f"{label}: {time.perf_counter() - t:.2f} s")

if __name__ == "__main__":
    print("python", sys.version.split()[0], "GIL enabled:", sys._is_gil_enabled())
    N, JOBS = 5_000_000, 4
    timed(f"CPU x{JOBS}, one thread (sequential)", lambda: [cpu(N) for _ in range(JOBS)])
    def threads():
        with ThreadPoolExecutor(JOBS) as ex: list(ex.map(cpu, [N] * JOBS))
    timed(f"CPU x{JOBS}, {JOBS} threads", threads)
    def procs():
        with ProcessPoolExecutor(2) as ex: list(ex.map(cpu, [N] * JOBS))
    timed(f"CPU x{JOBS}, 2 processes", procs)
    timed("I/O x20 (sleep 0.2 s), sequential", lambda: [io(0) for _ in range(20)])
    def io_threads():
        with ThreadPoolExecutor(20) as ex: list(ex.map(io, range(20)))
    timed("I/O x20, 20 threads", io_threads)

asyncio_io.py: many concurrent waits, and what a blocking call does to the event loop:

import asyncio, time
async def fetch(i):
    await asyncio.sleep(0.2)
    return i
async def main(n):
    t = time.perf_counter()
    await asyncio.gather(*(fetch(i) for i in range(n)))
    return time.perf_counter() - t
for n in (20, 1000, 10000):
    print(f"{n} concurrent waits of 0.2 s: {asyncio.run(main(n)):.2f} s")
async def blocking():
    t = time.perf_counter()
    async def bad(): time.sleep(0.2)          # blocks the whole event loop
    await asyncio.gather(*(bad() for _ in range(5)))
    return time.perf_counter() - t
print(f"5 tasks calling time.sleep(0.2): {asyncio.run(blocking()):.2f} s")

speed.py: loops against comprehensions, generators and NumPy; call and lookup costs; a cProfile run:

import timeit, cProfile, pstats, io, sys
import numpy as np
N = 1_000_000
data = list(range(N))
arr = np.arange(N, dtype=np.int64)

def loop():
    t = 0
    for x in data: t += x * x
    return t
def comp(): return sum([x * x for x in data])
def gen(): return sum(x * x for x in data)
def numpy_(): return int((arr * arr).sum())
for name, f in (("for loop", loop), ("sum(list comprehension)", comp), ("sum(generator)", gen), ("numpy arr*arr .sum()", numpy_)):
    t = min(timeit.repeat(f, number=3, repeat=3)) / 3
    print(f"{name:26s} {t*1e3:8.1f} ms")
assert loop() == comp() == gen() == numpy_()
print("int64 overflow check, numpy sum:", numpy_())

# numpy gotcha: fixed-width ints wrap around
print("np.int64 max + 1:", np.array([2**63 - 1], dtype=np.int64) + 1, "| python int:", 2**63)

# function call overhead and attribute lookups
def add(a, b): return a + b
def calls():
    for i in range(1_000_000): add(i, 1)
def inline():
    for i in range(1_000_000): i + 1
print(f"1M calls {min(timeit.repeat(calls, number=1, repeat=3))*1e3:.0f} ms vs inline {min(timeit.repeat(inline, number=1, repeat=3))*1e3:.0f} ms")
import math
def attr():
    for i in range(1_000_000): math.sqrt(i)
sq = math.sqrt
def local():
    for i in range(1_000_000): sq(i)
print(f"math.sqrt {min(timeit.repeat(attr, number=1, repeat=3))*1e3:.0f} ms vs local alias {min(timeit.repeat(local, number=1, repeat=3))*1e3:.0f} ms")

# cProfile: find the hot spot
def slow_unique(items):
    out = []
    for x in items:
        if x not in out: out.append(x)
    return out
def fast_unique(items): return list(dict.fromkeys(items))
def work():
    items = [i % 3000 for i in range(30000)]
    a = slow_unique(items); b = fast_unique(items)
    assert a == b
pr = cProfile.Profile(); pr.enable(); work(); pr.disable()
s = io.StringIO(); pstats.Stats(pr, stream=s).sort_stats("cumulative").print_stats(4)
print("\n".join(l for l in s.getvalue().splitlines() if "slow_unique" in l or "fast_unique" in l or "work" in l or "ncalls" in l))

pool_overhead.py: the cost of tiny tasks and large arguments in a process pool:

import time
from concurrent.futures import ProcessPoolExecutor
def tiny(x): return x + 1
def big(lst): return len(lst)
if __name__ == "__main__":
    items = list(range(2000))
    t = time.perf_counter(); [tiny(x) for x in items]; seq = time.perf_counter() - t
    with ProcessPoolExecutor(2) as ex:
        list(ex.map(tiny, range(4)))      # warm up the workers
        t = time.perf_counter(); list(ex.map(tiny, items)); par = time.perf_counter() - t
        t = time.perf_counter(); list(ex.map(tiny, items, chunksize=500)); chunked = time.perf_counter() - t
        payload = list(range(2_000_000))
        t = time.perf_counter(); ex.submit(big, payload).result(); ship = time.perf_counter() - t
    print(f"2000 tiny tasks: sequential {seq*1e3:.2f} ms | 2 processes {par*1e3:.0f} ms | chunksize=500 {chunked*1e3:.0f} ms")
    print(f"sending a 2-million-item list to a worker: {ship*1e3:.0f} ms")

Sources

  1. Python documentation, concurrent.futures, multiprocessing, threading and asyncio.
  2. Python documentation, The Python Profilers (cProfile) and timeit.
  3. Python documentation, Glossary: global interpreter lock, and Python support for free threading.
  4. Python Enhancement Proposals, PEP 703: Making the Global Interpreter Lock optional in CPython.
  5. NumPy documentation, Data types (fixed-width integers and overflow).
  6. David Beazley, Understanding the Python GIL (talk slides, 2010), for the background on how the lock works.

Practise this topic: Python

More guides