CS Fundamentals Trainer · Guides
Python performance and concurrency
Profiling first, what the interpreter costs per item, NumPy against loops, the GIL measured with threads and processes, process pool overhead, asyncio and blocking calls, and how to choose.
16 min read
"Python is slow" is true in one specific way and false in several others, and the useful skill is telling them apart. Plain Python code that does a little work per item, in a loop, over millions of items, pays a cost on every step. Python code that hands the same work to a built-in, a C library or the operating system can be as fast as anything. Python threads cannot run two pieces of Python code at once in the standard build, and still make I/O-heavy programs many times faster. This guide measures each of these on CPython 3.13 so you can decide with numbers: where the time goes, how to find it, how to remove it, and which kind of concurrency fits which kind of problem. It builds on how Python works; the experiment scripts are in the appendix.
All timings come from one small machine with two CPU cores, so the parallel speedups below top out at about 2. They show the shape of the results, not numbers to quote.
1. First, find out what is slow
Guessing wins rarely. Use the tools in this order:
- Measure the whole thing (wall-clock time of the real workload). If it is fast enough, stop.
- Profile to find where the time goes.
cProfilecounts every function call and the time spent in it; the standard library prints a table withpstats. - Time small alternatives for the hot spot with
timeit, which repeats a snippet many times and avoids one-off noise (take the minimum of several repeats). - For memory, use
tracemalloc(peak allocation, and which lines allocated it).
I profiled a function that removes duplicates by testing if x not in out on a list, next to one that uses dict.fromkeys, over 30,000 items with 3,000 distinct values:
| Function | Cumulative time |
|---|---|
slow_unique (membership test on a list) | 0.221 s |
fast_unique (list(dict.fromkeys(items))) | 0.001 s |
The profile pointed at one function out of the whole program, and the fix was a better data structure, not cleverer code: a list membership test scans the list, a dict or set does a hash lookup (see how Python works). That is the usual shape of a Python speedup. Changing the algorithm or data structure gives factors of hundreds; rewriting loops gives tens of percent.
2. What the interpreter costs
CPython runs your code by executing bytecode in a virtual machine, one instruction at a time, and checks types as it goes. I summed the squares of one million integers four ways:
| Approach | Time for 1,000,000 items |
|---|---|
for x in data: t += x * x | 44.4 ms |
sum([x * x for x in data]) (builds a list first) | 54.1 ms |
sum(x * x for x in data) (generator) | 37.0 ms |
NumPy: (arr * arr).sum() | 0.9 ms |
Three lessons:
- Roughly 40 ns per item is what the interpreter costs for even a trivial loop body. A million items is fine; a billion is a different design problem.
- The comprehension was not faster than the loop here. It had to build a million-element list before
sumcould start, and the generator, which builds nothing, was the fastest of the pure-Python versions. Folklore that "comprehensions are always faster" does not hold when the result is thrown away. Compare, don't assume. - NumPy was about 50 times faster because the loop runs inside compiled C over a contiguous array of machine integers, with no per-item bytecode and no per-item Python object. This is the main escape from interpreter overhead: put the loop in native code.
Other micro-costs I measured, to calibrate your intuition:
| Change | Result |
|---|---|
One million calls of add(i, 1) against the same addition written inline | 46 ms against 26 ms: a Python function call costs roughly 20 ns on top of the work |
math.sqrt(i) against a local alias sq = math.sqrt | 39 ms against 36 ms: a small gain, about 3 ns per call |
Both are real and small. Inlining code or caching attribute lookups is almost never the right first step; if a function is called in the hot loop a million times, the better question is whether the loop can be removed.
NumPy is fast, and different
Vectorised libraries change the rules, so know the differences:
- NumPy integers are fixed width.
np.array([2**63 - 1], dtype=np.int64) + 1wrapped around to-9223372036854775808, whereas Python's ownintgave9223372036854775808. Speed comes with overflow, so pick the dtype on purpose. - The gain needs whole-array operations. A Python
forloop that indexes into an array element by element is usually slower than a list, because each access creates a Python object. - Converting between lists and arrays copies the data; do it once, not inside a loop.
Pandas, SciPy, scikit-learn, PyTorch and similar libraries work the same way: Python describes what to do, compiled code does it in bulk. If you must write the loop yourself, tools such as Cython, Numba, PyO3 or a C extension compile it; and recent CPython releases include an adaptive interpreter that specialises hot bytecode (that is part of why a simple loop is faster on 3.13 than the folklore says), but none of this replaces choosing a good algorithm.
3. The GIL
The standard CPython build has a global interpreter lock (GIL): only one thread executes Python bytecode at any moment. It exists because the interpreter's memory management (reference counts, see how Python works) is not safe to run from two threads at once. A thread releases the lock while it waits for I/O, while it sleeps, and inside many C libraries that do long native work.
The consequence depends on what the threads are doing. I ran the same job on threads and on processes:
| Test | Time |
|---|---|
| CPU work, four jobs one after the other | 1.00 s |
| CPU work, four jobs on 4 threads | 1.06 s (no faster; slightly slower) |
| CPU work, four jobs on 2 processes | 0.63 s (two cores, plus process start-up) |
| I/O work (20 waits of 0.2 s), one after the other | 4.00 s |
| I/O work, 20 threads | 0.20 s |
- CPU-bound code gets no speedup from threads, because the threads take turns holding the lock. Processes have separate interpreters and separate locks, so they scale with cores (here two).
- I/O-bound code gets a large speedup from threads, because while one thread waits on the network or disk, others run. Twenty waits of 0.2 s took 0.2 s in total.
(sys._is_gil_enabled() printed True for this standard build. Python 3.13 added an optional free-threaded build with the GIL removed, described in PEP 703; it is experimental, many extension modules are not ready for it, and single-thread speed is somewhat lower. Check what your deployment actually uses before planning around it.)
Two caveats about the GIL: it does not make your data structures thread-safe (x += 1 on a shared variable is several bytecode steps and can still lose updates; use a threading.Lock or a queue, see the concurrency guide), and C code that holds the lock for a long time can block other threads.
4. Processes: real parallelism, with a cost
concurrent.futures.ProcessPoolExecutor (or multiprocessing) starts worker processes and sends them work. Everything crossing the process boundary is pickled, copied and unpickled, so overhead matters:
| Test | Result |
|---|---|
2,000 tiny tasks (x + 1) in a loop | 0.12 ms |
| The same 2,000 tasks through 2 worker processes, one task at a time | 366 ms (three thousand times slower) |
The same, but with chunksize=500 (four batches) | 1 ms |
| Sending a 2-million-item list to a worker | 117 ms |
Rules that follow:
- Use processes for big, CPU-heavy tasks; one task should take at least several milliseconds.
- Batch small tasks (
chunksize) so the per-task overhead is paid per batch. - Send small arguments and small results. Pass an id or a filename and let the worker read the data, or use shared memory (
multiprocessing.shared_memory) for large arrays. - The number of useful workers is about the number of CPU cores, not more.
- Functions given to a pool must be importable at module level, and the entry point needs an
if __name__ == "__main__":guard, because on some platforms workers re-import your module.
5. asyncio: many waits on one thread
asyncio runs many tasks on one thread, switching between them whenever a task reaches an await on something that is not ready. It is built for programs that spend their time waiting: network calls, sockets, queues.
I started many coroutines that each wait 0.2 s:
| Concurrent waits | Total time |
|---|---|
| 20 | 0.20 s |
| 1,000 | 0.21 s |
| 10,000 | 0.29 s |
Ten thousand waits cost about as much as one, with no thread per task. That is why asyncio fits servers with many mostly idle connections.
The rule that keeps it fast is that a coroutine must not block. Five tasks that each called the ordinary time.sleep(0.2) instead of await asyncio.sleep(0.2) took 1.00 s: the blocking call stopped the whole event loop, so the tasks ran one after another. Calling a blocking library (a synchronous HTTP client, a database driver, a big computation) inside a coroutine does the same. Options: use an async library, or run the blocking call in a thread with asyncio.to_thread() (or in a process pool for CPU work) and await its result.
6. Choosing
| Your problem | Reach for |
|---|---|
| One slow function in a large program | profile first (cProfile), then fix the hot spot |
| Repeated membership tests, lookups or sorting | the right data structure (set, dict, bisect, heapq) |
| Numeric loops over big arrays | NumPy or another vectorised library |
| Waiting on many network or disk operations | asyncio (many connections) or threads (simple code or blocking libraries) |
| CPU-heavy work that splits into big independent pieces | processes (ProcessPoolExecutor), batched |
| CPU-heavy code with a tight, un-vectorisable loop | compile it (Cython, Numba, a C or Rust extension) or rewrite that part |
| Need to share mutable state between workers | a queue, a database or a lock; avoid shared mutable globals |
7. Common mistakes
- Optimising without profiling. The slow part is usually somewhere else, and usually the algorithm.
- Using threads for CPU-bound work and expecting a speedup (1.06 s against 1.00 s above).
- Sending tiny tasks to a process pool (366 ms against 0.12 ms).
- Blocking calls inside
asyncfunctions, which turn concurrency into a queue (1.00 s instead of 0.2 s). - Using a list as a set (
x in some_listin a loop). - Looping in Python over NumPy arrays instead of using array operations, and forgetting that fixed-width integers overflow silently.
- Passing huge objects between processes repeatedly.
- Trusting folklore about what is fast. Comprehensions,
+=on strings and attribute lookups all behave differently across versions; measure on yours. - Reaching for concurrency first. A better algorithm usually gives more than any amount of parallelism.
8. Questions interviewers like to ask
"What is the GIL and why does it exist?"
A lock that lets only one thread run Python bytecode at a time in standard CPython, which keeps reference counting and the interpreter's internals safe without fine-grained locks. It limits CPU-bound threads but is released during I/O, so threads still help I/O-bound programs.
"Threads, processes or asyncio?"
I/O-bound with many connections: asyncio. I/O-bound with blocking libraries or simpler code: threads. CPU-bound: processes or native code. Mixed workloads combine them, for example an async server that sends heavy work to a process pool.
"How would you speed up a slow Python program?"
Measure, then profile; fix the algorithm or data structure first; move loops into built-ins or vectorised libraries; use concurrency that matches the bottleneck; compile only what remains hot.
"Why can multiprocessing be slower than a plain loop?"
Starting workers and pickling arguments and results costs far more than a tiny task. Batch the work, keep payloads small, and use processes only for tasks that are large enough.
"Does asyncio make code faster?"
It makes waiting concurrent, not computing faster. It uses one thread, so a CPU-heavy or blocking call stalls everything.
"Is x += 1 thread-safe with the GIL?"
No. It compiles to several bytecode steps (load, add, store), and a thread switch between them can lose an update. Use a lock, an atomic structure such as queue.Queue, or avoid shared state.
9. Things to mention in an interview
- Measure first: wall-clock time,
cProfile,timeit,tracemalloc. - Algorithm and data structure before micro-optimisation; the list-against-dict profile.
- What the interpreter costs per item and how vectorised libraries avoid it; fixed-width integer overflow.
- The GIL: what it protects, what it allows (I/O overlap), what it prevents (CPU parallelism in threads); the free-threaded build as an experiment.
- Process pools: pickling, chunking, small payloads, the number of cores.
asyncio: single thread, cooperative switching, never block the loop,to_threadfor blocking calls.- Matching the tool to the bottleneck.
Quick recap
- Profile before changing anything. A profile found one function responsible for 0.221 s, and a
dict.fromkeysfix made it 0.001 s. - The interpreter costs about 40 ns per item for a trivial loop; NumPy did the same work about 50 times faster by looping in C. Comprehensions are not automatically faster than loops.
- The GIL lets one thread run bytecode at a time: four CPU jobs on four threads took 1.06 s against 1.00 s sequentially; two processes took 0.63 s on two cores.
- Threads still make I/O fast: 20 waits of 0.2 s took 0.2 s on 20 threads instead of 4.0 s.
- Process pools need big tasks: 2,000 tiny tasks took 366 ms through a pool and 1 ms when batched.
asynciohandled 10,000 concurrent waits in 0.29 s, but five blockingtime.sleepcalls inside coroutines took 1.00 s.- Choose the tool that matches the bottleneck: algorithms, vectorisation,
asyncio, threads, processes or compiled code.
Appendix: the experiment code
Python 3.13 (CPython), standard library plus NumPy for speed.py. Run with python3 file.py. The machine had two CPU cores; with more cores the process results improve and the thread results for CPU work do not. Timings vary from run to run and between machines.
gil.py: CPU-bound and I/O-bound work on threads and processes:
import time, threading, sys
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
def cpu(n):
s = 0
for i in range(n):
s += i * i
return s
def io(_):
time.sleep(0.2)
def timed(label, fn):
t = time.perf_counter(); fn(); print(f"{label}: {time.perf_counter() - t:.2f} s")
if __name__ == "__main__":
print("python", sys.version.split()[0], "GIL enabled:", sys._is_gil_enabled())
N, JOBS = 5_000_000, 4
timed(f"CPU x{JOBS}, one thread (sequential)", lambda: [cpu(N) for _ in range(JOBS)])
def threads():
with ThreadPoolExecutor(JOBS) as ex: list(ex.map(cpu, [N] * JOBS))
timed(f"CPU x{JOBS}, {JOBS} threads", threads)
def procs():
with ProcessPoolExecutor(2) as ex: list(ex.map(cpu, [N] * JOBS))
timed(f"CPU x{JOBS}, 2 processes", procs)
timed("I/O x20 (sleep 0.2 s), sequential", lambda: [io(0) for _ in range(20)])
def io_threads():
with ThreadPoolExecutor(20) as ex: list(ex.map(io, range(20)))
timed("I/O x20, 20 threads", io_threads)
asyncio_io.py: many concurrent waits, and what a blocking call does to the event loop:
import asyncio, time
async def fetch(i):
await asyncio.sleep(0.2)
return i
async def main(n):
t = time.perf_counter()
await asyncio.gather(*(fetch(i) for i in range(n)))
return time.perf_counter() - t
for n in (20, 1000, 10000):
print(f"{n} concurrent waits of 0.2 s: {asyncio.run(main(n)):.2f} s")
async def blocking():
t = time.perf_counter()
async def bad(): time.sleep(0.2) # blocks the whole event loop
await asyncio.gather(*(bad() for _ in range(5)))
return time.perf_counter() - t
print(f"5 tasks calling time.sleep(0.2): {asyncio.run(blocking()):.2f} s")
speed.py: loops against comprehensions, generators and NumPy; call and lookup costs; a cProfile run:
import timeit, cProfile, pstats, io, sys
import numpy as np
N = 1_000_000
data = list(range(N))
arr = np.arange(N, dtype=np.int64)
def loop():
t = 0
for x in data: t += x * x
return t
def comp(): return sum([x * x for x in data])
def gen(): return sum(x * x for x in data)
def numpy_(): return int((arr * arr).sum())
for name, f in (("for loop", loop), ("sum(list comprehension)", comp), ("sum(generator)", gen), ("numpy arr*arr .sum()", numpy_)):
t = min(timeit.repeat(f, number=3, repeat=3)) / 3
print(f"{name:26s} {t*1e3:8.1f} ms")
assert loop() == comp() == gen() == numpy_()
print("int64 overflow check, numpy sum:", numpy_())
# numpy gotcha: fixed-width ints wrap around
print("np.int64 max + 1:", np.array([2**63 - 1], dtype=np.int64) + 1, "| python int:", 2**63)
# function call overhead and attribute lookups
def add(a, b): return a + b
def calls():
for i in range(1_000_000): add(i, 1)
def inline():
for i in range(1_000_000): i + 1
print(f"1M calls {min(timeit.repeat(calls, number=1, repeat=3))*1e3:.0f} ms vs inline {min(timeit.repeat(inline, number=1, repeat=3))*1e3:.0f} ms")
import math
def attr():
for i in range(1_000_000): math.sqrt(i)
sq = math.sqrt
def local():
for i in range(1_000_000): sq(i)
print(f"math.sqrt {min(timeit.repeat(attr, number=1, repeat=3))*1e3:.0f} ms vs local alias {min(timeit.repeat(local, number=1, repeat=3))*1e3:.0f} ms")
# cProfile: find the hot spot
def slow_unique(items):
out = []
for x in items:
if x not in out: out.append(x)
return out
def fast_unique(items): return list(dict.fromkeys(items))
def work():
items = [i % 3000 for i in range(30000)]
a = slow_unique(items); b = fast_unique(items)
assert a == b
pr = cProfile.Profile(); pr.enable(); work(); pr.disable()
s = io.StringIO(); pstats.Stats(pr, stream=s).sort_stats("cumulative").print_stats(4)
print("\n".join(l for l in s.getvalue().splitlines() if "slow_unique" in l or "fast_unique" in l or "work" in l or "ncalls" in l))
pool_overhead.py: the cost of tiny tasks and large arguments in a process pool:
import time
from concurrent.futures import ProcessPoolExecutor
def tiny(x): return x + 1
def big(lst): return len(lst)
if __name__ == "__main__":
items = list(range(2000))
t = time.perf_counter(); [tiny(x) for x in items]; seq = time.perf_counter() - t
with ProcessPoolExecutor(2) as ex:
list(ex.map(tiny, range(4))) # warm up the workers
t = time.perf_counter(); list(ex.map(tiny, items)); par = time.perf_counter() - t
t = time.perf_counter(); list(ex.map(tiny, items, chunksize=500)); chunked = time.perf_counter() - t
payload = list(range(2_000_000))
t = time.perf_counter(); ex.submit(big, payload).result(); ship = time.perf_counter() - t
print(f"2000 tiny tasks: sequential {seq*1e3:.2f} ms | 2 processes {par*1e3:.0f} ms | chunksize=500 {chunked*1e3:.0f} ms")
print(f"sending a 2-million-item list to a worker: {ship*1e3:.0f} ms")
Sources
- Python documentation,
concurrent.futures,multiprocessing,threadingandasyncio. - Python documentation, The Python Profilers (
cProfile) andtimeit. - Python documentation, Glossary: global interpreter lock, and Python support for free threading.
- Python Enhancement Proposals, PEP 703: Making the Global Interpreter Lock optional in CPython.
- NumPy documentation, Data types (fixed-width integers and overflow).
- David Beazley, Understanding the Python GIL (talk slides, 2010), for the background on how the lock works.
Practise this topic: Python