CS Fundamentals Trainer · Guides
Python errors, resources and testing
The exception hierarchy and try/else/finally, chaining with from, EAFP against LBYL (measured), context managers and ExitStack, and unittest with mocks: patching where a name is used, spec, test independence, flaky tests and what coverage means.
19 min read
Every program fails: files are missing, networks time out, users type nonsense, and code has bugs. What separates a reliable program is not that it avoids failure but that it fails in a way you can see, understand and recover from, and that you find the bugs before your users do. In Python that means three connected skills: using exceptions well, releasing resources no matter what happens, and writing tests that fail for the right reasons. This guide covers all three with scripts run on CPython 3.13 (in the appendix). It builds on the context managers and generators in how Python works and decorators, generators and async.
A limit on what I could run: the sandbox had the standard unittest framework but not pytest, so every test run shown here is unittest. The section on pytest describes it from its documentation and shows no output of its own.
1. How exceptions work
An exception is an object. When code raises one, Python stops the current function, looks for a matching except in the caller, then in its caller, and so on up the stack; if none matches, the program ends and prints a traceback, the list of calls that led there.
The hierarchy
All exceptions inherit from BaseException. Most you meet inherit from Exception, and the standard ones are organised in families: KeyError has the chain KeyError, LookupError, Exception, BaseException, object, so except LookupError catches both KeyError and IndexError. Three exceptions sit outside Exception, directly under BaseException: KeyboardInterrupt (Ctrl-C), SystemExit (sys.exit()), and GeneratorExit. That is deliberate: except Exception should not stop the user from interrupting the program.
I tested what a bare except: does with that. Code that raised KeyboardInterrupt passed straight through except Exception (correct), and was swallowed by a bare except:. So the rules are:
- Catch specific exceptions (
except FileNotFoundError), as narrow as the failure you can really handle. - Use
except Exceptiononly at a boundary where you log and continue (a request handler, a worker loop), never deep inside logic. - Never use a bare
except:, and never writeexcept Exception: passwithout a comment and a reason; silent failures are the hardest bugs to find. - When several
exceptclauses could match, the first match wins, so list specific classes before their parents. Withexcept LookupErrorwritten beforeexcept KeyError, the second clause was never reached.
The full try statement
try:
value = 10 / x
except ZeroDivisionError:
value = None
else:
... # runs only if the try body did NOT raise
finally:
... # always runs
The order of events I logged: with x = 2 it was try, else, finally; with x = 0 it was try, except, finally. Put as little as possible in the try body, so that an unrelated error is not caught by mistake; code that should run only on success belongs in else. finally runs on success, on handled errors, on unhandled errors, and on return, break and continue.
One trap: return inside finally overrides the function's result and discards any exception in flight. A function with try: return "try" and finally: return "finally" returned finally. Do not return, break or continue in a finally block.
Chaining: keep the original cause
When you catch one error and raise another, say how they relate:
| You write | __cause__ | __context__ | Traceback says |
|---|---|---|---|
raise AppError(...) from e | the original ValueError | the original | "The above exception was the direct cause of the following exception" |
raise AppError(...) inside except | None | the original ValueError | "During handling of the above exception, another exception occurred" |
raise AppError(...) from None | None | still set, but hidden (__suppress_context__ is True) | only the new exception |
Use from e when you translate a low-level error into your own (so the cause stays in the log), and from None only when the original is noise. A traceback that shows both is much easier to debug than one that shows only the final error.
Design your own exceptions
Define a small hierarchy for your application (class AppError(Exception), class ValidationError(AppError)), so callers can catch your errors without catching everything. Put data on the exception when the handler needs it: my ValidationError("email", "missing @") carried field = 'email' and printed email: missing @. Raise exceptions for exceptional conditions and let normal outcomes be return values; do not use exceptions as a way to return a result from deep inside normal logic.
Exception groups (3.11+)
When several things fail at once (concurrent tasks, a batch), ExceptionGroup bundles them, and except* handles the parts by type. A group of ValueError("a"), TypeError("b") and ValueError("c") gave ValueError part: ['a', 'c'] and TypeError part: ['b']. asyncio.TaskGroup raises these (see the async guide). You cannot return, break or continue inside an except* block.
Look before you leap, or ask forgiveness
Python style often prefers EAFP ("easier to ask forgiveness than permission"): try the operation and handle the exception, rather than LBYL (check first). Two reasons: it is often cleaner, and the check can be wrong by the time you act on it. I made a file, confirmed os.path.exists() was True, removed the file as another process might, then opened it: FileNotFoundError, although the check had passed. For files, locks and database rows, try the operation and handle the failure.
On speed, I timed a dictionary lookup a million times each way:
| Case | LBYL (if k in d) | EAFP (try: d[k]) |
|---|---|---|
| key present | 50 ms | 37 ms |
| key missing | 37 ms | 130 ms |
Handling an exception is cheap when nothing is raised and expensive when something is. So use EAFP when failure is the rare case, and a check (or dict.get) when a miss is common.
2. Resources: always release them
Files, sockets, locks, database connections and temporary directories must be released even when something fails. The tool is the with statement (see how Python works): the cleanup in __exit__ runs however the block ends.
- Open files with
with open(...) as f:. Relying on garbage collection to close them works in CPython by accident and fails on other implementations or when a traceback keeps the file alive. contextlib.suppress(FileNotFoundError)is a tidy way to ignore one specific, expected error (it printed nothing when removing a file that did not exist).contextlib.ExitStackmanages a variable number of resources: I opened three files in a loop withenter_context, and all three were closed when the stack exited. (If opening one of them fails, the ones already opened are still closed on the way out.)@contextmanager(from the decorators guide) turns a generator with atry/finallyinto a context manager; for transactions, timing, temporary settings.- Releasing in the reverse order of acquiring, and not letting one failed cleanup stop the others, is what
ExitStackdoes for you.
Also keep resources from leaking in long-running code: close database cursors and connections, put timeouts on network calls, and bound queues and caches.
Logging, not printing
At a boundary where you catch and continue, record the failure with the traceback: logging.exception("could not process order %s", order_id) inside an except block logs the message at ERROR level together with the full traceback. Include the identifiers you need to find the case later, and never log secrets or personal data (see observability and web application security). Decide for each error one of four responses: handle it (retry, fall back), translate it (raise your own with from e), let it propagate (the caller knows better), or crash (the program cannot continue safely). Logging and then re-raising at every level produces the same error ten times in the log; log once, where you handle it.
3. Testing
What a test is
A test is a small program that runs your code with known input and checks the output. Good tests are fast, independent, repeatable and readable: when one fails, the name and the message should tell you what broke.
I wrote a small module shop.py (a price with tax, a function that fetches a tax rate over the network, a total, and an opening-hours check) and tested it with unittest, which ships with Python. Run it with python3 -m unittest -v. All 11 tests passed in 0.3 s. What the tests demonstrate:
Table-driven cases. subTest runs one test body over many inputs and reports each failing row separately: (10, 0.2, 12.0), (0.1, 0.2, 0.12), (19.99, 0.07, 21.39), (0, 0.5, 0.0). Include boundaries and zero. (pytest does this with @pytest.mark.parametrize.)
Floating point. 0.1 + 0.2 == 0.3 is False; assertAlmostEqual passed. Never compare floats with == (use math.isclose or the framework's helper), and keep money in integers or decimal.Decimal.
Failure paths. assertRaises checks that the right exception is raised. A test that mocks a timeout and expects TimeoutError to propagate pins down behaviour that otherwise only appears in production.
Boundaries in time. The opening-hours function takes the time as a parameter, so the test passes 8, 9, 16 and 17 and checks False, True, True, False with no clock patching. Designing code so that dependencies are passed in (the clock, the random generator, the HTTP client) is the best way to make it testable, much better than patching later.
Test doubles and mocks
A test should not call the real network, send real email or depend on today's date. Replace those pieces with test doubles. unittest.mock.patch("shop.fetch_rate", return_value=0.1) replaced the network call, the total came out at the expected 33.0, and assert_called_once_with("BR") checked how the function was called. Three traps, two of which I reproduced:
- Patch the name where it is used, not where it is defined.
shop.pydoesfrom time import sleep, soshopholds its own reference namedsleep. Patchingtime.sleepchanged nothing: the call still took 0.30 s. Patchingshop.sleepmade it instant (0.0006 s). If a patch seems to do nothing, this is almost always why. - A plain
Mockaccepts anything.m.no_such_method(1, 2)raised nothing, so a typo in a test can pass silently.Mock(spec=shop)(orautospec=True) raisesAttributeErrorfor names the real object does not have. - Over-mocking tests the mock. If you replace everything around the code, the test verifies your assumptions about those parts, not the real behaviour. Mock the slow, non-deterministic edges (network, clock, randomness); use real objects, or fast fakes such as an in-memory database, for the rest. Add a small number of integration tests against the real thing.
Independence and order
Tests must not depend on each other. I made a class where test_a and test_b append to a shared class-level list and a third test asserts the list has two items. In the full run it passed, only because of the (alphabetical) execution order; run on its own it failed with AssertionError: 0 != 2. Create state per test in setUp (or fixtures), never share mutable state through class or module variables, and run tests in random order in CI to expose accidental dependencies. Likewise, give tests their own temporary files and database rows and clean them up.
What to test, and how much
- Test behaviour through the public interface, not private details, so refactoring does not break tests.
- Write the tests for a bug before fixing it; the failing test proves the fix and prevents regression.
- Aim for tests that are cheap to write and run in seconds; keep a few slow end-to-end tests for the critical paths. A common balance is many unit tests, fewer integration tests, and very few end-to-end tests.
- Coverage (which lines the tests run) finds untested code but does not prove correctness: a test that runs a line without asserting anything counts as coverage. Use it to find gaps, not as a score to maximise.
- Run the tests automatically on every change (continuous integration), and treat a flaky test (one that sometimes fails for no reason) as a bug, because people stop trusting a red build.
pytest
Most Python projects use pytest instead of unittest; it was not available where I ran these scripts, so this is from its documentation and not from a run. It finds functions named test_*, uses plain assert statements and rewrites them to show the values involved when they fail, provides fixtures (functions that create and clean up test state, injected by name into the tests that need them), and @pytest.mark.parametrize for tables. It can also run unittest tests, so migrating can be gradual. Everything in this section (injecting dependencies, mocking at the edges, independence, behaviour over implementation) applies identically.
4. Common mistakes
- Bare
except:andexcept Exception: pass, hiding the real failure. - A big
trybody, catching errors from unrelated lines. returnorbreakinsidefinally.- Raising a new exception without
from, losing the cause (or hiding it by accident). - Using exceptions for normal control flow in hot paths (130 ms against 37 ms per million misses).
- Check-then-act on files and shared state (
exists()thenopen()). - Opening resources without
with. - Logging an error and re-raising it at every layer.
- Comparing floats with
==, and storing money in floats. - Tests that depend on order, the clock, the network or each other.
- Patching the wrong name, or mocking so much that nothing real is tested.
- Treating coverage as the goal.
- Ignoring flaky tests.
5. Questions interviewers like to ask
"What is the difference between except Exception and a bare except?"
A bare except also catches BaseException subclasses such as KeyboardInterrupt and SystemExit, which makes the program hard to stop. except Exception catches ordinary errors only. Prefer a specific class.
"When does finally run, and what can go wrong?"
Whenever the try block is left: normally, by exception, or by return, break or continue. A return inside finally replaces the original return value and swallows any exception.
"What does raise ... from e do?"
It sets __cause__ so the traceback shows the original error as the direct cause. from None hides the original; without from, the original is shown as context only.
"EAFP or LBYL?"
EAFP (try and handle) is usually cleaner and avoids races between check and use, but a raised exception is slower than a check, so for a common failure case a check or dict.get is better.
"How would you test a function that calls an external API?"
Pass the client in (or patch the name the function actually uses), return controlled responses, test the success, error and timeout paths, and add a few integration tests against a sandbox. Use spec or autospec so a typo in the mock fails.
"What makes a test flaky, and what do you do about it?"
Shared state, order dependence, real time or network, randomness, race conditions. Isolate state per test, inject the clock and the random seed, replace the network with a double, and fix or delete the test rather than re-running it.
"Is 100% coverage enough?"
No. It shows lines were executed, not that results were checked or that edge cases were covered.
6. Things to mention in an interview
- The exception hierarchy, specific
exceptclauses, theelseandfinallysemantics, chaining withfrom. - Context managers and
ExitStackfor resources; releasing in reverse order; timeouts on I/O. - Log once with the traceback; handle, translate, propagate or crash.
- EAFP against LBYL and the race in check-then-act, with the cost of raising.
- Tests: fast, independent, repeatable; table-driven; float comparison; testing failure paths; injecting dependencies.
- Mocking at the edges, patching where a name is used,
spec, and the danger of over-mocking. - Fixtures and parametrisation in pytest; coverage as a tool, not a goal; flaky tests.
Quick recap
KeyboardInterruptandSystemExitare outsideException: a bareexcept:swallowed one,except Exceptionlet it through.try/except/else/finally:elseonly on success,finallyalways;returninfinallyoverrides everything.- Chain with
raise New from e;from Nonehides the cause. - Raising costs far more than checking when misses are common (130 ms against 37 ms per million), but check-then-act on files can race:
exists()wasTrueandopen()still failed. - Use
with,suppressandExitStackso cleanup runs on every path. - Eleven
unittesttests ran in 0.3 s; patchingtime.sleepdid nothing for code that importedsleep(0.30 s), patchingshop.sleepdid (0.0006 s). - A test that passed only because of run order failed alone with
0 != 2: keep state per test, mock only the edges, usespec, and treat coverage and green builds as signals, not proof.
Appendix: the experiment code
Python 3.13 (CPython), standard library only (ExceptionGroup and except* need 3.11 or later). Run python3 errors.py for the exception experiments. For the tests, put shop.py and test_shop.py in one folder and run python3 -m unittest -v test_shop; run python3 -m unittest test_shop.Isolation.test_z_depends_on_the_others on its own to see the order-dependent test fail. The test file never touches the real network: the one function that would is always patched. Timings vary by machine.
errors.py: the exception hierarchy, try/else/finally, chaining, groups, resource helpers and the EAFP timings:
import timeit, traceback, contextlib, os, tempfile
# 1. hierarchy
print("KeyError mro:", [c.__name__ for c in KeyError.__mro__])
print("KeyboardInterrupt is Exception:", issubclass(KeyboardInterrupt, Exception), "| BaseException:", issubclass(KeyboardInterrupt, BaseException))
# 2. try / except / else / finally order
def flow(x):
log = []
try:
log.append("try"); v = 10 / x
except ZeroDivisionError:
log.append("except"); v = None
else:
log.append("else")
finally:
log.append("finally")
return log
print(flow(2), flow(0))
# 3. finally runs even on return, and can override it
def f1():
try: return "try"
finally: print(" finally ran before returning")
def f2():
try: return "try"
finally: return "finally" # swallows the original return (and any exception)
print(f1(), f2())
# 4. except order: first match wins
try:
{}["k"]
except LookupError: print("caught as LookupError (parent listed first)")
except KeyError: print("never reached")
# 5. chaining
class AppError(Exception): pass
def load():
try: int("abc")
except ValueError as e: raise AppError("bad config") from e
try: load()
except AppError as e: print("cause:", repr(e.__cause__))
def load2():
try: int("abc")
except ValueError: raise AppError("bad config") # implicit chaining
try: load2()
except AppError as e: print("context:", repr(e.__context__), "| cause:", e.__cause__)
def load3():
try: int("abc")
except ValueError: raise AppError("bad config") from None
try: load3()
except AppError as e: print("from None -> suppressed:", e.__suppress_context__)
# 6. bare except swallows too much
def risky():
raise KeyboardInterrupt
try:
try: risky()
except Exception: print("except Exception did not catch it")
except KeyboardInterrupt: print("KeyboardInterrupt passed through `except Exception`")
try:
try: risky()
except: print("bare except swallowed KeyboardInterrupt")
except KeyboardInterrupt: print("(not reached)")
# 7. custom exception with data
class ValidationError(Exception):
def __init__(self, field, msg): super().__init__(f"{field}: {msg}"); self.field = field
try: raise ValidationError("email", "missing @")
except ValidationError as e: print(e, "| field:", e.field)
# 8. exception groups
try:
raise ExceptionGroup("batch", [ValueError("a"), TypeError("b"), ValueError("c")])
except* ValueError as eg: print("ValueError part:", [str(e) for e in eg.exceptions])
except* TypeError as eg: print("TypeError part:", [str(e) for e in eg.exceptions])
# 9. cleanup with ExitStack and suppress
with contextlib.suppress(FileNotFoundError): os.remove("/nonexistent/file"); print("(not printed)")
print("suppress ok")
d = tempfile.mkdtemp(); paths = [os.path.join(d, f"f{i}") for i in range(3)]
with contextlib.ExitStack() as st:
files = [st.enter_context(open(p, "w")) for p in paths]
for f in files: f.write("x")
print("all closed:", all(f.closed for f in files))
# 10. EAFP vs LBYL cost
d = {"a": 1}
def lbyl_hit(): return d["a"] if "a" in d else None
def eafp_hit():
try: return d["a"]
except KeyError: return None
def lbyl_miss(): return d["z"] if "z" in d else None
def eafp_miss():
try: return d["z"]
except KeyError: return None
n = 1_000_000
for name, fn in (("LBYL key present", lbyl_hit), ("EAFP key present", eafp_hit), ("LBYL key missing", lbyl_miss), ("EAFP key missing", eafp_miss)):
print(f"{name}: {timeit.timeit(fn, number=n)*1e3:.0f} ms per 1M")
# 11. TOCTOU: the check and the use are separate steps
tmp = os.path.join(tempfile.mkdtemp(), "x"); open(tmp, "w").close()
if os.path.exists(tmp): # check passes ...
os.remove(tmp) # ... then something else removes the file
try: open(tmp)
except FileNotFoundError: print("exists() was True, open() still failed: FileNotFoundError")
# 12. traceback
try: load2()
except AppError: print(traceback.format_exc().count("During handling of the above exception"), "chained section in traceback")
t/shop.py: the small module under test:
import json, urllib.request, time
from time import sleep
def price_with_tax(price, rate=0.2):
return round(price * (1 + rate), 2)
def fetch_rate(country):
with urllib.request.urlopen(f"https://rates.example/{country}") as r:
return json.load(r)["rate"]
def total(items, country):
rate = fetch_rate(country)
return round(sum(price_with_tax(p, rate) for p in items), 2)
def is_open(now=None):
now = now if now is not None else time.localtime()
return 9 <= now.tm_hour < 17
def pause():
sleep(0.3) # imported name: it lives in this module's namespace as shop.sleep
t/test_shop.py: the tests, with table-driven cases, mocks and the independence demonstration:
import time
import unittest
from unittest import mock
import shop
class PriceTests(unittest.TestCase):
def test_default_rate(self):
self.assertEqual(shop.price_with_tax(100), 120.0)
def test_rounding_table(self):
for price, rate, expected in [(10, 0.2, 12.0), (0.1, 0.2, 0.12), (19.99, 0.07, 21.39), (0, 0.5, 0.0)]:
with self.subTest(price=price, rate=rate):
self.assertEqual(shop.price_with_tax(price, rate), expected)
def test_float_comparison(self):
self.assertNotEqual(0.1 + 0.2, 0.3)
self.assertAlmostEqual(0.1 + 0.2, 0.3)
class TotalTests(unittest.TestCase):
def test_total_with_fake_rate(self):
with mock.patch("shop.fetch_rate", return_value=0.1) as fake: # patch where it is USED
self.assertEqual(shop.total([10, 20], "BR"), 33.0)
fake.assert_called_once_with("BR")
def test_network_failure_propagates(self):
with mock.patch("shop.fetch_rate", side_effect=TimeoutError("slow")):
with self.assertRaises(TimeoutError):
shop.total([1], "BR")
class OpeningHoursTests(unittest.TestCase):
def test_boundaries(self):
T = lambda h: mock.Mock(tm_hour=h) # pass the time in; no clock patching needed
self.assertFalse(shop.is_open(T(8)))
self.assertTrue(shop.is_open(T(9)))
self.assertTrue(shop.is_open(T(16)))
self.assertFalse(shop.is_open(T(17)))
class Isolation(unittest.TestCase):
shared = []
def setUp(self): self.local = []
def test_a(self): self.shared.append(1); self.local.append(1); self.assertEqual(len(self.local), 1)
def test_b(self): self.shared.append(2); self.local.append(2); self.assertEqual(len(self.local), 1)
def test_z_depends_on_the_others(self): self.assertEqual(len(self.shared), 2) # passes only after test_a and test_b
class MockTrap(unittest.TestCase):
def test_patch_where_it_is_used(self):
t = time.perf_counter()
with mock.patch("time.sleep"): # patches the time module, but shop already holds its own name
shop.pause()
slow = time.perf_counter() - t
t = time.perf_counter()
with mock.patch("shop.sleep"): # patches the name shop actually calls
shop.pause()
fast = time.perf_counter() - t
print(f"\n patch('time.sleep'): {slow:.2f} s | patch('shop.sleep'): {fast:.4f} s")
self.assertGreater(slow, 0.25); self.assertLess(fast, 0.05)
def test_mock_accepts_anything(self):
m = mock.Mock()
m.no_such_method(1, 2) # a plain Mock never complains
spec = mock.Mock(spec=shop)
with self.assertRaises(AttributeError):
spec.no_such_function() # spec turns typos into errors
if __name__ == "__main__":
unittest.main(verbosity=2)
Sources
- Python documentation, Errors and Exceptions, Built-in Exceptions (hierarchy, exception groups) and The
trystatement. - Python documentation,
contextlib(suppress,ExitStack,contextmanager) andlogging(Logger.exception). - Python documentation,
unittestandunittest.mock, including where to patch. - Python Enhancement Proposals, PEP 3134: Exception Chaining and Embedded Tracebacks and PEP 654: Exception Groups and
except*. - pytest documentation, Get started, fixtures and parametrize.
- Martin Fowler, Mocks Aren't Stubs (2007), on test doubles.
Practise this topic: Python