CS Fundamentals Trainer · Guides

Python errors, resources and testing

The exception hierarchy and try/else/finally, chaining with from, EAFP against LBYL (measured), context managers and ExitStack, and unittest with mocks: patching where a name is used, spec, test independence, flaky tests and what coverage means.

19 min read

Every program fails: files are missing, networks time out, users type nonsense, and code has bugs. What separates a reliable program is not that it avoids failure but that it fails in a way you can see, understand and recover from, and that you find the bugs before your users do. In Python that means three connected skills: using exceptions well, releasing resources no matter what happens, and writing tests that fail for the right reasons. This guide covers all three with scripts run on CPython 3.13 (in the appendix). It builds on the context managers and generators in how Python works and decorators, generators and async.

A limit on what I could run: the sandbox had the standard unittest framework but not pytest, so every test run shown here is unittest. The section on pytest describes it from its documentation and shows no output of its own.

1. How exceptions work

An exception is an object. When code raises one, Python stops the current function, looks for a matching except in the caller, then in its caller, and so on up the stack; if none matches, the program ends and prints a traceback, the list of calls that led there.

The hierarchy

All exceptions inherit from BaseException. Most you meet inherit from Exception, and the standard ones are organised in families: KeyError has the chain KeyError, LookupError, Exception, BaseException, object, so except LookupError catches both KeyError and IndexError. Three exceptions sit outside Exception, directly under BaseException: KeyboardInterrupt (Ctrl-C), SystemExit (sys.exit()), and GeneratorExit. That is deliberate: except Exception should not stop the user from interrupting the program.

I tested what a bare except: does with that. Code that raised KeyboardInterrupt passed straight through except Exception (correct), and was swallowed by a bare except:. So the rules are:

The full try statement

try:
    value = 10 / x
except ZeroDivisionError:
    value = None
else:
    ...        # runs only if the try body did NOT raise
finally:
    ...        # always runs

The order of events I logged: with x = 2 it was try, else, finally; with x = 0 it was try, except, finally. Put as little as possible in the try body, so that an unrelated error is not caught by mistake; code that should run only on success belongs in else. finally runs on success, on handled errors, on unhandled errors, and on return, break and continue.

One trap: return inside finally overrides the function's result and discards any exception in flight. A function with try: return "try" and finally: return "finally" returned finally. Do not return, break or continue in a finally block.

Chaining: keep the original cause

When you catch one error and raise another, say how they relate:

You write__cause____context__Traceback says
raise AppError(...) from ethe original ValueErrorthe original"The above exception was the direct cause of the following exception"
raise AppError(...) inside exceptNonethe original ValueError"During handling of the above exception, another exception occurred"
raise AppError(...) from NoneNonestill set, but hidden (__suppress_context__ is True)only the new exception

Use from e when you translate a low-level error into your own (so the cause stays in the log), and from None only when the original is noise. A traceback that shows both is much easier to debug than one that shows only the final error.

Design your own exceptions

Define a small hierarchy for your application (class AppError(Exception), class ValidationError(AppError)), so callers can catch your errors without catching everything. Put data on the exception when the handler needs it: my ValidationError("email", "missing @") carried field = 'email' and printed email: missing @. Raise exceptions for exceptional conditions and let normal outcomes be return values; do not use exceptions as a way to return a result from deep inside normal logic.

Exception groups (3.11+)

When several things fail at once (concurrent tasks, a batch), ExceptionGroup bundles them, and except* handles the parts by type. A group of ValueError("a"), TypeError("b") and ValueError("c") gave ValueError part: ['a', 'c'] and TypeError part: ['b']. asyncio.TaskGroup raises these (see the async guide). You cannot return, break or continue inside an except* block.

Look before you leap, or ask forgiveness

Python style often prefers EAFP ("easier to ask forgiveness than permission"): try the operation and handle the exception, rather than LBYL (check first). Two reasons: it is often cleaner, and the check can be wrong by the time you act on it. I made a file, confirmed os.path.exists() was True, removed the file as another process might, then opened it: FileNotFoundError, although the check had passed. For files, locks and database rows, try the operation and handle the failure.

On speed, I timed a dictionary lookup a million times each way:

CaseLBYL (if k in d)EAFP (try: d[k])
key present50 ms37 ms
key missing37 ms130 ms

Handling an exception is cheap when nothing is raised and expensive when something is. So use EAFP when failure is the rare case, and a check (or dict.get) when a miss is common.

2. Resources: always release them

Files, sockets, locks, database connections and temporary directories must be released even when something fails. The tool is the with statement (see how Python works): the cleanup in __exit__ runs however the block ends.

Also keep resources from leaking in long-running code: close database cursors and connections, put timeouts on network calls, and bound queues and caches.

Logging, not printing

At a boundary where you catch and continue, record the failure with the traceback: logging.exception("could not process order %s", order_id) inside an except block logs the message at ERROR level together with the full traceback. Include the identifiers you need to find the case later, and never log secrets or personal data (see observability and web application security). Decide for each error one of four responses: handle it (retry, fall back), translate it (raise your own with from e), let it propagate (the caller knows better), or crash (the program cannot continue safely). Logging and then re-raising at every level produces the same error ten times in the log; log once, where you handle it.

3. Testing

What a test is

A test is a small program that runs your code with known input and checks the output. Good tests are fast, independent, repeatable and readable: when one fails, the name and the message should tell you what broke.

I wrote a small module shop.py (a price with tax, a function that fetches a tax rate over the network, a total, and an opening-hours check) and tested it with unittest, which ships with Python. Run it with python3 -m unittest -v. All 11 tests passed in 0.3 s. What the tests demonstrate:

Table-driven cases. subTest runs one test body over many inputs and reports each failing row separately: (10, 0.2, 12.0), (0.1, 0.2, 0.12), (19.99, 0.07, 21.39), (0, 0.5, 0.0). Include boundaries and zero. (pytest does this with @pytest.mark.parametrize.)

Floating point. 0.1 + 0.2 == 0.3 is False; assertAlmostEqual passed. Never compare floats with == (use math.isclose or the framework's helper), and keep money in integers or decimal.Decimal.

Failure paths. assertRaises checks that the right exception is raised. A test that mocks a timeout and expects TimeoutError to propagate pins down behaviour that otherwise only appears in production.

Boundaries in time. The opening-hours function takes the time as a parameter, so the test passes 8, 9, 16 and 17 and checks False, True, True, False with no clock patching. Designing code so that dependencies are passed in (the clock, the random generator, the HTTP client) is the best way to make it testable, much better than patching later.

Test doubles and mocks

A test should not call the real network, send real email or depend on today's date. Replace those pieces with test doubles. unittest.mock.patch("shop.fetch_rate", return_value=0.1) replaced the network call, the total came out at the expected 33.0, and assert_called_once_with("BR") checked how the function was called. Three traps, two of which I reproduced:

  1. Patch the name where it is used, not where it is defined. shop.py does from time import sleep, so shop holds its own reference named sleep. Patching time.sleep changed nothing: the call still took 0.30 s. Patching shop.sleep made it instant (0.0006 s). If a patch seems to do nothing, this is almost always why.
  2. A plain Mock accepts anything. m.no_such_method(1, 2) raised nothing, so a typo in a test can pass silently. Mock(spec=shop) (or autospec=True) raises AttributeError for names the real object does not have.
  3. Over-mocking tests the mock. If you replace everything around the code, the test verifies your assumptions about those parts, not the real behaviour. Mock the slow, non-deterministic edges (network, clock, randomness); use real objects, or fast fakes such as an in-memory database, for the rest. Add a small number of integration tests against the real thing.

Independence and order

Tests must not depend on each other. I made a class where test_a and test_b append to a shared class-level list and a third test asserts the list has two items. In the full run it passed, only because of the (alphabetical) execution order; run on its own it failed with AssertionError: 0 != 2. Create state per test in setUp (or fixtures), never share mutable state through class or module variables, and run tests in random order in CI to expose accidental dependencies. Likewise, give tests their own temporary files and database rows and clean them up.

What to test, and how much

pytest

Most Python projects use pytest instead of unittest; it was not available where I ran these scripts, so this is from its documentation and not from a run. It finds functions named test_*, uses plain assert statements and rewrites them to show the values involved when they fail, provides fixtures (functions that create and clean up test state, injected by name into the tests that need them), and @pytest.mark.parametrize for tables. It can also run unittest tests, so migrating can be gradual. Everything in this section (injecting dependencies, mocking at the edges, independence, behaviour over implementation) applies identically.

4. Common mistakes

5. Questions interviewers like to ask

"What is the difference between except Exception and a bare except?"

A bare except also catches BaseException subclasses such as KeyboardInterrupt and SystemExit, which makes the program hard to stop. except Exception catches ordinary errors only. Prefer a specific class.

"When does finally run, and what can go wrong?"

Whenever the try block is left: normally, by exception, or by return, break or continue. A return inside finally replaces the original return value and swallows any exception.

"What does raise ... from e do?"

It sets __cause__ so the traceback shows the original error as the direct cause. from None hides the original; without from, the original is shown as context only.

"EAFP or LBYL?"

EAFP (try and handle) is usually cleaner and avoids races between check and use, but a raised exception is slower than a check, so for a common failure case a check or dict.get is better.

"How would you test a function that calls an external API?"

Pass the client in (or patch the name the function actually uses), return controlled responses, test the success, error and timeout paths, and add a few integration tests against a sandbox. Use spec or autospec so a typo in the mock fails.

"What makes a test flaky, and what do you do about it?"

Shared state, order dependence, real time or network, randomness, race conditions. Isolate state per test, inject the clock and the random seed, replace the network with a double, and fix or delete the test rather than re-running it.

"Is 100% coverage enough?"

No. It shows lines were executed, not that results were checked or that edge cases were covered.

6. Things to mention in an interview

Quick recap

Appendix: the experiment code

Python 3.13 (CPython), standard library only (ExceptionGroup and except* need 3.11 or later). Run python3 errors.py for the exception experiments. For the tests, put shop.py and test_shop.py in one folder and run python3 -m unittest -v test_shop; run python3 -m unittest test_shop.Isolation.test_z_depends_on_the_others on its own to see the order-dependent test fail. The test file never touches the real network: the one function that would is always patched. Timings vary by machine.

errors.py: the exception hierarchy, try/else/finally, chaining, groups, resource helpers and the EAFP timings:

import timeit, traceback, contextlib, os, tempfile

# 1. hierarchy
print("KeyError mro:", [c.__name__ for c in KeyError.__mro__])
print("KeyboardInterrupt is Exception:", issubclass(KeyboardInterrupt, Exception), "| BaseException:", issubclass(KeyboardInterrupt, BaseException))

# 2. try / except / else / finally order
def flow(x):
    log = []
    try:
        log.append("try"); v = 10 / x
    except ZeroDivisionError:
        log.append("except"); v = None
    else:
        log.append("else")
    finally:
        log.append("finally")
    return log
print(flow(2), flow(0))

# 3. finally runs even on return, and can override it
def f1():
    try: return "try"
    finally: print("  finally ran before returning")
def f2():
    try: return "try"
    finally: return "finally"      # swallows the original return (and any exception)
print(f1(), f2())

# 4. except order: first match wins
try:
    {}["k"]
except LookupError: print("caught as LookupError (parent listed first)")
except KeyError: print("never reached")

# 5. chaining
class AppError(Exception): pass
def load():
    try: int("abc")
    except ValueError as e: raise AppError("bad config") from e
try: load()
except AppError as e: print("cause:", repr(e.__cause__))
def load2():
    try: int("abc")
    except ValueError: raise AppError("bad config")      # implicit chaining
try: load2()
except AppError as e: print("context:", repr(e.__context__), "| cause:", e.__cause__)
def load3():
    try: int("abc")
    except ValueError: raise AppError("bad config") from None
try: load3()
except AppError as e: print("from None -> suppressed:", e.__suppress_context__)

# 6. bare except swallows too much
def risky():
    raise KeyboardInterrupt
try:
    try: risky()
    except Exception: print("except Exception did not catch it")
except KeyboardInterrupt: print("KeyboardInterrupt passed through `except Exception`")
try:
    try: risky()
    except: print("bare except swallowed KeyboardInterrupt")
except KeyboardInterrupt: print("(not reached)")

# 7. custom exception with data
class ValidationError(Exception):
    def __init__(self, field, msg): super().__init__(f"{field}: {msg}"); self.field = field
try: raise ValidationError("email", "missing @")
except ValidationError as e: print(e, "| field:", e.field)

# 8. exception groups
try:
    raise ExceptionGroup("batch", [ValueError("a"), TypeError("b"), ValueError("c")])
except* ValueError as eg: print("ValueError part:", [str(e) for e in eg.exceptions])
except* TypeError as eg: print("TypeError part:", [str(e) for e in eg.exceptions])

# 9. cleanup with ExitStack and suppress
with contextlib.suppress(FileNotFoundError): os.remove("/nonexistent/file"); print("(not printed)")
print("suppress ok")
d = tempfile.mkdtemp(); paths = [os.path.join(d, f"f{i}") for i in range(3)]
with contextlib.ExitStack() as st:
    files = [st.enter_context(open(p, "w")) for p in paths]
    for f in files: f.write("x")
print("all closed:", all(f.closed for f in files))

# 10. EAFP vs LBYL cost
d = {"a": 1}
def lbyl_hit():  return d["a"] if "a" in d else None
def eafp_hit():
    try: return d["a"]
    except KeyError: return None
def lbyl_miss(): return d["z"] if "z" in d else None
def eafp_miss():
    try: return d["z"]
    except KeyError: return None
n = 1_000_000
for name, fn in (("LBYL key present", lbyl_hit), ("EAFP key present", eafp_hit), ("LBYL key missing", lbyl_miss), ("EAFP key missing", eafp_miss)):
    print(f"{name}: {timeit.timeit(fn, number=n)*1e3:.0f} ms per 1M")
# 11. TOCTOU: the check and the use are separate steps
tmp = os.path.join(tempfile.mkdtemp(), "x"); open(tmp, "w").close()
if os.path.exists(tmp):          # check passes ...
    os.remove(tmp)               # ... then something else removes the file
    try: open(tmp)
    except FileNotFoundError: print("exists() was True, open() still failed: FileNotFoundError")
# 12. traceback
try: load2()
except AppError: print(traceback.format_exc().count("During handling of the above exception"), "chained section in traceback")

t/shop.py: the small module under test:

import json, urllib.request, time
from time import sleep

def price_with_tax(price, rate=0.2):
    return round(price * (1 + rate), 2)

def fetch_rate(country):
    with urllib.request.urlopen(f"https://rates.example/{country}") as r:
        return json.load(r)["rate"]

def total(items, country):
    rate = fetch_rate(country)
    return round(sum(price_with_tax(p, rate) for p in items), 2)

def is_open(now=None):
    now = now if now is not None else time.localtime()
    return 9 <= now.tm_hour < 17

def pause():
    sleep(0.3)            # imported name: it lives in this module's namespace as shop.sleep

t/test_shop.py: the tests, with table-driven cases, mocks and the independence demonstration:

import time
import unittest
from unittest import mock
import shop

class PriceTests(unittest.TestCase):
    def test_default_rate(self):
        self.assertEqual(shop.price_with_tax(100), 120.0)

    def test_rounding_table(self):
        for price, rate, expected in [(10, 0.2, 12.0), (0.1, 0.2, 0.12), (19.99, 0.07, 21.39), (0, 0.5, 0.0)]:
            with self.subTest(price=price, rate=rate):
                self.assertEqual(shop.price_with_tax(price, rate), expected)

    def test_float_comparison(self):
        self.assertNotEqual(0.1 + 0.2, 0.3)
        self.assertAlmostEqual(0.1 + 0.2, 0.3)

class TotalTests(unittest.TestCase):
    def test_total_with_fake_rate(self):
        with mock.patch("shop.fetch_rate", return_value=0.1) as fake:     # patch where it is USED
            self.assertEqual(shop.total([10, 20], "BR"), 33.0)
            fake.assert_called_once_with("BR")

    def test_network_failure_propagates(self):
        with mock.patch("shop.fetch_rate", side_effect=TimeoutError("slow")):
            with self.assertRaises(TimeoutError):
                shop.total([1], "BR")

class OpeningHoursTests(unittest.TestCase):
    def test_boundaries(self):
        T = lambda h: mock.Mock(tm_hour=h)          # pass the time in; no clock patching needed
        self.assertFalse(shop.is_open(T(8)))
        self.assertTrue(shop.is_open(T(9)))
        self.assertTrue(shop.is_open(T(16)))
        self.assertFalse(shop.is_open(T(17)))

class Isolation(unittest.TestCase):
    shared = []
    def setUp(self): self.local = []
    def test_a(self): self.shared.append(1); self.local.append(1); self.assertEqual(len(self.local), 1)
    def test_b(self): self.shared.append(2); self.local.append(2); self.assertEqual(len(self.local), 1)
    def test_z_depends_on_the_others(self): self.assertEqual(len(self.shared), 2)   # passes only after test_a and test_b

class MockTrap(unittest.TestCase):
    def test_patch_where_it_is_used(self):
        t = time.perf_counter()
        with mock.patch("time.sleep"):              # patches the time module, but shop already holds its own name
            shop.pause()
        slow = time.perf_counter() - t
        t = time.perf_counter()
        with mock.patch("shop.sleep"):              # patches the name shop actually calls
            shop.pause()
        fast = time.perf_counter() - t
        print(f"\n  patch('time.sleep'): {slow:.2f} s | patch('shop.sleep'): {fast:.4f} s")
        self.assertGreater(slow, 0.25); self.assertLess(fast, 0.05)

    def test_mock_accepts_anything(self):
        m = mock.Mock()
        m.no_such_method(1, 2)                      # a plain Mock never complains
        spec = mock.Mock(spec=shop)
        with self.assertRaises(AttributeError):
            spec.no_such_function()                 # spec turns typos into errors

if __name__ == "__main__":
    unittest.main(verbosity=2)

Sources

  1. Python documentation, Errors and Exceptions, Built-in Exceptions (hierarchy, exception groups) and The try statement.
  2. Python documentation, contextlib (suppress, ExitStack, contextmanager) and logging (Logger.exception).
  3. Python documentation, unittest and unittest.mock, including where to patch.
  4. Python Enhancement Proposals, PEP 3134: Exception Chaining and Embedded Tracebacks and PEP 654: Exception Groups and except*.
  5. pytest documentation, Get started, fixtures and parametrize.
  6. Martin Fowler, Mocks Aren't Stubs (2007), on test doubles.

Practise this topic: Python

More guides