Error Handling

Failing safely — exceptions, exception chaining, and custom exception hierarchies

Why

Production code fails — network requests time out, files are missing, APIs return unexpected shapes. The difference between hobby code and production code is how failures are handled. Catching the wrong exception silently swallows bugs; catching too broadly hides problems; not chaining exceptions loses the root cause.

Custom exception hierarchies are how you build APIs where callers can handle errors at the right level of specificity — catching a module's base exception type without needing to know all its internal subtypes.

Intuition

try/except separates the happy path from the failure path. The else block runs only when no exception occurred — it's for code that should run on success but shouldn't be inside try where its exceptions might be caught by the wrong handler. The finally block always runs — use it for guaranteed cleanup.

Explanation

1. try / except / else / finally

Python's error handling separates the happy path from failure. Each clause has a distinct role — catching specific exceptions prevents accidentally swallowing bugs you didn't intend to handle.

try: result = fetch_data(url) # operation that might fail except requests.Timeout: # most specific first result = retry(url) except (ValueError, KeyError) as e: # multiple types in a tuple raise DataError("bad response") from e # chain — preserves cause else: cache.store(result) # ONLY if no exception raised finally: metrics.increment("requests") # ALWAYS runs — even on re-raise # Exception hierarchy: # BaseException # SystemExit, KeyboardInterrupt ← don't catch these broadly # Exception # ValueError, TypeError, KeyError, AttributeError... # OSError → FileNotFoundError, PermissionError... # Never: except Exception: pass — silently swallows everything

Catch the most specific exception type you can handle. Use raise ... from e to chain exceptions so the original traceback isn't lost. The else block prevents a handler from masking errors from unrelated code.

2. Custom exception hierarchies

Defining your own exception classes lets callers catch errors at the right level — either a specific subtype or the whole module's base error — without knowing every internal detail.

class PipelineError(Exception): """Base for all pipeline failures — callers catch this.""" class DataValidationError(PipelineError): def __init__(self, column, reason): self.column = column super().__init__(f"Column '{column}': {reason}") class SchemaError(PipelineError): ... class IngestionError(PipelineError): ... # Raise with context — preserves original traceback try: validate_schema(df) except KeyError as e: raise DataValidationError("user_id", "missing") from e # Callers catch at the right level of specificity: except DataValidationError: # this specific error except PipelineError: # any pipeline failure except Exception: # nuclear — only at top level

Define a base exception per module. This lets you change internal exception types without breaking callers who catch the base. Always include enough context in the message to debug without a full stacktrace.

← Previous Next →