Python: decorators

A function is a value, just like a number#

Nothing else works without this: in Python you can put a function in a variable, pass it as an argument and return it from another function.

def shout(text):
    return text.upper()

f = shout                 # no parentheses — the function itself, not its result
print(f("hello"))         # HELLO
print(list(map(shout, ["a", "b"])))   # ['A', 'B']

Parentheses mean a call. f = shout is a reference; f = shout("x") is the result of a call. Mixing the two up causes half the bugs in this topic.

The simplest decorator#

A decorator takes a function, defines a wrapper inside and returns it.

def loud(func):
    def wrapper(*args, **kwargs):
        print("calling", func.__name__)
        result = func(*args, **kwargs)
        print("result", result)
        return result
    return wrapper

@loud
def add(a, b):
    return a + b

print(add(2, 3))
  # calling add
  # result 5
  # 5

Writing @loud above def add is equivalent to the line add = loud(add) after the definition. There is nothing more to it.

The wrapper takes *args, **kwargs so that it fits a function with any signature, and it must return the result — otherwise the decorated function starts silently returning None. That is the bug people spend the longest hunting for.

Keeping the function's name#

Without extra care, the decorated function is replaced by the wrapper, and anything that looks at the name or docstring notices:

print(add.__name__)   # wrapper

The fix is one line — functools.wraps copies the name, docstring and annotations from the original function to the wrapper:

import functools

def loud(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper

@loud
def add(a, b):
    """Add two numbers."""
    return a + b

print(add.__name__, "|", add.__doc__)   # add | Add two numbers.

A decorator with arguments#

When a decorator has its own settings, a third level appears: the outer function takes the settings and returns the decorator itself.

import functools

def repeat(times):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for _ in range(times):
                result = func(*args, **kwargs)
            return result
        return wrapper
    return decorator

@repeat(times=3)
def hello():
    print("hello")
    return "ok"

print(hello())
  # hello / hello / hello / ok

Hence the rule: a decorator with arguments is written with parentheses — @repeat(3) — and one without arguments has none — @loud. They are easy to mix up: @repeat without parentheses passes the function itself into the times parameter, and the crash happens not at definition time but on the first call.

Order and ready-made decorators#

Several decorators apply from the bottom up — the one closest to def wraps first:

@a
@b
def f():
    ...
  # equivalent to f = a(b(f))

The standard library already has useful decorators. functools.lru_cache remembers the results of calls:

import functools

@functools.lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

print(fib(30))            # 832040
print(fib.cache_info().hits)   # 28

Without the cache, the same fib(30) would make more than a million calls. Other familiar ones are @property, @staticmethod and @classmethod, which apply to methods and are covered in the article on Python classes.

Half an hour of practice: write a timed decorator that prints the run time using time.perf_counter, and a retry(times) decorator that repeats the call when an exception is raised — the second one relies on Python exceptions.

Step-by-step plan

1. A function in a variableAssign a function to a variable without parentheses, call it by the new name, compare with assigning the result.
2. A wrapper by handWrite a wrapper and apply it with the line f = deco(f), without the @ sign.
3. Switch to @Replace the manual assignment with the decorator syntax and check that the behaviour is the same.
4. Add functools.wrapsCheck __name__ before and after adding wraps.
5. A decorator with a parameterWrite repeat(times) and call the function with different parameter values.

Check yourself

Sources

• The functools modulewraps, lru_cache, cache and other ready-made wrappers
  free
• Language reference: function definitionsThe formal description of what the @ sign expands to
  free
• PEP 318The proposal that introduced decorator syntax and its reasoning
  free

Was this helpful?

Yes No