Guide to: Learning Python Decorators

Matt Harrison

Copyright © 2012
While every precaution has been taken in the preparation of this book, the
publisher and author assumes no responsibility for errors or omissions, or
for damages resulting from the use of the information contained herein.
Table of Contents
Introduction
Programming Styles
Functions
Function Parameters
Closures
Decorators
About the Author
Also Available
Introduction

Decorators are pretty common in Python. While not strictly necessary, they
can reduce code size while enabling control of function input, invocation
and output. Many explanations of decorators are brief and leave the reader
somewhat confused. This book is an attempt to remedy that. It is based off
of popular tutorials given at PyCon and Python user groups. Without fail,
attendees mention that “decorators now make sense”. Not only do they
understand them, but they can write them as well.
This is a quick yet detailed introduction to decorators. It starts off with an
introduction to many aspects of Python functions, and then proceeds to
closures. With the underlying foundation covered, decorators are discussed
in detail. Some might find this style pendantic, but the during teaching
people have commented that the review is useful and that they learned a
few things along the way.
It is assumed that the reader has some knowledge of Python.
Good luck in your quest to unlock the power of decorators!
– Matt
Programming Styles

Python supports multiple programming paradigms. The following

programming styles are available:

Imperative/Procedural
Object Oriented
Declarative/Functional

In the imperative style of programming, statements are executed to

mutate the state. Procedural programming is a subset of imperative that
relies on using procedures to evaluate these state changing computations. In
Python, procedures are akin to functions. State for the program is stored in
local or global data structures and is manipulated by invoking functions.
This is the same style that C uses.
In object oriented programming, objects are the main methods of
computation. In addition these objects hold program state, and allow
mutation of that state through invocation of thier methods. Java is object
oriented, as is C++. Though C++ can also be programmed in an imperative
(read C) style as well — because it is a superset of C.
In a declarative style of programming, the code describes the what rather
than the how. SQL, for example, is declarative because it describe the
relational algebra for the result of a query. In contrast, it is possible to write
Python code in an imperative style to join, select, and filter rows of data.
The SQL code would describe the result, while the Python version would
describe to process of computing the result — the code that does looping
and filtering. Other examples of declarative programming are XSLT and
regular expressions, because both describe the result rather the mechanics
of calculating it. Again, it is also possible to imperatively code similar
functionality. Python is commonly used to process XML and text without
resorting to XSLT or using regular expressions.
 Functional programming is a subset of declarative programming that
achieves the logic of the program by invoking functions. Usually, in strict
functional languages, side effects are kept to a minimum or eliminated.
Calling a function with the same input should be idempotent and always
return the same value. By eliminating side effects, compilers may be able to
make some optimizations to the running code. Features such as non-
mutable state help to ensure side effects are minimal in functional
languages. Because functions are first-class in Python, it includes support
for many functional features.
Functions

The basic definition of a function is simple in Python — it is a structure for

abstracting a block of code that accepts input, does some processing, and
returns output:

def function_name(input_1, input_2):

# some processing

return output
First Class Functions

Unlike some languages, Python supports first-class functions. The idea that
"functions [are] first-class citizens", coined in the 60's by British computer
scientist Christopher Strachey, allows for functions to be passed into other
functions, stored as variables and returned from functions. This is a hard
requirement for functional languages such as Lisp. In Lisp, the function map
takes a function as its first parameter and applies to members of the second
parameter. Python also has a map function to perform an analogous
operation. In contrast, the Java language does not support passing around
functions.
Function Instances

In Python, after a function is defined, the function name then refers to a

function instance:

>>> def foo():

... 'docstring for foo'

... print 'invoked foo'

>>> print foo

<function foo at 0x7f3916f02938>

Note that the above code did not invoke the function, it only printed out
the string representation of the function foo.
Using the type function, the foo object can be introspected:

>>> type(foo)

<type 'function'>

In Python, there is a built-in class, function, for representing functions.

Also, functions can be assigned to other variables:

>>> bar = foo

>>> print bar

<function foo at 0x7f3916f02938>

Both foo and bar could be passed into other functions as arguments.
They could also be returned as the result of invoking other functions:

>>> def get_foo():

... return foo

>>> print get_foo

<function get_foo at 0x7ffe7c4408c0>

>>> print get_foo()

<function foo at 0x7f3916f02938>

Because functions can return other functions, it is possible to create

function factories.
Invoking Functions

This might seem obvious, but it bears mentioning here. Functions are
callable. Because they are callable, they can be invoked. Python provides a
way to introspect whether objects are invokable with the callable
function:

>>> callable(foo)

True

Note
Python 3 removed the callable function. Two alternatives (that also
work in Python 2.6 and above) are:

>>> hasattr(foo, '__call__')

and:

>>> isinstance(foo, collections.Callable)

The invocation construct in Python is to add parentheses around any

arguments to a callable object. To invoke foo, simply add parentheses (with
no arguments since it has none) to it:

>>> foo()

invoked foo

Note
On a somewhat related note, if a class implements the __call__ method,
you can invoke instances of the class:

>>> class CallMe:

 ... def __call__(self):

... print "Called"

>>> c = CallMe()

>>> c()

Called

This is subtle but this feature allows for using instances as decorators (or
anything that is callable for that matter).
Note that functions also have __call__ methods, that are invoked under
the hood as well.
Functions have attributes

Instances of classes in Python have attributes. Functions, being instances of

function, likewise have attributes. Invoking dir on the function instance
will list the attributes:

>>> dir(foo)

['__call__', '__class__', '__closure__',

...

'__doc__',

...

'__name__',

'func_closure', 'func_code', 'func_defaults',

'func_dict', 'func_doc', 'func_globals', 'func_name']

As mentioned in the previous section, because functions are callable,

they have a __call__ method. Two other interesting attributes of functions
are the func_name attribute and the func_doc attribute, which have
alternative dunder spellings as well, __name__ and __doc__ respectively.
Because the func_* attributes are removed in Python 3 it is preferable to
use the dunder versions as they work in both Python 2 and 3:

>>> foo.__name__

'foo'

A variable holding a function will also list the function's name when
asked, not the variable name:

>>> bar = foo

>>> bar.__name__

'foo'

The attribute __doc__ is the docstring for the function:

>>> foo.__doc__
'docstring for foo'

Note
PEP 232, introduced Function Attributes in Python 2.1. Function
attributes allow getting and setting of arbitrary members to function
instances. This data is stored in the __dict__ dictionary:

>>> foo.note = 'more info'

>>> foo.__dict__

{'note': 'more info'}

Note that __dict__ is available in both Python 2 and 3.

Tip
Another function attribute is func_defaults. Python stores default
parameters in func_defaults. It is suggested that default parameters use
only non-mutable types. However, it is not uncommon to see Python
beginners use [] as a default parameter. This eventually leads them down a
debugging rabbit hole. To understand why, it is important to understand
when the default parameters are created. Default parameters are initialized
during function definition time, which occurs at either module load time or
during program execution. During module load time, globally defined
functions are created. Because Python has first-class functions, functions
can also be created and returned from inside the body of other function
during runtime. When the function is created, the default parameters are
examined and stored in the func_defaults attribute.
The following is a function that returns positive values from an iterable,
but will also append them to another list, seq, if seq is passed into the
function:

>>> def positive(items, seq=[]):

... for item in items:

... if item >= 0:

... seq.append(item)

... return seq

 >>> positive

<function positive at 0x7ffe7c440b18>

>>> positive.func_defaults

([],)

In this case, the first and only item in func_defaults is an empty list,
which is mutable. On the first invocation of positive, it appears to behave
correctly:

>>> positive([1])

[1]

At this point, examining the func_defaults no longer shows that the

default for seq should be an empty list. The list, being a mutable type, is
still the same instance defined during function creation, but it is now
populated from the interaction of the previous invocation:

>>> positive.func_defaults

([1],)

This leads to errors on subsequent invocations:

>>> positive([-2])

[1]

>>> positive([5])

[1, 5]

>>> positive.func_defaults

([1, 5],)

The general solution to resolve the need for mutable default parameters
is to shift their creation from module import time to runtime. A common
idiom is to have the parameter default to None then check for that value in
the body of the function:

>>> def positive2(items, seq=None):

... seq = seq or []

... for item in items:

 ... if item >= 0:

... seq.append(item)

... return seq

>>> positive2([1])

[1]

>>> positive2([-2])

[]

>>> positive2([5])

[5]

The above example was somewhat contrived to illustrate the problems

with using mutable types as default parameters. If a function was needed to
actually perform the logic of positive2, a more pythonic solution would be
to use a list comprehension:

>>> items = [3, 4, -5]

>>> pos = seq.extend([x for x in items if x >= 0])

Note
The function attribute func_defaults becomes __defaults__ in Python
3. Note that unlike the other attributes that have extra dunder spellings in
Python 2, __defaults__ does not exist in Python 2.
Function Scope

Functions are defined in a scope. Within the body of a function, you can
access the function itself, as well as anything that was in scope during the
definition of the function. This enables recursion (a function being able to
call itself), but also allows for reading and setting the attributes of a
function while it is executing:

>>> def foo2():

... print "NAME", foo2.__name__

>>> foo2()

NAME foo2

Any variable instances created within a function but not returned from it
are local to the function and will be garbage collected when the function
exits:

>>> def multiply(num1, num2):

... result = 0

... for i in range(num2):

... result = result + num1

... return result

>>> multiply(2, 10)

20

>>> print i

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

NameError: name 'i' is not defined

Given that the function name is in global scope within the body of that
function, it is possible to attach data to the function while the the function is
executing. It is also possible to attach data to a function outside of the
function, even before it is ever executed:
>>> def foo3():

... print foo3.stuff

>>> foo3.stuff = "Hello"

>>> foo3()

Hello

Note
The built-in functions locals and globals will return a mapping of
mapping of names in their respective namespaces to objects that the names
contain:

>>> def local_test():

... a = 1

... b = 'b'

... print locals()

>>> local_test()

{'a': 1, 'b': 'b'}

>>> globals()

{'bar': <function foo at 0x840b90>,

'__builtins__': {'bytearray': <type 'bytearray'>,

...

'foo': <function foo at 0x840b90>,

'foo3': <function foo3 at 0x9372a8>}

Functions can be nested

Another feature that falls out of this notion of first-class functions is the
ability to nest functions. In addition to just returning a function as a result of
an invocation, a function can be defined within another function and then
returned:

>>> def adder():

... def add(x, y):

... return x + y

... return add

>>> adder()

<function add at 0x7ffe7c440a28>

>>> adder()(2, 4)

Note
If the above example were in a Python program, the adder function
would have created at module import time. The inner function, add, on the
other hand, does not exist when the module is imported. It is created during
runtime, everytime the adder function is invoked. Thus every invocation of
adder creates a new instance of add as illustrated by thier changing id
below:

>>> adder()

<function add at 0x7ffe7c440aa0>

>>> adder()

<function add at 0x7ffe7c440a28>

Nested functions in Python also create the need for nested scope. Any
nested function has read/write access to built-ins and globals. Also nested
functions have read-only access to variables defined the enclosing
functions:

>>> x = 5 # "global" variable

>>> y = 3

>>> def wrapper():

... def inner():

... # can't write x unless

... # global is used ie

... # UnboundLocalError

... global x

... x = 6

... y = -2 # now local shadows global

... # z is a "free" variable in inner

... print "Inner", x, y, z

... y = 1 # now local

... z = 0

... inner()

... print "Wrap", x, y, z

>>> wrapper()

Inner 6 -2 0

Wrap 6 1 0

>>> print "Global", x, y

Global 6 3

In the previous example, x and y are global variables. Within wrapper,

read-only access is available for the global x. Inside of the inner function, x
is marked with the global keyword, which marks the x as a reference to the
global x. In both functions, wrapper and inner, y shadows the global y. At
any point inside a function when a variable is defined, it becomes local,
unless it was previously marked with the global keyword.
A free variable, is a computer science term for a variable that is neither
local nor passed in as a argument to a function. Within the function inner, z
is a free variable. In Python terms, a variable defined in an enclosing
function is a free variable inside any nested functions.
 Note
Though there can be multiple nested scopes within arbitrarily nested
functions, only the global scope is writeable.

Note
Python 3 introduces the nonlocal keywords which allows for finer
grained modification of non-global variables.
Function Parameters

Python supports four different types of parameters for functions.

Normal Parameters - Have a name and a position

Keyword(default/named) Paramenters - Have a name
Variable Parameters - Preceded by an *, have a position
Variable Keyword Parameters - Preceded by a **, have a name
Parameters vs Arguments

For computer science pedants, there is a slight distinction in the definition

of parameters and arguments. Parameters are the names of variables
accepted for input in the definition of a function. Arguments are the
variables passed into an invoked function:

>>> def mult(a, b):

... return a * b

>>> x = 3

>>> y = 2

>>> print mult(x, y)

In the above code, a and b are parameters because they appear in the
definition of the function. x and y are arguments.
This is a slight distinction in Python, and it is pretty common to see them
used interchangeably. In fact the common naming scheme for variable
parameters is *args and variable keyword parameters is **kwargs.
Normal and Keyword Parameters

Normal and keyword parameters are very common. Most Python

developers run into the mutable default parameter gotcha at some point.
The only real difference between this two parameter types is that normal
parameters are always required. Like keyword parameters, normal
parameters also support using the name=value argument style during
invocation:

>>> def normal(a, b, c):

... print a, b, c

>>> normal(1, 2, 3)

1 2 3

>>> normal(a=1, b=2, c=3)

1 2 3

If the arguments have thier name provided during invocation, the the
order of the parameters is not important:

>>> normal(c=3, a=1, b=2)

1 2 3
Variable Parameters

In addition to serving as the multiplication and power operators — the

asterisk, *, denotes variable parameters in function and method definitions.
Variable parameters allow a function to take an arbitrary number of position
based arguments. For example in the C world, the printf function, allows
for any number of arguments to be passed into it. A simple example in
Python follows:

>>> def printf(fmt, *args):

... done = False

... start = 0

... tup_idx = 0

... while not done:

... i = fmt.find('%s', start)

... if i == -1:

... done = True

... else:

... word = str(args[tup_idx])

... tup_idx = tup_idx + 1

... fmt = fmt[:i] + word + fmt[i+2:]

... start = i + 1 + len(word)

... print fmt

>>> printf('hello')

hello

>>> printf('My name: %s', 'Matt')

My name: Matt

>>> printf('nums: %s, %s, %s', *range(1, 4))

nums: 1, 2, 3

Note
Variable parameters are commonly labeled as *args. Much like self is
the accepted Python naming convention for the first parameter to a method
for an object, *args is the standard naming convention used for variable
parameters. In reality, the Python interpreter does not care what the name
of the parameter following the asterisk is. Because there can only be one
parameter defined as a variable parameter (not including the variable
keyword parameter), and it is common convention to spell the variable
parameter as *args.

By prefixing a parameter with an *, like *args, the function will allow

any number of arguments for that parameter (including 0 arguments).
Within the function itself, the variable args (without an *) will be a tuple
containing all the arguments passed into the function:

>>> def demo_args(*args):

... print type(args), args

>>> demo_args()

<type 'tuple'> ()

>>> demo_args(1)

<type 'tuple'> (1,)

>>> demo_args(3, 'foo')

<type 'tuple'> (3, 'foo')

Note
Variable parameters are commonly combined with variable keyword
parameters. Together they are often seen in the constructors of subclasses.
This allows a subclass to easily accept any of the arguments for a parent
class without enumerating any of them.
Variable parameters and variable keyword parameters are also used for
decorators which are described later in this book.
The * operator

The asterisk serves to:

apply multiplication (4*2)

apply the power operator (4**2)
mark variable parameters
flatten argument sequences, the splat operator
mark variable keyword parameters
flatten keywords dictionaries, the splat operator

The last two will be discussed later in the following sections. The
previous section discussed variable parameters. Flattening arguments is
next in the list. What if you already have appropriate arguments for a
function sitting around in a sequence? Is it necessary to pull each item out
during function invocation?:

>>> vars = ['John', 'Paul']

>>> demo_args(vars[0], vars[1])

<type 'tuple'> ('John', 'Paul')

The * operator is also overloaded to flatten a sequence of arguments

during invocation of a function. Some refer to this as splatting a sequence.
The following is equivalent to the previous line:

>>> demo_args(*vars)

<type 'tuple'> ('John', 'Paul')

If the * is left off the vars argument, args would be tuple containing a
single item, the list with the two parameters. This is probably not what was
intended if vars was meant to contain the parameters for a function:

>>> demo_args(vars)
<type 'tuple'> (['John', 'Paul'],)

In the definition of demo_args, the args parameter will accept any

number of arguments. Within the body of demo_args, args is a tuple
holding those values. In the previous case, a single argument containing a
list was passed into the function, therefore, args within the function will be
a tuple with only one item, the list that was passed in.

Tip
When invoking functions declared with variable arguments, make sure
you understand which arguments need to be flattened. (Usually for
decorators and constructors they should all be flattened).

It also is possible to flatten a sequence of arguments into a function that

does not have any variable parameters:

>>> def add3(a, b, c):

... return a + b + c

>>> add3(*[4, 5, 6])

15

Note
Only a single sequence may be flattened into a function:

>>> demo_args(*vars, *vars)

Traceback (most recent call last):

...

demo_args(*vars, *vars)

SyntaxError: invalid syntax

Note
If a function has normal, keyword and variable parameters, it may be
invoked with just a flattened sequence. In that case the sequence will
populate the normal and keyword parameters, and any left over variables
will be left in the variable argument:

>>> def func(a, b='b', *args):

... print [x for x in [a, b, args]]

>>> vars = (3, 4, 5)

>>> func(*vars)

[3, 4, (5,)]

Again, because the * flattens the arguments, they fill out the parameters.
The above invocation is the same as calling the function with the arguments
listed out:

>>> func(vars[0], vars[1], vars[2])

[3, 4, (5,)]
Variable Keyword Parameters

Similar to variable parameters, Python also allows variable keyword

parameters. Using the ** syntax, a parameter can be marked to allow any
number of keyword arguments:

>>> def demo_kwargs(**kwargs):

... print type(kwargs), kwargs

>>> demo_kwargs()

<type 'dict'> {}

>>> demo_kwargs(one=1)

<type 'dict'> {'one': 1}

>>> demo_kwargs(one=1, two=2)

<type 'dict'> {'two': 2, 'one': 1}

The **, when used within a function or method definition, indicates that
a function will take any number of keyword arguments. The arguments
arrive in a dictionary containing the names and their corresponding values.
Similar to args, kwargs is the standard convention for a parameter name
used for variable keyword parameters.

Note
Variable keyword parameters require arguments to provide a name:

>>> demo_kwargs(1)

Traceback (most recent call last):

...

demo_kwargs(1)

TypeError: demo_kwargs() takes exactly 0 arguments (1 given)

This error can be a little confusing, since demo_kwargs takes zero normal
parameters, but any number of keyword parameters.
Flattening dictionaries

The double asterisk also serves to flatten — or splat — a dictionary into

keyword arguments for a function:

>>> def distance(x1, y1, x2, y2):

... return ((x1-x2)**2 +

... (y1-y2)**2) ** .5

>>> points = {'x1':1, 'y1':1,

... 'x2':4, 'y2':5}

>>> distance(**points)

5.0

The above invocation of distance is the same as calling the function

with the items of the dictionary listed as keyword arguments:

>>> distance(x1=1, y1=1, x2=4, y2=5)

5.0

Dictionaries can also be flattened into functions with just normal

parameters — such as distance or functions defined to take variable
keyword parameters — such as demo_kwargs:

>>> demo_kwargs(**points)

<type 'dict'> {'y1': 1, 'x2': 4,

'x1': 1, 'y2': 5}
Arbitrary function parameters

A function that has both variable parameters and variable keyword

parameters can take an arbitrary number of arguments, be they passed in as
normal, keyword, or variable arguments. This makes using the combination
of them prime candidates for the parameters of subclass constructors and
decorators.
Here are the four types of parameters:

normal
keyword
variable
variable keyword

Here is a function that has all four types of parameters:

>>> def demo_params(normal, kw="Test", *args, **kwargs):

... print normal, kw, args, kwargs

Tip
Functions may only define one variable parameter and one variable
keyword parameter. Also, the order of the parameter definition must follow
the order of the four types of parameters listed above.

Tip
Remember when invoking functions with the splat operator — with either
variable arguments or variable keyword arguments — it is the same as if
those arguments are listed out individually. If variable keyword arguments
keys have the same name as normal or keyword parameters, they can be
used for them. Otherwise, variable arguments would come after the normal
and keyword arguments, and variable keyword arguments would appear
after that with their corresponding names:
 >>> args = (0, 1, 2)

>>> kw = {'foo': 3, 'bar': 4}

>>> demo_params(*args, **kw)

0 1 (2,) {'foo': 3, 'bar': 4}

Notice that the variable parameters flowed into the normal and keyword
parameters. Again, this invocation is equivalent to:

>>> demo_params(args[0], args[1], args[2],

... foo=3, bar=4)

0 1 (2,) {'foo': 3, 'bar': 4}

Closures

Closures often have an aura around them that makes them appear
unapproachable. It does not help that the descriptions of them are terse:
[A] closure (also lexical closure, function closure, function value or
functional value) is a function together with a referencing environment
for the non-local variables of that function. A closure allows a function
to access variables outside its typical scope. Such a function is said to
be "closed over" its free variables.
—Wikipedia
A closure in Python is simply a function that is returned by another
function:

>>> def add_x(x):

... def adder(num):

... # adder is a closure

... # x is a free variable

... return x + num

... return adder

>>> add_5 = add_x(5)

>>> add_5

<function adder at ...>

>>> add_5(10)

15

In the above example, the function add_x returns an inner function. The
inner function, adder, is “closed over” — hence a closure. Inside of adder,
the variable x is a free variable because it is non-local to adder and defined
outside of it. A simplified definition of a Python closure might be:
In Python functions can return new functions. The inner function is a
closure and any variable it accesses that are defined outside of that
function are free variables.
Common uses of closures

As illustrated in the previous section, closures are useful as function

generators. Here are some other uses:

To keep a common interface (the adapter pattern)

To eliminate code duplication
To delay execution of a function

A real-life example is creating a filter for tabular results:

>>> def db_results(query_filter, table):

... results = []

... for row in table:

... result = query_filter(row)

... if result:

... results.append(result)

... return results

Where a query_filter takes the form:

>>> def query_filter(row):

... # filter row of table

... return filtered_row or None

Assuming that tables are composed of lists of dictionaries, a filter for

name might look like this:

>>> def matt_filter(row):

... if row['name'].lower() == 'matt':

... return row

>>> matt_filter({'name':'Matt'})
{'name': 'Matt'}

>>> matt_filter({'name':'Fred'}) is None

True

But as often occurs, filtering by other names might be required as well. A

closure can be used to easily create different name filters:

>>> def name_filter(name):

... def inner(row):

... if row['name'].lower() == name.lower():

... return row

... return inner

>>> paul_filter = name_filter('Paul')

>>> john_filter = name_filter('John')

>>> george_filter = name_filter('George')

>>> ringo_filter = name_filter('Ringo')

Closures also enable filtering by multiple filters. An or operation that

takes multiple filters can be created using a closure as well:

>>> def or_op(filters):

... def inner(row):

... for f in filters:

... if f(row):

... return row

... return inner

>>> beatle = or_op([paul_filter,

... john_filter,

... ringo_filter,

... george_filter])

>>> beatle({'name':'Matt'}) is None

True
>>> beatle({'name':'John'})

{'name': 'John'}

These simple functions illustrate the power of closures. Closures quickly

enable generation of functions and conforming to a interface (the adapter
pattern).
Decorators

According to Wikipedia, a decorator is “a design pattern that allows

behavior to be added to an existing object dynamically”. In Python, a
decorator is a method for altering a callable. Closures enable the creation of
decorators. A decorated callable can be altered at the following times:

before invocation
during invocation — the implementation can be changed/replaced
after invocation

Normally the callables that are decorated in Python are either functions
or methods.
A simple decorator

Here is a decorator, verbose, that prints out the name of the function it
decorates before and after execution:

>>> def verbose(func):

... def wrapper():

... print "Before", func.__name__

... result = func()

... print "After", func.__name__

... return result

... return wrapper

Please make sure you understand what the verbose function does. It
accepts a function, func, and returns a new function, wrapper. When
wrapper is invoked, it will print the name, execute the original wrapped
function, print the name again and return the result of the original function.
This is about as simple as a decorator can get, and the others that follow
will build upon this same pattern.
A decorator is really only useful when applied to a function. There are
two ways to do this. The first is to simply invoke the decorator on a
function:

>>> def hello():

... print "Hello"

>>> hello = verbose(hello)

The above redefines the function hello, as the function returned by

verbose. Examining the __name__ attribute shows that the new function is
actually wrapper:

>>> hello.__name__

'wrapper'
 Now, when hello is invoked, it prints a message before and after
executing:

>>> hello()

Before hello

Hello

After hello

As described in PEP 318, Python 2.4 provided the second method of

wrapping a function. The syntactic sugar to decorate a function is illustrated
below:

>>> @verbose

... def greet():

... print "G'day"

Placing @verbose immediately before the function definition is the same

as writing greet = verbose(greet) following the function:

>>> greet()

Before greet

G'day

After greet

Tip
The PEP 318 style of decorating does not require parentheses following
verbose. It will result in an error if attempted:

>>> @verbose()

... def howdy()

... print "Howdy"

Traceback (most recent call last):

...

def howdy()

SyntaxError: invalid syntax

 The verbose decorator actually has an issue. It only works with functions
that do not have parameters:

>>> @verbose

... def add(x, y):

... return x + y

>>> add(2, 3)

Traceback (most recent call last):

...

add(2, 3)

TypeError: wrapper() takes no arguments (2 given)

As mentioned in the error, wrapper takes no arguments. The closure,

wrapper, was defined without parameters. Because wrapper is invoked
under the covers when add is invoked, the error is raised. The solution is
simple — add parameters. But how many parameters? The function add
takes two, but hello and greet take none. The answer is to use variable
parameters. As mentioned in the variable parameters section, using these
types of parameters enables a function to accept an arbitrary amount of
arguments:

>>> def chatty(func):

... def wrapper(*args, **kwargs):

... print "Before", func.__name__

... result = func(*args, **kwargs)

... print "After", func.__name__

... return result

... return wrapper

>>> @chatty

... def mult(x, y):

... return x * y

>>> mult(2, 4)

Before mult

After mult
8

>>> @chatty

... def goodbye():

... print "Later"

>>> goodbye()

Before goodbye

Later

After goodbye
A decorator template

What follows is a template for a decorator. This template accepts functions

that take an arbitrary amount of arguments.

>>> def decorator(func_to_decorate):

... def wrapper(*args, **kwargs):

... # do something before invocation

... result = func_to_decorate(*args, **kwargs)

... # do something after

... return result

... wrapper.__doc__ = func_to_decorate.__doc__

... wrapper.__name__ = func_to_decorate.__doc__

... return wrapper

There are two lines in this template that weren’t present in the previous
examples. In well-behaved decorators the __doc__ and __name__ of the
wrapper function need to be updated with the values from the function that
is being decorated. For pickling (serialization to disk) of objects, it is
required that __name__ is updated. The __doc__ attribute is updated so the
function is friendly to introspection.

Note
The function wraps (which is a decorator itself) found in the functools
module, will update __doc__ and __name__ as well. So another template is:

>>> import functools

>>> def decorator(func_to_decorate):

... @functool.wraps(func_to_decorate)

... def wrapper(*args, **kwargs):

... # do something before invocation

... result = func_to_decorate(*args, **kwargs)

... # do something after

 ... return result

... return wrapper

The __module__ attribute is also updated by wraps.

Note
As was discussed in the closure section, any callable can wrap another
function. So a callable can also serve as a decorator. Here is an example of
a class that can be used to decorate functions:

>>> class decorator_class(object):

... def __init__(self, function):

... self.function = function

... def __call__(self, *args, **kwargs):

... # do something before invocation

... result = self.function(*args, **kwargs)

... # do something after

... return result

This class decorator could be used as follows:

>>> @decorator_class

... def function():

... # implementation

Note
The above decorator is not the same as what are known as Class
Decorators in Python. These are discussed in PEP 3129.
Parameterized decorators

Often decorators need to be customized on a per function basis. For

example the Django require_http_methods decorator decorates a view
and ensures that it uses the correct http methods — GET, HEAD, POST,
PUT, etc — or a combination of those methods. This decorator is a
parameterized decorator that can be customized for the specific function it
is wrapping. To enforce GET on a view, wrapping with
@require_http_methods(["GET"]) is sufficient. If a different function
required GET or POST, @require_http_methods(["GET", "POST"])
customizes the behavior for that function. How does this work, when it was
mentioned previously that decorators will throw an error if they use
parentheses when wrapping a function?
Previous chapters have discussed how to implement function generators
in Python. How is it done? The usual manner is — use a closure to generate
a new function. Hence the method to generate parameterizable decorators
is to wrap them with another function! The parameterized decorator itself if
actually a decorator generator.
Make sure you understand the previous paragraph because it tends to
cause people’s heads to hurt until they understand what is going on.
If business logic required truncating results from functions to a certain
length — 5, a single generator could do that. Any function that possibly
generated results longer than 5 could be decorated to limit the length of the
results:

>>> def trunc(func):

... def wrapper(*args, **kwargs):

... result = func(*args, **kwargs)

... return result[:5]

... return wrapper

>>> @trunc

... def data():

... return "foobar"

>>> data()

'fooba'

Now assume that the requirement for truncation is changed. Some

functions must be truncated to a length of 3 while other functions might
need to have a length of 6. What follows is a simple parameterized
decorator — or a function that generators customized decorators — that
limits the length of the result of the decorated function:

>>> def limit(length):

... def decorator(func):

... def wrapper(*args, **kwargs):

... result = func(*args, **kwargs)

... return result[:length]

... return wrapper

... return decorator

>>> @limit(3)

... def data3():

... return "limit to 3"

>>> data3()

'lim'

>>> @limit(6)

... def data6():

... return "limit to 6"

>>> data6()

'limit '

Note
The syntactic sugar for decorating with parameterized decorators is
@limit(3). It is the same as the following:
 >>> def data3():

... return "limit to 3"

>>> data3 = limit(6)(data3)

When the function limit is invoked (with 6 as its parameter), it returns

(or generates) a decorator. This decorator, aptly named decorator, is then
invoked with the function to decorate, data3.
Parameterized Template

Here is the template to follow for parameterized decorators:

>>> def param_dec(option):

... def decorator(function):

... def wrapper(*args, **kwargs):

... # probably use option in here

... # before

... result = function(*args, **kwargs)

... # after

... return result

... wrapper.__doc__ = function.__doc__

... wrapper.__name__ = function.__name__

... return wrapper

... return decorator

Multiple decorators

Just as functions can be nested arbitrarily inside of other functions,

functions can be wrapped by multiple decorators:

>>> @chatty

... @limit(2)

... def greet():

... return 'Greetings'

>>> print greet()

Before wrapper

After wrapper

Gr

The decorating syntactic sugar is the same as:

>>> greet = chatty(limit(2)(greet))

Another way to think of these nested decorators is that the topmost

decorator is the outermost decorator, and wraps the result of any inner
decorator.
Common uses for decorators

Remember that decorators can alter or inspect the:

function arguments
function being wrapped
results of the function

With that in mind there are common instances where decorators are used.

Caching expensive calculations

Retrying a function that might fail
Redirecting sys.stdout to capture what a function prints to it
Logging the amount of time spent in a function
Timing out a function call
Access control
About the Author
Matt Harrison has over 11 years Python experience across the domains of
search, build management and testing, business intelligence and storage.
He has presented and taught tutorials at conferences such as SCALE,
PyCON and OSCON as well as local user conferences. The structure and
content of this book is based off of first hand experience teaching Python to
many individuals.
He blogs at hairysun.com and occasionally tweets useful Python related
information at __mharrison__.
Also Available

Treading on Python Volume 1: Foundations

 Treading on Python Volume 1 is a book designed to bring developers and
others who are anxious to learn Python up to speed quickly. Not only does it
teach the basics of syntax, but it condenses years of experience gained from
programming, tutoring and teaching. You will learn warts, gotchas, best
practices and hints that have been gleaned through the years in days. You
will hit the ground running and running in the right way.
Available on Amazon and Barnes and Noble.
http://hairysun.com/books/tread/
Reviews

“Very informative … an awesome resource”

—Grig G
...
“Matt clearly knows his Python. He has peppered the book with
helpful tips that compelled me to whip out my Python interpreter to
experiment. Many of the tips were very handy, even for a semi-
experienced Python programmer such as myself.
The book reads smoothly and quickly. Matt is very careful to keep
his explanations succinct and clear, such that you don't feel like you're
reading a college text book or a reference manual. Even still, the book
does contain a high information density.”
—David Smith
...
“The reader will get a nice overview of the language without the
confusion of 3rd party packages or even very much of the included
modules. They get a little introduction to Python introspection tools,
which is pretty nice. ... this is one you may want to keep in mind for
your budding Pythonista.”
—Mike Driscoll