Lambda Functions, map, filter and reduce in Python

Lambda, map, filter and reduce come from functional programming, and Python supports them without insisting on them. Used in the right places they make code shorter and clearer. Used everywhere, they make it cryptic. This guide is about telling the difference.

lambda: a function in an expression

A lambda creates a small, anonymous function whose body is a single expression (Python reference: lambdas). lambda price: price * 1.18 is equivalent to a def that returns price * 1.18, but it can be written inline, exactly where a function is expected.

That is its only real advantage, and it points to where lambdas belong: as arguments to other functions. The most common case in everyday Python is a sort key:

orders = [
    {"id": 101, "customer": "Asha", "total": 2400, "items": 3},
    {"id": 102, "customer": "Ravi", "total": 900, "items": 5},
    {"id": 103, "customer": "Meena", "total": 2400, "items": 1},
]

by_total = sorted(orders, key=lambda o: (-o["total"], o["id"]))
print([o["id"] for o in by_total])
print(max(orders, key=lambda o: o["items"])["customer"])

[101, 103, 102]
Ravi

The key function returns a tuple, so orders sort by total descending (the minus sign) and then by id. max, min and sorted all take a key, and a lambda is the natural way to say "compare by this".

What a lambda should not be is a named function in disguise. PEP 8 recommends against writing tax = lambda p: p * 1.18; use def, which gives the function a real name in tracebacks and room for a docstring (PEP 8).

The operator module: keys without lambda

For the most common keys, the operator module has ready-made functions that read even more clearly (Python docs: operator):

from operator import itemgetter

print([o["customer"] for o in sorted(orders, key=itemgetter("total", "id"))])

['Ravi', 'Asha', 'Meena']

itemgetter("total", "id") fetches both fields and returns them as a tuple. attrgetter does the same for object attributes, and methodcaller calls a method.

map and filter: lazy, and single-use

map(function, iterable) applies a function to every item; filter(function, iterable) keeps the items for which the function returns a true value. Both return iterators, not lists (Python docs: built-in functions). Nothing is computed until you consume them, and they can be consumed only once:

raw = [" 42 ", "7", "", " 19"]

cleaned = map(str.strip, raw)
print(cleaned)
print(list(cleaned))
print(list(cleaned))            # already used up

<map object at 0x...>
['42', '7', '', '19']
[]

(The memory address in the first line varies.) The empty second list is the classic surprise. If you need the results more than once, store them with list().

map and filter read well when you already have a named function: map(str.strip, raw) or filter(None, values) (which keeps truthy values) say exactly what they do. With a lambda inside, a comprehension is usually clearer:

numbers = [int(s) for s in map(str.strip, raw) if s]

via_functions = list(filter(lambda n: n % 2, map(lambda n: n * n, numbers)))
via_comprehension = [n * n for n in numbers if n * n % 2]
print(via_functions, via_comprehension)

[49, 361] [49, 361]

Both lines compute the odd squares; the second can be read left to right. The list comprehensions guide covers that syntax in depth.

reduce: usually replaced by a built-in

functools.reduce folds a sequence into one value by applying a two-argument function cumulatively (Python docs: functools.reduce). It was moved out of the built-ins in Python 3 because most uses have a clearer alternative:

from functools import reduce
import math
import operator

prices = [120, 80, 45]
print(reduce(operator.add, prices), sum(prices))
print(reduce(operator.mul, [2, 3, 4]), math.prod([2, 3, 4]))

try:
    reduce(operator.add, [])
except TypeError as exc:
    print(exc)
print(reduce(operator.add, [], 0))

245 245
24 24
reduce() of empty iterable with no initial value
0

Before reaching for reduce, check for a built-in: sum, math.prod, max, min, any, all and "".join cover most folds and handle empty input sensibly. When reduce is right, for example merging a list of dictionaries or applying a list of functions in sequence, pass an initial value. Without one, an empty input raises the TypeError above.

settings = reduce(lambda acc, d: acc | d, [{"env": "dev"}, {"debug": True}, {"env": "prod"}], {})
print(settings)

{'env': 'prod', 'debug': True}

The late-binding trap

Lambdas (like any nested function) look up variables from the enclosing scope when they are called, not when they are created. A loop that builds lambdas therefore produces functions that all see the loop variable's final value (Python programming FAQ):

multipliers = [lambda x: x * factor for factor in (1, 2, 3)]
print([m(10) for m in multipliers])

fixed = [lambda x, factor=factor: x * factor for factor in (1, 2, 3)]
print([m(10) for m in fixed])

[30, 30, 30]
[10, 20, 30]

The default argument factor=factor is evaluated at creation time, which freezes each value. functools.partial(operator.mul, factor) achieves the same without the trick.

Quick guide

When in doubt, write the plain loop first. Convert it to a comprehension, a key function or a reduce only if the result is shorter and still obvious to the next person who reads it.

Functions that take or return other functions are also the basis of decorators; see Python decorators explained with real examples. Multi-key sorting comes up in coding interviews too, as in the Python interview questions for data roles. Functional programming, including map(), filter(), reduce(), lambda expressions and higher-order functions, is a module of the Python Programming course.

Sources

• Lambdas (language reference) — Python documentation
• Built-in functions: map and filter — Python documentation
• functools.reduce — Python documentation
• operator — Standard operators as functions — Python documentation
• PEP 8: Programming Recommendations — Python Enhancement Proposals
• Programming FAQ: lambdas defined in a loop — Python documentation