Regular Expressions in Python: A Practical Guide

A regular expression is a small pattern language for text: "three capital letters, a dash, then digits" or "a date written as day, slash, month, slash, year". String methods handle fixed text such as a known prefix or a comma; regular expressions handle text that varies within a pattern. Python's re module provides them (Python docs: re), and a dozen features cover nearly every everyday job. This guide focuses on those, and on the mistakes that make patterns slow or wrong.

Always use raw strings

Regular expressions use backslashes heavily (\d for a digit, \b for a word boundary), and so do Python string literals. Write patterns as raw strings, r"\d+", so Python passes the backslashes through untouched instead of interpreting them first; the regex HOWTO recommends this for every pattern (Python regex HOWTO). Without the r, "\b" is a backspace character and the pattern silently matches nothing.

Finding things: search, findall and finditer

import re

text = "Refund INV-1042 and INV-98 (too short), then order ORD-555001 on 25/09/2026."
ORDER_ID = r"\b[A-Z]{3}-\d{4,6}\b"

print(re.search(ORDER_ID, text).group())
print(re.findall(ORDER_ID, text))
for m in re.finditer(ORDER_ID, text):
    print(m.group(), m.start(), m.end())

INV-1042
['INV-1042', 'ORD-555001']
INV-1042 7 15
ORD-555001 51 61

The pattern reads: a word boundary, three capital letters, a dash, four to six digits, a word boundary. INV-98 does not match because it has only two digits. search returns the first match as a match object (or None), findall returns every match as strings, and finditer yields match objects with positions, which is what you want when you need to know where each match was.

Validating: match versus fullmatch

PIN = r"[1-9]\d{5}"                    # Indian PIN code: six digits, not starting with 0
for candidate in ["500081", "5000812", "050081", "500 081"]:
    print(candidate, bool(re.match(PIN, candidate)), bool(re.fullmatch(PIN, candidate)))

500081 True True
5000812 True False
050081 False False
500 081 False False

re.match only anchors at the start, so it happily accepts 5000812, whose first six characters look valid. For validation, use re.fullmatch, which requires the whole string to match. This is the most common regex bug in validation code. Note that a pattern this simple checks the shape of a PIN code, not that the code exists.

Groups: pulling out the parts

Parentheses capture parts of a match. Named groups, (?P<name>...), make the result self-describing:

LOG = re.compile(r"(?P<date>\d{4}-\d{2}-\d{2}) (?P<level>INFO|WARN|ERROR) (?P<message>.*)")
m = LOG.match("2026-09-25 ERROR payment gateway timeout after 30s")
print(m.group("level"), "|", m.group("message"))
print(m.groupdict())

ERROR | payment gateway timeout after 30s
{'date': '2026-09-25', 'level': 'ERROR', 'message': 'payment gateway timeout after 30s'}

re.compile stores the pattern as an object you can reuse; Python caches recently used patterns anyway, so compiling is mostly about naming the pattern and keeping it in one place. groupdict() turns a matched line into a dictionary, a quick way to parse structured logs.

Greedy and lazy quantifiers

*, + and {m,n} are greedy: they match as much as possible and then back off. Adding ? makes them lazy, matching as little as possible (Python regex HOWTO):

html_ish = "<b>total</b> and <b>tax</b>"
print(re.findall(r"<b>(.*)</b>", html_ish))
print(re.findall(r"<b>(.*?)</b>", html_ish))

['total</b> and <b>tax']
['total', 'tax']

The greedy version swallowed everything between the first <b> and the last </b>. When a pattern captures too much, a lazy quantifier or a narrower character class such as [^<]* is usually the fix. For real HTML, use an HTML parser instead; see web scraping with BeautifulSoup.

Replacing: re.sub, with backreferences or a function

dates = "Due 25/09/2026, paid 03/10/2026."
print(re.sub(r"(\d{2})/(\d{2})/(\d{4})", r"\3-\2-\1", dates))

def mask(m):
    return m.group()[:2] + "*" * (len(m.group()) - 4) + m.group()[-2:]

print(re.sub(r"\b\d{10}\b", mask, "Call 9876543210 or 040-12345678"))

Due 2026-09-25, paid 2026-10-03.
Call 98******10 or 040-12345678

In the replacement string, \1, \2 and \3 refer to the captured groups, which makes reordering day, month and year a one-liner. Passing a function instead of a string gives full control: here each ten-digit number is masked, while the landline with a dash is left alone.

Readable patterns with VERBOSE

Long patterns are hard to read on one line. The re.VERBOSE flag ignores whitespace in the pattern and allows comments, so a pattern can be laid out and explained:

IFSC = re.compile(r"""
    ^[A-Z]{4}      # bank code
    0              # reserved character, always zero
    [A-Z0-9]{6}$   # branch code
""", re.VERBOSE)
print([bool(IFSC.match(c)) for c in ["SBIN0001234", "SBIN1001234", "HDFC0ABC123"]])

[True, False, True]

Other useful flags are re.IGNORECASE and re.MULTILINE, which makes ^ and $ match at every line in a multi-line string (Python docs: re).

Pitfalls

• The digit class matches more than 0 to 9. In string patterns, \d matches any Unicode decimal digit, including Arabic-Indic and Devanagari digits. Use [0-9] or the re.ASCII flag when only ASCII digits are acceptable.
• User input inside a pattern. Characters such as ., + and ( have special meanings; wrap untrusted text with re.escape() before building a pattern from it.
• Catastrophic backtracking. Nested quantifiers such as (a+)+$ can take exponential time on inputs that almost match, which is a known denial-of-service risk in web applications (OWASP: ReDoS). Keep patterns specific and avoid quantifiers nested inside quantified groups.
• The wrong tool. JSON, HTML, CSV and dates all have parsers in the standard library or popular packages. They handle edge cases a pattern will not.

print(bool(re.fullmatch(r"\d{6}", "५००००१")), bool(re.fullmatch(r"[0-9]{6}", "५००००१")))
print(re.escape("price (INR) 1.5+"))

True False
price\ \(INR\)\ 1\.5\+

The first line shows \d accepting six Devanagari digits that [0-9] rejects.

When to reach for re, and when not

Use string methods when the text is fixed: startswith, in, split, replace are simpler and faster; see Python string methods. Use re when the text follows a pattern that varies, such as IDs, codes, dates in text, or log lines. Use a real parser when the text is a structured format. Test every pattern against examples that should match and examples that should not, as the PIN code check above does.

Regular expressions show up constantly in data cleaning; the automation scripts guide uses one to pull email addresses out of text files. Regular expressions and pattern matching are part of the strings module of the Python Programming course.

Sources

• re — Regular expression operations — Python documentation
• Regular Expression HOWTO — Python documentation
• Regular expression Denial of Service (ReDoS) — OWASP