Find PESEL, BSN and other national ID numbers in text with Python

Why a regex is not enough

A PESEL is 11 digits, a BSN is 9. So are invoice numbers, order numbers, phone numbers without a prefix and many reference codes. A pattern like \d{11} flags all of them. Validating the check digit removes most false matches, but not all: when we tested random 11-digit numbers, about 1.5% still passed the PESEL check digit and birth-date test, and about 1 in 11 random 9-digit numbers pass the Dutch elfproef.

Check digits and birth dates

Most European ID numbers carry a check digit, and many encode a date of birth that has to exist. The open-source library python-stdnum implements both for dozens of countries, so you do not have to write each algorithm yourself.

Require a label for weak check digits

Schemes with a single check digit, like PESEL and BSN, are only reliable when the number follows its label: PESEL:, BSN, burgerservicenummer. Schemes with stronger checks, like the Swedish personnummer (with its separator) or the Spanish DNI letter, can be matched on their own.

A small, working finder

import re

from stdnum.nl import bsn
from stdnum.pl import pesel

# (name, candidate pattern, stdnum module, labels that must precede the number)
SCHEMES = [
    ("PL PESEL", re.compile(r"(?<!\d)\d{11}(?!\d)"), pesel, "PESEL"),
    ("NL BSN", re.compile(r"(?<!\d)\d{9}(?!\d)"), bsn, "BSN|burgerservicenummer"),
]


def find_ids(text):
    found = []
    for name, pattern, module, labels in SCHEMES:
        for m in pattern.finditer(text):
            before = text[max(0, m.start() - 30):m.start()]
            # Weak check digits: only count a number that follows its label.
            if not re.search(rf"(?i)\b(?:{labels})\b[^\d\n]{{0,15}}$", before):
                continue
            if module.is_valid(m.group()):  # check digits and, for PESEL, a real birth date
                found.append((name, m.group(), m.start()))
    return found


print(find_ids("Order 44051401359 shipped. PESEL: 44051401359, BSN 111222333, ref 111222333."))

Output:

[('PL PESEL', '44051401359', 34), ('NL BSN', '111222333', 51)]

The same digits as an order number, or after ref, are ignored.

Scaling it up

Real text needs more: number formats with spaces and dots, labels in every language, more countries, and names next to the numbers. That is what the EU PII Redaction API does for 21 EU/EEA countries, returning each number's country and scheme. The quickstart shows it in five lines.

To check a single number by hand, use the PESEL, BSN and other validators.