Evaluation and test data¶
evaluate ¶
Evaluate detection against labelled text (the spec's "Evaluation" rules).
TypeScore
dataclass
¶
TypeScore(type: str, tp: int, fp: int, fn: int, precision: float | None, recall: float | None, f1: float | None)
Counts and rates for one type (or overall).
EvaluationReport ¶
EvaluationReport(rows: Iterable[TypeScore], n_documents: int, match: str)
load_corpus ¶
load_corpus(path: str | PathLike[str]) -> list[dict[str, Any]]
Read labelled documents from a JSON Lines file.
evaluate ¶
evaluate(documents: Iterable[Mapping[str, Any]] | str | PathLike[str], min_confidence: float = 0.0, match: str = 'exact', detectors: Sequence[str] | None = None) -> EvaluationReport
Measure detection against labelled documents.
Each document is a mapping with text and spans (a list of
{"start", "end", "type"} with 0-based, end-exclusive offsets); every
PII span must be labelled. Returns per-type and overall true positives,
false positives, false negatives, precision, recall and F1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
documents
|
Iterable[Mapping[str, Any]] | str | PathLike[str]
|
Labelled documents, or a path to a JSON Lines file. |
required |
min_confidence
|
float
|
Ignore predictions scoring below this value. |
0.0
|
match
|
str
|
|
'exact'
|
detectors
|
Sequence[str] | None
|
Detector ids to run; |
None
|
load_corpus ¶
load_corpus(path: str | PathLike[str]) -> list[dict[str, Any]]
Read labelled documents from a JSON Lines file.
EvaluationReport ¶
EvaluationReport(rows: Iterable[TypeScore], n_documents: int, match: str)
TypeScore
dataclass
¶
TypeScore(type: str, tp: int, fp: int, fn: int, precision: float | None, recall: float | None, f1: float | None)
Counts and rates for one type (or overall).
synth_indian_pii ¶
synth_indian_pii(n: int = 10, seed: int | None = None) -> list[dict[str, str]]
Generate n synthetic records of Indian identifiers.
Values pass the same structural and checksum validation as real ones.
They are random, not taken from any real record, but a random number
with a valid checksum can still coincide with a real one by chance, so
treat generated data as test data and never as anonymised data. Names
combine common Indian first names and surnames; email addresses use the
reserved example.com/example.org domains.
Uses its own random generator; the global random state is untouched.
Checksums¶
verhoeff_valid ¶
verhoeff_valid(x: str | None) -> bool
Whether a number (with its final check digit) passes Verhoeff.
Used by Aadhaar and Aadhaar Virtual ID.
verhoeff_check_digit ¶
verhoeff_check_digit(x: str | None) -> str | None
The Verhoeff check digit to append to x, or None if invalid.
luhn_valid ¶
luhn_valid(x: str | None) -> bool
Whether a number passes the Luhn checksum (payment cards).
luhn_check_digit ¶
luhn_check_digit(x: str | None) -> str | None
The Luhn check digit to append to x, or None if invalid.
gstin_valid ¶
gstin_valid(x: str | None) -> bool
Whether x is a structurally valid GSTIN with a correct check character.
A GSTIN is a 2-digit state code, the holder's PAN, an entity number, the
letter Z and the check character.
gstin_check_char ¶
gstin_check_char(x: str | None) -> str | None
The mod-36 check character for the first 14 characters of a GSTIN.