Adding Recognizers

Recognizers are the core extension point of SovereignGuard. They define how the gateway detects PII before tokenization.

This guide explains the structure, expectations, and workflow for adding a new recognizer safely.

Recognizer Contract

All recognizers inherit from BaseRecognizer and must provide:

1. entity_types: the PII classes returned by the recognizer
2. locale: the recognizer locale such as universal, tn, fr, or ma
3. priority: ordering hint for registry execution
4. analyze(text): returns a list of RecognizerResult

Each RecognizerResult must include:

• entity_type
• start
• end
• score
• text
• locale

Where to Put New Recognizers

Universal Pattern

Use sovereignguard/recognizers/universal/ when the pattern is not country-specific.

Examples:

• email
• IBAN
• credit card
• IP address

Locale-Specific Pattern

Use sovereignguard/recognizers/<locale>/ when the pattern depends on local formats or local contextual language.

Examples:

• Tunisian national ID
• French NIR
• Moroccan ICE

Step-by-Step Workflow

1. Create the Recognizer Module

Create a file under the correct locale package.

Example path:

sovereignguard/recognizers/universal/passport.py

2. Implement the Recognizer

Example skeleton:

from typing import List

from sovereignguard.recognizers.base import BaseRecognizer, RecognizerResult


class PassportRecognizer(BaseRecognizer):
	@property
	def entity_types(self) -> List[str]:
		return ["PASSPORT"]

	@property
	def locale(self) -> str:
		return "universal"

	@property
	def priority(self) -> int:
		return 50

	def analyze(self, text: str) -> List[RecognizerResult]:
		return self._regex_analyze(
			text,
			[
				(r"\b[A-Z]{2}\d{7}\b", "PASSPORT", 0.85),
			],
		)

3. Register the Recognizer

Add it to the registry in sovereignguard/recognizers/registry.py.

If it is universal, add it to the universal recognizer list. If it is locale-specific, add it to that locale list.

4. Add Tests

Minimum test coverage should include:

1. valid positive examples
2. invalid negative examples
3. edge cases near neighboring punctuation or whitespace
4. locale-specific variations if relevant

5. Validate End-to-End

Run the recognizer tests and at least one masking engine integration test to ensure the entity is actually tokenized and restored correctly.

Design Guidelines

Prefer Precision Over Recall

A recognizer that misses rare edge cases is usually safer than a recognizer that masks normal business text incorrectly.

Use Context for Ambiguous Data

Fields like names, dates, and account references are often ambiguous. Add contextual keywords or formatting expectations so the recognizer does not over-mask unrelated text.

Keep Offsets Correct

start and end must point into the original input string. Do not rewrite, normalize, or mutate the input before computing offsets unless you can map the offsets back correctly.

Respect Priority

Locale-specific recognizers should generally have a higher priority than broad universal recognizers so they win overlap resolution when both match the same text.

Do Not Log Raw PII

Recognizer code must never emit sensitive values into logs.

Scoring Guidance

Recognizer scores should be meaningful because they interact with CONFIDENCE_THRESHOLD.

Typical guidance:

• 0.9 - 1.0: explicit identifiers with rigid structure and checksum-like confidence
• 0.8 - 0.9: highly structured identifiers or strong contextual clues
• 0.7 - 0.8: useful but somewhat ambiguous patterns
• below 0.7: generally avoid unless you want operators to lower the threshold intentionally

Common Mistakes

Avoid these failures:

1. returning overlapping matches with inconsistent scores
2. matching generic numbers or dates without context
3. using patterns that capture surrounding punctuation unnecessarily
4. forgetting to register the recognizer in the registry
5. adding a recognizer without negative tests

When to Create a Custom Recognizer

Create a custom recognizer when your domain includes identifiers that general privacy tooling will miss.

Examples:

• internal customer numbers
• contract references
• invoice IDs
• healthcare record IDs
• local governmental formats not yet covered by the project