Fix Presidio with LangChain Experimental Not Detecting Polish Names

The issue is that Presidio's default NLP configuration does not map spaCy's Polish named entity labels (e.g., persName, placeName) to Presidio's standard entity types (e.g., PERSON, LOCATION). When you pass a languages_config to PresidioAnonymizer or PresidioReversibleAnonymizer, the underlying AnalyzerEngine is created without the required model_to_presidio_entity_mapping, so spaCy correctly identifies entities but Presidio ignores them. The fix is to explicitly configure the NLP engine with the correct entity mapping and then inject that analyzer into the LangChain anonymizer.

The Full Answer

Why the Default Configuration Fails

Presidio uses spaCy as its NLP engine for entity recognition. When you specify "nlp_engine_name": "spacy" and "models": [{"lang_code": "pl", "model_name": "pl_core_news_lg"}], Presidio loads the Polish model. However, spaCy's Polish model outputs entity labels like persName (person name), placeName (location), orgName (organization), geogName (geographical name), and date (date/time). Presidio internally maps these spaCy labels to its own entity types (PERSON, LOCATION, ORGANIZATION, DATE_TIME). If this mapping is missing or incomplete, Presidio discards the recognized entities.

As confirmed by the user in Source 1, running spaCy directly on the text works correctly:

import spacy

nlp = spacy.load("pl_core_news_lg")
doc = nlp("Jan Kowalski mieszka w Warszawie i ma e-mail jan.kowalski@example.com.")
for ent in doc.ents:
    print(ent.text, ent.label_)

Output:

Jan Kowalski persName
Warszawie placeName

This proves the spaCy model is installed and functioning. The problem is purely in the Presidio configuration layer.

Solution 1: Configure the NLP Engine with Entity Mapping (Recommended)

Source 2 provides the definitive solution: create an NlpEngineProvider with a YAML configuration that includes the ner_model_configuration section. This maps spaCy's Polish entity labels to Presidio's standard types.

Here is the complete code:

from presidio_analyzer import AnalyzerEngine, RecognizerRegistry
from presidio_analyzer.nlp_engine import NlpEngineProvider
import tempfile

with tempfile.NamedTemporaryFile(delete=False) as languages_config:
    languages_config.write(b"""
nlp_engine_name: spacy
models:
-
  lang_code: en
  model_name: en_core_web_lg
-
  lang_code: pl
  model_name: pl_core_news_lg

ner_model_configuration:
  model_to_presidio_entity_mapping:
    persName: PERSON
    placeName: LOCATION
    orgName: ORGANIZATION
    geogName: LOCATION
    date: DATE_TIME
"""
)

# Create NLP engine based on configuration file
provider = NlpEngineProvider(conf_file=languages_config.name)
nlp_engine_with_polish = provider.create_engine()

# Pass created NLP engine and supported_languages to the AnalyzerEngine
analyzer = AnalyzerEngine(
    nlp_engine=nlp_engine_with_polish,
    supported_languages=["en", "pl"]
)

# Analyze in different languages
results_polish = analyzer.analyze(text="Jan Kowalski mieszka w Warszawie i ma e-mail jan.kowalski@example.com.", language="pl")
print(results_polish)

results_english = analyzer.analyze(text="My name is David", language="en")
print(results_english)

Output:

[type: EMAIL_ADDRESS, start: 45, end: 69, score: 1.0, type: PERSON, start: 0, end: 12, score: 0.85, type: LOCATION, start: 23, end: 32, score: 0.85, type: URL, start: 58, end: 69, score: 0.5]
[type: PERSON, start: 11, end: 16, score: 0.85]

Notice that EMAIL_ADDRESS and URL are detected by Presidio's built-in regex-based recognizers (not spaCy), so they work without any mapping. The key addition is the ner_model_configuration block. The mapping geogName: LOCATION ensures that geographical names like "Warszawie" are treated as locations.

When to use this approach: This is the cleanest, most maintainable solution. It works with any spaCy model and any language. You can reuse the same AnalyzerEngine instance across your application.

Solution 2: Inject the Custom Analyzer into LangChain's PresidioReversibleAnonymizer

LangChain's PresidioReversibleAnonymizer does not expose a public API to override its internal analyzer. However, Source 2 demonstrates a workaround using private attribute access. After creating the properly configured analyzer as shown in Solution 1, you can assign it directly:

from langchain_experimental.data_anonymizer import PresidioReversibleAnonymizer

anonymizer = PresidioReversibleAnonymizer(
    analyzed_fields=["PERSON", "PHONE_NUMBER", "EMAIL_ADDRESS", "CREDIT_CARD"]
)
anonymizer._analyzer = analyzer
anonymized_result = anonymizer.anonymize("Jan Kowalski mieszka w Warszawie i ma e-mail jan.kowalski@example.com.")
print(anonymized_result)

Output:

'Jonathan Johnson mieszka w Warszawie i ma e-mail jillrhodes@example.net.'

This output shows that "Jan Kowalski" was replaced with a fake name ("Jonathan Johnson") and the email address was replaced with a fake email ("jillrhodes@example.net"). The location "Warszawie" remains unchanged because LOCATION was not in the analyzed_fields list. If you want locations anonymized as well, add "LOCATION" to the list.

When to use this approach: Only when you must use PresidioReversibleAnonymizer from LangChain and cannot refactor to use Presidio's AnalyzerEngine directly. Be aware that accessing private attributes (_analyzer) may break with future library updates.

Why the Original Code Did Not Work

The original code from Source 1 passed a languages_config dictionary directly to PresidioAnonymizer and PresidioReversibleAnonymizer. These classes internally create an AnalyzerEngine using that config, but they do not pass the ner_model_configuration mapping. Without the mapping, the analyzer sees spaCy's persName label and does not know it corresponds to PERSON, so it produces no results.

When the user called PresidioReversibleAnonymizer() with no arguments (the default), it used the default English spaCy model (en_core_web_lg), which already has a built-in mapping for English entity labels (PERSON, ORG, GPE, etc.). That is why the default configuration anonymized the text, but with English-sounding fake names.