Skip to content

Software Quality

Personal-data detection

Finding the names, contact details, identifiers and account numbers a dataset holds before it is shared, published or used for training.

Personal-data detection scans a dataset for information that identifies a person, such as names, email addresses, phone numbers, national identifiers and account numbers, so that it can be removed, masked or restricted before the dataset is shared, published or used to train a model. Personal information reaches datasets that were never meant to hold it: a patient's name in a free-text note, a customer's card number in a support ticket, a user's email in a log line. Nobody declared those fields, so no schema says where to look, and the check has to read the values.

How a value is recognized

  • Patterns. A regular expression recognizes a type with a rigid format, such as an email address, an IP address or a phone number. A checksum separates a real card or bank account number from a string of the same shape: without one, a pattern for international bank account numbers flags far more strings than are real accounts (Subramani et al. 2023)1.
  • Named-entity models. A model trained to tag names, places and organizations in text recognizes types with no fixed format. Such models make errors, so some personal information stays unrecognized (Lukas et al. 2023)2.
  • Hybrids. Most tools combine the two, taking a pattern's verdict for rigid types and a model's for the rest. The most successful systems in a clinical de-identification challenge combined statistical sequence models with hand-written rules (Stubbs et al. 2015)3.

In a table, a tool tags whole columns, from the column's name and type or from a sample of its values, so the result is a map of where personal information lives, not a list of every occurrence.

Credentials committed to a repository are found by secret scanning, which applies the same patterns and thresholds to code and its history.

What it catches

  • Identifiers in free text. Names, dates, addresses and record numbers in notes, tickets and messages.
  • Contact details and account numbers in scraped data. Email addresses, phone numbers, card and bank account numbers in a corpus collected for training.
  • Personal data in logs. A user's email, address or session identifier written to a log line by a debugging statement.
  • Undeclared personal columns. A free-text or generic column in a warehouse table that turns out to hold identifiers.

What it does not catch:

  • information that identifies a person only in combination, such as a rare diagnosis together with a postcode and an age;
  • a description of a person rather than an identifier, which no entity type names;
  • personal information encoded or transformed, such as a hashed email that can be matched against a list of known addresses.

Tools

  • Presidio — Microsoft's open-source library that detects personal information in text and images with pattern recognizers, checksums and named-entity models, and masks or replaces what it finds.
  • OpenMetadata — open-source data catalog whose auto-classification tags a column as personal information from its name, or from a named-entity scan of sampled values above a confidence threshold.
  • DataHub — open-source data catalog that proposes a personal-information type for each column at ingestion from its name, description and data type.
  • Cloud data-loss-prevention services (commercial) — Google Cloud Sensitive Data Protection and Amazon Macie scan data held in their providers' storage services for personal information.

When to use, when not

Use:

  • Before a dataset leaves the team that collected it: a publication, a vendor share, a training run.
  • On free-text fields, logs and scraped data, where nothing declares what the values contain.
  • As a recurring scan of a warehouse, to keep the catalog's record of personal columns current.

Don't:

  • As the only safeguard for data that must be anonymous. A detector misses combinations of values that identify a person, and a clean scan is not a guarantee.
  • To redact automatically without checking precision on the dataset at hand. A pattern that flags more strings than real identifiers deletes data the dataset needed.

Evidence

  • Clinical de-identification is effective and not solved. In the 2014 i2b2/UTHealth shared task, 3 of 10 teams' systems reached F1 scores above 0.90 and seven of the top 10 scored above 0.75, and the organizers concluded that de-identification is not yet a solved problem (Stubbs et al. 2015)3.
  • Accuracy depends on the type. On a manual audit of detections in C4, Presidio and regular expressions were near-perfect on email addresses, while Presidio was 98.5% accurate on international bank account numbers against 15.2% for regular expressions without a checksum, and both were below 15% on credit card numbers (Subramani et al. 2023)1.
  • Pattern precision varies by corpus. In ten corpora used to train language models, regular expressions identified email addresses with over 80% precision in eight corpora and phone numbers in five; in LAION-2B-en, 7% of phone-number matches were real phone numbers (Elazar et al. 2024)4.

Classification

  • Data-quality characteristics: Confidentiality (personal information found before a dataset is shared, published or used for training can be removed or restricted).
  • Area: Datasets that may hold personal information nobody intended to keep: free text such as clinical notes, support tickets and logs, corpora scraped for model training, and the columns of warehouse tables.
  • Guarantee: Empirical — a span is flagged when a pattern or a model recognizes it; personal information in a form neither recognizes passes, and a clean scan is not a proof of absence.

Referenced by

References

  1. Subramani, Nishant, Sasha Luccioni, Jesse Dodge, and Margaret Mitchell. 2023. "Detecting Personal Information in Training Corpora: an Analysis." Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023). https://doi.org/10.18653/v1/2023.trustnlp-1.18. ↩↩

  2. Lukas, Nils, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. 2023. "Analyzing Leakage of Personally Identifiable Information in Language Models." 2023 IEEE Symposium on Security and Privacy (SP). https://doi.org/10.1109/SP46215.2023.10179300. ↩

  3. Stubbs, Amber, Christopher Kotfila, and Özlem Uzuner. 2015. "Automated Systems for the De-identification of Longitudinal Clinical Narratives: Overview of 2014 i2b2/UTHealth Shared Task Track 1." Journal of Biomedical Informatics 58. https://doi.org/10.1016/j.jbi.2015.06.007. ↩↩

  4. Elazar, Yanai, Akshita Bhagia, Ian Magnusson, et al. 2024. "What's In My Big Data?" The Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/pdf/2310.20707. ↩