Data Discovery to Prevent Sensitive Data Exposure in AI Agents

Auto-discover and classify sensitive data for AI agents.

Use AI Developer Edition to auto-discover PII and secrets across datasets, logs, and prompts before agents use them. Data Discovery automatically finds and tags sensitive data at source, giving agents only safe, compliant data to work with.

The Problem

AI agents routinely access:

  • Datasets with hidden PII, such as SSN, emails, phone numbers in unexpected columns.
  • Logs containing credentials, session tokens, user IDs.
  • Code with hardcoded API keys, connection strings, passwords.
  • Prompts that include sensitive data from copy-paste.

Without Data Discovery, agents unknowingly process, store, and potentially expose this sensitive data.

How Data Discovery Works

The Data Discovery Engine processes data from any source, such as datasets, logs, code, and prompts, through three sequential stages:

  1. Pattern Recognition - Applies regex patterns, such as SSN, credit card formats; named entity recognition, and contextual analysis to identify candidate sensitive values across all input.

  2. Classification - Categorizes each detected item into one of four tiers:

    • PII: It includes names, email addresses, phone numbers, and physical addresses that can identify an individual.
    • Highly Sensitive: It includes SSNs and medical records that require the strictest level of protection.
    • Secrets: It includes API keys, tokens, and credentials used to access systems or services.
    • Financial: It includes credit card numbers and bank account details subject to regulatory controls.

  3. Confidence Scoring - Assigns a confidence level between 0.0 and 1.0 to each finding using context-aware, multi-signal validation to minimize false positives.

The engine outputs a classification report with recommended actions, such as tokenize, mask, anonymize, or block for each detected field.

Best Practices

  1. Scan before any AI processing - Run Data Discovery on all data before sending to agents
  2. Automate in CI/CD - Add Data Discovery scanning to your pipeline
  3. Set confidence thresholds - Configure minimum confidence for automated actions
  4. Review edge cases - Manually review items with confidence 0.6-0.8
  5. Continuous monitoring - Re-scan when data sources change
  6. Least privilege - Only expose fields the agent actually needs

Example: Employee Data Containing Sensitive Data

Cursor Prompt

In Cursor’s AI chat, enter the following prompt:

Using AI Developer Edition, run data discovery on my employee dataset to identify all sensitive fields before using it in AI workflows.

Implementation

import protegrity_developer_python

protegrity_developer_python.configure(
    endpoint_url="http://localhost:8580/pty/data-discovery/v2/classify",
    named_entity_map={
        "PERSON": "NAME", "EMAIL_ADDRESS": "EMAIL", "SOCIAL_SECURITY_ID": "SSN",
        "PHONE_NUMBER": "PHONE", "LOCATION": "ADDRESS"
    },
    classification_score_threshold=0.6,
    enable_logging=True,
    log_level="info"
)

# Sample employee data as text
employee_text = """
Name: John Smith, Email: john.smith@company.com, SSN: 123-45-6789, Phone: 555-0142
Name: Sarah Johnson, Email: sarah.j@company.com, SSN: 987-65-4321, Phone: 555-0198
"""

# Run data discovery and redact PII
redacted = protegrity_developer_python.find_and_redact(employee_text)
print("DATA DISCOVERY + REDACTION REPORT")
print("=" * 70)
print(f"Original:\n{employee_text}")
print(f"Redacted:\n{redacted}")

Output

DATA DISCOVERY + REDACTION REPORT
======================================================================
Original:
Name: John Smith, Email: john.smith@company.com, SSN: 123-45-6789, Phone: 555-0142
Name: Sarah Johnson, Email: sarah.j@company.com, SSN: 987-65-4321, Phone: 555-0198

Redacted:
Name: ##########, Email: ######################, SSN: ###########, Phone: ########
Name: #############, Email: ###################, SSN: ###########, Phone: ########

Automated Protection with Tokenization

# Instead of redaction, tokenize sensitive data so it can be restored later
protected = protegrity_developer_python.find_and_protect(employee_text)
print(f"Protected (tokenized):\n{protected}")

# Authorized unprotect to restore original values
original = protegrity_developer_python.find_and_unprotect(protected)
print(f"Restored:\n{original}")

Detect & Prevent Secret Leakage in Code Using Cursor

The Scenario

A developer asks Cursor: “Analyze this repo and generate a deployment script.”

Without Data Discovery, the AI agent might:

  • Read config files with hardcoded API keys
  • Include secrets in generated deployment scripts
  • Expose tokens in output logs

Cursor Prompt

In Cursor’s AI chat, enter the following prompt:

Using AI Developer Edition's Data Discovery, scan my codebase for secrets and PII before generating the deployment script. Block or mask any sensitive data found.

Implementation

"""
Data Discovery: Scan codebase for secrets before AI processing
"""
import protegrity_developer_python
import os

protegrity_developer_python.configure(
    endpoint_url="http://localhost:8580/pty/data-discovery/v2/classify",
    classification_score_threshold=0.6,
    method="redact",
    enable_logging=True,
    log_level="info"
)

# Read config files and scan for secrets
config_files = ["config/database.yml", ".env", "src/services/payment.py"]
for filepath in config_files:
    if os.path.exists(filepath):
        with open(filepath, "r") as f:
            content = f.read()
        redacted = protegrity_developer_python.find_and_redact(content)
        print(f"\n--- {filepath} ---")
        print(f"Redacted:\n{redacted}")

print("=" * 70)
print("RECOMMENDATION: Apply protection before sending to AI agent")

Scan Results

REPOSITORY SCAN RESULTS
======================================================================
config/database.yml
   Line 5: connection_string = "postgresql://user:P@ssw0rd123@prod-db:5432/app"
      Classification: Credential (password in connection string)
      Confidence: 0.99
      Action: BLOCK - Do not send to AI agent
.env
   Line 2: STRIPE_API_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc
      Classification: API Key (Stripe live key)
      Confidence: 0.99
      Action: BLOCK
   Line 5: AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
      Classification: API Key (AWS secret)
      Confidence: 0.99
      Action: BLOCK
src/services/payment.py
   Line 42: api_key = "pk_test_TYooMQauvdEDq54NiTphI7jx"
      Classification: Hardcoded API Key
      Confidence: 0.97
      Action: TOKENIZE before AI processing
logs/app.log
   Line 156: "user_email": "john.smith@company.com"
      Classification: PII (email)
      Confidence: 0.95
      Action: ANONYMIZE
   Line 203: Session token: eyJhbGciOiJIUzI1NiIs...
      Classification: Session Token (JWT)
      Confidence: 0.93
      Action: MASK
======================================================================
SUMMARY:
   Files scanned: 147
   Secrets found: 5 (CRITICAL)
   PII found: 12 (WARNING)
   Total sensitive items: 17
   
RECOMMENDATION: Apply protection before sending to AI agent

Protected Deployment Script Generation

import protegrity_developer_python

protegrity_developer_python.configure(
    endpoint_url="http://localhost:8580/pty/data-discovery/v2/classify",
    classification_score_threshold=0.6,
    enable_logging=True,
    log_level="info"
)

# Step 1: Read config files with secrets
with open("config/database.yml", "r") as f:
    config_content = f.read()

# Step 2: Protect secrets before AI processing (tokenize)
safe_content = protegrity_developer_python.find_and_protect(config_content)
# Cursor sees tokenized values, not real secrets

# Step 3: AI agent generates deployment script with SAFE data
deployment_script = agent.generate_deployment(context=safe_content)

# Step 4: For actual deployment, restore tokens using authorized API
if authorized_deploy:
    real_script = protegrity_developer_python.find_and_unprotect(deployment_script)

What the AI Agent Sees as Protected

# deployment.yml (as seen by AI agent)
database:
  connection: "postgresql://TKN-USER-001:TKN-PASS-001@TKN-HOST-001:5432/app"
  
api_keys:
  stripe: "TKN-API-001"
  aws_secret: "TKN-API-002"
# Agent generates correct deployment logic without seeing real secrets

What Gets Deployed after Authorized Unprotect

# deployment.yml (restored for actual deployment)
database:
  connection: "postgresql://user:P@ssw0rd123@prod-db:5432/app"
  
api_keys:
  stripe: "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
  aws_secret: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Data Discovery for Different Data Sources

Text Data

import protegrity_developer_python

protegrity_developer_python.configure(
    endpoint_url="http://localhost:8580/pty/data-discovery/v2/classify",
    classification_score_threshold=0.6,
    method="redact",
    enable_logging=True,
    log_level="info"
)

# Scan text content from any source
with open("app.log", "r") as f:
    log_content = f.read()
redacted_log = protegrity_developer_python.find_and_redact(log_content)

Scan and Protect Source Code

import os

# Scan source files for hardcoded secrets
for root, dirs, files in os.walk("./src"):
    for file in files:
        if file.endswith((".py", ".js", ".java", ".yml", ".env")):
            filepath = os.path.join(root, file)
            with open(filepath, "r") as f:
                content = f.read()
            protected = protegrity_developer_python.find_and_protect(content)
            if protected != content:
                print(f"  Secrets found and protected in: {filepath}")

Last modified : August 12, 2026