Data Protection to Tokenize Sensitive Data Before AI Agents

Tokenize sensitive data before AI agents process it to prevent data leakage.

Data Protection with AI Developer Edition prevents data leakage in prompts, logs, and tool chains by replacing sensitive values with tokens. It keeps data usable and compliant using tokenization, letting AI agents work on real workflows while sensitive data stays protected end-to-end.

Key Principles

Tokenization replaces real values with reversible tokens.
AI agents reason on safe data tokens as if they are functionally equivalent.
Real data restored only for approved API calls through authorized unprotect.
Works across the entire pipeline prompts, reasoning, tool calls, and outputs.

Key Benefits

Prevents secret leakage in prompts, logs, and outputs.
Maintains full functionality using reversible tokens.
AI agents work normally logic is preserved.
Zero trust architecture secrets never leave your environment.
Audit trail all protect or unprotect operations are logged.

How It Works

Real Data                    Tokenized Data
─────────                    ──────────────
John Smith          →        TKN-NAME-001
4532-1234-5678-9012 →        TKN-CC-A8F2
123-45-6789         →        TKN-SSN-B3C1
sk_live_abc123      →        TKN-API-D4E5
AI Agent processes tokens → Same logic, zero risk
Real values restored only for authorized execution

Best Practices

Keep the following guidelines in mind when using Data Protection to handle tokens safely and keep sensitive data within authorized boundaries.

Protect before the AI sees the data. Call find_and_protect() or session.protect() before passing any input to the model. Tokenize prompts, tool arguments, and log entries at the point of entry.
Scope unprotect operations narrowly. Only unprotect what the current operation requires.
Validate AI responses before acting on them. Scan model output for sensitive data using the Semantic Guardrail API. Block or sanitize the response before passing it downstream.
Choose the right protection method for each context. Use tokenization where you need reversibility, masking for logs and display surfaces, and format-preserving tokenization when downstream APIs validate the value’s shape.
Never log or print unprotected values. Treat any variable that has been unprotected as sensitive. Do not print it, write it to a file, or pass it to a logging framework.
Require authorization for every unprotect call. Always pass valid credentials. Do not cache or reuse authorization tokens across sessions or scopes.
Treat tokens as opaque identifiers. Do not parse, decode, or reverse tokens in application logic. They are functionally equivalent to the original values for reasoning only.

Example: AI Agent Updates Customer Records

The Scenario

An AI agent processes customer support requests that involve:

Updating customer records
Processing credit card transactions
Accessing account details

Without protection: Agent sees real credit card numbers, SSNs, addresses

With AI Developer Edition: Agent reasons on tokenized data safely

Cursor Prompt

In Cursor’s AI chat, enter the following prompt:

Using AI Developer Edition Data Protection, tokenize all sensitive customer data before my AI agent processes support tickets. The agent should be able to reason about the workflow without seeing real PII or payment data.

Implementation

from appython import Protector

protector = Protector()
session = protector.create_session("superuser")

# Customer record with sensitive data
customer_record = {
    "customer_id": "CUST-12345",
    "name": "Sarah Johnson",
    "email": "sarah.j@email.com",
    "ssn": "123-45-6789",
    "credit_card": "4532-1234-5678-9012",
    "address": "123 Main St, New York, NY 10001",
    "phone": "+1-555-0142",
    "account_balance": 5420.00,
    "support_ticket": "Customer wants to update billing address and process refund"
}

# Protect each sensitive field before AI agent processes
protected_record = {
    "customer_id": customer_record["customer_id"],
    "name": session.protect(customer_record["name"], "name"),
    "email": session.protect(customer_record["email"], "email"),
    "ssn": session.protect(customer_record["ssn"], "ssn"),
    "credit_card": session.protect(customer_record["credit_card"], "ccn"),
    "address": session.protect(customer_record["address"], "address"),
    "phone": session.protect(customer_record["phone"], "phone"),
    "account_balance": customer_record["account_balance"],
    "support_ticket": customer_record["support_ticket"]
}

What the AI Agent Sees

# Agent receives tokenized record
{
    "customer_id": "CUST-12345",           # Non-sensitive: kept as-is
    "name": "TKN-NAME-A1B2",              # Tokenized
    "email": "TKN-EMAIL-C3D4",            # Tokenized
    "ssn": "TKN-SSN-E5F6",               # Tokenized
    "credit_card": "TKN-CC-G7H8",         # Tokenized
    "address": "TKN-ADDR-I9J0",           # Tokenized
    "phone": "TKN-PHONE-K1L2",            # Tokenized
    "account_balance": 5420.00,            # Non-sensitive: kept as-is
    "support_ticket": "Customer wants to update billing address and process refund"
}

Agent Reasons Safely

# AI agent processes the support ticket
agent_decision = agent.process(protected_record)
# Agent output: "Update address for TKN-NAME-A1B2, process refund to TKN-CC-G7H8"

# Only when executing the actual API call, restore real data
if agent_decision.requires_execution:
    # Authorized unprotect for approved operation only
    real_name = session.unprotect(protected_record["name"], "name")
    real_cc = session.unprotect(protected_record["credit_card"], "ccn")
    billing_api.update({"name": real_name, "credit_card": real_cc})

Protect Secrets in Cursor Prompts with Tokenization

The Scenario

Developers use Cursor to analyze code or debug issues. Prompts often contain:

API keys from config files
Database connection strings
Authentication tokens
Customer data from logs

AI Developer Edition replaces secrets with tokens before sending to the model.

Cursor Prompt

In Cursor’s AI chat, enter the following prompt:

Using AI Developer Edition Data Protection, tokenize any secrets in my prompts before they're sent to the AI model. I want to debug code safely without leaking credentials.

Example: Debug Code with API Key

Developer’s original prompt:

Debug this code - the API call is failing:
import requests
API_KEY = "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
response = requests.post(
    "https://api.stripe.com/v1/charges",
    headers={"Authorization": f"Bearer {API_KEY}"},
    data={"amount": 2000, "currency": "usd"}
)
print(response.status_code)  # Returns 401

What AI Developer Edition sends to the model (tokenized):

Debug this code - the API call is failing:
import requests
API_KEY = "TKN-API-A1B2C3D4"
response = requests.post(
    "https://api.stripe.com/v1/charges",
    headers={"Authorization": f"Bearer {API_KEY}"},
    data={"amount": 2000, "currency": "usd"}
)
print(response.status_code)  # Returns 401

Cursor processes safely - Diagnoses the 401 error, for example, “Check if the API key has correct permissions” without ever seeing the real secret.

Real value restored only for execution:

# When developer runs the fix, real key is restored
real_code = protegrity_developer_python.find_and_unprotect(fixed_code)

Example: Debug Database Connection

Developer’s prompt (before protection):

This connection keeps timing out:
conn = psycopg2.connect("postgresql://admin:Sup3rS3cret!@prod-db.internal:5432/customers")

What the model sees (tokenized):

This connection keeps timing out:
conn = psycopg2.connect("postgresql://TKN-USER-001:TKN-PASS-001@TKN-HOST-001:5432/customers")

AI diagnoses: “The timeout might be caused by network configuration, connection pool exhaustion, or firewall rules. Try adding connect_timeout=10 parameter.”

Full Protection Pipeline

"""
Complete Data Protection workflow for Cursor development
"""
import protegrity_developer_python
import requests

protegrity_developer_python.configure(
    endpoint_url="http://localhost:8580/pty/data-discovery/v2/classify",
    classification_score_threshold=0.6,
    enable_logging=True,
    log_level="info"
)

GUARDRAIL_URL = "http://localhost:8581/pty/semantic-guardrail/v1.1/conversations/messages/scan"

# ═══════════════════════════════════════════════════════
# Step 1: PROTECT - Before sending to AI
# ═══════════════════════════════════════════════════════
def protect_prompt(user_input: str) -> str:
    """Tokenize any secrets/PII in the prompt before AI processing."""
    return protegrity_developer_python.find_and_protect(user_input)

# ═══════════════════════════════════════════════════════
# Step 2: PROCESS - AI reasons on safe data
# ═══════════════════════════════════════════════════════
def process_with_ai(safe_prompt: str) -> str:
    """AI processes the tokenized prompt safely."""
    response = ai_model.generate(safe_prompt)
    return response

# ═══════════════════════════════════════════════════════
# Step 3: VALIDATE - Check response for leaks
# ═══════════════════════════════════════════════════════
def validate_response(response: str) -> str:
    """Ensure AI response doesn't contain sensitive data."""
    payload = {
        "messages": [{"role": "assistant", "content": response}],
        "processors": ["pii"]
    }
    check = requests.post(GUARDRAIL_URL, json=payload).json()
    if check.get("action") == "BLOCK":
        return "Response blocked: contained sensitive information"
    return response

# ═══════════════════════════════════════════════════════
# Step 4: UNPROTECT - Restore for execution only
# ═══════════════════════════════════════════════════════
def execute_safely(ai_response: str) -> str:
    """Restore real values only for authorized execution."""
    real_response = protegrity_developer_python.find_and_unprotect(ai_response)
    return real_response

# ═══════════════════════════════════════════════════════
# Usage Flow
# ═══════════════════════════════════════════════════════
# Developer writes prompt with real API key
raw_prompt = 'Fix this: requests.get(url, headers={"X-API-Key": "ak_prod_9f8e7d6c5b4a"})'

# Step 1: Protect
safe_prompt = protect_prompt(raw_prompt)
# → 'Fix this: requests.get(url, headers={"X-API-Key": "TKN-API-M3N4"})'

# Step 2: AI processes safely
ai_response = process_with_ai(safe_prompt)
# → "Add error handling: try/except around the request with TKN-API-M3N4"

# Step 3: Validate response
clean_response = validate_response(ai_response)

# Step 4: Execute with real values
result = execute_safely(clean_response)

Protection Methods

Method	Use Case	Reversible	Format-Preserving
Tokenization	Prompts, tool calls	Yes (authorized)	Optional
Masking	Logs, displays	No	Yes
Encryption	Storage at rest	Yes (with key)	No
Format-Preserving Tokenization	APIs that validate format	Yes	Yes

Format-Preserving Examples

# Credit card: same format, different value
"4532-1234-5678-9012" → "4916-8823-4455-1107"
# SSN: same format
"123-45-6789" → "987-65-0123"
# Email: same structure
"john@company.com" → "tkn_a1b2@example.com"

Feedback

Was this page helpful?

Last modified : August 12, 2026