Appendix

This section provides supplemental details for working with the product.

1: Input Sanitization
2: Working with the Data Discovery containers

2.1: Understanding the Docker Compose File
2.2: Deploying the Application

3: Supported Sensitive Entity Types
4: Data Security Policy
5: Removing AI Developer Edition
6: Known Issues

1 - Input Sanitization

Rejecting unsanitized data.

The Classification service in Data Discovery offers a security feature that rejects unsanitized data. Data that is malformed, non-normalized, containing homoglyphs, hieroglyphs, mixed Unicode variants, or control characters are considered as unsanitized data. These are rejected for classification.

The following are few examples of data that will be rejected:

Ⅷ
𝓉𝑒𝓍𝓉
Ｐｅｐ

Before invoking the Classification endpoint, ensure that the input text is normalized. Replace invalid characters by their corresponding normalized plaintext characters. If the input text contains any invalid character, a status code of 422 and a message Untrusted input is returned.

For security purposes, the application rejects unsanitized data by default. It is recommended that this feature remains enabled. However, to override this feature, perform the following steps.

Navigate to the docker_compose directory.
Edit the docker-compose.yaml file.
Under the environment section of classification_service, append the security parameter as follows.

- SECURITY_SETTINGS={"ENABLE_ALL_SECURITY_CONTROLS":false}

Save the changes.
Run the docker compose down command to undeploy the application.
Run the docker compose up command to redeploy the application.

2 - Working with the Data Discovery containers

Using the Data Discovery containers.

Use Data Discovery by setting up and deploying the containers.

2.1 - Understanding the Docker Compose File

Details of the configurable parameters in the docker-compose.yml file.

The following variables can be configured in the docker-compose.yml file.

Variable	Description	Mandatory
networks:name	Specify the name of the Docker network.	No
services:enviroment	Specify the location for the logs in the `logging_config` parameter.	No
classification_service:ports	Specify the listening port for the classification service. By default, the port is set to 8580.	No

2.2 - Deploying the Application

Deploying the Data Discovery container.

Ensure that the prerequisites are completed before deploying the application.

Run the following steps to deploy the Data Discovery application on Docker.

Open a command prompt.
Navigate to the AI Developer Edition package directory.
Run the command to start the containers. For example, the following command starts the Classification service container.

docker compose up -d

3 - Supported Sensitive Entity Types

PII entities supported by Protegrity AI Developer Edition.

Entity Name	Data Element	Description
ABA_ROUTING_NUMBER	number	Routing number used to identify financial institutions in the United States.
ACCOUNT_NAME	string	Name associated with a financial account.
ACCOUNT_NUMBER	number	Bank account number used to identify financial accounts.
AGE	number	Age information used to identify individuals.
AMOUNT	int	Specific amount of money, which can be linked to financial transactions.
AU_ABN	number	Australian Business Number used to identify businesses in Australia.
AU_ACN	number	Australian Company Number used to identify businesses in Australia.
AU_MEDICARE	number	Medicare number used to identify individuals for healthcare services in Australia.
AU_TFN	number	Tax File Number used to identify taxpayers in Australia.
BIC	number	Bank Identifier Code used to identify financial institutions.
BITCOIN_ADDRESS	address	Bitcoin wallet address used for digital transactions.
BUILDING	address	Building information used to identify specific locations.
CITY	city	City information used to identify geographic locations.
COMPANY_NAME	string	Name of a company used to identify businesses.
COUNTRY	string	Country information used to identify geographic locations.
COUNTY	string	County information used to identify geographic locations.
CREDIT_CARD	ccn	Credit card number used for financial transactions.
CREDIT_CARD_CVV	number	Card Verification Value used to secure credit card transactions.
CRYPTO	address	Cryptocurrency wallet address used for digital transactions.
CURRENCY	string	Currency information used in financial transactions.
CURRENCY_CODE	string	Code representing currency used in financial transactions.
CURRENCY_NAME	string	Name of currency used in financial transactions.
CURRENCY_SYMBOL	string	Symbol representing currency, sometimes linked to financial transactions.
DATE	datetime	Specific date that can be linked to personal activities.
DATE_OF_BIRTH	datetime	Date of birth used to identify individuals.
DATE_TIME	datetime	Specific date and time that can be linked to personal activities.
DRIVER_LICENSE	number	Driver’s license number used to identify individuals.
EMAIL_ADDRESS	email	Email address used for communication and identification.
ES_NIE	nin	Foreigner Identification Number used to identify non-residents in Spain.
ES_NIF	nin	Tax Identification Number used to identify taxpayers in Spain.
ETHEREUM_ADDRESS	address	Ethereum wallet address used for digital transactions.
FI_PERSONAL_IDENTITY_CODE	nin	Personal identity code used to identify individuals in Finland.
GENDER	string	Gender information used to identify individuals.
GEO_CCORDINATE	address	Geographic coordinates used to identify specific locations.
IBAN_CODE	iban	International Bank Account Number used to identify bank accounts globally.
ID_CARD	number	Identity card number used to identify individuals.
IN_AADHAAR	nin	Unique identification number used to identify residents in India.
IN_PAN	number	Permanent Account Number used to identify taxpayers in India.
IN_PASSPORT	passport	Passport number used to identify individuals in India.
IN_VEHICLE_REGISTRATION	number	Vehicle registration number used to identify vehicles in India.
IN_VOTER	number	Voter ID number used to identify registered voters in India.
IP_ADDRESS	address	Internet Protocol address used to identify devices on a network.
IPV4	address	IPv4 address used to identify devices on a network.
IPV6	address	IPv6 address used to identify devices on a network.
IT_DRIVER_LICENSE	number	Driver’s license number used to identify individuals in Italy.
IT_FISCAL_CODE	nin	Fiscal code used to identify taxpayers in Italy.
IT_IDENTITY_CARD	number	Identity card number used to identify individuals in Italy.
IT_PASSPORT	passport	Passport number used to identify individuals in Italy.
LITECOIN_ADDRESS	address	Litecoin wallet address used for digital transactions.
LOCATION	address	Specific location or address that can be linked to an individual.
MAC	address	Media Access Control address used to identify devices on a network.
MEDICAL_LICENSE	number	License number used to identify medical professionals.
NRP	number	A person’s nationality, religious or political group.
ORGANIZATION	string	Name or identifier used to identify an organization.
PASSPORT	passport	Passport number used to identify individuals.
PASSWORD	string	Password used to secure access to personal accounts.
PERSON	string	Name or identifier used to identify an individual.
PHONE_NUMBER	phone	Number used to contact or identify an individual.
PIN	number	Personal Identification Number used to secure access to accounts.
PL_PESEL	nin	Personal Identification Number used to identify individuals in Poland.
SECONDARYADDRESS	address	Additional address information used to identify locations.
SG_NRIC_FIN	nin	National Registration Identity Card number used to identify residents in Singapore.
SG_UEN	number	Unique Entity Number used to identify businesses in Singapore.
SOCIAL_SECURITY_NUMBER	ssn	Social Security Number used to identify individuals.
STATE	string	State information used to identify geographic locations.
STREET	address	Street address used to identify specific locations.
TIME	datetime	Specific time that can be linked to personal activities.
TITLE	string	Title or honorific used to identify individuals.
UK_NHS	number	National Health Service number used to identify individuals for healthcare services in the United Kingdom.
URL	address	Web address that can sometimes contain personal information.
US_BANK_NUMBER	number	Bank account number used to identify financial accounts in the United States.
US_DRIVER_LICENSE	number	Driver’s license number used to identify individuals in the United States.
US_ITIN	number	Individual Taxpayer Identification Number used to identify taxpayers in the United States.
US_PASSPORT	passport	Passport number used to identify individuals in the United States.
US_SSN	ssn	Social Security Number used to identify individuals in the United States.
USERNAME	string	Username used to identify individuals in online systems.
ZIP_CODE	zipcode	Postal code used to identify specific geographic areas.

4 - Data Security Policy

Data Security Policy configuration.

This section describes the Policy configuration used by the AI Developer Edition API Service.

The superuser has all permissions, that is, protect, unprotect, and reprotect operations. Users assigned the admin role will receive protected data when performing an unprotect operation, except in the case of the text data elements, which will return null. All other user roles will receive null as the output for any unprotect operation.

Policy Definition

Generic Data Elements

Data Element	Method	Use Case	UTF Set	LP	PP	eIV	Role
							Admin		Finance		Marketing		HR
							P	U	P	U	P	U	P	U
datetime	Tokenization	A date or datetime string. Formats accepted: YYYY/MM/DD HH:MM:SS and YYYY/MM/DD. Delimiters accepted: /, - (required).	N/A	N/A	N/A	No	✓	Ｘ	Ｘ	Ｘ	Ｘ	✓	Ｘ	Ｘ
datetime_yc	Tokenization	A date or datetime string. Formats accepted: YYYY/MM/DD HH:MM:SS and YYYY/MM/DD. Delimiters accepted: /, - (required). Leaves the year in the clear.	N/A	N/A	N/A	No	✓	Ｘ	Ｘ	Ｘ	Ｘ	✓	Ｘ	Ｘ
int	Tokenization	An integer string (4 bytes).	Numeric	No	No	Yes	✓	Ｘ	Ｘ	Ｘ	Ｘ	✓	Ｘ	Ｘ
number	Tokenization	A numeric string. May produce leading zeroes.	Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	Ｘ	Ｘ	✓	Ｘ	Ｘ
string	Tokenization	An alphanumeric string.	Latin + Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	Ｘ	Ｘ	✓	Ｘ	Ｘ
text	Encryption	A long string (e.g., a comment field) using any character set. Use hex or base64 encoding to utilize.	All	No	No	Yes	✓	Ｘ	Ｘ	Ｘ	Ｘ	✓	Ｘ	Ｘ

PCI DSS Data Elements

Data Element	Method	Use Case	UTF Set	LP	PP	eIV	Role
							Admin		Finance		Marketing		HR
							P	U	P	U	P	U	P	U
ccn	Tokenization	Credit card numbers.	Numeric	No	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	X	Ｘ	✓
ccn_bin	Tokenization	Credit card numbers. Leaves 8-digit BIN in the clear.	Numeric	No	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	X	Ｘ	✓
iban	Tokenization	IBAN numbers. Preserves the length, case, and position of the input characters but may create invalid IBAN codes.	Latin + Numeric	Yes	Yes	No	✓	Ｘ	Ｘ	✓	Ｘ	X	Ｘ	✓
iban_cc	Tokenization	IBAN numbers. Leaves letters in the clear.	Latin + Numeric	No	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	X	Ｘ	✓

PII Data Elements

Data Element	Method	Use Case	UTF Set	LP	PP	eIV	Role
							Admin		Finance		Marketing		HR
							P	U	P	U	P	U	P	U
address	Tokenization	Street names	Latin + Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	Ｘ	Ｘ	✓
city	Tokenization	Town or city name	Latin	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓
email	Tokenization	Email address. Leaves the domain in the clear.	Latin + Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓
nin	Tokenization	National Insurance Number. Preserves the length, case, and position of the input characters but may create invalid NIN codes.	Latin + Numeric	Yes	Yes	No	✓	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ
name	Tokenization	Person's name	Latin	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓
passport	Tokenization	Passport codes. Preserves the length, case, and position of the input characters but may create invalid passport numbers.	Latin + Numeric	Yes	Yes	No	✓	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ
phone	Tokenization	Phone number. May produce leading zeroes.	Latin + Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ
postcode	Tokenization	Postal codes with digits and characters. Preserves the length, case, and position of the input characters but may create invalid post codes.	Latin + numeric	Yes	Yes	No	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓
ssn	Tokenization	Social Security Number (US)	Latin + Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ	Ｘ
zipcode	Tokenization	Zip codes with digits only. May produce leading zeroes.	Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓

PII Data Elements

Data Element	Method	Use Case	UTF Set	LP	PP	eIV	Role
							Admin		Finance		Marketing		HR
							P	U	P	U	P	U	P	U
address_de	Tokenization	Street names (German)	Latin + German + Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	Ｘ	Ｘ	✓
address_fr	Tokenization	Street names (French)	Latin + French + Numeric	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	Ｘ	Ｘ	✓
city_de	Tokenization	Town or city name (German)	Latin + German	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓
city_fr	Tokenization	Town or city name (French)	Latin + French	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓
name_de	Tokenization	Person's name (German)	Latin + German	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓
name_fr	Tokenization	Person's name (French)	Latin + French	Yes	No	Yes	✓	Ｘ	Ｘ	✓	Ｘ	✓	Ｘ	✓

LEGEND

eIV: External IV
LP: Length Preservation
PP: Position Preservation
P: User group can protect data
U: User group can unprotect data

5 - Removing AI Developer Edition

Steps for removing the product.

Open a command prompt.
Navigate to the cloned repository location.
Run the following command to remove the containers and images.
```
docker compose down --rmi all
```
Run the following command to remove the Python module.
```
pip uninstall protegrity-developer-python
```

6 - Known Issues

Issues and workaround information.

Issue: SSL errors in the Data Discovery container

Description: The tldextract tries to download the following public Suffix lists files:

When these lists cannot be downloaded, then the default files included in the package are used and no issue in observed in the classification.