Running the Data Protection samples
Applications are provided out-of-the-box to test and understand the capabilities of AI Developer Edition.
Before running the samples, verify that the AI Developer Edition Service is running. The service availability can be monitored on the status page, refer to the AI Developer Edition Status page.
Running the sample find application
This sample requires that the Data Discovery feature is installed and running.
- Open a command prompt.
- Navigate to the directory where AI Developer Edition is cloned.
- Run the sample application using the following command.
python solutions/find-and-redact/sample-app-find.py
bash solutions/find-and-redact/sample-app-find.sh
- View the output of the files processed on the screen. The output displays a list of sensitive items in the source file.
Running the sample find and redact application
This sample requires that the Data Discovery feature is installed and running.
- Open a command prompt.
- Navigate to the directory where AI Developer Edition is cloned.
- Run the sample application using the following command.
python solutions/find-and-redact/sample-app-find-and-redact.py
bash solutions/find-and-redact/sample-app-find-and-redact.sh
- View the output of the files processed on the screen. The output displays a list of sensitive items in the source file. It also displays the location and name of the output file with the redacted output.
- View the processed output file in the output directory.
Using the protection notebook
The online notebook provides a quick way to test tokenization using just a browser.
Ensure that the required credentials are obtained and environment variables specified, using the steps from Optional - Obtaining access to the AI Developer Edition API Service.
Navigate to the online notebook, refer to Protegrity Data Protection Jupyter notebook.
Click the Play button to progress through the notebook. Specify the email address, password, and API key when prompted.
Running the sample find and protect application
This sample requires that the Data Discovery feature is installed and running.
- Ensure that the required credentials are obtained and environment variables specified, using the steps from Optional - Obtaining access to the AI Developer Edition API Service.
- Open a command prompt.
- Navigate to the directory where AI Developer Edition is cloned.
- Run the sample application using the following command.
python solutions/find-and-protect/sample-app-find-and-protect.py
bash solutions/find-and-protect/sample-app-find-and-protect.sh
View the output of the files processed on the screen. The output displays the protected data and unprotected data.
View the processed output file in the output directory. The
solutions/find-and-protect/output-protect.txtfile is generated with protected, tokenized-like, values.To obtain the original data, run the following command.
python solutions/find-and-protect/sample-app-find-and-unprotect.py
bash solutions/find-and-protect/sample-app-find-and-unprotect.sh
This reads the `solutions/find-and-protect/output-protect.txt` file and produces the `solutions/find-and-protect/output-unprotect.txt` file with original values.
Running the script for protecting data
The sample-app-protection showcases the various scenarios to protect, unprotect, and reprotect data.
Understanding Users and Roles
The users and roles are built-in for impersonate testing. Leverage any of the preconfigured users to showcase Protegrity’s Role-Based Access Controls. Using a different user will result in distinct views over sensitive data. Some users will only be able to protect data but will not be able to reverse the operation. Some users will only be able to re-identify selected attributes.
To use any of the roles, simply pass the chosen value to the payload in the user attribute during the protect or unprotect operation. If the user is not specified, the request will default to superuser.
The following roles and users have been configured and are available for use:
| Role | User | Description |
|---|---|---|
| ADMIN | admin, devops, jay.banerjee | The role can protect all data but cannot unprotect. If this role attempts to unprotect, they will only see protected values. |
| FINANCE | finance, robin.goodwill | The role can unprotect all PII and PCI data. The role cannot protect any data. If this role attempts to unprotect data without authorization they will only see null values. |
| MARKETING | marketing, merlin.ishida | The role can unprotect some PII data that is required for analytical research and campaign outreach. When attempting to unprotect data without authorization, they will only see null values. The role cannot protect any data. |
| HR | hr, paloma.torres | The role can unprotect all PII data but cannot view any PCI data. When attempting to unprotect data without authorization, they will only see null values. The role cannot protect any data. |
| OTHER | superuser | The role can perform any protect and unprotect operation. This superuser role has been made available for testing only. It is strongly advised that superuser roles should not be created. |
Additionally, it is possible to enter in any username to simulate unauthorized user behavior.
Understanding the Data Elements
Provided here is a list of supported data elements. For a mapping of the Data Element and the Entity Type, refer to Supported Sensitive Entity Types.
For more information about the data elements policy, refer to Policy Definition.
| Name | Description |
|---|---|
| name | Protect or unprotect name of a person. |
| name_de | Protect or unprotect name of a person in the German language. |
| name_fr | Protect or unprotect name of a person in the French language. |
| address | Protect or unprotect an address. |
| address_de | Protect or unprotect an address in the German language. |
| address_fr | Protect or unprotect an address in the French language. |
| city | Protect or unprotect a town or city. |
| city_de | Protect or unprotect a town or city name in the German language. |
| city_fr | Protect or unprotect a town or city name in the French language. |
| postcode | Protect or unprotect a postal code with digits and characters. |
| zipcode | Protect or unprotect a postal code with digits only. |
| phone | Protect or unprotect a phone number. |
| Protect or unprotect an email. | |
| datetime | Protect or unprotect all components of a datetime string date, month, and year. The input for the datetime data element must be in the yyyy-mm-dd [hh:mm:ss] format. |
| datetime_yc | Protect or unprotect a datetime string. Year will be in the clear. The input for the datetime data element must be in the yyyy-mm-dd [hh:mm:ss] format. |
| int | Protect or unprotect a 4-byte integer string. |
| nin | Protect or unprotect a National Insurance Number UK. |
| ssn | Protect or unprotect a Social Security Number US. |
| ccn | Protect or unprotect a Credit Card Number. |
| ccn_bin | Protect or unprotect a Credit Card Number. Leaves 8-digit BIN in the clear. |
| passport | Protect or unprotect a passport number. |
| iban | Protect or unprotect an International Banking Account Number. |
| iban_cc | Protect or unprotect an International Banking Account Number. Leaves letters in the clear. |
| string | Protect or unprotect a string. |
| number | Protect or unprotect a number. |
| text | Protect or unprotect text using encryption. |
| mask | Unprotect with any user not having permission to perform unprotect operation. The output is masked. |
| fpe_numeric | Protect or unprotect a number using a Format Preserving Encryption data element. |
| fpe_alpha | Protect or unprotect a string containing alphabets using a Format Preserving Encryption data element. |
| fpe_alphanumeric | Protect or unprotect a string containing alphabets and numbers using a Format Preserving Encryption data element. |
| fpe_latin1_alpha | Protect or unprotect a string containing basic latin and latin-1 supplement characters using a Format Preserving Encryption data element. |
| fpe_latin1_alphanumeric | Protect or unprotect a string containing numbers, basic latin and latin-1 supplement characters using a Format Preserving Encryption data element. |
| no_encryption | When applied, the No Encryption protection method lets sensitive data be stored in the clear. It is highly transparent, which means that the implementation of this method does not cause any changes in the target environment. |
| short | Protect or unprotect a 2-byte integer string. |
| long | Protect or unprotect a 8-byte integer string. |
Testing the sample file
- Ensure that the required credentials are obtained and environment variables specified, using the steps from Optional - Obtaining access to the AI Developer Edition API Service.
- Open a command prompt.
- Navigate to the directory where AI Developer Edition is cloned.
- Protect data using the following command.
python data-protection/samples/python/sample-app-protection.py --input_data "John Smith" --policy_user superuser --data_element name --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Smith" --policy_user superuser --data_element name --protect
View the protected output.
Unprotect the data obtained from the earlier step using the following command.
python data-protection/samples/python/sample-app-protection.py --input_data "<protected_data>" --policy_user superuser --data_element name --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "<protected_data>" --policy_user superuser --data_element name --unprotect
View the unprotected output.
Encrypt data using the following command.
python data-protection/samples/python/sample-app-protection.py --input_data "John Smith" --policy_user superuser --data_element text --enc
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Smith" --policy_user superuser --data_element text --enc
View the encrypted output.
Decrypt the data obtained from the earlier step using the following command.
python data-protection/samples/python/sample-app-protection.py --input_data "<encrypted_data>" --policy_user superuser --data_element text --dec
bash data-protection/samples/java/sample-app-protection.sh --input_data "<encrypted_data>" --policy_user superuser --data_element text --dec
View the decrypted output.
Use the help command for more information about using the sample file.
python data-protection/samples/python/sample-app-protection.py --help
bash data-protection/samples/java/sample-app-protection.sh --help
FPE, Masking, and No Encryption Samples
- Open a command prompt.
- Navigate to the directory where AI Developer Edition is cloned.
- Run the Format Preserving Encryption (FPE) using the following command.
python data-protection/samples/python/sample-app-protection.py --input_data "ELatin1_S+NSABC¹º»¼½¾¿ÄÅÆÇÈAlice1234567Bob" --policy_user superuser --data_element fpe_latin1_alphanumeric --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "ELatin1_S+NSABC¹º»¼½¾¿ÄÅÆÇÈAlice1234567Bob" --policy_user superuser --data_element fpe_latin1_alphanumeric --protect
View the protected output.
Unprotect the data obtained from the earlier step using the following command.
python data-protection/samples/python/sample-app-protection.py --input_data "VðÈuXñ5_À+Áîg1ÿ¹º»¼½¾¿12ÔP1ëÕÖlgxÏHóFÚ6O3W" --policy_user superuser --data_element fpe_latin1_alphanumeric --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "VðÈuXñ5_À+Áîg1ÿ¹º»¼½¾¿12ÔP1ëÕÖlgxÏHóFÚ6O3W" --policy_user superuser --data_element fpe_latin1_alphanumeric --unprotect
- View the unprotected output.
- Use the no_encryption data element using the following command.
python data-protection/samples/python/sample-app-protection.py --input_data "John Smith" --policy_user superuser --data_element no_encryption --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Smith" --policy_user superuser --data_element no_encryption --protect
View the output. The output data will be in clear.
Unprotect the data using masking data element.
python data-protection/samples/python/sample-app-protection.py --input_data "John Smith" --policy_user hr --data_element mask --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Smith" --policy_user hr --data_element mask --unprotect
Additional use cases
This section demonstrates the expected behavior of various user roles when running the sample-app-protection.py. Each section describes the permissions and restrictions for a role, followed by example commands and their outputs.
ADMIN
Users: admin, devops, jay.banerjee
This role can protect all data but cannot unprotect. When attempting to unprotect, protected values are displayed.
python data-protection/samples/python/sample-app-protection.py --input_data "Protegrity$" --policy_user devops --data_element name --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "Protegrity$" --policy_user devops --data_element name --protect
python data-protection/samples/python/sample-app-protection.py --input_data "2839874358655598" --policy_user admin --data_element ccn --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "2839874358655598" --policy_user admin --data_element ccn --protect
python data-protection/samples/python/sample-app-protection.py --input_data "CxWHeztVNp$" --policy_user jay.banerjee --data_element name --protect --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "CxWHeztVNp$" --policy_user jay.banerjee --data_element name --protect --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "6211214171366290" --policy_user admin --data_element ccn --protect --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "6211214171366290" --policy_user admin --data_element ccn --protect --unprotect
FINANCE
Users: finance, robin.goodwill
This role can unprotect all PII and PCI data. The role cannot protect any data. When attempting to unprotect data without authorization, the value Null is displayed.
python data-protection/samples/python/sample-app-protection.py --input_data "xzrT sqdVc" --policy_user finance --data_element name --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "xzrT sqdVc" --policy_user finance --data_element name --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "4321567898765432" --policy_user finance --data_element ccn --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "4321567898765432" --policy_user finance --data_element ccn --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "John Smith" --policy_user finance --data_element name --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Smith" --policy_user finance --data_element name --protect
python data-protection/samples/python/sample-app-protection.py --input_data "2839874358655598" --policy_user robin.goodwill --data_element ccn --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "2839874358655598" --policy_user robin.goodwill --data_element ccn --protect
python data-protection/samples/python/sample-app-protection.py --input_data "1998/10/11" --policy_user finance --data_element datetime --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "1998/10/11" --policy_user finance --data_element datetime --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "1998/10/11" --policy_user robin.goodwill --data_element datetime --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "1998/10/11" --policy_user robin.goodwill --data_element datetime --unprotect
MARKETING
Users: marketing, merlin.ishida
This role can unprotect some PII data that is required for analytical research and campaign outreach. The role cannot protect any data. When attempting to unprotect data without authorization, the value Null is displayed.
python data-protection/samples/python/sample-app-protection.py --input_data "DnZQHKcpVJ, J.G." --policy_user marketing --data_element city --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "DnZQHKcpVJ, J.G." --policy_user marketing --data_element city --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "4321567898765432" --policy_user merlin.ishida --data_element ccn --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "4321567898765432" --policy_user merlin.ishida --data_element ccn --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "Washington, D.C." --policy_user marketing --data_element city --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "Washington, D.C." --policy_user marketing --data_element city --protect
python data-protection/samples/python/sample-app-protection.py --input_data "2839874358655598" --policy_user merlin.ishida --data_element ccn --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "2839874358655598" --policy_user merlin.ishida --data_element ccn --protect
HR
Users: hr, paloma.torres
This role can unprotect all PII data but cannot view any PCI data. The role cannot protect any data. When attempting to unprotect data without authorization, the value Null is displayed.
python data-protection/samples/python/sample-app-protection.py --input_data "2839874358655598" --policy_user paloma.torres --data_element ccn --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "2839874358655598" --policy_user paloma.torres --data_element ccn --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "CIF123654987" --policy_user hr --data_element passport --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "CIF123654987" --policy_user hr --data_element passport --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "John Doe" --policy_user hr --data_element name --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Doe" --policy_user hr --data_element name --protect
python data-protection/samples/python/sample-app-protection.py --input_data "John Doe" --policy_user paloma.torres --data_element name --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Doe" --policy_user paloma.torres --data_element name --protect
python data-protection/samples/python/sample-app-protection.py --input_data "4321567898765432" --policy_user paloma.torres --data_element ccn --protect
bash data-protection/samples/java/sample-app-protection.sh --input_data "4321567898765432" --policy_user paloma.torres --data_element ccn --protect
OTHER
User: superuser
This role can perform any protect and unprotect operation. The role is only made available for testing. It is strongly advised against creating superuser roles in an environment.
python data-protection/samples/python/sample-app-protection.py --input_data "John Smith" --policy_user superuser --data_element name --protect --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "John Smith" --policy_user superuser --data_element name --protect --unprotect
python data-protection/samples/python/sample-app-protection.py --input_data "2839874358655598" --policy_user superuser --data_element ccn --protect --unprotect
bash data-protection/samples/java/sample-app-protection.sh --input_data "2839874358655598" --policy_user superuser --data_element ccn --protect --unprotect
Feedback
Was this page helpful?