Synthetic Data Generation on

Synthetic Data Architecture

Tue, 20 Feb 2024 00:00:00 +0000

Protegrity’s Synthetic Data solution is a Synthetic Data generator which generates artificial data that is realistic, statistically accurate, and privacy-safe. This data unlocks the full potential of AI and analytics. By creating entirely new data that mirrors the patterns of your original datasets but contains no sensitive information, you can train and test AI models without risk. You can also scale these models without exposure or compliance violations.

An overview of the communication is shown in the following figure.

Prerequisites for Synthetic Data

Tue, 20 Feb 2024 00:00:00 +0000

Ensure that the following prerequisites are met before running these examples for Synthetic Data:

Docker CLI, Docker Compose, and Python are installed. For more information, refer to AI Developer Edition, Pre-requisites Guide.
For shell samples: Bash version greater than or equal to 5.1.8 and curl version greater than or equal to 7.76.1.
For notebook samples: JupyterLab version greater than or equal to 4.5.6.

Setting up Synthetic Data

Tue, 20 Feb 2024 00:00:00 +0000

Use the containers to set up the Synthetic Data feature for data generation.

Open a command prompt.
Navigate to the cloned repository location for protegrity-ai-developer-edition.
Run the following command to download and start the containers. The dependent containers are large in size. Based on the network connection, the containers might take time to download and deploy.

Running the Synthetic Data samples

Tue, 20 Feb 2024 00:00:00 +0000

The example scripts under the synthetic-data/ folder demonstrate the usage of Synthetic Data APIs. For more information about the Synthetic Data APIs, refer to the section Synthetic Data APIs.

Note: A dedicated synthetic-data/docker-compose.yml is provided to start the Synthetic Data services.

Open a command prompt.
Navigate to the directory where AI Developer Edition is cloned.
Run the following command to start Jupyter Lab.
```
jupyter lab
```
Copy the URL displayed and navigate to the site from a web browser. Ensure that localhost is replaced with the IP address of the system where the AI Developer Edition is set up.

Using the Synthetic Data APIs

Tue, 20 Feb 2024 00:00:00 +0000

base

Base HTTP client for SDK communication

Provides low-level HTTP communication utilities shared across all SDK clients. Handles request construction, response parsing, error handling, and retries.

dataframe_to_base64

def dataframe_to_base64(df: pd.DataFrame) -> str

Convert DataFrame to base64-encoded CSV string for inline data transfer.

Arguments:

df - DataFrame to encode.

Returns:

str - Base64-encoded CSV string.

Examples:

```
df = pd.DataFrame({"a": [1, 2], "b": [3, 4]})
encoded = dataframe_to_base64(df)
print(encoded[:20])
YSxiCjEsMwoyLDQK
```

Uninstalling Synthetic Data

Tue, 20 Feb 2024 00:00:00 +0000

Open a command prompt.
Navigate to the cloned repository location.
Navigate to the synthetic-data directory.
```
cd synthetic-data
```
Run the following command to remove the containers and images.
```
docker compose down --rmi all
```