Synthetic Data Agent: Agent Workflow & Artifacts

The Synthetic Data Agent is the core component within the Datapunkt ecosystem responsible for generating high-fidelity, privacy-preserving synthetic datasets. By analyzing source schemas, learning statistical distributions, and leveraging advanced generative machine learning models, the agent enables teams to provision massive volumes of production-grade test data on demand.

This guide walks you through the step-by-step operational workflow of using the Synthetic Data Agent, defines the core output artifacts, and details the benefits of incorporating it into your pipelines.

Supported Capabilities

The Synthetic Data Agent natively integrates advanced data security and synthesis techniques to ensure your target datasets are safe, realistic, and fully compliant:

• PII DETECTION: Automatically scans incoming schemas, column names, and sample metadata to identify columns containing Personally Identifiable Information (such as names, addresses, credit cards, SSNs, and phone numbers).
• PRIVACY PRESERVING: Implements specialized noise injection, hashing, and tokenization logic to ensure that no real-world identity can be reconstructed from the generated records.
• GDPR/CCPA: Generates compliant datasets that fall completely outside the scope of regulatory restrictions like GDPR and CCPA, enabling risk-free training and testing.
• ML SYNTHESIS: Employs deep generative architectures, including Generative Adversarial Networks (GANs) and variational autoencoders, to capture non-linear relationships and complex correlation matrices in multidimensional datasets.
• DIFFERENTIAL PRIVACY: Adds mathematical noise guarantees (Epsilon-Delta differential privacy) to make certain that no single individual's production record can be reverse-engineered or isolated.
• SCHEMA FIDELITY: Preserves the exact structural constraints, database column types, foreign-primary key relations, nullability requirements, and field length limitations of your source database.
• DATA INTEGRITY: Enforces relational consistency across multiple related tables, preventing key collisions, orphaned rows, and referential constraint violations.
• DATA PRIVACY & FIDELITY: Balances absolute privacy security against statistical utility, ensuring that synthetic datasets behave identically to production data during analysis and modeling.

Operational Workflow

The Synthetic Data Agent utilizes a dual-pane workspace designed to simplify configuration while protecting sensitive source metadata.

Step 1: Subscribing to the Agent

To start using the agent, you must obtain a subscription via the platform marketplace:

1. Log in to your Agentpunkt platform account.
2. Search for the Synthetic Data Agent in the platform marketplace catalog.
3. Choose the appropriate subscription plan (e.g., Weekly, Bi-weekly, or Monthly).
4. Access the agent through your Hired Agents Page.

Step 2: Workspace Session Initialization

Open a new workspace session to establish your sandboxed environment:

1. Click "Start Session" next to the Synthetic Data Agent on the Hired Agents Page.
2. The user interface splits into two primary panes:
   • Left Pane (Interactive Chat & Logs): Here you communicate with the agent in natural language, define generation requirements, and monitor step-by-step progress logs.
   • Right Pane (Workspace & Code Preview): This pane displays schema layouts, generated configuration YAMLs/JSONs, data preview tables, and ready-to-run injection scripts.

Step 3: Source Schema Analysis

Introduce the agent to your data structures without sharing sensitive production records:

1. Copy and paste your database DDL (e.g., CREATE TABLE scripts) or a JSON-based schema representation into the Left Pane.
2. You can also provide a safe, scrubbed CSV containing 5 to 10 rows of dummy headers and data format examples.
3. The agent triggers its PII DETECTION scans to flag sensitive columns and presents a visual schema mapping in the Right Pane.

Example prompt in the Left Pane:

"Please analyze the following schema for our orders database. Identify any PII columns and verify foreign key relationships: [Insert DDL SQL statement here]"

Step 4: Configuring Generation Constraints

Specify your data requirements, statistical expectations, and volume targets:

1. Instruct the agent on how many rows you need (e.g., "Generate 50,000 orders and 10,000 customers").
2. Define distribution targets (e.g., "Ensure customer age is normally distributed between 18 and 75, and 15% of transactions fail or use promo codes").
3. Set your privacy parameters, specifying the level of DIFFERENTIAL PRIVACY noise (Epsilon value) or requesting strict GDPR/CCPA anonymization.
4. The agent compiles these rules into a synthesis configuration file shown in the Right Pane.

Step 5: High-Fidelity Data Generation

Trigger the core generation routine:

1. Direct the agent to execute the synthesis process.
2. The agent executes ML SYNTHESIS models locally in-memory, mapping statistical correlations and generating rows that match the schema's shape.
3. Review the execution logs in the Left Pane. When complete, a preview of the generated dataset appears in the Right Pane.

Step 6: Target Export or Injection

Export your newly synthesized dataset or schedule direct database injection:

1. Download the generated data in standard formats like CSV, Parquet, or JSON.
2. Alternatively, copy the generated Python injection script or database insert statements to load the data directly into your staging warehouse or target tables.

Benefits: What Makes It Good?

• Accelerated QA Cycles: Removes dependencies on manual scrubbing, database sanitization scripts, or lengthy approvals, letting developers spin up test environments in seconds.
• Flawless Schema Matching: Strict enforcement of schema rules prevents load failures, key collision, and primary-foreign key mismatch.
• Uncompromising Privacy Safeguards: Native PII detection combined with differential noise guarantees that downstream developers and test environments remain completely compliance-secure.
• High-Fidelity Machine Learning Utility: Generates complex mathematical correlations and edge cases, ensuring that analytical models trained on synthetic data perform reliably when deployed against real-world systems.