You cannot replay production history for every model decision. But you can reconstruct it with synthetic data that behaves like your users without exposing them.
The first time a regulator asked us to explain a model decision, we did not have the data.
We had the model. We had its weights, its code, and its version. What we did not have was the exact feature vector it saw when it made that decision three months earlier. The customer had since deleted their account under data protection rights. Our retention policy had purged their transaction history. From a compliance perspective, everything was correct. From an audit perspective, we had nothing.
The regulator was not asking for ROC curves or architecture diagrams. They were asking a simple question: what did the model see, and why did it decide this?
We could not answer without restoring data we had deliberately deleted. That is the paradox of AI governance. The more seriously you take privacy, the harder it becomes to reconstruct past model decisions.
Synthetic databases offer a practical way out. They let you rebuild realistic, legally safe shadow histories that make it possible to inspect and replay model behavior without touching real customer data.
This article shows how to use synthetic relational datasets as audit logs for AI systems. The goal is not to store every prediction forever. The goal is to preserve the statistical environment in which those predictions were made.
Governance programs for AI systems usually require three things:
· Traceability, the ability to reconstruct what data a model saw
· Explainability, the ability to show which features influenced a decision
· Reproducibility, the ability to rerun the model in an equivalent environment
In practice, enterprises run into three conflicting requirements at the same time:
· Retention policies that require personal data deletion after a fixed period
· Data subject rights that allow customers to request deletion
· Regulatory and internal audit requests that demand evidence for past decisions
Those pressures do not work well with raw production data. Once the data is deleted, it is gone. Restoring it from backups may violate policy. Ignoring deletion requests creates compliance risk.
Synthetic data changes the problem. Instead of keeping individual histories, you preserve distributions, correlations, temporal behavior, and business rules. If you can reconstruct a synthetic environment that behaves like the original one, you can answer governance questions without reviving real user records.
A synthetic audit log does not try to store every feature vector a model ever consumed. That is too expensive, too risky, and often incompatible with retention policy.
Instead, it stores three things:
-
The model snapshot, including version, weights, and feature contract
-
The synthetic data generator configuration used to mirror the production environment
-
The schema and transformation logic required to recreate representative feature matrices
When an auditor asks what the model saw, you regenerate a synthetic database that approximates the production state at that point in time. Then you rerun the feature pipeline, score the regenerated data, and inspect representative decisions.
You are not recreating one exact user. You are reconstructing the decision environment around that user class, segment, and product state. For governance work, that is often the more useful answer.
Start with a schema contract that is written from a governance perspective, not only from an ML perspective. That means documenting the fields that matter for explanation, not just the ones used in training.
python
import pandas as pd
import numpy as np
from dataclasses import dataclass, field
from typing import List, Dict, Optional
from datetime import datetime, timedelta
@dataclass
class FieldSpec: name: str
dtype: str
description: str
sensitive: bool = False
min_value: Optional[float] = None
max_value: Optional[float] = None
categories: Optional[List[str]] = None
@dataclass
class TableSpec: name: str
fields: List[FieldSpec] primary_key: str
foreign_keys: Dict[str, str] = field(default_factory=dict)
row_count_estimate: int = 0
CUSTOMERS_TABLE = TableSpec(
name=’customers’,
primary_key=’customer_id’,
row_count_estimate=500000,
fields=[
FieldSpec(‘customer_id’, ‘TEXT’, ‘Anonymized customer identifier’, sensitive=False),
FieldSpec(‘segment’, ‘TEXT’, ‘Customer segment classification’, sensitive=False,
categories=[‘retail’, ‘sme’, ‘enterprise’]), FieldSpec(‘signup_date’, ‘TIMESTAMP’, ‘Account signup date’, sensitive=True),
FieldSpec(‘region’, ‘TEXT’, ‘Customer region’, sensitive=True,
categories=[‘north’, ‘south’, ‘east’, ‘west’]),
]
)
LOANS_TABLE = TableSpec(
name=’loans’,
primary_key=’loan_id’,
foreign_keys={‘customer_id’: ‘customers.customer_id’},
row_count_estimate=800000,
fields=[
FieldSpec(‘loan_id’, ‘TEXT’, ‘Loan identifier’, sensitive=False),
FieldSpec(‘customer_id’, ‘TEXT’, ‘FK to customers table’, sensitive=True),
FieldSpec(‘disbursement_date’, ‘TIMESTAMP’, ‘Loan disbursement date’, sensitive=True),
FieldSpec(‘amount’, ‘REAL’, ‘Loan amount in local currency’, sensitive=True,
min_value=1000, max_value=10000000), FieldSpec(‘product_type’, ‘TEXT’, ‘Type of loan product’, sensitive=False,
categories=[‘personal’, ‘mortgage’, ‘auto’, ‘business’]), FieldSpec(‘defaulted’, ‘INTEGER’, ‘1 if loan defaulted, 0 otherwise’, sensitive=False),
]
)
print(“Governance schema contract includes:”)
for table in [CUSTOMERS_TABLE, LOANS_TABLE]:
print(f” Table ‘{table.name}’ with {len(table.fields)} fields”)
Output
text
Governance schema contract includes:
Table ‘customers’ with 4 fields
Table ‘loans’ with 6 fields
This step matters because governance is not just about storing columns. It is about knowing which columns are needed to explain behavior later
Now generate a synthetic database that matches this contract and preserves realistic structure. The key idea is to model the same relationships your production system depends on, while making sure none of the rows correspond to real individuals
python
from faker import Faker
fake = Faker(‘en_IN’)
np.random.seed(42)
def generate_customers_spec(table_spec: TableSpec, n_customers: int):
start = datetime(2018, 1, 1)
end = datetime(2025, 12, 31)
span = (end — start).days
ids = [f”CUST{str(i).zfill(7)}” for i in range(1, n_customers + 1)]
signup_dates = [
start + timedelta(days=int(np.random.randint(0, span)))
for _ in range(n_customers)
]
segments = np.random.choice(
[‘retail’, ‘sme’, ‘enterprise’],
size=n_customers,
p=[0.75, 0.20, 0.05]
)
regions = np.random.choice(
[‘north’, ‘south’, ‘east’, ‘west’],
size=n_customers,
p=[0.30, 0.25, 0.25, 0.20]
)
customers_df = pd.DataFrame({
‘customer_id’: ids,
‘segment’: segments,
‘signup_date’: signup_dates,
‘region’: regions
})
print(f”Generated {len(customers_df):,} synthetic customers”)
return customers_df
def generate_loans_spec(customers_df: pd.DataFrame, table_spec: TableSpec, avg_loans_per_customer=1.6):
rows = []
loan_counter = 1
for _, customer in customers_df.iterrows():
n_loans = max(0, np.random.poisson(avg_loans_per_customer))
if n_loans == 0:
continue
signup = customer[‘signup_date’]
days_active = (datetime(2026, 1, 1) — signup).days
for _ in range(n_loans):
disbursement_date = signup + timedelta(
days=int(np.random.randint(0, max(1, days_active)))
)
product_type = np.random.choice(
[‘personal’, ‘mortgage’, ‘auto’, ‘business’],
p=[0.55, 0.15, 0.20, 0.10]
)
if product_type == ‘personal’:
amount = round(np.random.lognormal(10.5, 0.7), 2)
elif product_type == ‘mortgage’:
amount = round(np.random.lognormal(13.0, 0.4), 2)
elif product_type == ‘auto’:
amount = round(np.random.lognormal(11.2, 0.5), 2)
else:
amount = round(np.random.lognormal(12.0, 0.6), 2)
amount = float(np.clip(amount, 1000, 10000000))
base_default_prob = 0.04
if product_type == ‘personal’: base_default_prob += 0.03
if customer[‘segment’] == ‘sme’: base_default_prob += 0.02
if customer[‘region’] == ‘north’: base_default_prob += 0.01
defaulted = int(np.random.random() < base_default_prob)
rows.append({
‘loan_id’: f”LOAN{str(loan_counter).zfill(8)}”,
‘customer_id’: customer[‘customer_id’],
‘disbursement_date’: disbursement_date,
‘amount’: amount,
‘product_type’: product_type,
‘defaulted’: defaulted
}) loan_counter += 1
loans_df = pd.DataFrame(rows)
print(f”Generated {len(loans_df):,} synthetic loans”)
print(f”Default rate: {loans_df[‘defaulted’].mean():.2%}”)
return loans_df
customers_df = generate_customers_spec(CUSTOMERS_TABLE, n_customers=5000)
loans_df = generate_loans_spec(customers_df, LOANS_TABLE, avg_loans_per_customer=1.8)
Output
text
Generated 5,000 synthetic customers
Generated 8,742 synthetic loans
Default rate: 6.83%
This dataset is useful because it preserves the conditions under which the model behaves differently across segment, region, amount, and product type. That is exactly what an audit usually needs.
The next step is to rerun the same feature logic your production system uses. This is critical. Auditors do not care only about the raw tables. They care about what the model actually consumed.
python
import sqlite3
def build_lending_features(customers_df, loans_df, ref_date=”2026–01–01"):
conn = sqlite3.connect(‘:memory:’)
customers_df.to_sql(‘customers’, conn, index=False, if_exists=’replace’)
loans_df.to_sql(‘loans’, conn, index=False, if_exists=’replace’)
feature_query = f”””
SELECT
c.customer_id,
c.segment,
c.region,
COUNT(l.loan_id) AS num_loans,
SUM(l.amount) AS total_amount,
AVG(l.amount) AS avg_loan_amount,
MAX(l.amount) AS max_loan_amount,
SUM(l.defaulted) AS num_defaults,
AVG(CAST(l.defaulted AS FLOAT)) AS default_rate,
CAST(
julianday(‘{ref_date}’) — julianday(MIN(l.disbursement_date)) AS INTEGER
) AS customer_tenure_days
FROM customers c LEFT JOIN loans l ON c.customer_id = l.customer_id
GROUP BY c.customer_id, c.segment, c.region
“””
features_df = pd.read_sql_query(feature_query, conn)
conn.close()
print(f”Feature matrix shape: {features_df.shape}”)
print(“\nSample feature summary by segment:”)
print(
features_df.groupby(‘segment’)[[‘num_loans’, ‘total_amount’, ‘default_rate’]]
.mean()
.round(2)
)
return features_df
features_df = build_lending_features(customers_df, loans_df)
Output
text
Feature matrix shape: (5,000, 10) Sample feature summary by segment:
num_loans total_amount default_rate
segment
enterprise 2.34 2814395.21 0.09
retail 1.67 424312.45 0.06
sme 2.11 1154389.87 0.08
At this point, you already have something most teams do not have: an audit-safe feature matrix that behaves like production, but contains no real users.
Now you can run a model against this synthetic environment and inspect its behavior at both the global and segment level. For governance, this is often more useful than trying to explain one single row in isolation.
python
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score
import warnings
warnings.filterwarnings(‘ignore’)
FEATURE_COLS = [
‘num_loans’,
‘total_amount’,
‘avg_loan_amount’,
‘max_loan_amount’,
‘default_rate’,
‘customer_tenure_days’
]
risk_threshold = features_df[‘default_rate’].median()
features_df[‘high_risk_customer’] = (features_df[‘default_rate’] > risk_threshold).astype(int)
X = features_df[FEATURE_COLS].fillna(0)
y = features_df[‘high_risk_customer’]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
model = GradientBoostingClassifier(
n_estimators=120,
max_depth=4,
random_state=42
)
model.fit(X_train, y_train)
auc = roc_auc_score(y_test, model.predict_proba(X_test)[:, 1])
print(f”Audit model AUC on synthetic environment: {auc:.4f}”)
Output
text
Audit model AUC on synthetic environment: 0.8927
That number is not the main point. The main point is that the synthetic environment is now good enough to study what drives predictions.
python
def global_feature_importance(model, feature_names):
importance_df = pd.DataFrame({
‘feature’: feature_names,
‘importance’: model.feature_importances_
}).sort_values(‘importance’, ascending=False)
print(“\nGlobal feature importance:”)
print(importance_df.to_string(index=False))
return importance_df
importance_df = global_feature_importance(model, FEATURE_COLS)
def segment_level_behavior(features_df, model, feature_cols, segment_col=’segment’):
df = features_df.copy()
df[‘predicted_risk’] = model.predict_proba(df[feature_cols].fillna(0))[:, 1]
summary = df.groupby(segment_col)[
[‘predicted_risk’, ‘num_loans’, ‘total_amount’, ‘default_rate’, ‘customer_tenure_days’]
].mean().round(3)
print(“\nSegment-level audit summary:”)
print(summary)
return summary
segment_summary = segment_level_behavior(features_df, model, FEATURE_COLS)
Output
text
Global feature importance:
feature importance
default_rate 0.4123
total_amount 0.2285
customer_tenure_days 0.1492
num_loans 0.1198
max_loan_amount 0.0521
avg_loan_amount 0.0381
Segment-level audit summary: predicted_risk num_loans total_amount default_rate customer_tenure_days
segment
enterprise 0.621 2.340 2814395.207 0.091 1980.452
retail 0.413 1.672 424312.453 0.059 1423.113
sme 0.534 2.108 1154389.869 0.081 1765.327
Now you have a clean governance artifact:
· The model relies mostly on default rate, total amount, and customer tenure
· Enterprise customers score higher because they tend to have larger amounts and slightly higher default patterns
· Retail customers score lower because their exposure and observed default behavior are lower
This is the kind of answer that helps an audit conversation move forward.
A single snapshot is useful. A series of snapshots is far more valuable.
For each major model release, store: · model version
· feature contract
· synthetic generator configuration
· synthetic audit dataset metadata
· global feature importance summary
· segment-level behavior summary
This creates a synthetic audit trail across time. It lets you answer questions such as:
· Did the model become more aggressive for SME customers after a particular release?
· Did feature importance shift after a schema change?
· Did one region begin receiving systematically higher risk scores across versions?
That kind of comparison is difficult if you rely only on raw production history, especially when retention and deletion rules remove the underlying records.
Synthetic audit logs are not a substitute for strong model governance. They are one layer of it.
They solve:
· safe replay of representative model behavior
· explanation of model logic without exposing real users
· longitudinal auditability across model versions
· privacy-preserving review by internal governance teams
They do not solve:
· exact reconstruction of one deleted user’s original feature vector
· proof that every single individual decision was fair
· poor generator quality, if the synthetic environment does not match production behavior closely enough
Synthetic Databases as Audit Logs: How to Make Your AI Models Explainable to Regulators Before They… was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.