{"slug": "synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo", "title": "Synthetic Data Generation for Financial AI Research with NVIDIA NeMo", "summary": "NVIDIA released a pipeline combining NeMo Data Designer, NeMo Curator, and Nemotron models to generate 500,000 unique synthetic financial news headlines across 13 categories for AI research. The iterative approach addresses data imbalance by generating, deduplicating, and adjusting category weights over 82 iterations, producing a dataset for trading, risk modeling, and surveillance applications.", "body_md": "Fine-tuning LLMs for financial natural language processing (NLP) is constrained by limited, imbalanced data. Real-world financial news overrepresents earnings and stock movements, while rarer events such as credit-rating changes, product approvals, and labor issues are harder to capture at scale. Synthetic generation can help fill those gaps for trading research, risk modeling, and surveillance, but diversity requires more than a single large generation run.\n\nThis post walks through an iterative pipeline for generating 500,000 unique financial news headlines across 12 topics plus an “Other” category. The workflow combines [NVIDIA NeMo Data Designer](https://nvidia-nemo.github.io/DataDesigner/latest/) for structured generation, [NVIDIA NeMo Curator](https://github.com/NVIDIA/NeMo-Curator) for semantic deduplication, and NVIDIA Nemotron models for high-throughput headline synthesis. It also adds a farthest-from-centroid few-shot strategy to steer each round toward novel outputs.\n\nIn a naive run of 50,000 headlines, 65% were removed as near-duplicates. Instead of scaling a single batch, the pipeline iterates: generate, filter, deduplicate globally, select distinctive few-shot examples, correct category weights, and repeat until the corpus reaches the target size. Our full run produced 502,536 unique headlines across 13 categories in 82 iterations, using approximately 6 days of compute on a single 8-way NVIDIA B200 node with checkpointing, crash recovery, and SLURM job chaining.\n\nBy the end, you’ll understand how to reproduce the generation-deduplication loop, adapt it for your own financial research workflows, or start directly from the open-sourced dataset.\n\n## Reproducibility\n\n### Versions\n\nComponent | Version |\n|---|---|\n| NeMo Curator | 1.0.0rc0.dev0 |\n| NeMo Data Designer | 0.1.5 |\n| vLLM | 0.12.0 |\n| Model |\n|\n\n*Table 1. Software and model versions used in this guide*\n\n**Hardware:** Single 8-way NVIDIA B200 node. GPUs 0–3 dedicated to vLLM inference (4-way tensor parallelism, 448 concurrent requests); GPUs 4–7 ran NeMo Curator semantic deduplication with Ray.\n\n**Pipeline parameters:** 35K headlines per batch (50K in early iterations), 90% cosine-similarity deduplication threshold, 500 K-means clusters, 3 few-shot examples per category, 80% cross-iteration similarity cutoff for example selection.\n\n**Checkpointing:** Each iteration persisted corpus state, category weights, few-shot examples, metrics, and used-example embeddings for crash recovery and SLURM job chaining.\n\n## Pipeline overview\n\nFinancial headline generation fails as a single-pass problem because real-world feeds are imbalanced: earnings reports and stock movements dominate, while credit-rating changes, product approvals, and other rarer events are sparse. In a 50K-headline baseline, semantic deduplication retained only 17,348 unique headlines, showing that larger batches add many near-duplicates rather than proportionally more coverage.\n\nThe iterative pipeline addresses this with a closed loop. Each iteration generates a category-weighted batch, filters malformed outputs, deduplicates against the full accumulated corpus, selects diverse few-shot examples, and adjusts category weights before the next iteration.\n\nFigure 1, below, illustrates the overall architecture.\n\nThe key design decision is **global deduplication**. Each batch is compared with all previously retained headlines, not just with itself, preventing cross-batch duplicates and preserving semantic uniqueness across the 500K corpus.\n\n## Step 1. Generate headlines with NeMo Data Designer\n\nWe serve Nemotron 3 Nano (30B parameters, 3B active) via vLLM with 4-way tensor parallelism and 448 concurrent requests. The model’s Mixture-of-Experts (MoE) architecture balances throughput and quality, using 3B active parameters per forward pass within a 30B-parameter model.\n\nNeMo Data Designer orchestrates generation through a declarative configuration: a weighted category sampler across 12 topics plus “Other,” and an LLM column that generates headlines conditioned on the sampled category and current few-shot examples.\n\nCategory weights are updated after each iteration through distribution correction (Step 5).\n\n**Data Designer configuration:**\n\n```\nconfig_builder = DataDesignerConfigBuilder()\n\n# Model config for local vLLM-served Nemotron\nlocal_model = ModelConfig(\n    alias=\"local-nemotron\",\n    model=\"nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8\",\n    provider=\"local-vllm\",\n    inference_parameters=InferenceParameters(\n        max_parallel_requests=448,\n        temperature=0.95,\n        top_p=0.95,\n    ),\n)\nconfig_builder.add_model_config(local_model)\n\n# Category sampler with DYNAMIC weights (updated each iteration)\nconfig_builder.add_column(\n    SamplerColumnConfig(\n        name=\"category\",\n        sampler_type=SamplerType.CATEGORY,\n        params=CategorySamplerParams(\n            values=CATEGORIES,        # 12 topics + \"Other\"\n            weights=current_weights,   # Adjusted after each iteration\n        ),\n    )\n)\n\n# LLM column for headline generation with few-shot examples\nconfig_builder.add_column(\n    LLMTextColumnConfig(\n        name=\"headline\",\n        model_alias=\"local-nemotron\",\n        system_prompt=\"You are a financial news headline writer.\",\n        prompt=f\"\"\"Generate a realistic financial news headline \\\nfor the category: {{{{ category }}}}\n\n{format_examples_for_prompt(examples_by_category)}\n\nGenerate a single headline for the \"{{{{ category }}}}\" category.\nHeadline:\"\"\",\n    )\n)\n```\n\nWe use high-diversity sampling (temperature=0.95, frequency penalty 0.3, presence penalty 0.2) and rely on downstream filtering and semantic deduplication to remove malformed or redundant outputs.\n\n## Prompt structure\n\nThe prompt includes a role-setting system prompt, few-shot examples grouped by category, and an instruction for the sampled category. Data Designer injects the category at runtime through Jinja templating, and the few-shot examples are refreshed each iteration (Step 4).\n\n**Prompt structure (simplified):**\n\n```\nSYSTEM:\n  You are a financial news headline writer. Generate realistic,\n  professional financial news headlines. Output ONLY the headline\n  text, nothing else.\n\nUSER:\n  Generate a realistic financial news headline for the category:\n  {{ category }}\n\n  Examples by category:\n  - Earnings: \"Google Parent Alphabet Posts $1.84 EPS, $63.5B Revenue\"\n  - Earnings: \"Myriad Genetics Beats Q4 Estimates with 15% Revenue Surge\"\n  - Earnings: \"Codec Solutions Posts Q3 Adjusted EPS $0.31, Surpasses ...\"\n  - Price Targets: \"J.P. Morgan Cuts Target for Thermo Fisher, Downgrades ...\"\n  - Price Targets: \"Mitsubishi UFJ Secures Buy Rating on Mitsui & Co. ...\"\n  - Price Targets: \"Credence Resources Upgrades Target to $12.50 ...\"\n  - Stock price movement: \"Equities Rise Broadly, Dow Gains 180 Points ...\"\n  - ...\n  (3 examples x 12 topics + Other = up to 39 few-shot examples)\n\n  Generate a single headline for the \"{{ category }}\" category.\n  The headline should be 10-25 words long, realistic, and match\n  the style of professional financial news.\n\n  Headline:\n```\n\n## Step 2. Quality filtering\n\nBefore semantic deduplication, a lightweight rule-based pass removes malformed outputs such as truncated, runaway, or garbled headlines.\n\nFilter | Threshold | Purpose |\n|---|---|---|\n| Minimum word count | >= 5 words | Remove truncated or fragment outputs |\n| Minimum char count | >= 25 characters | Remove very short outputs |\n| Maximum word count | <= 100 words | Remove runaway generation |\n| Maximum char count | <= 1,000 characters | Catch edge cases |\n| Bracket count | < 5 brackets | Remove formatting artifacts |\n| Special character ratio | <= 25% | Remove garbled outputs |\n\n*Table 2. Quality filters applied to generated headlines, with thresholds and rationale for each*\n\nIn practice, these filters removed fewer than 1% of headlines per batch, so most quality control comes from semantic deduplication in the next step.\n\n## Step 3. Semantic deduplication with NeMo Curator\n\nThis is the core quality mechanism. After each batch, the pipeline combines newly generated headlines with the accumulated corpus and runs NeMo Curator’s `TextSemanticDeduplicationWorkflow`\n\n. Headlines are embedded with `all-MiniLM-L6-v2`\n\n, clustered with K-means, and compared within clusters using pairwise cosine similarity. Any headline above the 90% similarity threshold is removed.\n\n**NeMo Curator semantic deduplication configuration:**\n\n```\nworkflow = TextSemanticDeduplicationWorkflow(\n    input_path=input_path,\n    output_path=output_path,\n    cache_path=cache_path,\n    perform_removal=True,\n    # Embedding parameters\n    text_field=\"headline\",\n    model_identifier=\"sentence-transformers/all-MiniLM-L6-v2\",\n    embedding_max_seq_length=512,\n    embedding_model_inference_batch_size=1024,\n    # Deduplication parameters\n    n_clusters=500,     # Higher = smaller clusters = faster pairwise\n    eps=0.10,           # eps = 1 - similarity_threshold (0.90)\n    id_field=\"id\", \n    # Among duplicates, keep the more representative copy\n    ranking_strategy=RankingStrategy(   \n        metadata_cols=[\"cosine_dist_to_cent\"],\n        ascending=True,  #Keep the most representative (closest to centroid)\n    ),\n    pairwise_batch_size=4096,\n    input_filetype=\"parquet\",\n    output_filetype=\"parquet\",\n)\nworkflow.run()\n```\n\n## Why 500 clusters?\n\nPairwise similarity is expensive because comparisons scale quadratically within each cluster. At the 500K-headline endpoint, 500 clusters keep the average cluster near ~1,000 headlines and total comparisons near ~500M. Using 13 clusters, one per category, would push total comparisons toward ~19B. The higher cluster count keeps global deduplication tractable as the corpus grows.\n\n## Similarity distribution monitoring\n\nAfter each iteration, the pipeline plots an ECDF (Empirical Cumulative Distribution Function) of pairwise cosine similarities. The red line marks the 0.90 deduplication threshold: a steep rise near 1.0 signals many near-duplicates, while a smoother left-shifted curve indicates broader coverage.\n\nFigure 2, below, overlays representative iterations 5, 20, 40, 60, and 82. Early curves skew right because the model repeats common patterns from limited few-shot examples. Later curves shift left and stabilize, showing that updated few-shot examples continue to push generation toward novel content even as the corpus approaches 500K headlines.\n\n## Step 4. Extract diverse few-shot examples for the next iteration\n\nAfter deduplication, the pipeline selects few-shot examples for the next batch. The goal is to steer the model away from patterns it has already produced.\n\n### Farthest-from-centroid selection\n\nK-means clustering from the deduplication step gives each headline a cluster centroid. We rank headlines by cosine distance from that centroid and select farthest examples, which are atypical within their semantic neighborhood and useful for prompting novel outputs.\n\n### Cross-iteration semantic filtering\n\nDistance alone is not enough. The pipeline also compares each candidate against all previously used few-shot examples and rejects any candidate with >=80% cosine similarity. This prevents semantically repeated prompt signals across iterations.\n\nTable 3, below, summarizes the selection priority. Three examples are selected per category, for 39 total per iteration, and the used-example set plus embeddings are checkpointed for recovery.\n\nPriority | Strategy | Purpose |\n|---|---|---|\n| 1. Primary | Farthest from K-means centroid | Most atypical = most informative few-shot signal |\n| 2. Semantic filter | < 80% cosine similarity to all used examples | Prevents semantic repetition across iterations |\n| 3. Fallback | Random sample from category pool | Ensures coverage when outliers are exhausted |\n| 4. Last resort | Seed examples (manually curated) | Cold-start baseline for hard-to-fill categories |\n\n*Table 3. Few-shot example selection priority*\n\n**Semantic similarity check for few-shot candidate filtering:**\n\n``` python\ndef is_semantically_distinct(\n    candidate_embedding, used_embeddings, threshold=0.80\n):\n    \"\"\"Reject candidates >= threshold similar to ANY used example.\"\"\"\n    if used_embeddings.size == 0:\n        return True\n    candidate_norm = candidate_embedding / (\n        np.linalg.norm(candidate_embedding) + 1e-9\n    )\n    used_norms = used_embeddings / (\n        np.linalg.norm(used_embeddings, axis=1, keepdims=True) + 1e-9\n    )\n    similarities = np.dot(used_norms, candidate_norm)\n    return np.max(similarities) < threshold\n```\n\nTable 4, below, shows how selected examples become more specific by the final iteration.\n\nCategory | Iteration 1 Example | Iteration 82 Example |\n| Earnings | Google Parent Alphabet Posts $1.84 EPS, $63.5B Revenue, Exceeds Forecasts | ThermoGlow Posts Q2 Beat, Lifts FY Guidance on Record Orders for Ultra-Low-VOC Automotive Adhesive |\n| M&A | Macy’s to Acquire Nordstrom’s Home Division in $1.8B Deal | Siemens Agrees to Acquire Mentor Graphics in $3.5B Deal, EU Demands Divestiture of Overlapping EDA Tools |\n| Credit Ratings | S&P Upgrades Moody’s Outlook for Major European Bank to Stable | Fitch Downgrades Argentina’s State Oil Company YPF to B-, Citing Fiscal Pressures and Falling Production |\n\n*Table 4. Few-shot examples from iteration 1 (early) and iteration 82 (final). Later examples feature more specific entities, niche financial instruments, and longer, more detailed structures, reflecting the pipeline’s drift toward frontier content.*\n\n## Step 5. Dynamic distribution correction\n\nWeighted sampling alone doesn’t keep categories on target because the model favors easier classes such as “Other” and “Stock Movement.” After each iteration, the pipeline compares target and actual category proportions, boosts underrepresented categories, clamps extremes, and normalizes weights for the next batch.\n\n**Weight adjustment algorithm:**\n\n``` python\ndef compute_adjusted_weights(current_dist, target_dist, floor=0.005):\n    adjusted = []\n    for category in CATEGORIES:\n        target = target_dist.get(category, 0.01)\n        current = current_dist.get(category, 0.0)\n        if current > 0:\n            ratio = target / current           # > 1 = under-represented\n            ratio = max(0.2, min(5.0, ratio))  # Clamp to avoid extremes\n            weight = target * ratio\n        else:\n            weight = target * 2.0              # Missing category: boost\n        adjusted.append(max(floor, weight))\n    total = sum(adjusted)\n    return [w / total for w in adjusted]       # Normalize to 1.0\n```\n\nBy iteration 82, rare categories such as “Credit Ratings” and “Product Approval” closely match their 1% targets, while “Other” remains overrepresented. The correction loop still reduces model bias enough to preserve coverage across all categories.\n\n## Results\n\n**Pipeline run summary: **The 500K target is slightly overshot because generation proceeds in fixed 35K batches and halts only after the threshold is crossed; the excess 2,536 examples are trimmed from the over-represented “Other” category.\n\n**Total unique headlines:** 502,536 (target: 500,000)**Total iterations:** 82**Batch size:** 35,000 headlines per iteration**Total generated (pre-deduplication):**~2.87M headlines** Overall deduplication rate:**~82% (cumulative)** Net new per iteration:**~5,000–6,000 unique headlines** Categories:**12 broad topics + Other** Similarity threshold:**90% cosine similarity** Few-shot examples per iteration:**39 (3 per category)\n\n### Deduplication efficiency\n\nSemantic deduplication was the dominant filter, and its role shifted over the run. In iteration 1, seeded by only one manual example per category, the pipeline retained 17,348 of 50K headlines; the remaining 32,652 (65%) were near-duplicates. Iterations 2–3 used farthest-from-centroid examples from previous outputs and maintained higher yields of 15K–17K new headlines per 50K batch, though cross-batch collisions began rising as the corpus grew.\n\nFrom iteration 4 onward, we reduced batch size from 50K to 35K once the deduplication drop rate exceeded ~80%. Larger SLURM jobs were mostly producing headlines discarded downstream, while smaller 35K batches delivered similar net-new yield with less wasted compute.\n\nAfter the corpus passed ~100K headlines, each new headline had to be distinct from a much larger pool. By iteration 40+, yield stabilized at ~5K–6K new headlines per 35K batch, or roughly 15%, with about 30K discarded per batch. Table 4 shows the transition: early duplicates are mostly within-batch repetition, while later duplicates are mostly collisions with the accumulated corpus. The steady nonzero yield near 500K shows that evolving few-shot examples continued to steer generation into novel regions.\n\nIteration | Batch Size | Net New | Discarded | Yield | Corpus Size After |\n|---|---|---|---|---|---|\n| 1 | 50,000 | 17,348 | 32,652 | 35% | 17,348 |\n| 2 | 50,000 | 17,199 | 32,801 | 34% | 34,547 |\n| 3 | 50,000 | 15,209 | 34,791 | 30% | 49,756 |\n| 5 | 50,000 | 16,917 | 33,083 | 34% | 80,028 |\n| 42 | 35,000 | ~5,700 | ~29,300 | 16% | 268,724 |\n| 82 | 35,000 | ~5,155 | ~29,845 | 15% | 502,536 |\n\n*Table 5. Deduplication yield across representative iterations. Yield drops from ~35% (early, small corpus) to ~15% (late, large corpus) as cross-batch collisions dominate*\n\n## Downstream application: Model distillation\n\nThe generated dataset supports the [AI Model Distillation for Financial Data developer example](https://build.nvidia.com/nvidia/ai-model-distillation-for-financial-data), where a 49B–70B teacher model labels synthetic headlines and smaller 1B, 3B, and 8B student models are fine-tuned with LoRA. With 25K labeled examples, a 3B student reaches 95% of the teacher’s F1-score on this financial headline classification task.\n\nDataset diversity matters here: semantic uniqueness and category balance expose the student to rare financial events and edge cases that are difficult to collect from real-world sources.\n\n## A practical path\n\nFour principles emerged for iterative synthetic data generation with global semantic duplication:\n\n**Iterate, don’t batch.** A single 500K generation pass produces mostly duplicates. Eighty-two smaller iterations with evolving prompts yield a far more diverse corpus.**Deduplicate globally, not locally.** Each iteration must deduplicate against the full accumulated dataset to prevent cross-batch duplicates.**Few-shot examples are the steering wheel.** Farthest-from-centroid selection with semantic similarity filtering is the most impactful lever for generation diversity.**Distribution correction is essential.** Without dynamic weight adjustment, models over-generate popular categories and starve rare ones.\n\n## Getting started\n\n**Use the dataset**— Download[FinHeadlineMix on Hugging Face](https://huggingface.co/datasets/nvidia/FinHeadlineMix)and use it directly for headline classification, fine-tuning, or as training data for your own distillation pipeline.**Run the recipe**— Reproduce or extend this pipeline using[NeMo Data Designer](https://github.com/NVIDIA/NeMo-Curator)for structured synthetic generation and[NeMo Curator](https://github.com/NVIDIA/NeMo-Curator)for scalable semantic deduplication.**Train a distilled model**— Follow the[AI Model Distillation for Financial Data](https://developer.nvidia.com/blog/build-efficient-financial-data-workflows-with-ai-model-distillation/)developer example to fine-tune a compact student model on this dataset.\n\n## Going further\n\n[Nemotron-3-Nano-30B-A3B on NVIDIA Build](https://build.nvidia.com/nvidia/nemotron-3-nano-30b-a3b)— the MoE model used for generation, approved for distillation workflows[Scaling Laws for Task-Specific LLM Distillation](https://arxiv.org/abs/2606.24747)— research paper leveraging the produced[FinHeadlineMix dataset](https://huggingface.co/datasets/nvidia/FinHeadlineMix)to derive distillation scaling laws[Build Efficient Financial Data Workflows with AI Model Distillation](https://developer.nvidia.com/blog/build-efficient-financial-data-workflows-with-ai-model-distillation/)— companion blog[How to Build License-Compliant Synthetic Data Pipelines](https://developer.nvidia.com/blog/how-to-build-license-compliant-synthetic-data-pipelines-for-ai-model-distillation/)— related reading", "url": "https://wpnews.pro/news/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo", "canonical_source": "https://developer.nvidia.com/blog/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo/", "published_at": "2026-07-09 19:40:37+00:00", "updated_at": "2026-07-09 20:10:24.094878+00:00", "lang": "en", "topics": ["artificial-intelligence", "natural-language-processing", "large-language-models", "ai-tools", "ai-research"], "entities": ["NVIDIA", "NeMo Data Designer", "NeMo Curator", "Nemotron", "vLLM", "B200"], "alternates": {"html": "https://wpnews.pro/news/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo", "markdown": "https://wpnews.pro/news/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo.md", "text": "https://wpnews.pro/news/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo.txt", "jsonld": "https://wpnews.pro/news/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo.jsonld"}}