From STTM to Snowflake SQL: Building a Metadata-Driven Data Engineering Copilot

A developer built Data Engineering Copilot, a metadata-driven tool that automates the generation of engineering artifacts from source-to-target mapping documents. The tool parses STTM files, normalizes metadata into a canonical model, and outputs artifacts like Snowflake SQL, aiming to reduce repetitive manual work in enterprise data engineering programs.

Most data engineering teams do not struggle because they lack smart people. They struggle because too much of the delivery process is still repetitive. A source-to-target mapping document comes in. Then someone has to manually create: For one or two tables, this is manageable. For a real enterprise program with many tables, changing requirements, multiple source systems, and repeated delivery cycles, this becomes a major productivity problem. That is the problem I am exploring with Data Engineering Copilot . Website: https://dataengineeringcopilot.com https://dataengineeringcopilot.com The idea is simple: text Upload STTM ↓ Parse metadata ↓ Normalize into a canonical metadata model ↓ Generate engineering artifacts