Skip to content

Data Pilot AI

An AI co-worker that builds and fixes data pipelines, safely

Key highlights

  • 21data models built automatically

  • 23 + 25dbt tests + quality checks

  • 1 → 1break detected, fixed automatically

  • 0human interventions · 100% evidence complete

Vision

Data teams spend hours on repetitive work and on fixing broken pipelines. Data Pilot automates that loop on top of the tools companies already use (Snowflake, dbt), with proof that every step is correct.

Key features

  • Maps the data

    digital twin of the estate

  • Builds models

    dbt project generated

  • Auto-testing

    tests, checks, reconciliation

  • Spots breakages

    e.g. a renamed column

  • Safe auto-repair

    one allowed fix type

  • Evidence pack

    PASS / FAIL proof

From business request to verified data product

  1. Request

    customer revenue

  2. Map the data

    digital twin

  3. Build

    21 dbt models

  4. Test

    tests & checks

  5. Detect & repair

    1 safe fix

  6. Verify

    PASS + evidence

Architecture

  • Request(User)

    business ask · CLI · API

    • 1 request Supervisor

  • Supervisor(AI workflow)

    runs the steps · state machine

    • each step Policy check
    • pins Digital twin

  • Policy check(Checks & controls)

    every action · rules engine

    • allowed Safe gateway

  • Safe gateway(AI workflow)

    the only door · governed tools

    • run Snowflake
    • build dbt
    • test Tests & quality

  • Snowflake(External tool)

    read raw, write dev · warehouse

  • Digital twin(Stored record)

    snapshot of the data · pinned

    • stored Control record

  • Safe repair(AI workflow)

    1 allowed fix

    • re-check Policy check

  • Tests & quality(AI workflow)

    dbt tests, checks

    • break found Safe repair
    • evidence Verifier

  • dbt(External tool)

    builds models · dbt Core

  • Control record(Stored record)

    PostgreSQL 16

  • Verifier(Checks & controls)

    independent, read-only

    • verdict Evidence pack

  • Evidence pack(Stored record)

    PASS / FAIL

  • User
  • AI workflow
  • Checks & controls
  • Stored record
  • External tool

Design choice: deliberately a controlled workflow, not a free-roaming agent. One safe gateway, minimum access rights and an independent verifier make every result reproducible and auditable.

Security

  • Works only in a safe dev area
  • Minimum access rights (read vs write)
  • Policy check before every action
  • Verifier cannot change anything

Testing

  • ~100 test files incl. security tests
  • Benchmark runs twice, results identical
  • No AI grading itself: code decides PASS/FAIL

Tech stack

Interface
  • Typer CLI
  • FastAPI
Backend
  • Python 3.12
  • uv
  • SQLAlchemy
  • Alembic
  • sqlglot
AI approach
  • Controlled workflow
  • Model design validated by code
  • No AI self-grading
Data
  • Snowflake
  • dbt Core 1.11
  • DuckDB (offline)
  • PostgreSQL 16
Deploy
  • Docker
  • Railway demo
  • GitHub Actions

Deployment

  • One-command demo (CLI)
  • Docker + synthetic demo on Railway
  • Optional live Snowflake smoke test
Next case studyAI Career Intelligence Assistant