DataOps

The platform
beneath the pipeline

A pipeline should prove what is running, not assume it. Every deployment system has a state where the automation reports success and the old code is still serving traffic — and if nothing compares the two, nobody finds out from a dashboard.

System: Passenger profiling platform
Scale: Two environments, one declared source
Role: Sole architect and engineer
Stack: Terraform / AWS ECS / GitHub Actions

The challenge

A release passed every check and was then silently reverted by the platform's own safety mechanism. Nothing in the pipeline was built to notice.

The approach

Declare the whole platform as code, make the quality gates ratchet, and end every deployment by proving what is running.

The result

An entire class of green-pipeline failures became impossible to miss, and the data platform can be rebuilt from source rather than from memory.

01DeclaredEverything the pipeline needs to run — the workers, the queues, the storage, the cache, the alarms — is written down as code rather than clicked together. A rule like this data must never expire becomes a reviewable line protected by a test, instead of a setting nobody re-checks.Terraform / 11 reusable modules / S3 remote state 02BuildThe quality gates only move one way. Test coverage, bundle size, lint debt and security findings can hold or improve but never quietly slip, because each one is measured against the last accepted value rather than a fixed target someone can lower.GitHub Actions / coverage ratchet / checkov / gitleaks 03DeployThe rollout has a deliberate order: the half of the pipeline that consumes data is upgraded before the half that produces it, so the two are never briefly incompatible. Credentials are short-lived and issued per run, so there is no long-term key to leak.ECS rolling deploy / GitHub OIDC federation 04VerifyThe last step asks the only question that matters: is the version actually running the one we just built? A deployment can pass every check and still be silently rolled back by the platform, leaving last week's code in production with all the lights green. Comparing the two turns an invisible failure into a red build.Running-task inspection / image digest comparison

Fig. 06 — The last step is the one that checks the other three told the truth

Hover any step to read what it doesTap any step to read what it does

The platformbeneath the pipeline

The challenge

The approach

The result

The platform
beneath the pipeline