Data & Distributed Systems

I build fault-tolerant systems.

Autonomous data systems, open-source tools anyone can run, and operations calm enough to be boring. The log records the engineering decisions, and the research is pre-registered before the results.

See the Systems Read the Log

Data is fuel. I build the engines.

Deployment Stack

Hetzner Cloud Tailscale Dagster Polars dbt Docker Caddy BigQuery SOPS
Pre-Registered Research · Distributed Systems

Distributed Livelock & Systemic Network Fragility

An exchange's limit order book is a distributed system: an asynchronous network of algorithmic finite state machines (FSMs). This protocol asks whether its worst failures are the faults such a network produces under clock skew, and states its prediction before the results.

Thesis: Liquidity exhaustion events are not exogenous price shocks, but deterministic state-transition faults within the network's global state scheduler caused by multivariable clock skew.

Philosophy

Calm Systems

My goal is to make the engineering boring, deterministic and cost-efficient. The first lesson below comes from an earlier stack. Each of the others is one a system here enforces today.

FinOps Is Architecture

The most expensive mistake in the cloud is a lazy configuration. On an earlier AWS stack, I cut network ingest costs by ~90% with one VPC Gateway Endpoint.

A Green Check Is Not Proof

A check on data that never arrives never runs, so a stopped schedule stays green. Catalyst compares each schedule's declared state with the live one, and raises an alert when a source's data does not arrive in time.

Refuse Rather Than Overdraw

A shared pool has a ceiling. A ration is refused rather than overdrawn, and the dex service refuses a billed call when it cannot reach the ledger that meters its spend.

Compute Apart from Data

Catalyst's warehouse lives apart from the host that computes it, so the host can be destroyed and rebuilt from code without touching the data. A deploy runs only from a release I cut, and each one is smoke-tested.

A Fix Is a Plan First

The dex service proposes a fix as a stored plan that can be reviewed. Proposing writes nothing to the project, and the agent cannot apply a plan: applying is a separate step that a person approves.

Private by Default

Catalyst's host is reached over Tailscale and its disk is encrypted with LUKS. The scheduled workflows and the Slack agent that call the dex service hold no warehouse credential.

Hiring for systems work?

If your team builds data systems that have to keep running, I'd like to talk. The systems, the log and the research show how I work, and the about page shows where I come from.