~/projects

Projects

Systems, tools and storage engines I've built to understand how the real thing works. Everything is open source; most of it is also running in my homelab, which is where the interesting bugs show up.

    flywheel(opens on GitHub)

    distributed job scheduler

    Fault-tolerant scheduler built on a hand-rolled Raft implementation: leader election, log replication, and at-least-once dispatch with idempotency keys. Cron expressions and DAG dependencies included, because every team eventually reinvents Airflow badly.

    1.2M jobs/day on a 3-node cluster, p99 dispatch < 80ms

    • rust
    • raft
    • grpc
    • sqlite

    cask(opens on GitHub)

    append-only key-value store

    Bitcask-style storage engine: an in-memory hash index over append-only segment files. Recovers from kill -9 at any write boundary, merges segments in the background, and never mutates a byte in place.

    120k ops/s single node, p99 read < 600µs, 8GB dataset

    • go
    • storage-engine
    • crash-recovery

    tailpipe(opens on GitHub)

    log ingestion pipeline

    Kafka → ClickHouse pipeline with size- and time-based batching, backpressure propagated end to end, and a dead-letter queue with replay. Rewriting the flush path cut daily part count 4x and made midnight merges boring.

    1M events/s on 3 nodes, end-to-end lag < 2s at peak

    • go
    • kafka
    • clickhouse
    • otel

    pgstat(opens on GitHub)

    postgres internals exporter

    Prometheus exporter for the parts of PostgreSQL people forget to watch: WAL generation rate, replication slot lag, bloat estimates and lock waits by query fingerprint. One static binary, zero external dependencies.

    used by 40+ people on GitHub and one very nervous homelab

    • go
    • postgres
    • prometheus

    swarmd(opens on GitHub)

    a tiny container orchestrator

    Small enough to read in one sitting: desired-state reconciliation loop, gossip-based membership, health-gated rescheduling. Built to understand Kubernetes, not to replace it.

    reconciles 50 containers across 5 nodes in < 1s

    • rust
    • gossip
    • grpc