REFERENCEupdated 2026-07-20

The Junkyard Field Guide

Every term and tool I throw around in these posts, explained for readers who know nothing, written by an author who until recently also knew nothing. Updated whenever the project forces me to learn a new word.

Signals & protocols

OpenTelemetry (OTel): The industry-standard framework for producing telemetry (logs, metrics, traces) from your code. Vendor-neutral, so instrumenting once means you can send data anywhere. Including to a junkyard.
OTLP: The OpenTelemetry Protocol: the wire format telemetry travels in. Speaks gRPC on port 4317 and HTTP on 4318. If Junkyard v2 speaks one language fluently, it's this one.
gRPC: A remote procedure call framework built on HTTP/2 and Protocol Buffers. Fast, typed, and the reason Go services can talk to each other without anyone writing JSON parsing code at 2am.
Log: A timestamped record of something that happened. The humblest signal, and the one that ate a 2GB VM's disk and my dignity along with it.
Metric: A numeric measurement over time, identified by a name plus labels (e.g. http_requests_total{status="500"}). Small data, enormous volume.
Trace / span: A trace follows one request through every service it touches. Each unit of work inside it is a span. Spans share a trace_id, which is how you reassemble the story afterwards.
Cardinality: How many unique combinations of metric name + labels exist. The silent killer of metrics systems: one user_id label and your "small" TSDB is suddenly keeping a series per user. We will not be doing that. Probably.

Storage

TSDB: Time-series database: a store specialized for (timestamp, value) pairs. Junkyard ships its own small one, partly for learning, mostly so I can say I did.
Head block: The in-memory part of a TSDB where fresh writes land before being flushed to disk. Fast to write, fast to query, gone if you crash, which is why it has a bodyguard (see WAL).
WAL (write-ahead log): An append-only file every write hits before the fancy data structures. After a crash you replay it and pretend nothing happened. Not to be confused with SQLite's WAL mode, which is the same idea wearing a different hat.
Segment files: Immutable chunks of flushed data on disk. Immutable means "never modified, only created and deleted," which makes retention a matter of deleting old folders. My favorite kind of feature: the kind rm -rf can implement.
Downsampling / rollup: Throwing away resolution over time: keep every data point for a week, 1-minute averages for a month, 1-hour averages for a year. Nobody needs per-second CPU data from March.
Retention: How long data is allowed to live before deletion. In v1, retention was "until the disk is full and the VM dies." v2 aims slightly higher.
SQLite: An entire SQL database in a single file, embedded in the process. Unfairly dismissed as a toy; it powers a shocking amount of the world's software and both of Junkyard's versions.
FTS5: SQLite's full-text search extension. Turns "find logs containing this word" from a table scan into an index lookup.

Infrastructure

Kubernetes (k8s): A container orchestrator: you declare what should run, it keeps it running. The de facto standard for deploying serious systems, and the thing my M.Sc. keeps making me draw diagrams of.
kind: Kubernetes IN Docker: a whole cluster running in Docker containers on your laptop. How you test k8s deployments without paying a cloud provider for the privilege.
Helm: The package manager for Kubernetes. A Helm chart is a templated bundle of everything an app needs to run, so helm install junkyard just works. That's the goal, anyway.
CI (continuous integration): Robots that run your tests and builds on every push. v1 had none and it showed. v2 has them from day one and it will also show, but positively.

Concepts

Backpressure: Telling an over-eager client to slow down instead of falling over silently. In Junkyard: a full queue returns "come back later" (503 / RESOURCE_EXHAUSTED) rather than eating memory until the process dies.
Monolith: One binary that does everything. Usually said as an insult. Here it's a deliberate choice: the interesting problems live inside the process, and one binary is hard to misconfigure.
Dogfooding: Using your own product. Junkyard's twist: Datadog's free tier monitors Junkyard, meaning the real product watches my tribute act.
ClickHouse / VictoriaMetrics / Elasticsearch: Excellent, battle-tested storage engines I am deliberately not using, because they would hide everything I'm trying to learn. This is a feature of the project, not an oversight. I keep telling myself that.