Getting Started

This page is the path a design partner walks the first week: get a toolchain, install the CLI, prove the contracts are sound, run a service against local Postgres, and put that service on your own infrastructure. Every command below was run against the real cafe binary and the real service repositories, and the output is quoted where it matters.

Status, first

cafaye is in early development. Of the thirteen cafe subcommands, eleven do real work today — backup, contract lint/contract resolve, deploy, dev, doctor, env, gen telemetry, lock, mcp, reclaim, version. The remaining two (init, new) parse their flags, check their argument count, and return not implemented in v0. That is stated on every step below rather than hidden, because a command that silently half-works is worse than one that refuses. cafe/internal/cli/stub_test.go holds the table this sentence is derived from, and it is checked rather than remembered.

What is real: the contracts are frozen and validated, the services have HTTP surfaces you can call, container images you can build, a local stack you can run, cafe dev will render and bring up a compose file from a manifest, and cafe deploy will build, push and roll a service out. What is not real: cafe gen, and the broker. The runbooks assume you have already got the services running on your own infrastructure.

The launch deploy path is the substrate, not Kamal. The customer-facing deployment substrate — k3s + Traefik + cert-manager + ArgoCD + CloudNativePG + Redis + Alloy/Loki/Grafana + Prometheus — is decided in DEPLOYMENT.md in the moon repository (recorded publicly in Deployment) and is what the launch stack ships on. cafe deploy drives Kamal today; the k3s/Argo implementation lands behind the same deploy.Runner interface, so the command does not change — the infrastructure it drives does. Kamal retires at parity, not before. Deployment is the runbook for the substrate.

Step 1 — check the toolchain

cafe doctor is the first command to run, and it is safe anywhere. It resolves binaries on PATH and never executes them: no subprocess, no container, no network.

go install github.com/cafaye/cafe/cmd/cafe@latest
cafe doctor

tool            status  found at
git             ok      /opt/homebrew/bin/git
docker          ok      /usr/local/bin/docker
docker compose  ok      /usr/local/bin/docker-compose
tilt            ok      /Users/kaka/.local/share/mise/shims/tilt
go              ok      /Users/kaka/.local/share/mise/shims/go
ruby            ok      /Users/kaka/.local/share/mise/shims/ruby
elixir          ok      /Users/kaka/.local/share/mise/shims/elixir
python          ok      /Users/kaka/.local/share/mise/shims/python3
bun             ok      /Users/kaka/.local/share/mise/shims/bun
rust            ok      /Users/kaka/.cargo/bin/rustc
checked 10 tools, 10 ok, 0 missing

It checks ten tools, in that order: git, docker, docker compose, tilt, go, ruby, elixir, python (looked up as python3 then python), bun, rust (rustc then cargo). That list is the tools table in cafe/internal/cli/doctor.go, one row each, in that order. doctor exits 1 when any row in the project or reclamation tables is fail — a warn never moves it — so a red doctor is a real signal, not a decoration.

You do not need all ten. You need the languages of the services you intend to run, plus git and docker. identity alone needs go; parlor needs node.

Run it with no argument and, having no project to plan, the project table still prints — with a plan row that fails:

project .
check  state  detail  fix
plan   fail   could not be planned: no manifest: no cafaye.yml in .; a project declares its service in one (run "cafe init" to create it, or pass the project directory as the argument)  cafe contract lint .

That is a failure — doctor exits 1. Point it at a project directory to get the full project table, which is the one the tool table cannot answer — whether the container runtime is answering, whether the machine has the memory and CPUs a stack needs, whether the ports the stack publishes are free, and whether the manifest is one that plans:

cafe doctor ../identity

project /path/to/identity: identity (go), 3 services in identity-dev
check  state  detail  fix
runtime  ok    server 29.4.0  -
machine  ok    16 GiB and 8 CPUs  -
ports    ok    every published port is free and inside 15000-15999  -
plan     ok    3 services in identity-dev  -
4 state check(s): 4 ok, 0 warn, 0 fail

Four rows there are invisible from the tool table. runtime asks the Docker server, so an installed-but-stopped runtime reads fail while the table above says ok. machine holds a floor of 4 GiB and 4 CPUs for a database, a cache and a service at once. And the ports come from the same plan cafe dev would build, so the two cannot drift. A third table, reclamation, follows: whether the ledger holds stacks and port reservations nothing has reclaimed, and whether anything outside cafe’s ledger is holding a port in 15000-15999. -tools-only prints the first table alone.

What the check is not

doctor proves a binary is on PATH. It does not prove the version, and in the first table it cannot prove that Docker is running — only the second table can, by asking the server. Every service repository pins its own toolchain in its own mise.toml; mise install inside that repository is the versioned answer, and this page does not repeat those pins.

Step 2 — install cafe

There is no tagged release, so @latest resolves to a pseudo-version built from master. That works today:

go install github.com/cafaye/cafe/cmd/cafe@latest
cafe version

cafe 0.0.0-dev (commit unknown)

That version string is not a mistake. The semver and commit are injected at link time by the release pipeline; a go install from an untagged commit has neither, so the binary reports the development identity. When a release lands, cafe version reports the release instead.

To build from a clone instead — which is what you want if you are going to read the source:

git clone git@github.com:cafaye/cafe.git
cd cafe
go build ./cmd/cafe

go build ./cmd/cafe, not go build -o cafe .

The main package is at cmd/cafe. The repository root holds no Go files, so go build -o cafe . from the root fails with no Go files in <path> and exit 1. go build ./cmd/cafe writes ./cafe in the current directory. go install github.com/cafaye/cafe/cmd/cafe@latest writes to $GOBIN instead.

There is no Homebrew tap and no cafe install command. brew install cafaye/tap/cafe and cafe install are both Coming soon; neither exists today, and neither has a workaround beyond the two commands above.

Step 3 — read what the CLI can do

cafe help

cafe - the cafaye platform CLI

Usage:
  cafe <command> [flags] [arguments]
  cafe help <command>

Commands:
  backup    take one real backup and prove it restores, into a scratch database it then drops
  contract  validate cafaye manifests and resolve core version constraints
  deploy    deploy a service with Kamal: build, push, roll out behind kamal-proxy
  dev       run the local development stack for a project
  doctor    report which cafaye toolchains this machine has, and whether it can run this project
  env       provision the local stack for a command, and reclaim it when the command is done
  gen       generate code and config from cafaye contracts
  init      create a cafaye.yml manifest for the current project
  lock      pin this tree's specs, vendored schemas and generated clients, and check the pins
  mcp       serve the cafaye tools over the Model Context Protocol
  new       scaffold a new cafaye service or app
  reclaim   reclaim what a cafe session left behind: containers first, then volumes
  version   print the cafe version
  help      show help for a command

cafe help <command> is the same as <command> --help, and it is the authority on flags — this page does not restate them, because a flag list copied into prose is a flag list that drifts.

Command	Takes	Works today
cafe version	—	yes
cafe doctor	0 or 1	yes
cafe contract lint <path>	1 argument	yes
cafe contract resolve <constraint> <version>	2 arguments	yes
cafe dev [project]	0 or 1	yes
cafe backup <service>	1 argument	yes
cafe env <verb>	1 argument	yes
cafe gen <target>	1 argument	yes
cafe lock <path>	1 argument	yes
cafe mcp	0 arguments	yes
cafe reclaim	0 arguments	yes
cafe init	0 arguments	flags only
cafe new <name>	1 argument	flags only

Flags only means exactly what it says: the flags parse, the argument count is checked, and the command returns not implemented in v0 with exit code 1.

doctor and dev both take an optional path — cafe doctor alone reports the machine, and cafe doctor ./my-service reports the machine and whether it can run this particular project.

Two things that will bite you

cafe init takes no positional argument. It operates on a directory. This is the argument-count check firing:

$ cafe init my-saas
cafe: usage: cafe init wants 0 arguments, got 1 (usage: cafe init [flags])
$ echo $?
2

The command that takes a name is cafe new:

$ cafe new my-saas
cafe: cafe new: not implemented in v0
$ echo $?
1

Flags come before positional arguments. cafe uses Go’s standard flag package, so the flags must precede them:

cafe deploy --dry-run identity     # works
cafe deploy identity --dry-run     # usage error, exit 2

Exit codes

These are a contract — scripts branch on them, so they do not change without a version bump.

Code	Meaning
0	the command succeeded, or you asked for help
1	the command ran and failed
2	cafe was invoked wrongly: unknown command, unknown flag, wrong argument count

Step 4 — cafe init a project

Coming soon — the flags parse, the command returns not implemented in v0.

$ cafe init
cafe: cafe init: not implemented in v0
$ echo $?
1

cafe init is the command that will create a cafaye.yml for the current project. It takes -dir (default .) and -force, and nothing else:

cafe help init

cafe init - create a cafaye.yml manifest for the current project

Usage:
  cafe init [flags]

Flags:
  -dir string
        directory to initialize (default ".")
  -force
        overwrite an existing cafaye.yml

Until it ships, write the manifest by hand. Copy the shape from Contracts, then prove it with the one validator that does work. docs/cafaye.yml is a complete, schema-valid example you can read in the repository.

cafe contract lint ./cafaye.yml

OK ./cafaye.yml

Point it at a directory and it walks the tree, checking every cafaye.yml it finds and skipping .git, node_modules, deps, _build, and target:

cafe contract lint /path/to/checkouts

OK /path/to/checkouts/billing/cafaye.yml
INVALID /path/to/checkouts/parlor/cafaye.yml: is missing required fields ["name", "language", "core", "repository", "owner"]

It prints one line per manifest and nothing else, so the output is greppable and diffable. It exits 1 if any manifest is invalid, and also if the path holds no manifest at all — a tree that validated nothing is not a pass:

$ cafe contract lint /tmp/empty-dir
cafe: cafe contract lint: no cafaye.yml found under /tmp/empty-dir

Validation is two things: the core JSON Schema, vendored into the cafe binary at internal/contract/schemas/manifest-0.2.json and pinned by sha256 (cafe never fetches it), plus the three cross-field rules a JSON Schema cannot state — a published event type must start with the publisher’s own name, a service never consumes its own events, and an exposes must name an OpenAPI document or an event. The schema is checked first; the cross-field rules only run on a document that passed it.

Checking a core version constraint

cafe contract resolve answers one question — does this service’s core: range admit this core release — and is shaped for a CI gate.

cafe contract resolve '^0.2.0' 0.2.0

yes  ^0.2.0 allows 0.2.0: 0.2.0 is in [0.2.0, 0.3.0)

cafe contract resolve '^0.1.0' 0.2.0

no   ^0.1.0 allows 0.2.0: 0.2.0 is not in [0.1.0, 0.2.0)

Exit 0 when the version is inside the constraint, 1 when it is not, 2 when cafe cannot read what you typed. The constraint grammar is deliberately tiny: 1.2.3 exactly, ^1.2.3 caret, ~1.2.3 tilde, >=1.2.3 at-or-above. No ranges, no ||, no x-ranges. Note the pre-1.0 caret: on a 0.x service ^0.2.0 is >=0.2.0 <0.3.0, so a 0.3.0 core is not admitted by ^0.2.0.

Step 5 — run a service locally

cafe dev works. It reads a project’s cafaye.yml, validates it against the same rules cafe contract lint applies, renders a compose file from it, writes that file and prints it in full, brings the stack up, waits for it, and reports what came up and what did not.

cafe dev --dry-run ./hello      # render and print; do not bring anything up

project hello-dev: hello (go)
skipped legacy: optional dependency, and the local registry does not know how to run it
wrote /work/hello/hello.compose.yaml

The report is the message: cafe dev prints its verdict for a stack that came up with something in it that did not, on stdout, once, and does not repeat it on stderr. A generated compose file is marked do not edit — every run rewrites it, and the manifest is the source of truth.

What it does not do yet: it plans from the manifest and the local registry, so a dependency the registry does not know how to run is reported as skipped rather than silently omitted. It does not deploy, and it does not start a broker — nothing in the platform publishes events off the outbox.

No service has a Compose stack you can bring up on its own

Every service’s docker-compose.yml is an override, not a stack: the service, its database name and role, and nothing else. None of them can be brought up with a bare docker compose up -d — the Postgres cluster, NATS, Redis, the collector and the observability backends all come from kit, at the commit named in the service’s kit.ref. Run alone, each one stops before starting anything:

git clone git@github.com:cafaye/identity.git
cd identity
docker compose up -d

service "postgres" has neither an image nor a build context specified: invalid compose project

(courier fails one step earlier still, on a missing COURIER_SECRET_BOX_KEY.)

identity, courier and billing carry bin/dev, which does the merge for you: it fetches kit’s stack at the ref in kit.ref and merges your file beside it. It reads the pin, resolves that exact ref of kit — from a local checkout, a cache, a vendored copy, or a fetch — runs compose with kit’s stack as the base file and yours as the override, then migrates, seeds the local admin and prints the URLs that actually work:

bin/dev                              # fetch kit at kit.ref, merge, up --wait, migrate, print URLs
curl -s localhost:8080/healthz
curl -s localhost:8080/readyz

identity is the shortest example of the bin/dev shape, and the difference shows in the readiness answer: the database is in kit’s stack rather than the service’s, so readiness carries the dependency by name — {"status":"ok","deps":"postgres"}.

darkroom, muse and gateway do the merge by hand. They carry a kit.ref but no bin/dev, so the two-file form is yours to run — and KIT_COMPOSE_DIR is not optional, because compose resolves kit’s initdb mount against the project directory without it:

kit=/path/to/kit    # a checkout of kit at the ref in kit.ref
KIT_COMPOSE_DIR=$kit/templates/compose \
docker compose --project-directory . \
  -f $kit/templates/compose/docker-compose.yml \
  -f ./docker-compose.yml up -d --wait

darkroom then applies its own migrations with psql and runs DARKROOM_OBJECT_STORE=memory go run ./cmd/darkroom; muse applies its with docker compose … exec -T postgres psql. The probes answer the same way once the service is up.

With the service actually up — through bin/dev or the two-file merge — both probes answer:

{"status":"ok"}
{"status":"ok","deps":"postgres"}

Without a database readyz answers {"status":"ok","deps":"none"} — there is nothing to check, and saying so beats an empty list that looks identical to a healthy dependency check. The /v1 routes are only registered when DATABASE_URL is set, so a missing database is a clean 404 rather than a pile of 500s.

To run identity without Docker at all — the faster loop while you are changing code, and it needs no database:

mise install          # reads identity/mise.toml -> the pinned Go
go run ./cmd/identity # http://localhost:8080

The other services

The command depends on which kind of stack the repository ships, and the two kinds are not interchangeable — that is the whole table:

Service	Local stack	Command	Port	What answers
identity	kit override	bin/dev — see the caution above	8080	{"status":"ok","deps":"postgres"}
courier	kit override	bin/dev, or mix phx.server — see the caution above	4000	{"status":"ok"}
billing	kit override	bin/dev — see the caution above	3000	{"status":"ok","checks":{"database":"ok"}}
darkroom	kit override	two-file merge, psql migrations, go run ./cmd/darkroom	8080	{"status":"ok"}
muse	kit override	two-file merge (needs MUSE_VAULT_KEY), psql migrations	8000	{"status":"ok","deps":{"db":"ok"}}
gateway	kit override	two-file merge	8080	{"deps":{…}}
parlor	none — no Compose file	npm run dev	3000	{"status":"ok","deps":"none"}

The first three ship bin/dev and a kit.ref pin; the next three ship a kit.ref but no bin/dev, so the two-file merge is yours to run. parlor ships no Compose file at all — it is a Vite app and npm run dev is the whole story.

Two stacks refuse to start until you tell them a secret, and that refusal is the right one: muse will not render without MUSE_VAULT_KEY, and courier will not render without COURIER_SECRET_BOX_KEY. (courier also reads COURIER_INBOUND_RESEND_SECRET, but that one has a default — it is not a refusal, just a placeholder you should replace.) The compose message names the variable and, for muse, the command that generates a valid one.

billing is the one that used to be described as “database only, application on the host”. It is not: its Compose file builds and publishes billing on 3000:3000 like every other service, and bin/dev brings the whole thing up. The commands are in that repository’s README if you want the host-side loop instead.

Which bring-up command is which

The table above is per-service local development. When somebody says “bring up the platform”, there are three different stacks, and they are not interchangeable — with four commands, because install.sh drives bin/launch rather than being a stack of its own:

bin/launch (in site) is the launch tier — the product’s local face. It brings up the launch four (identity + gateway + courier + parlor) on kit’s pinned stack (one Postgres cluster, NATS with JetStream, Redis, the OTel collector), plus an SMTP sink a person can read (mailpit), built from the checkouts beside site. scripts/walk-the-loop.sh then walks the loop the product is sold on against that stack — signup, the confirmation mail, confirm, signin, signin with MFA, forgot password, the reset mail redeemed, create-an-account/invite-a-member, settings — with every mail read out of mailpit over its HTTP API. COURIER_FORCE_SSL is unset there on purpose: the launch network is plain HTTP (see courier).

bin/launch                  # fetch kit, mint the courier key, up --wait, print URLs
scripts/walk-the-loop.sh    # the loop, with real curls against the launch stack

What	Host port	Container port
identity, the auth service	${LAUNCH_IDENTITY_PORT:-15080}	8080
gateway, the API edge	${LAUNCH_GATEWAY_PORT:-15081}	8080
courier, the mail service	${LAUNCH_COURIER_PORT:-15082}	4000
parlor, the product UI — the front door	${LAUNCH_PARLOR_PORT:-15083}	3000
mailpit web UI, the SMTP sink (COURIER_SMTP_HOST: mailpit, port 1025 inside the network)	${KIT_MAILPIT_PORT:-15025}	8025

The port block is kit’s 15000-15999, declared in site/e2e/docker-compose.launch.yml, which is an override on kit’s stack — read that file’s header before changing anything about the topology.

install.sh (in site) is the stranger path onto the launch tier. It clones the launch four (identity, gateway, courier, parlor — at their default branch tips, because the tier builds whatever is checked out) plus kit at master beside the site checkout, then drives bin/launch — so what comes up is the launch stack above, not a second topology: the launch four on kit’s pinned stack (one Postgres cluster, NATS with JetStream, Redis, the OTel collector), plus the mailpit SMTP sink. It is safe to re-run: a checkout that is already beside site is left alone, and a stack that is already up is reported with its URLs rather than rebuilt. When it finishes, bin/launch prints where to look — parlor on 15083 (the front door), mailpit on 15025, identity on 15080, gateway on 15081, courier on 15082 — including the launch owner’s sign-in for the stack it just brought up. It does not clone pantry or oauth-stub: those are the e2e tier’s services, and the launch compose names neither.

bin/up (in site) is the e2e tier, not the launch stack. It brings up identity + gateway + pantry + oauth-stub (+ site) from published artifacts, on its own 16000-16099 port block — so the two tiers can run on one machine without meeting. Do not describe it as bringing up the launch tier: it runs pantry and oauth-stub, which the launch stack does not contain, and it does not run courier, parlor, NATS, Redis, or mailpit in the launch shape.

curl https://cafaye.com/box/install | sh (kit) is the production path. It installs the substrate — k3s + Traefik + cert-manager + ArgoCD + CloudNativePG + Redis + Alloy/Loki/Grafana + Prometheus — on one box. Deployment is the runbook for it.

The ports, and the one collision that is not a port collision

8080	identity, gateway, darkroom — three services, one port
3000	billing, parlor — two services, one port
15500	the Postgres every kit override uses

No service publishes its own Postgres host port. darkroom and muse used to bind 5432 and 5433; they no longer do — their postgres: services carry no ports: at all, and the cluster they share with identity, courier and billing is the one kit publishes, on ${KIT_POSTGRES_PORT:-15500}. The variable that moves it is KIT_POSTGRES_PORT, it is kit’s rather than a service’s, it lives in kit’s .env, and it moves kit’s shared container rather than any service’s. Set it there if 15500 is taken.

The real reason you run one of identity, courier and billing at a time is not a port. They share one Postgres container and each names its own database — KIT_POSTGRES_DATABASES: identity, then courier, then billing — and compose merges those overrides last-one-wins. Bring two up together and one service’s DSN points at a database that does not exist, which surfaces as role "…" does not exist and looks exactly like a wrong password. darkroom and muse share that same cluster, so the same rule holds for them.

To stop one, without losing its data: bin/dev down. bin/dev nuke is the destructive one and says so.

kit’s Postgres is built, not pulled, and it is Debian

kit’s stack writes image: kit-postgres:${KIT_POSTGRES_TAG:-17} — an image built from templates/compose/postgres/Dockerfile on a postgres:17 (Debian) base, because postgres:17-alpine cannot carry pgvector: pglayers publishes glibc-linked layers and the alpine base is musl, so the extension fails to load at run time on an image that builds cleanly. The .env that bin/dev creates on a first run comes from kit’s .env.example, and that file sets KIT_POSTGRES_TAG=17 — the same default, so there is nothing to change.

darkroom’s compose stack defaults to DARKROOM_OBJECT_STORE=memory, which is what makes the local path work with no bucket and no credentials. Its deployment build defaults to the s3 object store — docker/Dockerfile sets DARKROOM_OBJECT_STORE=s3 — so docker build needs no flag to get it (see the container table below).

Migrations are a step you run, not something that runs itself

Every service applies its schema separately, in its own language, with its own tool. None of them migrates on boot — a rolling deploy with two versions live would race, and a half-applied migration would take the process down with it.

On bin/dev you do not run this step at all: bin/dev runs the repository’s own migration command between bringing the stack up and printing the URLs, and refuses to do anything else if the stack is not healthy first. The commands below are for cafe dev, for a host-side loop, or for a Kamal deploy — where they are a job that runs before the new image rolls out.

identity uses goose, which is a command-line tool rather than a module dependency. The port is 15500, not 5432 — that is kit’s Postgres, which is where bin/dev put it:

export DATABASE_URL=postgres://identity:identity@localhost:15500/identity
goose -dir migrations postgres "$DATABASE_URL" status   # what has been applied
goose -dir migrations postgres "$DATABASE_URL" up       # apply everything pending

billing uses Rails migrations, against the same Postgres and the same port:

DATABASE_URL=postgres://billing:billing@localhost:15500/billing bin/rails db:prepare

muse applies plain SQL:

psql muse -f migrations/00001_outbox_events.sql
psql muse -f migrations/00002_vault_secrets.sql

In production, run up-by-one (or the equivalent) as a single ordered job before the new image rolls out, and fail the deploy on a non-zero exit.

On a cafe dev stack, run them from inside the compose network. The rendered file publishes only the service’s own port, so goose on the host has nothing to connect to, and the first request you make before migrating fails with a bare 500 whose real reason — relation "users" does not exist — is only in the service’s logs. Hosted pilot onboarding has the loop that was run, and why it does not record a goose version.

Step 6 — deploy to your own infrastructure

cafe deploy works. It drives Kamal: it installs Docker on the host if the host lacks it, boots the accessories, builds and pushes the image, and rolls the release out behind kamal-proxy only once the new container answers its healthcheck. What it does not have is a cafaye-operated remote, so it deploys to infrastructure you already pay for.

This is the retiring path. The launch path is the substrate

The Kamal mechanics below are real and they are what runs today — but Kamal retires at parity, not before, and the launch stack does not ship on it. The customer-facing substrate (k3s + ArgoCD + CloudNativePG, installed by curl https://cafaye.com/box/install | sh) is decided in DEPLOYMENT.md in the moon repository and is documented in Deployment. The k3s/Argo implementation of cafe deploy lands behind the same deploy.Runner interface (cafe/internal/cli/deploy.go), so the command you type does not change — the infrastructure it drives does. Read this section to understand the shape; read the Deployment runbook for the path that replaces it.

cafe deploy --dry-run identity

Run it from the project directory — the one holding config/deploy.yml. In a directory without one, the real answer is:

cafe: cafe deploy: /tmp/cafdry has no config/deploy.yml, so there is nothing to deploy.
  Copy it from cafaye/kit at the ref your kit.ref names: kit/templates/kamal/deploy.yml.erb -> config/deploy.yml

It takes -env, -version, -dry-run and -yes, and its usage line is cafe deploy [flags] <service>. Flags come before the service name, so cafe deploy --dry-run identity and not cafe deploy identity --dry-run; the latter is a usage error. -env names an overlay merged over the base config (config/deploy.<env>.yml over config/deploy.yml) and defaults to empty rather than to a named stage. -dry-run prints every command it would run and changes nothing. Without -yes it asks first, and a machine with nobody to answer is told to pass it rather than left to guess.

cafe does not deploy a cafaye service. kamal does, and kit ships the configuration — three files, copied per service

kit/templates/kamal/ holds the three files a Kamal deployment is made of:

File in kit	Copied to	What it is
templates/kamal/deploy.yml.erb	config/deploy.yml	the Kamal config: service, registry, servers, proxy, accessories
templates/kamal/kamal-backup.yml.erb	config/kamal-backup.yml	what to back up, where, and for how long
templates/kamal/drill.sh	bin/drill	a restore drill that drops its own scratch database

Adoption is partial, and what exists today is measured rather than claimed. Three repositories have a config/deploy.yml: identity and courier carry kit’s template (as of 2026-10-01), and billing’s is the stock Rails-generated file from its first commit, with the whole proxy: and accessories: blocks commented out. Two have a config/kamal-backup.yml — identity and courier. No repository has bin/drill. And no service has been deployed through any of it: there is no web server behind any of these files. So docker build + docker run below is still the real path, and the Kamal files are the shape to adopt when you want backups. Backup and restore is where this gap was measured, and it tells you which of the seven services have adopted what.

Three things about the template that are worth knowing before you copy it, because all three are counter-intuitive:

• deploy.yml.erb is ERB because five values differ per operator rather than per service — KIT_SERVICE, KIT_REGISTRY_ORG, KIT_REPO, KIT_WEB_HOST, KIT_APP_DOMAIN — and it raises by name if one is unset rather than rendering an empty string, which YAML reads as a null list item and which surfaces three layers away as “cannot find a host”. Copy the file and export the five; do not fork it per language, because the only thing that genuinely differs is the migration command.
• deploy_timeout is 180 seconds rather than Kamal’s 30 for the same class of reason: a container booting against a loaded host needs longer than 30 seconds to answer its first probe, and a window that is too small produces a deploy that cannot succeed rather than one that fails. And its proxy.healthcheck.path is /up, which is wrong — see the caution further down.
• kamal is a Ruby gem, so your machine needs Ruby. Not the service image — a service image never needs Ruby, whatever its language — and not the server. The backup accessory ships its own Ruby inside the container. Only the operator’s machine carries the requirement, and it comes from Kamal rather than from cafaye.
• kit’s deploy tooling is not remote. templates/bin/deploy.sh runs a production-shaped deployment — health gates, rollback, credential injection — against a local Docker daemon. Pointed at your own machine it is a local production-shaped run, and it does not become a remote deploy by being called from a laptop.

And, independently of all that, you can still build and run an image yourself. Every service ships a multi-stage, non-root Dockerfile, and they are not interchangeable — each pins its own base image and runtime:

git clone git@github.com:cafaye/identity.git
cd identity
docker build -t identity .
docker run --rm -p 8080:8080 \
  -e DATABASE_URL='postgres://identity:***@postgres:5432/identity?sslmode=require' \
  -e LOG_LEVEL=info \
  identity

The images run as uid 65532 on gcr.io/distroless/static-debian12:nonroot, so there is no shell in the container. That is deliberate, and it means you cannot docker exec sh into a running service to debug it: use the probes and the logs.

Service	docker build	Runtime port	Base
identity	docker build -t identity .	8080	gcr.io/distroless/static-debian12:nonroot
billing	docker build -t billing .	80	ruby:4.0.1-slim, entrypoint /rails/bin/docker-entrypoint
courier	docker build -t courier .	4000	debian:trixie-20260918-slim, USER nobody
darkroom	docker build -f docker/Dockerfile -t darkroom .	8080	a Go binary on the same distroless base as identity; the Dockerfile is under docker/, not the root
muse	docker build -t muse .	8000	python:3.14-slim
gateway	docker build -t gateway .	8080	gcr.io/distroless/static-debian12:nonroot
parlor	docker build -t parlor .	3000	node:22.22.2-slim (pinned in every stage), ENV PORT=3000

Two of those build lines used to carry flags that do not exist. courier’s Dockerfile declares no SERVICE_NAME build argument, and its compose file passes no args:, so --build-arg SERVICE_NAME=courier builds the same image as without it while looking like it configures something. And --build-arg --features s3 is not a command at all: --build-arg requires a NAME[=VALUE] after it, so Docker refuses at the flag parser before reading any Dockerfile.

$ docker build --build-arg
flag needs an argument: --build-arg

darkroom needs no flag for its deployment build either way: docker/Dockerfile already sets DARKROOM_OBJECT_STORE=s3, so the default build is the S3 build. There is no dev-auth verifier to enable — the Go tree’s only verifier is auth.New, and it is always on.

The order that works, and the order that bites:

1. Migrate first, as a job. The service never migrates on boot. A schema the new image needs must exist before the new image starts.
2. Roll out. The probes below decide whether traffic is held back.
3. Read /readyz, not /healthz. Liveness answers whenever the process can dispatch. Readiness answers 503 while the service cannot do work. Restarting on a readiness failure is how a dependency outage becomes a crash loop.

Service	Liveness	Readiness
identity	GET /healthz → always 200	GET /readyz → 200, or 503 with each probe bounded at 2s
billing	GET /healthz → always 200	GET /readyz → 200, or 503 {"status":"error","checks":{"database":"error"}}
courier	GET /healthz → always 200, never touches the database	GET /readyz → 503 while the database is unreachable
darkroom	GET /healthz → consults nothing	GET /readyz → really runs a query
muse	GET /healthz → 200, consults nothing	GET /readyz → reports the db slot
gateway	GET /healthz → always 200	GET /readyz → 200, or 503 if a registered probe is down
parlor	GET /healthz → 200	GET /readyz → {"status":"ok","deps":"none"}

One warning about courier: with a database that has gone away underneath a running pool, /readyz takes about 4.4 seconds to answer 503 — the pool’s queue backpressure, not the query timeout. An orchestrator with a shorter probe timeout will time out and read courier as not ready, which is the same verdict. Set the probe timeout above 5s or you will restart a service that is answering correctly.

gateway’s probes sit on the same port as its API, and it is not in the table above for /v1/* because gateway forwards only what ROUTE_TABLE configures: unset, it serves its own routes and nothing else. /v1/me is one of its own, and it proves the auth chain without forwarding anything.

kit’s proxy healthcheck path is /up, and /up is a boot check

config/deploy.yml.erb sets proxy.healthcheck.path: /up with a 2-second interval and a 5-second timeout, and kamal deploy gates the rollout on it. The comment immediately above that line calls it “A READINESS endpoint, not a liveness one”, and for every cafaye service that comment is wrong:

• /up is not a cafaye path at all. identity, courier, muse, darkroom, gateway and parlor serve /healthz and /readyz, so a proxy pointed at /up gets a 404 and the rollout never goes green. identity and courier are the two that carry kit’s template, and both changed the path to /readyz while they were in it — the change is in each file, commented.
• billing is the one service that serves /up, and it is Rails’ built-in boot check (get "up" => "rails/health#show" in its routes). It answers 200 if the application booted and does not touch the database; billing’s readiness is /readyz, and that is the endpoint that names {"status":"error","checks":{"database":"error"}}. billing does not use kit’s template, though — its config/deploy.yml is the stock Rails-generated file with the whole proxy: block commented out, so it has no healthcheck path at all today.

So no service in the fleet is deploying against /up; the template is simply wrong about its own default. Change the path in config/deploy.yml to whatever your service actually serves, and prefer /readyz. A container that answers 200 while its database is unreachable is a container the deploy will happily call healthy — and with a 2-second interval and a 5-second timeout that window is small enough to look fine and large enough to matter.

What is not in this path yet

The broker. core specifies a transactional outbox and NATS as the transport, and identity, billing, muse, courier and darkroom all write outbox_events correctly, in the same transaction as the domain change — but no service starts a publisher loop, and identity’s only Publisher implementation is a deliberate no-op that would mark every event published and drain the outbox into nowhere. Events land in the table and stay there.

That is not a bug to work around; it is the state of the platform, and it is the first thing to know if you are waiting for a downstream reaction that never comes. Troubleshooting has the entry, and Upgrading opens by saying how much of the event surface this actually affects — it matters if you wrote your own publisher loop, and not at all if you did not.

Observability is instrumented and not deployed, and those are two different sentences. core ships seven telemetry schemas and enforces the contract; courier, billing, identity and muse each wire an OTel SDK and export traces to whatever <SERVICE>_OTEL_ENDPOINT names, defaulting to the collector that ships with the stack. No collector is deployed anywhere in a cafaye environment, so a service started from its own docker-compose.yml is exporting to a host that is not there.

Getting a collector takes bin/dev, which three of the seven repositories already carry (identity, courier, billing) — it fetches kit’s whole stack at the ref in kit.ref, brings it up, and prints the URLs. The other four have no bin/dev, and copying it is the fix: cp <kit>/templates/bin/dev.sh ./bin/dev (plus a kit.ref pinning a 40-character commit sha, because bin/dev refuses a branch name before it makes any network call). Observability has the per-service table and the three states, and says what the gap costs you.

What to read next

• Hosted pilot onboarding — the same path, aimed at somebody who has already decided, with the licence position and every gap stated before you commit.
• Upgrading — if you already run a deployment and are moving to the current contracts.
• Architecture — what each service owns, and why the boundaries are where they are.
• Topology — the operator’s table: ports, probes, environment variables, the HTTP surface of each service, and the cross-repo drift audit.
• Contracts — cafaye.yml, the event envelope, OpenAPI. The one thing worth understanding before anything else.
• Observability — the telemetry contract, and the gap between it and a running stack.
• Services — one page per service, each stating what is built and what is not.
• Runbooks — provisioning, backup and restore, secret rotation, a service down, and billing webhooks. Read them before the first incident, not during it.
• Running the gates — the real gate for every repository, including the tiers that do not run by default.
• Troubleshooting — keyed by symptom.