Back up and restore

This recipe covers what actually needs backing up on a Meridian instance and the restore sequence to bring it back. For why the durable state lives in Restate rather than in application code, see explanation/why-durable-execution.md; for the process/volume layout referenced below, see reference/topology.md.

What to back up

Data	Where	Criticality
Durable run state (journal, suspended runs)	`restate-data` volume	Critical — this is the only copy
Run index (run id, workflow file, slug and version, subjects, delivered event, `source_event_id`, `event_type`, `route_match`, dates, final status)	`proxy-data` volume (SQLite) or Postgres	Rebuildable empty, but holds event deduplication
Workflow memory (`workflow_memory` table)	Application database (`DEFINITION_STORE_URL`)	Critical — the only copy, read by running workflows
Reception inbox (`inbound_messages` table): messages of the inbound connections and records polled by event sources	Application database (`DEFINITION_STORE_URL`)	Critical — a pending message or record is kept only there until it is delivered; holds the content as received (health data), and serves the deduplication of the records a source reads again
Workflows, agents, instance manifests	YAML files in git	Already backed up (the repo is the backup)
Patient record	The hospital's own FHIR warehouse	Out of platform scope

The distinction that matters: Restate's journal is the only place a run's state actually lives. Losing the run index loses the list the API shows at GET /api/runs, not any run's execution state. It also loses the deduplication record: an event redelivered after the loss (a caller resending the same eventId, a source re-reading a record the reception inbox no longer holds) starts its workflows again — see reference/run-store.md for what that index does and does not hold. Losing restate-data loses in-flight and suspended runs outright.

Nothing else needs a backup procedure:

Workflow, agent, and instance-manifest YAML are versioned in git — the clone/checkout you deploy from is already the backup.
The plugin cache (MERIDIAN_PLUGIN_CACHE, default .data/plugins) only holds copies re-downloadable from the registry at next boot — see Set up a plugin registry. Losing it costs a re-download, not data.
The patient record lives in the hospital's FHIR warehouse, connected per Connect a FHIR warehouse — it is outside this platform's backup responsibility.

Back up Restate's durable state

Restate persists its journal to the path mounted at /restate-data, backed by the restate-data volume in docker-compose.yml. Snapshot that volume with Restate stopped, so nothing is mid-write:

1. Stop Restate

docker compose stop restate

Stopping only the restate service is enough — endpoint and proxy will simply fail their calls to it until it comes back, they don't need to be stopped to take a consistent snapshot of restate-data.

2. Snapshot the volume

docker run --rm \
  -v restate-data:/data:ro \
  -v "$(pwd)":/backup \
  alpine tar czf /backup/restate-data-$(date +%Y%m%d-%H%M%S).tar.gz -C /data .

Store the resulting archive wherever your other backups go (off-host, per your retention policy) — there is nothing Meridian-specific about handling this tarball.

3. Restart Restate

docker compose start restate

Back up the run index

SQLite (default)

The proxy's proxy-data volume holds the SQLite file at the container path configured by RUN_DB_URL (default sqlite:/data/runs.sqlite in compose, sqlite:.data/runs.sqlite for a bare process — see apps/api/src/server/run-store.ts). Copy the file directly; a stopped or running proxy can both be copied since SQLite runs in WAL mode, but stopping the proxy first avoids capturing a WAL file mid-checkpoint:

docker compose stop proxy
docker run --rm -v proxy-data:/data:ro -v "$(pwd)":/backup \
  alpine cp /data/runs.sqlite /backup/runs-$(date +%Y%m%d-%H%M%S).sqlite
docker compose start proxy

Postgres

If you've switched the run index to Postgres (see Use Postgres for the run index), back it up the same way as any other Postgres database in your stack. The index is not the source of truth for run state, but it holds event deduplication, so dump it often enough to cover the overlap window of your sources:

pg_dump "$RUN_DB_URL" -f runs-$(date +%Y%m%d-%H%M%S).sql

The application database is a different matter: besides the definitions and the reception inbox (EAI messages and the records polled by event sources), it holds the workflow memory, which nothing can rebuild. Back it up on the same schedule as any database holding patient data:

pg_dump "$DEFINITION_STORE_URL" -f meridian-$(date +%Y%m%d-%H%M%S).sql

Restore

Restore order matters: Restate first (it's the source of truth), then the processes that depend on it, then the run index — which, if lost, starts empty rather than blocking anything; events redelivered afterwards are then not recognized as duplicates.

1. Restore Restate

docker compose stop restate
docker run --rm -v restate-data:/data -v "$(pwd)":/backup alpine sh -c \
  "rm -rf /data/* && tar xzf /backup/restate-data-<timestamp>.tar.gz -C /data"
docker compose start restate

Keep RESTATE_NODE_NAME identical to what it was when the volume was written. Restate otherwise derives its node name from the container's hostname; a new container gets a new hostname and refuses to start against the restored data (error RT0002) — this is exactly why docker-compose.yml pins RESTATE_NODE_NAME: restate-dev for the dev service instead of leaving it to default. Pin the same fixed name in whatever compose file or manifest you restore into.

2. Restart the endpoint

docker compose restart endpoint

The endpoint self-registers with Restate's admin API on startup with retries (apps/api/src/restate/register.ts, autoRegister) — no manual re-registration step is needed. Restate resumes any run left suspended or in-flight in the restored journal from wherever it was left off; this is the same guarantee an ordinary endpoint restart relies on, see Upgrade the platform.

3. Restart the proxy and UI

docker compose restart proxy ui

4. Restore the run index (optional)

Do this to get old entries back in GET /api/runs and to keep recognizing redelivered events as duplicates — it does not affect any run's actual state:

# SQLite
docker compose stop proxy
docker run --rm -v proxy-data:/data -v "$(pwd)":/backup \
  alpine cp /backup/runs-<timestamp>.sqlite /data/runs.sqlite
docker compose start proxy

# Postgres
psql "$RUN_DB_URL" -f runs-<timestamp>.sql

Verify

Confirm the stack is healthy and the restored journal is actually being served:

curl http://localhost:4000/readyz

/readyz returns 200 only once both the Restate ingress and the run index respond (apps/api/src/server/routes/health.ts); on failure it returns 503 with a { checks } detail naming which one is down. Then confirm a run that existed before the backup is still resolvable:

curl http://localhost:8080/meridianWorkflow/<runId>/getRun   # runId: the run's id in the index, URL-encoded

reference/run-store.md — what the run index contains and why it's separate from Restate's state.
reference/topology.md — the four processes and their volumes.
explanation/why-durable-execution.md — why the journal, not application code, is the source of truth.
Use Postgres for the run index — switching backends before you need to scale the proxy.
Upgrade the platform — the restart order for a routine version upgrade, as opposed to a disaster restore.