> ## Documentation Index > Fetch the complete documentation index at: https://iii.dev/docs/llms.txt > Use this file to discover all available pages before exploring further. # Compose > The compose daemon and its worker surface: the compose::* functions, the worker-compose.yaml schema, and the environment every worker is started with. Compose runs a group of workers as one project. The daemon reads a `worker-compose.yaml`, is able to resolve, start, and stop each worker in dependency order, handles engine registration tasks, and supervises running workers. ## The daemon The daemon is itself a worker. It registers under the name `compose` and exposes the `compose::*` functions, so every project operation is a standard [trigger](./triggers). Bare `iii compose` is the command to start the compose worker/daemon. It reads `worker-compose.yaml` in the working directory and starts only the compose worker. `iii compose --up` is provided as a convenience. It starts the compose worker, the iii engine, and the workers specified in `worker-compose.yaml`. It is approximately the equivalent of running `iii`, `iii compose`, and `iii trigger compose::up` as separate commands. `iii compose --up` combines engine and project startup for development. Independent `iii` and `iii compose` daemons provide separate lifecycle control for production deployments. ### Starting a project with the daemon ```text theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii compose [OPTIONS] ``` | Option | Description | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--engine ` | Existing engine WebSocket address. Overrides the compose file and III\_URL. The local default is used when none of them supplies a URL | | `-n, --namespace ` | Namespace this daemon answers `compose::*` in and applies to every project it loads. Several daemons attach to one engine; this is what tells them apart | | `--up` | Serve with one project brought up first, starting its declared engine unless `--engine` selects an existing one | | `--frozen` | With `--up`, require the compose file and existing lock to match and skip package resolution | | `-f, --file ` | The compose file. Only valid with `--up`. Defaults to `./worker-compose.yaml`, the same fallback `compose::up` uses when a call names no file | `Ctrl^C`, `SIGINT` and `SIGTERM` all gracefully stop the daemon, every worker run by the daemon, and the iii engine if compose was started with `--up`. When `--up` has started the engine, every project and worker stops before the engine process is stopped. `compose::stop` is the function equivalent of this operation. `compose::*` functions as documented below are the intended way to manage a running compose daemon. ### Process names on Linux Compose includes its resolved namespace in the process label. The daemon uses `iii:c:`, and an engine started by that daemon uses `iii:e:`. The namespace comes from `--namespace`, then `namespace:` in the compose file, then `default`. For example, `iii compose --namespace orders --up` produces these labels when it starts an engine: ```text theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii:c:orders +-- iii:e:orders ``` Use `ps -p -o pid,ppid,comm,args` to inspect both fields. The `args` field retains the full namespace and the original command arguments. Linux limits `comm` to 15 bytes, so namespaces longer than nine characters use their first two characters, `~`, and six hexadecimal hash characters in that field. Use `args` to read the full namespace when a short label is abbreviated. The daemon sets its command label through an early re-exec with the same PID, before starting the runtime or any children. Existing external engines keep their names. On other operating systems, process names retain their previous behavior. ### Compose logs Compose logs stdout and stderr output from started workers to `logs/` in the state directory. The default state directory is `/.iii/compose//`. When `III_COMPOSE_STATE_DIR` is set, it is `$III_COMPOSE_STATE_DIR///`. The managed engine writes to `engine.log` in that same state directory. Logs are rotated every 10 MiB. Compose keeps up to 40 MiB of logs. Compose strips terminal control sequences before persisting the engine output. You can use the logs client for a recent snapshot or a live view: ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii compose logs # last 100 lines from every worker iii compose logs queue --tail 200 # one worker iii compose logs queue --follow # keep waiting for new output iii compose logs queue --stream stderr # only stderr iii compose logs queue --namespace dev --engine ws://127.0.0.1:49134 ``` | Option | Description | | ---------------------- | -------------------------------------------------------------------------------------- | | `[WORKER]` | Worker to read. Omit to read every worker in the project | | `--engine ` | Existing engine WebSocket address. The compose file and III\_URL are used when omitted | | `-n, --namespace ` | Namespace of the Compose daemon that owns the project | | `-f, --file ` | Compose file path on the daemon host. The daemon's default file is used when omitted | | `--tail ` | Number of recent lines to show before following new output \[default: 100] | | `-F, --follow` | Continue waiting for new output until interrupted | | `--stream ` | Restrict output to one process stream \[possible values: stdout, stderr] | Each line is prefixed with the worker name, and stderr uses a bold prefix on a terminal. ### Running it in the background Compose does not background or daemonize itself. Standard process-management tooling, such as shell backgrounding or systemd, runs the daemon in the background. Bash and zsh support the following invocation: ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii compose --namespace dev --engine ws://127.0.0.1:49134 >> ~/iii-compose.log 2>&1 & ``` Or on a server with systemd support you can use a unit file. This is a basic example: ```ini theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} [Unit] Description=iii compose After=network.target [Service] Type=simple ExecStart=/usr/local/bin/iii compose --namespace prod --engine ws://127.0.0.1:49134 Restart=always [Install] WantedBy=multi-user.target ``` Use `Type=simple`. `Type=notify` waits for an `sd_notify` readiness message, which compose does not send. ## The `compose::*` functions These are the functions that control the compose worker and are the canonical way of interacting with it and making basic changes to the `worker-compose.yaml` file. | Function | Takes | Returns | | ------------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------- | | `compose::up` | `file`, `container`, `frozen` | An operation result. | | `compose::down` | `file`, `container` | An operation result. | | `compose::status` | `file` | The project's namespace, file, state directory, daemon pid, worker states. | | `compose::logs` | `file`, `container`, `cursors`, `tail`, `stream`, `wait_ms` | Bounded stdout/stderr entries and one cursor per worker. | | `compose::list` | nothing | The daemon name, its namespace, its pid, and every project it holds. | | `compose::validate` | `file` | A validation report. | | `compose::add` | `file`, `workers` | What the edit did, changed workers restarted, and new workers started. | | `compose::remove` | `file`, `worker` | The worker removed, its targeted stop, and the idempotent `up`. | | `compose::restart` | `file`, `container` | The `down` and the `up`, or one worker's restart. | | `compose::update` | `file`, `worker` | The resolved version and a restart when the package content changed. | | `compose::stop` | nothing | The daemon name, its pid, and the projects it is about to stop. | | `compose::schema` | `function_id` | Request/response JSON Schemas, descriptions, timeouts, and retry safety. | `file` is not required. Left out, it falls back to a `worker-compose.yaml` in the daemon's own working directory; without one, the call fails with `NO_COMPOSE_FILE`. A relative `file` path is considered relative to the daemon's directory, not the caller's working directory. You may pass an absolute path as well. ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii trigger compose::up --namespace dev file=./worker-compose.yaml iii trigger compose::up --namespace dev file=./worker-compose.yaml frozen=true iii trigger compose::up --namespace dev file=./worker-compose.yaml container=api iii trigger compose::status --namespace dev file=./worker-compose.yaml iii trigger compose::logs --namespace dev file=./worker-compose.yaml worker=api tail=100 iii trigger compose::down --namespace dev file=./worker-compose.yaml iii trigger compose::list --namespace dev iii trigger compose::add --namespace dev file=./worker-compose.yaml worker=database worker=web iii trigger compose::remove --namespace dev file=./worker-compose.yaml worker=state iii trigger compose::restart --namespace dev file=./worker-compose.yaml iii trigger compose::schema --namespace dev function_id=compose::up iii trigger compose::stop --namespace dev ``` ### Starting a project `compose::up` starts every worker in the compose file, in dependency order. Workers that are already ready stay as they are. | Field | Description | | ----------- | -------------------------------------------------------------------------------------------- | | `file` | The project to start. | | `container` | Starts that worker and the workers it depends on, and nothing else. | | `frozen` | Requires a current `worker-compose.lock` and does not resolve package selectors when `true`. | #### compose::up failures A worker is not required by default. If it fails to start, its failure is reported against that worker, nothing is rolled back, and the operation still returns `ok`. Workers that name it in `start_after` start anyway, because `start_after` is a start order and not a claim that the dependent cannot run without it. The response lists every worker that failed this way in `not_required_failures`. A response with `status: ok` has no top-level `error`; that field is present only when the operation returns `status: failed`. Set `required: true` when a worker must fail the operation: ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} containers: database: worker: path://./workers/database required: true ``` A required failure ends the command with `PROJECT_DID_NOT_START`. Partial starts are rolled back in reverse dependency order. Use `required_default` to change the fallback for all containers in the file. An explicit container value wins over the file value: ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} required_default: true containers: queue: worker: package://queue version: "0.21.9" required: false state: worker: package://state version: "0.22.5-rc.1" ``` In this example, `state` inherits `required_default: true`, while `queue` remains false. If both fields are absent, the effective value is false. #### Retrying a worker that fails By default, a failed first start settles immediately. A worker that exits after it was ready takes its dependents down with it and stays down. To retry either case, declare a restart policy: ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} containers: api: worker: path://./workers/api restart: on-failure ``` `no` is the default. `on-failure` retries a failed start or a worker that exits with a non-zero status. `always` also restarts it when it exits successfully after it was ready. The short form uses five attempts, a 500ms base delay, a 30-second maximum delay, and a 60-second stability window. Use the object form to change these values for one worker: ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} containers: api: worker: path://./workers/api restart: condition: on-failure delay: 500ms max_delay: 30s max_attempts: 5 window: 60s ``` | Field | Description | Default | | -------------- | ------------------------------------------------------------------------------- | ---------------------- | | `condition` | Required in the object form. Accepts `no`, `on-failure`, or `always`. | `no` in the short form | | `delay` | Base delay for exponential backoff after a replacement fails. | `500ms` | | `max_delay` | Maximum delay between replacement attempts. | `30s` | | `max_attempts` | Maximum replacement attempts after the original process fails. | `5` | | `window` | Time a ready worker must stay active before its run-time attempt budget resets. | `60s` | The first replacement is immediate. If it fails, later attempts use exponential backoff from `delay` up to `max_delay`. During `up`, the progress row shows the current attempt and wait. For a run-time exit, only the named worker bounces: workers that name it in `start_after` keep running and see their connection drop and reconnect. `compose::status` reports the worker as `restarting` while it waits for a run-time replacement. Once `max_attempts` is spent, `required` controls whether the startup operation fails. At run time, the worker is marked `failed`, its dependents are stopped, and `last_error` says the supervisor gave up. ### Stopping a project `compose::down` stops the project in reverse dependency order. | Field | Description | | ----------- | ---------------------------------------------------------------------- | | `file` | The project to stop. | | `container` | Stops that worker and the workers that depend on it, and nothing else. | ### Adding workers `compose::add worker=state worker=./workers/api` declares one or more workers in the compose file and reconciles the project once. On the CLI, repeat `worker=` for each worker. Each value takes a registry package name (`state`), a package name with a version (`state@0.21.4`), or a directory (`./workers/api`). A JSON payload can mix worker names and container objects in the same list. Each object accepts the container fields from `worker-compose.yaml`. The container key comes from the last part of the worker name or directory path. ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii trigger compose::add --namespace dev file=./worker-compose.yaml --json '{ "workers": [ "database", { "worker": "./workers/api", "start_after": ["database"], "scripts": { "pre_run": "pnpm build", "pre_run_timeout": "60s", "run": "pnpm start", "post_run": "echo stopped" }, "config_name": "api", "config_override": { "port": 3000 }, "working_dir": "./workers/api", "environment": { "NODE_ENV": "development" }, "env_file": ["./api.env"], "startup_timeout": "30s" } ] }' ``` | Field | Description | | --------- | ------------------------------------------------- | | `file` | The project to edit. | | `workers` | A JSON list of worker names or container objects. | | `worker` | One worker. Repeatable on the CLI. | A container object accepts these fields: | Field | Description | | ----------------- | ----------------------------------------------------------------------------------------------------------------------------- | | `worker` | Required. A package name, `name@version`, registry reference, or local path. Also accepts `package://` and `path://` sources. | | `version` | Package version. Must agree with a version included in `worker`. | | `start_after` | Container keys to start first. Compose adds the package's resolved dependencies to this list. | | `scripts` | `pre_run`, `pre_run_timeout`, `run`, and `post_run`. `run` is valid only for local workers. | | `config_name` | Name of the worker's base configuration. | | `config_override` | Configuration value applied over the base configuration. | | `working_dir` | Working directory, relative to the compose file. | | `environment` | Environment variables with string values. | | `env_file` | List of environment file paths, relative to the compose file. | | `startup_timeout` | Maximum wait for worker registration, such as `30s`. | A package without a version pins to the latest available version, for example `0.23.1`. An explicit selector such as `state@next` stays as `next` in `worker-compose.yaml`. Unknown fields and invalid field types are rejected. Compose validates the complete edited file before it writes any changes. For an existing container, omitted fields other than `version` keep their values. A supplied field replaces that entire field, including maps such as `scripts`, `environment`, and `config_override`. Use `{}` or `[]` to clear maps or lists. An omitted `start_after` keeps existing dependencies; a supplied list replaces them and includes any required package dependencies. Workers whose declarations did not change remain running. Existing workers whose source, version, dependencies, or settings changed restart in place. Newly declared workers start through the normal dependency plan. If the resolved declaration already matches the file, the call makes no file changes and causes no restart. ### Package lock Compose writes `worker-compose.lock` beside `worker-compose.yaml`. The compose file keeps the requested selector. The lock keeps the resolved dependency graph, versions, package types, artifact URLs, SHA-256 digests, and default configuration returned by the registry. ```text theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} worker-compose.yaml: next | v worker-compose.lock: 0.22.8 + SHA-256 | v package cache: verified artifact ``` Normal starts and restarts use the lock without resolving `next` again. If the cache is empty, Compose downloads the URL in the lock and verifies its SHA-256 digest. Compose also verifies the extracted cache contents before reuse. A changed cache entry is downloaded again from the locked URL. Use `iii compose build --frozen` in CI to require the compose file and lock to match. Use `iii compose --up --frozen` or `compose::up frozen=true` to apply the same rule during startup. Frozen mode never resolves selectors or changes the lock. It can download a missing artifact only from the URL already in the lock. Commit the lock so development, CI, and deployments use the same package content. ### Removing a worker `compose::remove worker=state` removes the named worker and every reference to it. The changes are validated before a worker is removed. Compose then stops only that worker and runs an idempotent `up`. | Field | Description | | -------- | --------------------- | | `file` | The project to edit. | | `worker` | The worker to remove. | Removal does not resolve the registry graph or remove other workers that were added with this worker. Those remain declared until they are removed explicitly. ### Restarting one worker `compose::restart worker=state` stops that specified worker and starts it again. All other workers, including dependencies, are left unchanged. `compose::restart` without a worker argument restarts the entire compose project. It is approximately the equivalent of `compose::down` followed by `compose::up`. | Field | Description | | -------- | ----------------------- | | `file` | The project to restart. | | `worker` | The worker to restart. | ### Updating workers `compose::update` without `worker` or `workers` updates every declared `package://` worker to its registry's latest version. It keeps each worker's registry reference and skips `path://` workers. ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii trigger compose::update iii trigger compose::update file=worker-compose.yaml ``` `compose::update worker=state` resolves the selector already in the compose file. A tag such as `next`, a range, and an exact version all remain unchanged in the file. Use `worker=state@` to change the selector. For example, use `worker=state@latest` to move an exact version to the registry's latest channel. Update resolves the complete dependency graph. It adds new dependencies, updates changed dependencies, and removes stale dependencies that Compose generated and no remaining package root uses. Manually declared workers are not removed. Compose downloads and verifies the new artifact before it changes the lock or stops a worker. A failed resolve or download leaves the prior lock and running workers unchanged. If the resolved artifact and default configuration did not change, Compose updates lock metadata when needed and does not restart the project. If all selected workers already use the requested versions, the operation leaves the file and running processes unchanged. A project with only `path://` workers also stays unchanged. | Field | Description | | --------- | ----------------------------------------------------------------- | | `file` | The project to edit. | | `worker` | Optional worker spec: `name` or `name@version`. | | `workers` | Optional non-empty list of specs. Takes precedence over `worker`. | ```text theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} worker=state resolve the selector already declared in the compose file worker=state@0.21.4 that version, which is also how a downgrade is spelled worker=state@next the version currently selected by the next tag worker=state@latest the version currently selected by the latest tag ``` The worker has to be declared already in order to be updated, and it has to be a `package://`. Use [`compose::add`](#adding-workers) to add new workers. Workers specified with `path://` are not versioned, any updates to these workers will be reflected the next time the worker is restarted. An update that changes package content or graph topology restarts the whole project once. ### Checking status `compose::status` reports each declared worker with its `state`, the active process `pid` when one exists, an `owned` flag, its rotating `log_path`, and `last_error` when there is one. `owned` is `false` for a worker this daemon has knowledge of but does not manage (ie. was not started by the compose daemon). #### Worker states | State | Meaning | | ------------ | ------------------------------------------------------------------------------------------------ | | `starting` | Spawned. The engine has not registered it yet. | | `ready` | Registered in the engine under `(namespace, container)`. | | `restarting` | Waiting for the next configured retry. No process or PID is active. | | `failed` | Exited unsuccessfully with no eligible retry, exhausted its retries, or one of its hooks failed. | | `stopped` | Stopped by the daemon, or exited successfully without an eligible restart. | ### Viewing logs `compose::logs` returns recent stdout and stderr for the workers of one project. | Field | Description | | --------- | ------------------------------------------------------------------------------------ | | `file` | The project to read. | | `worker` | One worker. Returns every compose-orchestrated worker when omitted. | | `tail` | Recent lines returned when no cursor is sent. Default 100, maximum 1000. | | `cursors` | The `cursor` from the last response, keyed by worker. The call continues from there. | | `stream` | `stdout` or `stderr`. Omit for both. | | `wait_ms` | Wait this many milliseconds for new output. Maximum 5000. | The response holds one entry per worker with `container`, `entries`, `cursor`, and `truncated`. Each entry in `entries` has `stream` and `message`. A `cursor` has `generation` and `offset`. `truncated` is `true` when the cursor sent is older than the retained archives. ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii trigger compose::logs --namespace dev worker=api tail=200 iii trigger compose::logs --namespace dev worker=api stream=stderr wait_ms=5000 ``` ### Listing projects `compose::list` returns the daemon name, its namespace, its pid, and other project information. ### Validating a file `compose::validate` validates a compose file and is intended for package develpment work. It takes `file`. Validation is offline, so `package://` workers are reported under `deferred_packages` and not resolved. #### Validation reports | Field | Type | Description | | ------------------- | ------ | --------------------------------------------------------- | | `namespace` | string | Namespace the project's workers register in. | | `start_order` | array | Worker names in dependency order. | | `deferred_packages` | array | `package://` workers, which need the registry to resolve. | ### Stopping the daemon `compose::stop` stops the compose project, all associated workers, and optionally the engine if started with (`--up`). The compose daemon will also exit. Before exiting the daemon will return its name, pid, and the projects it is about to stop. There is no `compose::start` equivalent to `compose::stop`. Stopping a compose daemon means it must be restarted from the server it is running on. ### Viewing schema `compose::schema` takes a `function_id` argument. With no `function_id`, it returns every `compose::*` schema. Pass a function id to return the schema for a given function\_id. The pseudo-id `worker-compose.yaml` returns the file's JSON Schema as `request` and a complete small example as `response`. Each entry holds `function_id`, `description`, `request`, `response`, `default_timeout_ms`, and `idempotent`. The same schemas, descriptions, and metadata are also published through `engine::functions::info`. ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii trigger compose::schema --namespace dev iii trigger compose::schema --namespace dev function_id=compose::up iii trigger compose::schema --namespace dev function_id=worker-compose.yaml ``` ## Configure a namespace Set the namespace in `worker-compose.yaml` or pass `--namespace` when starting Compose. Use the same namespace in Trigger and Function calls that target the project's workers. For the daemon namespace, project namespace, and routing model, see [Compose architecture](../understanding-iii/compose). ### Precedence | Value | Sets | | ------------------------ | ----------------------------------------------------------------------------- | | `-n, --namespace ` | The daemon namespace, and the project namespace of every project it loads. | | `namespace:` in the file | The daemon namespace when `--namespace` is absent, and the project namespace. | | Neither | `default`. | `namespace` is commonly defined in `worker-compose.yaml` but can be overridden on compose daemon startup with the `--namespace` flag. Likewise, compose's own `compose::*` functions will exist within the same declared namespace. ### What a namespace may hold A valid namespace is made up of the lowercase characters `a-z`, `0-9`, `-` and `_`. All other characters are not permitted and will result in an `INVALID_NAMESPACE` error; including uppercase letters. To prevent naming conflicts there is no coercion of invalid namespaces to valid namespaces. ### One daemon to a namespace Two compose daemons with different namespaces can share an engine. However, a second daemon claiming a namespace that is already served is refused at registration with `DAEMON_ALREADY_SERVING`. Only one Compose daemon can serve a namespace on a iii engine. Set `namespace:` in the compose file or pass `--namespace` when several daemons must share one iii engine. ## `worker-compose.yaml` Below is an example of version 1 of a worker compose file. Unknown keys and duplicate keys are errors. Durations can specify a unit such as: `500ms`, `30s`, `2m`. ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} namespace: shop startup_timeout: 60s stop_timeout: 10s engine: url: ws://127.0.0.1:49134 workers: configuration: adapter: name: fs config: directory: ./config containers: database_1: # Worker names do not need to match package names worker: package://database version: "0.21.4" api: worker: path://./workers/api start_after: [database_1] config_name: shop-api config_override: log_level: debug env_file: [./.env] # specify .envs, later in list overwrites earlier declarations environment: RUST_LOG: info # specify environment variables directly MY_ENV: ${MY_ENV:-defaultValue} # assign an environment variable from the execution environment scripts: pre_run: npm run migrate pre_run_timeout: 2m run: npm start post_run: ./scripts/cleanup.sh state: worker: package://state version: "0.21.4" ``` ### Top-level fields | Field | Type | Default | Description | | ------------------ | ------- | ------- | --------------------------------------------------------------------------------------------- | | `namespace` | string | absent | Namespace the project's workers register in. A project that declares none lands in `default`. | | `startup_timeout` | string | `60s` | Readiness budget for every worker. A worker may override it. | | `stop_timeout` | string | `10s` | Grace between the polite stop and the forced kill. | | `required_default` | boolean | `false` | Fallback for workers that omit `required`. An explicit worker value wins. | | `engine` | map | absent | Present when this Compose invocation owns and configures the engine. | | `containers` | map | empty | Project workers. May be empty only when `engine:` is present. | ### Engine fields | Field | Type | Default | Description | | --------------------------------- | ------- | ---------------------- | ---------------------------------------------------------------------------------------- | | `url` | string | `ws://127.0.0.1:49134` | Managed engine endpoint used by Compose and its workers. | | `registration_namespace_grace_ms` | integer | engine default | Namespace-registration grace passed to the engine. | | `workers` | map | empty | Direct configs for engine-owned workers. Values must be mappings; use `{}` for defaults. | Allowed worker keys are `configuration`, `iii-worker-manager`, `iii-http-functions`, `iii-stream`, and `iii-sandbox`. Use `#instance` for another instance of an allowed type, for example `iii-worker-manager#rbac`. The engine injects `iii-engine-functions`, `iii-telemetry` for anonymous usage analytics, and `iii-observability` for OpenTelemetry traces, metrics, and logs. Do not declare these workers. Changes to these worker configurations take effect only after the engine restarts. #### Which engine Compose uses Compose takes the engine address from the first of these that supplies one: the `--engine` flag, the `III_URL` environment variable, the `engine:` section of the compose file, then `ws://127.0.0.1:49134`. An address from `--engine` or `III_URL` names an engine that already runs, so Compose connects to it and starts no engine of its own, even with `--up`. Only the compose file can give Compose an engine to own, and only `--up` takes it: an `engine:` section carries the engine's whole configuration, not only an address. ### Worker fields Each key under `containers` is the name the worker registers under. | Field | Type | Default | Description | | ----------------- | -------------- | -------------------- | ---------------------------------------------------------------------------------------------------------- | | `worker` | string | required | `path://` or `package://`. A package may name its registry: `package:///`. | | `version` | string | absent | Version range. Required for `package://`. | | `start_after` | array | empty | Workers that must start first (ie. a worker dependency). Self-dependencies and cycles are rejected. | | `config_name` | string | absent | The [configuration worker](./configuration) entry this worker owns. | | `config_override` | mapping | absent | Merged on top of the fetched configuration; a mapping whose `name` changes is replaced whole. | | `working_dir` | path | the worker directory | Resolved against the compose file's directory. | | `environment` | map | empty | Environment variables for this worker. | | `env_file` | array of paths | empty | Read at start time, in declaration order. A later file wins on conflicting entries. | | `startup_timeout` | string | the file's value | Readiness budget for this worker. | | `required` | boolean | `false` | Whether a failed start fails the operation. `false` reports the failure and lets the operation carry on. | | `restart` | string | `no` | What happens after a failed start or a run-time exit. `no`, `on-failure`, or `always`. | | `scripts` | mapping | absent | See below. | `path://` directories resolve against the compose file's directory. A missing directory fails with `MISSING_WORKER_DIRECTORY`, and a missing `env_file` fails with `MISSING_ENV_FILE` during validation. A path worker normally runs as a host process. A non-empty `runtime.base_image` in its `iii.worker.yaml` selects a local VM instead. The worker's `scripts.install` and start command then run inside that image, and compose keeps the VM state inside the project. An invalid image reference fails the start instead of falling back to the host or to another image. ### Worker kinds Worker packages can be released in multiple different "kinds". A kind compose cannot run fails with `UNSUPPORTED_PACKAGE_KIND`. | Kind | How it runs | | -------- | ------------------------------------------------------------------------------- | | `binary` | A child process on the host. | | `bundle` | A VM. The start command is the bundle's own `scripts.start`, read in the guest. | | `path` | A path to a worker stored locally on disk. | A bundle's VM is booted by `iii-worker`, which the installer ships beside `iii` and which needs glibc on Linux. Compose runs it as a process rather than linking it, so the engine stays portable; a bundle worker on a machine without `iii-worker` fails saying so, and every other worker kind is unaffected. Bundles need a VM, and windows has none: a bundle worker there fails with `BUNDLE_NEEDS_A_VM` before anything is downloaded. Run compose under WSL, where the VM has KVM to run on. Every other worker kind runs on windows as it always has. Bundle support can be refused machine-wide with `III_BUNDLE_WORKERS_DISABLED=1`, which compose honours. ### Scripts | Field | Type | Default | Description | | ----------------- | ------ | ------- | --------------------------------------------------- | | `pre_run` | string | absent | Runs to completion before the worker is spawned. | | `pre_run_timeout` | string | `60s` | Budget for `pre_run`. Rejected without a `pre_run`. | | `run` | string | absent | Start command. Rejected for `package://` workers. | | `post_run` | string | absent | Runs after the worker's exit is confirmed. | Both hooks run with the worker's environment, working directory, and their own process group. A `post_run` runs after the worker stops but before the compose daemon exits. The top-level `stop_timeout` argument is a global timer for a compose daemon to stop. If this time is exceeded all scripts will be exited along with the compose daemon. A path worker's start command is `scripts.run` in `worker-compose.yaml`, or `scripts.start` in the worker's own `iii.worker.yaml` when `run` is absent. A worker with neither fails with `MISSING_START_COMMAND`. ### Configuration precedence Lowest to highest: the configuration a package ships, the entry in the configuration worker, then `config_override`. Maps merge key by key; arrays and scalars replace. A mapping whose `name` the override changes is replaced whole: the keys beside `name` belong to the variant it picks. The merged result is delivered through `configuration::set` with `flush: false` before the worker starts. `III_CONFIG_NAME` identifies the entry read through `configuration::get`. Compose does not create an execution snapshot file or set `III_CONFIG`. The persistent `config/.yaml` remains unchanged by this memory-only update. `configuration::ensure` seeds defaults only into the base and preserves the active value. `configuration::set` with `flush: true` (the default) persists the complete submitted object, including values originally supplied by an override. Both forms notify consumers. Each start reads the current configuration with `configuration::get` and `raw: true`, then applies the declared override. Removing a field from the override keeps its current value; it does not restore the value from disk. Stopping a worker leaves the active value available. Restarting the configuration service discards unsaved memory values and reloads its adapter. Read or update failures stop startup with `CONFIG_FETCH_FAILED`, rather than silently falling back to defaults. ### Readable configuration names and migration Without an explicit `config_name`, the entry id is exactly `-`. For example, `default` plus `harness` uses `default-harness` and the filesystem adapter stores `config/default-harness.yaml`. The id must match `[a-z0-9_-]{1,64}`. Compose rejects an invalid or long generated name with `INVALID_CONFIG_NAME` and asks for an explicit `config_name`; it never sanitizes, truncates, or adds a hash. Explicit names remain unchanged and are never auto-migrated. Before reading configuration or starting the child, Compose asks the configuration authority to migrate the exact hashed id produced by the previous algorithm for that namespace and key. `default-harness-a14f3656efb8d5ea` therefore becomes `default-harness`. Stored raw values (including `${VAR}`, `false`, `0`, and `null`), name, description, metadata, and available schema are preserved. The filesystem adapter updates both filename and internal id, and the authority updates its caches and notifies subscribers. It re-reads the source so manual edits awaiting the watcher are retained. Migration gives the legacy source priority over an existing destination. After that, the `default` namespace adopts the exact bare container key (`state` becomes `default-state`), even when the destination already exists. The bare source replaces the whole destination entry, preserving raw values and metadata rather than merging defaults. Another container's explicit ownership blocks this adoption; other namespaces and explicit `config_name` values never adopt bare entries. After publishing the destination, the fs adapter archives the original source as `.yaml.bak` (for example, `state.yaml.bak`). The previous destination is replaced, not backed up. Files ending in `.yaml.bak`, `.bak.yaml`, or `.bkup.yaml` are ignored during loading, watching, and legacy directory migration. Repeated starts with no source perform no writes. An existing identical backup permits recovery after interrupted cleanup; a conflicting backup is never overwritten and stops migration with an error. Migration commits the complete destination before archiving the source. I/O failures stop startup; failure after publication can leave two recoverable copies. The filesystem adapter requires same-directory hard-link support and may normalize YAML formatting or remove comments on the one migration rewrite; values and unknown document fields are retained. Stop source consumers before migration: Compose checks that the child is not already registered, and its restart path stops the old child first. Do not run two configuration authorities against the same directory or edit the source concurrently with migration. The bridge delegates migration to the remote authority. Upgrade that authority together with Compose: an absent `configuration::migrate` or unsupported adapter fails with `CONFIG_MIGRATION_FAILED`, never a read/copy/delete fallback that could reset stored values. Within one project, a generated name colliding with another container's explicit name is rejected before startup. Sharing is allowed only when both names are explicit. Across namespaces, `a-b` / `c` and `a` / `b-c` both produce `a-b-c`; choose unambiguous namespaces or distinct explicit `config_name` values. Readable names do not claim cross-project ownership. ## The worker environment A worker's environment combines the following sources, from lowest to highest precedence: 1. The machine environment visible to the Compose daemon. Variables are inherited even when neither `env_file` nor `environment` is declared. Non-Unicode names and values are skipped. 2. The worker's `env_file` entries, in declaration order. Later files override earlier files. 3. The worker's `environment` map. Nonempty values override env files; an empty string preserves an existing env-file value, or supplies an empty value when no env file defines the key. 4. The reserved variables, which the daemon owns. Export machine variables before starting the daemon. Changing another shell's environment does not update an already-running daemon. Workers and hooks inherit the daemon's Unicode machine variables, including credentials, subject to the daemon-owned values above and the project identity exception below; only start workers you trust with that environment. `III_HOST_USER_ID` is not inherited from the machine: it comes from the current project's `.iii/project.ini`, or is absent when the project has no device ID. An explicit `env_file` or `environment` value can still override it, including an empty value. | Variable | Value | | ----------------------- | ------------------------------------------------------------------------------- | | `III_URL` | The engine address the daemon is connected to. | | `III_NAMESPACE` | The project's namespace. | | `III_COMPOSE_NAMESPACE` | The supervising Compose daemon's namespace for explicit `compose::*` routing. | | `III_COMPOSE_FILE` | Canonical path of the compose file that owns this worker. | | `III_COMPOSE_DIR` | Canonical directory that contains the owning compose file. | | `III_WORKER_NAME` | The key under `containers`. | | `III_CONFIG_NAME` | The explicit configuration id or `-`, always present. | Declaring a reserved variable in `environment` or an `env_file` fails with `RESERVED_ENV_OVERRIDE`. ### Expanding values from environment variables Variables can be expanded with `${VAR}`. References expand in any value, not only in `environment`. For example: ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} containers: queue: worker: path://${WORKERS_DIR}/queue environment: RUST_LOG: ${RUST_LOG:-info} ``` | Written | Means | | -------------- | ----------------------------------------------------------------------------------------------- | | `${VAR}` | The value from compose's own environment. Unset, the file is refused with `UNDEFINED_VARIABLE`. | | `${VAR:-text}` | The value, or `text` when it is unset. `${VAR:-}` makes it optional and empty. | | `$VAR` | Nothing. A bare name is left alone, so a `scripts.run` holding `$PWD` still reaches the shell. | | `$${VAR}` | A literal `${VAR}`. | `config_override` is never expanded, all values are treated as literals. ```yaml theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} config_override: # Reaches the worker exactly as written. api_key: ${ANTHROPIC_API_KEY} ``` ## Build registry packages ```text theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii compose build [-f, --file ] [--frozen] ``` | Option | Description | | ------------------- | ----------------------------------------------------------------------------------------- | | `-f, --file ` | Compose file whose registry packages should be downloaded \[default: worker-compose.yaml] | | `--frozen` | Require an existing current lock and skip package selector resolution | `build` reads and validates the compose file, prepares `worker-compose.lock`, then downloads every `package://` worker into the same cache used by `compose::up`. The file defaults to `./worker-compose.yaml`. The command does not connect to an engine, start a worker, or run lifecycle hooks. Local `path://` workers and engine-managed workers need no registry download and are skipped. ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii compose build --file worker-compose.yaml iii compose build --file worker-compose.yaml --frozen iii compose --up --file worker-compose.yaml --frozen ``` The cache is shared under `~/.iii/compose/packages`, or under `$III_COMPOSE_STATE_DIR/packages` when that variable is set. A later `compose::up` reuses a valid cached artifact if it exists. ## Readiness A worker is considered up when the engine reports a worker of that name in the project's namespace. Compose polls `engine::workers::list` every 200 ms until the worker's `startup_timeout` runs out. | Outcome | Code | | -------------------------------------------------------------- | ---------------------------------- | | The worker never appeared. | `STARTUP_TIMEOUT` | | The process exited while compose was waiting. | `CHILD_EXITED_BEFORE_REGISTRATION` | | It registered in `default` instead of the project's namespace. | `WORKER_IGNORED_NAMESPACE` | | It registered under a different name in the right namespace. | `WORKER_NAME_MISMATCH` | | Its functions landed outside the project's namespace. | `FUNCTIONS_IN_WRONG_NAMESPACE` | | A worker already held that name in the namespace. | `CONTAINER_NAME_TAKEN` | After a worker is ready, the daemon checks it every 250 ms. Its restart policy determines what happens after it exits. An eligible retry keeps its dependents running and reports `restarting` between attempts. A successful exit without an eligible retry is recorded as `stopped`. An unsuccessful exit without an eligible retry, or one that exhausts its retries, is recorded as `failed`. When no retry remains, its transitive dependents stop. When the engine connection drops and comes back, every running worker gets its `startup_timeout` to register again. Dependency shutdown is dependent upon when a shutdown happens: | When | What comes down | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | During `up` | The operation ends. Everything it started is stopped in reverse order, and workers later in the start order are never attempted. | | After the worker is ready | Its transitive dependents. Everything else keeps running. | ## Where compose keeps state Compose state is stored in `/.iii/compose//`. `` is the directory containing the canonical compose file, including when `--file` points outside the current working directory or follows a symbolic link. `` is the daemon's namespace. | Path | Contents | | -------------------- | ---------------------------------------------------------------------------------------------- | | `engine.lock` | Lock for the managed engine in this project and namespace. | | `engine-config.yaml` | Generated engine configuration, removed after clean shutdown. | | `engine.log` | Rotating stdout and stderr for the managed engine. | | `state.json` | Child process records for the project. Owner-only. | | `config/` | Resolved worker configuration files. | | `logs/` | Rotating stdout and stderr for project workers. | | `vm/` | VM state for bundle and local-image workers, including rootfs, boot scripts and guest configs. | Two projects can use `default` with separate engines on different ports. Starting the same project and namespace twice is refused. Two daemons on the same engine must use different namespaces. With the default state layout, two compose files in the same directory must also use different namespaces. For a read-only project directory, set `III_COMPOSE_STATE_DIR` to a writable directory. Project state is then stored at `$III_COMPOSE_STATE_DIR///`. The slug combines the project directory name with a hash of the canonical compose file path, keeping projects with the same directory name separate. With this override, two compose files in the same directory have separate state directories and can use the same namespace on separate engines. The package cache stays shared at `~/.iii/compose/packages`, or `$III_COMPOSE_STATE_DIR/packages`. Add `**/.iii/compose/` to the repository's `.gitignore` to exclude generated state, including state from nested projects, from version control. ```bash theme={"theme":{"light":"catppuccin-latte","dark":"dark-plus"}} iii trigger compose::status --namespace dev file=./worker-compose.yaml # ... "state_dir": "/home/you/shop/.iii/compose/dev" ls /home/you/shop/.iii/compose/dev/logs/ ``` Before upgrading from the shared namespace layout, stop existing Compose daemons with the old version and confirm that their engines and workers have stopped. The new version does not migrate old process records or logs. Previous state remains under `~/.iii/compose//` (or the old `III_COMPOSE_STATE_DIR` layout); keep any logs or VM data you need before removing it. ## Error codes Compose can output the following error codes: | Area | Codes | | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Compose file | `COMPOSE_FILE_UNREADABLE`, `INVALID_COMPOSE_FILE`, `EMPTY_CONTAINERS`, `INVALID_DURATION`, `UNKNOWN_DEPENDENCY`, `SELF_DEPENDENCY`, `DEPENDENCY_CYCLE`, `UNSUPPORTED_WORKER_SOURCE`, `UNDEFINED_VARIABLE`, `INVALID_REFERENCE`, `UNTERMINATED_REFERENCE` | | Worker declaration | `MISSING_VERSION_FOR_PACKAGE`, `RUN_NOT_ALLOWED_FOR_PACKAGE`, `NOT_A_PACKAGE_CONTAINER`, `PRE_RUN_TIMEOUT_WITHOUT_PRE_RUN`, `RESERVED_ENV_OVERRIDE`, `MISSING_ENV_FILE` | | Worker resolution | `MISSING_WORKER_DIRECTORY`, `MISSING_START_COMMAND`, `INVALID_MANIFEST`, `INVALID_WORKER_SPEC`, `WORKER_SOURCE_CHANGED`, `ENGINE_WORKER_IS_BUILTIN` | | Packages | `REGISTRY_UNREACHABLE`, `PACKAGE_NOT_RESOLVED`, `PACKAGE_NOT_INSTALLED`, `PACKAGE_DOWNLOAD_FAILED`, `PACKAGE_DIGEST_MISMATCH`, `PACKAGE_ARTIFACT_EMPTY`, `REGISTRY_NAME_REFUSED`, `UNSUPPORTED_PACKAGE_KIND`, `UNSUPPORTED_PLATFORM`, `BUNDLE_NEEDS_A_VM` | | Start and readiness | `SPAWN_FAILED`, `HOOK_SPAWN_FAILED`, `HOOK_FAILED`, `HOOK_TIMEOUT`, `STARTUP_TIMEOUT`, `CHILD_EXITED_BEFORE_REGISTRATION`, `WORKER_IGNORED_NAMESPACE`, `WORKER_NAME_MISMATCH`, `FUNCTIONS_IN_WRONG_NAMESPACE`, `CONTAINER_NAME_TAKEN`, `CONFIG_FETCH_FAILED`, `CONFIG_PUBLISH_FAILED`, `ENGINE_CALL_FAILED`, `PROJECT_DID_NOT_START` | | Managed engine | `ENGINE_SECTION_REQUIRES_MANAGED_START`, `ENGINE_ALREADY_OWNED`, `ENGINE_RESTART_REQUIRED`, `ENGINE_WORKER_IS_INJECTED`, `UNSUPPORTED_ENGINE_WORKER`, `INVALID_ENGINE_WORKER_CONFIG`, `INVALID_MANAGED_ENGINE_URL`, `MANAGED_ENGINE_ENDPOINT_MISMATCH`, `MANAGED_ENGINE_LISTENER_UNAVAILABLE`, `ENGINE_SPAWN_FAILED`, `ENGINE_STARTUP_TIMEOUT`, `ENGINE_EXITED` | | Daemon and project | `NO_COMPOSE_FILE`, `WRONG_DAEMON`, `INVALID_NAMESPACE`, `UNKNOWN_CONTAINER`, `UNKNOWN_PROJECT`, `INVALID_STATE_FILE`, `STATE_DIR_UNAVAILABLE`, `DAEMON_ALREADY_SERVING`, `DAEMON_NAMESPACE_TAKEN`, `IO_ERROR` | | Command line | `FILE_REQUIRES_UP`, `BUILD_CONFLICTS_WITH_SERVE_OPTIONS` | These are literal diagnostic identifiers emitted by Compose, not worker categories. `ENGINE_WORKER_IS_BUILTIN` reports that a worker declared under `containers` is already supplied by the engine. Its name is retained for compatibility; it does not define a separate worker type. All workers follow the same Function/Trigger/Worker model. ## Related For why compose is a worker see [Understanding iii / Compose](../understanding-iii/compose). For understanding namespaces see [Understanding iii / Namespaces](../understanding-iii/namespaces). For the `iii compose` entry in the command tree, see the [CLI reference](../cli-reference/index#iii-compose).