Part 3 of 5

From Weights on Disk to API

This page turns the chosen artifact and engine into a callable local API, then shows where readiness and request addressing can still fail.

Broken intuition

Having the bytes on disk still does not make a model callable. The repo chooses the candidate/profile port or runtime default first, refuses to start if that selected port is already occupied, then starts the right server binary against the right artifact shape and waits for the OpenAI-compatible readiness endpoint before routing requests. src/coding_agent/profile.clj:32-36 src/coding_agent/serve.clj:136-166 src/coding_agent/serve.clj:62-66

Experiment

Step through the connection states from "files only" to a ready local server and watch where local API readiness or the request model id can still break.

A callable server here means a local OpenAI-compatible endpoint at http://127.0.0.1:<port>/v1 with a matching model id and a passing /v1/models check.

Revealed constraint

Local API readiness is stricter than "the process exists." The repo first refuses to reuse an occupied port, then starts the server, records the pid, and waits for /v1/models. After that, the frontend still has to send the right request model id: MLX validates the repo id it reports, while llama.cpp uses the resolved GGUF filename as the meaningful request label. src/coding_agent/serve.clj:62-66 src/coding_agent/serve.clj:136-166 src/coding_agent/bench.clj:46-60

Named concept

The useful boundary is a callable local API: one OpenAI-compatible provider entry pointing at 127.0.0.1, one server process that has passed readiness, and one request model id that matches what the server exposes. In Pi, that means a provider with baseUrl set to http://127.0.0.1:<port>/v1, one model entry, the correct input list, and the correct model value for the chosen runtime. src/coding_agent/agent_config.clj:6-47

Repo implementation

The serve layer fixes the host to 127.0.0.1, builds runtime-specific args, and treats /v1/models as the shared readiness surface for llama-server, mlx_lm.server, and mlx_vlm.server. The Pi config layer then writes one OpenAI-compatible provider whose baseUrl points at that local server and whose model entry uses the right request model id plus text-only or text-and-image inputs. src/coding_agent/serve.clj:9-66 src/coding_agent/agent_config.clj:6-47

Recognition

Choose the statement that proves the API is truly callable.

Select a statement to see whether it proves local API readiness.

Sources

src/coding_agent/serve.clj:9-66 src/coding_agent/serve.clj:136-166 src/coding_agent/bench.clj:46-60 src/coding_agent/agent_config.clj:6-47

Why keep readiness separate from request routing?
A server can be up enough to answer /v1/models and still reject requests that use the wrong model id. Keeping those checks separate makes the failure mode easier to recognize.