Interior · Format choices constrain the runtime
Engines and Formats
This interior page separates the thing that runs inference from the file shape it can load, the executable that exposes it, and the optional features that stay off until the run enables them.
Broken intuition
One artifact on disk does not answer the question "what should run this?" A GGUF exact-file path belongs to llama.cpp, while an MLX snapshot directory belongs to the MLX servers; then the serving command decides whether companions like mmproj or --model-draft are actually enabled for this run. src/coding_agent/serve.clj:9-43 src/coding_agent/serve.clj:45-60
Experiment
Pick an inference engine, then inspect its server binary and see which artifact/package combinations are viable, incomplete, or wrongly addressed.
The table below keeps inference engine, server binary, packaging format, supported feature, configured feature, and request model id visibly separate.
Revealed constraint
A supported feature is not the same thing as a configured feature. The Gemma GGUF artifact can carry vision and MTP companions, but the current run uses them only when the command includes --mmproj, --model-draft, and speculative-decoding flags. Meanwhile, the M1 MLX candidate is text-only and the M3 MLX candidate is text and image, so the request shape and Pi input list also differ by runtime/modality. profiles/m3-max-128gb.edn:28-35 src/coding_agent/serve.clj:9-43 profiles/m1-16gb.edn:17-27 profiles/m3-max-128gb.edn:36-39 src/coding_agent/agent_config.clj:6-9
Named concept
Inference engine: the implementation that loads weights and runs inference.
Server binary: the executable that exposes an API for an engine path.
Packaging format: the artifact shape the engine can load.
Supported feature: something the artifact or engine can do.
Configured feature: something this run actually enables.
Request model id: the value the frontend sends in model after the server is already running.
The useful comparison here is not "which model is better?" but "which engine/server pair can load this artifact shape, which features are merely available, and which are actually turned on for the current run?" src/coding_agent/bench.clj:46-60 src/coding_agent/agent_config.clj:13-21
Repo implementation
The repo already encodes these splits explicitly. llama-server gets a concrete -m GGUF file path and optional --mmproj, --model-draft, and speculative flags; mlx_lm.server and mlx_vlm.server get a snapshot directory through --model. Later, request routing keeps the model id meaningful by sending the MLX repo id for MLX candidates or the resolved GGUF filename for llama.cpp candidates. src/coding_agent/serve.clj:27-43 src/coding_agent/serve.clj:45-60 src/coding_agent/bench.clj:46-60
Recognition
Choose the claim that keeps all six distinctions intact.
Select a claim to see whether it collapses engine, server, artifact, feature, or model-id boundaries.
Sources
src/coding_agent/serve.clj:9-43 src/coding_agent/serve.clj:45-60 src/coding_agent/bench.clj:46-60 src/coding_agent/agent_config.clj:6-21 profiles/m1-16gb.edn:17-27 profiles/m3-max-128gb.edn:28-35 profiles/m3-max-128gb.edn:36-39