Server Configuration

Command Line Options

./atelico-server [OPTIONS]

Option	Default	Description
-p, --port	11434	Port to listen on

Environment Variables

Model & Asset Management

Variable	Description
ATELICO_CACHE_DIR	Local model cache directory (default: ~/Library/Caches/atelico on macOS, ~/.cache/atelico on Linux)
ATELICO_ASSET_STORE_URL	Remote asset store URL (S3, R2, HTTP, local path)
ATELICO_ASSET_ACCESS_KEY	Access key for private S3/R2 stores
ATELICO_ASSET_SECRET_KEY	Secret key for private S3/R2 stores
HF_TOKEN	HuggingFace token for gated models (fallback source)

Proxy Backends

Variable	Description
OPENAI_API_KEY	Enables the openai:: model prefix
OPENAI_BASE_URL	OpenAI endpoint (default: https://api.openai.com/v1)
PROXY_<NAME>_API_KEY	Enables <name>:: prefix for any OpenAI-compatible API
PROXY_<NAME>_BASE_URL	Endpoint for the named proxy

Content Safety

Variable	Description
GUARDRAILS_ENABLED	Enable content filtering (true/false)
GUARDRAILS_POLICY	strict or permissive

Classifiers

Variable	Description
ATELICO_CLASSIFIERS	Comma-separated classifier IDs to load on startup
CLASSIFIER_DIR	Custom classifier directory

Data Logging

Variable	Description
ATELICO_DATA_LOG	Enable request/response logging (true/false)
ATELICO_DATA_LOG_DIR	Output directory for logs
ATELICO_DATA_LOG_SAMPLE_RATE	Sampling rate (0.0 - 1.0)
ATELICO_DATA_LOG_REDACT_SYSTEM	Remove system messages from captured chat requests (true/false)

See Model Interaction Capture for record schema, performance characteristics, SDK configuration, filtering, and fine-tuning preparation.

Inference Tuning

Variable	Description
ATELICO_BONSAI_PREFILL	speed (default) or memory — Bonsai 1-bit models pre-dequantize weights to F16 for fast prefill at the cost of ~170 MB extra RAM (1.7B) or ~2.8 GB (8B). Set memory to skip pre-dequantization.
ATELICO_CUDA_GRAPHS	Set to 0 to disable CUDA graph capture/replay (useful for debugging).
ATELICO_Q4_USE_DMMV	CUDA only. Overrides the direct-dequant mat-vec (DMMV) path for Qwen 3.5 quantized (Q*_K) decode. 1 forces it on, 0 forces it off. Default is auto: on for Ada / Blackwell (sm_89+) GPUs except the 9B model, off on older arches (Turing/Ampere) and for the 9B, where the Q8_1 mat-vec is faster or more accurate.
ATELICO_GEMMA4_MOE_F32	Set to 1 to force the legacy F32-input MoE path on Gemma 4 (Metal). The default F16-input path is ~36% faster at numerically identical output; this override is for users who need bit-identical output to releases ≤ 0.9.0.
ATELICO_NO_CLOCK_LOCK	Set to 1 to disable GPU clock locking on Windows (WDDM performance mitigation).
ATELICO_GPU_ACQUIRE_TIMEOUT_SECS	How long a request waits for the shared GPU before failing with a "GPU busy" error rather than blocking. Default 30; 0 waits forever. Set it below your application's own request timeout, so the engine gives up first and can tell you which section is holding the GPU rather than leaving you with a silent hang. The engine shares the GPU with everything else on the machine — a browser, a video call, the compositor — so under heavy external load a request that can never start should tell you, not hang.
ATELICO_GPU_WATCHDOG_SECS	Threshold (seconds) for the GPU watchdog, which logs which section is holding the device when it is held longer than this. 0 disables.
ATELICO_LEASE_DRAIN	Apple Metal only. The engine drains the GPU when a modality finishes a section, so no modality can hand the GPU on with work still in flight. On by default; set to 0 to disable. Leave it on unless you are A/B testing — with it off, concurrent modalities can corrupt each other's results.
CANDLE_CUDA_MEMPOOL_RELEASE_THRESHOLD	CUDA only. Bytes of freed memory the CUDA pool may hold between requests. Read once, when the first CUDA device is created, so it must be set before startup. Unset, candle hoards everything on a discrete GPU — which also makes nvidia-smi report the card as full when it is not. The SDK's pool_reserve_mb sets this for you; a value you export yourself wins.
ATELICO_PER_FORWARD_LEASE	CUDA only. The engine releases the shared-GPU lease between decode tokens, so other modalities (speech, embeddings, images) interleave with a running generation instead of waiting for it to finish. On by default; set to 0 to hold the GPU for a whole generation instead.
ATELICO_DEVICE_SEED_ISOLATION	Set to 0 to disable per-section RNG-seed save/restore on the shared GPU. On by default — without it, one modality's sampling can perturb another's. An A/B escape hatch, not something to turn off in production.
ATELICO_EMBED_DTYPE	f32 / bf16 / f16 — weight precision for the CLIP/SigLIP, CLAP, EBind, ImageBind and DINOv2 embedding towers. Defaults to bf16 on CUDA GPUs with working BF16 kernels, f32 elsewhere. Does not apply to the text embedders (all-MiniLM-L6-v2, BGE, Granite r2), which always run F32.

:::note Automatic memory budget On CPU and Metal (Apple Silicon unified memory), the engine automatically caps model memory usage at ~65% of total system RAM and evicts least-recently-used models before the budget is exceeded. This is detected at startup with no configuration required; the remaining headroom covers the OS, the application, and the KV cache. :::

Audio

Variable	Description
ATELICO_KOKORO_QUANT	q8_0 or q4_0 — opt-in quantization of Kokoro's linear layers (~12% / ~18% memory saving). Default: F16 on Metal/CUDA, F32 on CPU. Convolutional decoder is unaffected.
ATELICO_TTS_DTYPE	Pocket TTS runtime dtype. bf16 (default on GPU), f32 (default on CPU, parity reference), or f16 (~12% faster on Metal but minor quality drift).
ATELICO_POCKET_TTS_MAX_TOKENS	Maximum SentencePiece tokens per Pocket TTS chunk. Default 50 (the model's training distribution boundary); raising this produces out-of-distribution audio.

Debug

Variable	Description
RUST_LOG	Log level: error, warn, info, debug, trace

Example: Production Configuration

ATELICO_ASSET_STORE_URL=https://models.yourgame.com \
GUARDRAILS_ENABLED=true \
GUARDRAILS_POLICY=strict \
RUST_LOG=info \
./atelico-server --port 11434

Example: Development with OpenAI Fallback

OPENAI_API_KEY=sk-... \
RUST_LOG=debug \
./atelico-server

This lets you use local models (in-memory::...) and OpenAI models (openai::gpt-4o-mini) side by side during development.