Unity API Reference

AtelicoSpeechListener

Microphone-to-transcription helper: records from a microphone and raises speech and transcription events on the main thread, so a game never handles capture, resampling or voice-activity detection itself.

Add to any GameObject in a scene with an AtelicoEngine, subscribe to the events, then call StartListening. Uses UnityEngine.Microphone (the app needs microphone permission on mobile). Events fire from Update.

Methods

bool StartListening(string model)

Start recording and transcribing with the given Whisper model.

• model: Whisper model id, e.g. "in-memory::whisper".

Returns: True once recording has started.

void StopListening()

Stop recording. The utterance in progress is still transcribed and its events still fire over the next frames.

AtelicoEngine

Main Atelico AI Engine singleton. Add to a GameObject to initialize the engine. Persists across scene loads via DontDestroyOnLoad. Provides access to all AI subsystems (LLM, images, embeddings, classifiers, guardrails, LoRA, KV store, ANN) through typed accessor properties.

Properties

Property	Type	Description
Instance	AtelicoEngine	Global singleton instance. Set during Awake and cleared on destroy. Returns null if no AtelicoEngine exists in the scene.
Llm	AtelicoLlm	LLM subsystem for chat completions, text completions, the Responses API, and templated LM function calls. Null if the engine failed to initialize.
Images	AtelicoImages	Image generation subsystem for text-to-image generation and background removal. Null if the engine failed to initialize.
Audio	AtelicoAudio	Audio subsystem — text-to-speech (Kokoro / Pocket TTS) and speech-to-text (Whisper). Null if the engine failed to initialize.
AudioToFace	AtelicoAudioToFace	Audio-to-Face subsystem — converts speech audio into per-frame facial blendshape weights for lip-sync. Null if the engine failed to initialize.
Embeddings	AtelicoEmbeddings	Embedding subsystem for generating text embeddings and computing semantic similarity. Null if the engine failed to initialize.
Classifiers	AtelicoClassifiers	Classifier subsystem for text, image and audio classification (centroid, KNN/HNSW, SetFit, or the embedder-free n-gram perceptron). Null if the engine failed to initialize.
Guardrails	AtelicoGuardrails	Guardrails subsystem for content safety checking of user inputs, model outputs, and image generation prompts. Null if the engine failed to initialize.
Lora	AtelicoLora	LoRA adapter management subsystem for loading, unloading, and scaling adapters on top of loaded base models. Null if the engine failed to initialize.
KvStore	AtelicoKvStore	Key-value store subsystem with embedding-backed semantic queries for storing and searching game data (NPC dialogue, lore, inventory). Null if the engine failed to initialize.
Ann	AtelicoAnn	Approximate Nearest Neighbor (ANN) index subsystem for vector search, backed by HNSW. Pure data structure that operates on pre-computed vectors. Null if the engine failed to initialize.

Methods

void RegisterStream(ulong streamId, StreamCallbacks callbacks)

Registers a stream for per-frame polling in Update. Called by subsystems (e.g., AtelicoLlm.ChatCompletionStream) after starting a streaming operation. Not intended for direct use by game code.

• streamId: Native stream handle returned by the C ABI.

• callbacks: Callback set invoked as stream data arrives, completes, or errors.

void SetSchedulingMode(SchedulingMode mode)

Set the GPU scheduling mode to control how rendering and inference share GPU time. Takes effect immediately for subsequent inference operations. PrioritizeCompute minimizes AI latency; Balance is the default; PrioritizeGraphics maximizes FPS.

• mode: The desired scheduling priority balance (PrioritizeCompute, Balance, or PrioritizeGraphics).

SchedulingMode GetSchedulingMode()

Get the current GPU scheduling mode. Returns Balance if the engine is not initialized.

Returns: The active SchedulingMode value (PrioritizeCompute, Balance, or PrioritizeGraphics).

void SetVramBudgetMb(uint mb)

Set the maximum VRAM budget in megabytes for AI model storage. The engine will avoid loading models that would exceed this limit.

• mb: VRAM budget in MB. Use 0 for unlimited.

void SetTargetTps(uint tps)

Set the target tokens-per-second rate for inference pacing. The engine will yield GPU time between tokens to stay near this rate, freeing GPU cycles for rendering.

• tps: Target tokens per second. Use 0 for unlimited (fastest possible).

void SetFrameTimeMs(uint ms)

Set the target frame time in milliseconds for frame-budget-aware scheduling. The engine uses this to decide how much GPU work to perform per frame.

• ms: Target frame time in milliseconds. For example, 16 for 60fps, 33 for 30fps. Use 0 to disable frame budgeting.

bool LoadModel(string modelId)

Load a model synchronously. Blocks the calling thread until the model is fully loaded into memory and ready for inference. The model will be downloaded from HuggingFace Hub if not already cached locally.

• modelId: Model identifier, typically in the format "in-memory::org/model-name" (e.g., "in-memory::meta-llama/Llama-3.2-1B-Instruct-GGUF").

Returns: True if the model loaded successfully; false on error (e.g., invalid model ID, download failure, insufficient VRAM).

bool UnloadModel(string modelId)

Unload a model, freeing its GPU and system memory. Any in-flight inference using this model will be cancelled.

• modelId: Model identifier previously passed to LoadModel.

Returns: True if the model was found and unloaded; false if the model was not loaded or on error.

string ListModelsJson()

List all currently loaded models. Returns a JSON array of model descriptors. Returns "[]" if no models are loaded or on error.

Response JSON schema:

[
  {
    "id": "in-memory::meta-llama/Llama-3.2-1B-Instruct-GGUF",
    "type": "llm",
    "size_bytes": 1234567890
  }
]

• id (string): The model identifier.
• type (string): Model type ("llm", "embedding", "image", "classifier").
• size_bytes (int): Approximate memory usage in bytes.

Returns: JSON array string of model descriptor objects (see Remarks for schema).

bool IsLoaded(string modelId)

Check whether a model is currently loaded and ready for inference.

• modelId: Model identifier to check.

Returns: True if the model is loaded and ready; false otherwise.

AtelicoJson

Stateless JSON helpers. No engine instance required.

Methods

string RepairJson(string input, string defaultJson = null)

Repair a possibly-truncated or malformed JSON string.

The common case is a structured-generation response cut off by a token limit (finish_reason == "length"), e.g. {"a": 1, "b": "hel — valid up to the cut but unparseable as a whole. This completes it best-effort: complete fields are kept, a truncated string value is closed and kept, a dangling key with no value is dropped, and open objects/arrays are closed.

Caveat: a value truncated mid-token is kept as-is. Repair is a safety net — prefer checking finish_reason and raising max_tokens.

• input: The JSON string to repair.

• defaultJson: Optional JSON string whose fields fill any missing/null fields in the result (fields present in win). Pass null for no default.

Returns: The repaired JSON string.

AtelicoAnn

Approximate Nearest Neighbor (ANN) index for vector search, backed by HNSW. Pure data structure that operates on pre-computed vectors -- no GPU or models required. Access via AtelicoEngine.Ann.

Methods

ulong Create(string configJson)

Create a new ANN index with the specified vector dimensionality and capacity.

Configuration JSON schema:

{
  "dim": 384,
  "max_elements": 10000,
  "m": 16,
  "ef_construction": 200,
  "ef_search": 50
}

• dim (int, required): Vector dimensionality (must match inserted vectors).

• max_elements (int, required): Maximum number of vectors the index can hold.

• m (int): HNSW connections per node (default: 16). Higher values improve recall at the cost of memory.

• ef_construction (int): Build-time search width (default: 200). Higher values improve index quality.

• ef_search (int): Query-time search width (default: 50). Higher values improve recall at query time.

• configJson: JSON string containing the index configuration (see Remarks for schema).

Returns: Index handle (ulong) for use with other ANN methods, or 0 on error.

bool Build(ulong indexId)

Build the HNSW index graph. Must be called after all insertions and before any searches. This operation is O(n * log(n)) and may take noticeable time for large indices.

• indexId: Index handle returned by Create.

Returns: True on success; false on error.

bool Destroy(ulong indexId)

Destroy an ANN index and free all associated memory.

• indexId: Index handle returned by Create.

Returns: True if the index was found and destroyed; false otherwise.

AtelicoAudio

Audio subsystem providing text-to-speech (TTS) via Kokoro / Pocket TTS and speech-to-text (STT) via Whisper. Access via AtelicoEngine.Audio.

Methods

string Synthesize(string requestJson)

Synthesize speech from text synchronously (blocking).

Request JSON schema:

{
  "model": "in-memory::tts",
  "input": "Hello from Atelico.",
  "voice": "af_heart",
  "speed": 1.0
}

• model (string, required): Model id, e.g. "in-memory::tts", "in-memory::kokoro-82m", or "in-memory::pocket-tts". Bare ids accepted after the in-memory:: prefix: tts (default → kokoro-82m), kokoro, kokoro-82m, pocket, pocket-tts.
• input (string, required): Text to synthesize.
• voice (string): Voice id (default "af_heart"). Pocket TTS ships 31 built-in English voices (alba, ian, morgan, kate, …).
• speed (float): Speech speed multiplier 0.25–4.0 (default 1.0).

Response JSON schema:

{
  "audio_b64": "UklGRn...",
  "duration_seconds": 1.42,
  "format": "wav",
  "sample_rate": 24000
}

audio_b64 decodes to a complete WAV file (RIFF + PCM).

• requestJson: JSON string with the speech request (see Remarks for schema).

Returns: JSON string with base64-encoded WAV audio, or null on error.

string Transcribe(string requestJson)

Transcribe audio to text synchronously (blocking).

Request JSON schema:

{
  "model": "in-memory::whisper",
  "audio_b64": "UklGRn...",
  "language": "en"
}

• model (string, required): Model id, e.g. "in-memory::whisper", "in-memory::whisper-large-v3-turbo", "in-memory::distil-large-v3". Default whisper resolves to whisper-base.en.
• audio_b64 (string, required): Base64-encoded WAV file (RIFF + PCM). The native layer decodes the WAV header to obtain sample rate.
• language (string): ISO 639-1 code ("en", "ja", …) or omit for auto-detect.
• response_format (string): "json" (default) or "verbose_json" (adds segments).
• temperature (float): Decoder sampling temperature (0.0 = greedy).
• timestamp_granularities (string[]): ["segment"] and/or ["word"].

Response JSON schema:

{
  "text": "Hello from Atelico.",
  "language": "en",
  "duration": 1.4
}

• requestJson: JSON string with the transcription request (see Remarks for schema).

Returns: JSON string with the transcribed text, or null on error.

AtelicoTranscriptionStream StartTranscriptionStream(string configJson)

Open a live transcription stream for microphone audio. Feed samples as they arrive and poll speech / transcription events; utterances are found by voice-activity detection and transcribed through the engine's Whisper as they end. For a ready-made microphone hookup use AtelicoSpeechListener.

• configJson: JSON TranscriptionStreamConfig: \{"model": "in-memory::whisper", "endpointing": "command" | "dialogue" | "dictation", "energy_threshold": 0.02, "language": "en", "partial_interval_ms": 700\} — only model is required.

Returns: The stream (dispose it when done), or null on error (see the log).

AtelicoAudioToFace

Audio-to-Face (A2F) subsystem that turns speech audio into per-frame facial blendshape weights for lip-sync and facial animation. Drives a 52-weight ARKit-style blendshape rig from raw PCM audio. Access via AtelicoEngine.AudioToFace.

A2F owns its own native handle, created lazily on the first call to Process and destroyed when the owning engine is torn down.

Methods

string Process(float[] pcm, int sampleRate, string character, ulong? noiseSeed = null)

Process speech PCM audio into per-frame facial blendshape weights (blocking). Lazily creates the underlying native A2F instance on first use.

Response JSON schema:

[
  {
    "time_seconds": 0.0,
    "weights": [0.0, 0.12, ...]
  },
  {
    "time_seconds": 0.033,
    "weights": [0.0, 0.18, ...]
  }
]

• time_seconds (float): Timestamp of the frame relative to the start of the clip.
• weights (float[52]): ARKit-style blendshape weights for the frame, in the range 0.0 to 1.0.

Apply each frame's weights to the matching blendshapes of a SkinnedMeshRenderer at the corresponding time_seconds to drive lip-sync.

• pcm: Mono PCM audio samples in the range -1.0 to 1.0.

• sampleRate: Sample rate of in Hz (e.g. 16000).

• character: Character model to animate: "claire" (0), "james" (1), or "mark" (2). Case-insensitive; unknown values fall back to "claire".

• noiseSeed: Optional deterministic noise seed. Pass a value for reproducible animation, or leave null for random per-call noise.

Returns: JSON array string of animation frames, or null on error (see Remarks for schema).

void Dispose()

Destroy the native A2F handle. Called automatically when the owning engine is torn down; safe to call multiple times.

AtelicoClassifiers

Classifier subsystem for text, image and audio classification. Supports centroid, KNN/HNSW, SetFit, or the embedder-free n-gram perceptron. The n-gram perceptron has no embedder, so it loads offline from a bundled directory with no model download; the others need their encoder weights. Access via AtelicoEngine.Classifiers.

Methods

bool Load(string classifierId, string directory)

Load a classifier model from a filesystem directory (blocking).

• classifierId: Unique name to register the classifier under, used in subsequent PredictText/PredictImage/PredictAudio calls.

• directory: Absolute filesystem path to the classifier model directory.

Returns: True on success, false on error.

string PredictText(string requestJson)

Predict the class of input text using a loaded classifier model (blocking).

Request JSON schema:

{
  "model_id": "intent-classifier",
  "text": "I want to buy a health potion",
  "top_k": 3
}

• model_id (string, required): Classifier model ID.
• text (string, required): Input text to classify.
• top_k (int): Number of top predictions to return (default: 1).

Response JSON schema:

{
  "label": "purchase",
  "probability": 0.92,
  "top": [["purchase", 0.92], ["inquiry", 0.05], ["combat", 0.03]]
}

• label (string): Predicted class label with highest probability.

• probability (float): Confidence score for the top prediction.

• top (array): Top-k predictions as [label, probability] PAIRS, not objects. Read top[i][0] for the label and top[i][1] for the score.

• requestJson: JSON string containing the classification request (see Remarks for schema).

Returns: JSON string containing the classification result with predicted label and confidence, or null on error (see Remarks for schema).

string PredictImage(string requestJson)

Classify an image using a multi-modal classifier (e.g. DINOv2-backed centroid / KNN / SetFit) (blocking).

The classifier referenced by model_id must have been trained with an image embedder. Calling this against a text-only classifier returns null and sets the engine error.

Request JSON schema:

{
  "model_id": "animals",
  "image_path": "/abs/path/to/cat.jpg",
  "top_k": 3
}

• model_id (string, required): Image classifier ID.
• image_path (string, required): Absolute filesystem path to the image (PNG / JPEG).
• top_k (int): Number of top predictions to return (default: all classes).

Response JSON schema is the same as PredictText — label, probability, top.

• requestJson: JSON image classification request (see Remarks for schema).

Returns: JSON ClassifierResult, or null on error.

string PredictAudio(string requestJson)

Classify an audio clip using a multi-modal classifier (e.g. an audio embedder-backed centroid / KNN / SetFit) (blocking).

The classifier referenced by model_id must have been trained with an audio embedder. Calling this against a text- or image-only classifier returns null and sets the engine error.

Request JSON schema:

{
  "model_id": "ambience",
  "audio_path": "/abs/path/to/clip.wav",
  "top_k": 3
}

• model_id (string, required): Audio classifier ID.
• audio_path (string, required): Absolute filesystem path to the audio clip (WAV / RIFF).
• top_k (int): Number of top predictions to return (default: all classes).

Response JSON schema is the same as PredictText — label, probability, top.

• requestJson: JSON audio classification request (see Remarks for schema).

Returns: JSON ClassifierResult, or null on error.

bool Create(string classifierId, string configJson)

Create an empty classifier to teach during play with TrainIncremental(string, string, string) (blocking). Each label is created by its first example; persist with Save and restore with Load.

• classifierId: Name to register it under; must not already exist.

• configJson: JSON config: kind ("centroid", "knn" or "ngram_perceptron"), embedder (embedding model id, required for centroid/knn, absent for ngram_perceptron), optional multilabel (bool) and k_neighbors (knn only, default 7).

Returns: True on success, false on error (see the engine's last error).

bool TrainIncremental(string classifierId, string label, string text)

Teach a classifier one example while the game runs (blocking): folds into , creating the label if it is new. No retraining pass.

• classifierId: A created or loaded classifier.

• label: The example's label (non-empty).

• text: The example utterance.

Returns: True on success, false on error, including a classifier that cannot learn incrementally (check supports_incremental_training in GetInfo).

bool TrainIncremental(string classifierId, string label, float[] embedding)

Teach one example from an embedding you already computed, skipping a second embedding of the same text (blocking). The vector must come from the classifier's own embedder model and have embedding_dim entries (see GetInfo).

• classifierId: A created or loaded classifier.

• label: The example's label (non-empty).

• embedding: The example's embedding.

Returns: True on success, false on error (e.g. an ngram_perceptron, which has no embedding space).

string PredictEmbedding(string classifierId, float[] embedding, int topK = -1)

Classify an embedding you already computed (blocking). Same rules for the vector as TrainIncremental(string, string, float[]).

• classifierId: A created or loaded classifier.

• embedding: The input's embedding.

• topK: How many ranked predictions top holds: N > 0 the best N, 0 none (just the winning label), negative (the default) every label.

Returns: JSON ClassifierResult (same schema as PredictText), or null on error.

bool RemoveLabel(string classifierId, string label)

Forget a label and everything learned for it (blocking).

• classifierId: A created or loaded classifier.

• label: The label to remove.

Returns: True on success, false on error (e.g. an unknown label).

string GetInfo(string classifierId)

Describe a classifier for a debug panel: kind, labels and examples per label (blocking).

{
  "classifier_id": "dragon_cues", "kind": "centroid", "modality": "text",
  "embedding_dim": 384, "supports_incremental_training": true,
  "labels": [{"label": "come", "example_count": 6}, {"label": "sit", "example_count": 6}]
}

embedding_dim is null for classifiers without an embedding space; example_count is null for classifiers saved before counts were recorded.

• classifierId: A created or loaded classifier.

Returns: JSON info, or null on error (see Remarks for schema).

bool Save(string classifierId, string directory)

Save a classifier to a directory (e.g. inside the player's save slot), in the format Load reads back (blocking). The directory is created if missing.

• classifierId: A created or loaded classifier.

• directory: Absolute filesystem path.

Returns: True on success, false on error.

AtelicoEmbeddings

Embedding subsystem for generating text embeddings and computing semantic similarity. Access via AtelicoEngine.Embeddings.

Methods

string EmbedText(string requestJson)

Generate embedding vectors for one or more input texts (blocking). Follows the OpenAI Embeddings API format.

Request JSON schema:

{
  "model": "in-memory::sentence-transformers/all-MiniLM-L6-v2",
  "input": ["The knight draws his sword.", "A warrior unsheathes a blade."]
}

• model (string, required): Embedding model ID in "backend::org/model" format.
• input (string or array of strings, required): Text(s) to embed.

Response JSON schema:

{
  "data": [
    { "embedding": [0.012, -0.034, ...], "index": 0 },
    { "embedding": [0.011, -0.033, ...], "index": 1 }
  ],
  "model": "in-memory::sentence-transformers/all-MiniLM-L6-v2",
  "usage": { "prompt_tokens": 14, "total_tokens": 14 }
}

Each element in data contains the float embedding vector and its index matching the input order.

• requestJson: JSON string containing the embedding request (see Remarks for schema).

Returns: JSON string containing embedding vectors for each input text, or null on error (see Remarks for schema).

string EmbedImage(string requestJson)

Generate an embedding vector for a single image (blocking). Uses a multimodal model (e.g. CLIP) that maps images into the same vector space as text, so image and text embeddings can be compared directly with cosine similarity.

Request JSON schema:

{
  "model": "clip",
  "image": "iVBORw0KGgo..."
}

• model (string, required): Multimodal embedding model ID (e.g. "clip").
• image (string, required): Base64-encoded PNG image bytes.

Response JSON schema:

{
  "data": [
    { "embedding": [0.012, -0.034, ...], "index": 0 }
  ],
  "model": "clip",
  "usage": { "prompt_tokens": 0, "total_tokens": 0 }
}

data[0].embedding is the float embedding vector for the image.

• requestJson: JSON string containing the image embedding request (see Remarks for schema).

Returns: JSON string containing the image embedding vector, or null on error (see Remarks for schema).

string EmbedAudio(string requestJson)

Generate an embedding vector for a single audio clip (blocking). Uses a multimodal model (e.g. CLAP) that maps audio into the same vector space as text, so audio and text embeddings can be compared directly with cosine similarity.

Request JSON schema:

{
  "model": "clap",
  "audio": "UklGRn..."
}

• model (string, required): Multimodal embedding model ID (e.g. "clap").
• audio (string, required): Base64-encoded WAV file bytes (RIFF + PCM).

Response JSON schema:

{
  "data": [
    { "embedding": [0.012, -0.034, ...], "index": 0 }
  ],
  "model": "clap",
  "usage": { "prompt_tokens": 0, "total_tokens": 0 }
}

data[0].embedding is the float embedding vector for the audio clip.

• requestJson: JSON string containing the audio embedding request (see Remarks for schema).

Returns: JSON string containing the audio embedding vector, or null on error (see Remarks for schema).

float Similarity(string modelId, string textA, string textB)

Compute cosine similarity between two texts using the specified embedding model. Embeds both texts and computes their similarity in a single call.

• modelId: Embedding model ID to use, in "backend::org/model" format.

• textA: First text to compare.

• textB: Second text to compare.

Returns: Cosine similarity score (float) in the range -1.0 to 1.0, where 1.0 means identical meaning and -1.0 means opposite.

AtelicoGuardrails

Guardrails subsystem for content safety checking of user inputs, model outputs, and image generation prompts. Access via AtelicoEngine.Guardrails.

Methods

bool BuildCustom(string preset, string blocklistsJson, string classifierConfigJson)

Build a custom guardrails cascade: keyword blocklists, then an optional classifier. The checkers run in order and short-circuit on the first block, so text a blocklist rejects never reaches the classifier. Call once after engine creation to replace the preset guardrails; CheckInput/CheckOutput then evaluate against this configuration.

• preset: Base preset: "game-safe", "child-safe", or "developer-sdk".

• blocklistsJson: JSON array of ["id", "file_path"] pairs for keyword blocklists.

• classifierConfigJson: Classifier stage config JSON, or "" to skip. Margin rule: {"model_dir":"...","benign_label":"innocent","margin":0.04}. Absolute rule: {"model_dir":"...","class_thresholds":{"sexual":0.6,...}}.

Returns: true on success, false on error (bad path, missing model, or unbuildable config).

string CheckInput(string text)

Check user input text against configured safety guardrails before sending to a model.

Response JSON schema:

{
  "action": "allow",
  "checker_name": "content-safety",
  "score": 0.01,
  "reason": "Content is safe."
}

• action (string): "allow", "block", or "warn".

• checker_name (string): Name of the guardrail checker that triggered.

• score (float, optional): Confidence score from the checker.

• reason (string, optional): Human-readable reason for the verdict.

• text: The user input text to check.

Returns: JSON SafetyVerdict string, or null on error (see Remarks for schema).

string CheckOutput(string text)

Check model output text against configured safety guardrails before displaying to the user.

Response JSON schema:

{
  "action": "allow",
  "checker_name": "content-safety",
  "score": 0.02,
  "reason": "Content is safe."
}

• action (string): "allow", "block", or "warn".

• checker_name (string): Name of the guardrail checker that triggered.

• score (float, optional): Confidence score from the checker.

• reason (string, optional): Human-readable reason for the verdict.

• text: The model output text to check.

Returns: JSON SafetyVerdict string, or null on error (same schema as CheckInput).

string CheckImagePrompt(string prompt)

Check an image generation prompt against safety guardrails before generating.

Response JSON schema:

{
  "action": "allow",
  "checker_name": "image-safety",
  "score": 0.01,
  "reason": "Content is safe."
}

• action (string): "allow", "block", or "warn".

• checker_name (string): Name of the guardrail checker that triggered.

• score (float, optional): Confidence score from the checker.

• reason (string, optional): Human-readable reason for the verdict.

• prompt: The image generation prompt text to check.

Returns: JSON SafetyVerdict string, or null on error (same schema as CheckInput).

AtelicoImages

Image generation subsystem providing text-to-image generation and background removal. Access via AtelicoEngine.Images.

Methods

string Generate(string requestJson)

Generate an image from a text prompt synchronously (blocking). Follows the OpenAI Images API format.

Request JSON schema:

{
  "model": "in-memory::PixArt-alpha/PixArt-Sigma-XL-2-1024-MS",
  "prompt": "A medieval castle at sunset, fantasy art style",
  "size": "512x512",
  "n": 1,
  "response_format": "b64_json"
}

• model (string, required): Image model ID in "backend::org/model" format.
• prompt (string, required): Text description of the desired image.
• size (string): Image dimensions as "WxH" (e.g., "512x512", "1024x1024"). Default: "512x512".
• n (int): Number of images to generate (default: 1).
• response_format (string): "b64_json" (default) or "url".

Response JSON schema:

{
  "created": 1700000000,
  "data": [
    { "b64_json": "iVBORw0KGgoAAAANSUhEUg..." }
  ]
}

Each element in data contains a base64-encoded PNG image.

• requestJson: JSON string containing the image generation request (see Remarks for schema).

Returns: JSON string containing the generated image data as base64, or null on error (see Remarks for schema).

string RemoveBackground(string requestJson)

Remove the background from an image synchronously (blocking). Returns a PNG with transparent background.

Request JSON schema:

{
  "image": "iVBORw0KGgoAAAANSUhEUg...",
  "model": "optional-model-id"
}

• image (string, required): Base64-encoded input image (PNG or JPEG).
• model (string): Background removal model ID (uses default if omitted).

Response JSON schema:

{
  "b64_json": "iVBORw0KGgoAAAANSUhEUg..."
}

The b64_json field contains a base64-encoded PNG with transparent background.

• requestJson: JSON string containing the base64-encoded input image (see Remarks for schema).

Returns: JSON string containing the base64-encoded result image with transparent background, or null on error.

AtelicoKvStore

Key-value store subsystem with embedding-backed semantic queries. Stores text entries with automatic embedding generation, enabling semantic search over game data (e.g., NPC dialogue, lore, inventory). Access via AtelicoEngine.KvStore.

Methods

ulong Create(string configJson)

Create a new LanceDB-backed KV store for vector search over game data.

This store is vector-in, vector-out: you embed text yourself with AtelicoEmbeddings and pass vectors to Insert/Query. The store's embed_dim must match your embedding model's output dimension (384 for MiniLM/bge-small, 768 for bge-base).

Configuration JSON schema:

{
  "store_id": "npc-dialogue",
  "db_path": "./data/npc-dialogue",
  "embed_dim": 384
}

• store_id (string): Store identifier (default "default").

• db_path (string): On-disk LanceDB directory, persisted across restarts (default "./data").

• table_name (string): Table within the database (default "entries").

• embed_dim (int): Embedding vector dimension; must match your model (default 384).

• has_priority (bool): Blend a priority score into ranking (default false).

• similarity_weight / priority_weight (float): Blend weights when has_priority (default 0.5/0.5).

• configJson: JSON string containing the store configuration (see Remarks for schema).

Returns: Store ID handle (ulong) for use with other KvStore methods, or 0 on failure.

bool Insert(string storeId, string entriesJson)

Insert one or more entries into a KV store. You supply each entry's embedding vector (produce it with EmbedText).

Entries JSON schema:

[
  {
    "id": "mem-001",
    "key_text": "The guard saw a thief enter the keep at midnight.",
    "key_embedding": [0.1, -0.2, 0.05],
    "facets": { "npc_id": "guard-01", "location": "keep" },
    "priority": 0.8
  }
]

• id (string, required): Unique entry identifier.

• key_text (string, required): Searchable text (kept for display/reranking).

• key_embedding (float[], required): Pre-computed vector; length must equal the store's embed_dim.

• facets (object, optional): String key/value pairs for exact-match filtering.

• priority (float, optional): Ranking priority (0.0-1.0), blended when has_priority.

• storeId: Store identifier returned by Create.

• entriesJson: JSON array of entry objects to insert (see Remarks for schema).

Returns: True on success; false on error.

string Query(string storeId, string queryJson)

Query a KV store by vector similarity. You supply the query embedding (embed the query text first); results are ranked by cosine similarity.

Query JSON schema:

{
  "query_embedding": [0.1, -0.2, 0.05],
  "query_text": "who broke into the keep?",
  "facet_filters": { "npc_id": "guard-01" },
  "vector_search_limit": 20,
  "limit": 5,
  "use_prefilter": true
}

• query_embedding (float[], required): Query vector (length = store's embed_dim).
• query_text (string, optional): Raw text, passed to the reranker / logs.
• facet_filters (object, optional): Exact-match facet filters.
• vector_search_limit (int): ANN candidate pool size (default 20).
• limit (int): Number of results to return (default 5).
• use_prefilter (bool): Apply facet filters before the ANN search (default true).

Response JSON schema:

[
  {
    "id": "mem-001",
    "key_text": "The guard saw a thief enter the keep at midnight.",
    "similarity": 0.89,
    "priority": 0.8,
    "combined_score": 0.87
  }
]

Each result contains id, key_text, similarity (cosine), priority, and the combined_score used for ordering.

• storeId: Store identifier returned by Create.

• queryJson: JSON string containing the query parameters (see Remarks for schema).

Returns: JSON array of matching entries sorted by descending similarity, or null on error (see Remarks for response schema).

string Scan(string storeId, string filter, uint limit)

Scan entries in a KV store with an optional metadata filter. Unlike Query, this does not perform semantic search -- it returns entries matching the filter in insertion order.

Response JSON schema:

[
  {
    "id": "mem-001",
    "key_text": "The guard saw a thief enter the keep at midnight.",
    "priority": 0.8
  }
]

Each entry contains its id, key_text, and priority.

• storeId: Store identifier returned by Create.

• filter: Optional metadata filter expression, or null for no filter.

• limit: Maximum number of entries to return.

Returns: JSON array of matching entries in insertion order, or null on error.

ulong Count(string storeId, string filter)

Count entries in a KV store matching an optional metadata filter.

• storeId: Store identifier returned by Create.

• filter: Optional metadata filter expression, or null to count all entries.

Returns: Number of matching entries (ulong), or 0 on error.

bool Destroy(string storeId)

Destroy a KV store and release all associated resources (embeddings, index, storage).

• storeId: Store identifier returned by Create.

Returns: True if the store was found and destroyed; false otherwise.

bool CreateFtsIndex(string storeId, string column)

Create a full-text-search (FTS) index on a string column of a KV store.

Required before calling LexicalQuery or HybridQuery. Indexing is a one-time per-store operation; subsequent inserts are indexed automatically.

• storeId: Store identifier returned by Create.

• column: Column name to index. Typical choices: "key_text" (search keys) or any string-typed value column.

Returns: True on success.

string LexicalQuery(string storeId, string queryJson)

Lexical (full-text) retrieval against an FTS-indexed column. Returns rows ranked by BM25 relevance.

The target column must have an FTS index — see CreateFtsIndex.

Request JSON schema:

{
  "text": "dragon attack",
  "column": "key_text",
  "filter": null,
  "limit": 10
}

The combined_score field on each result row carries the BM25 score.

• storeId: Store identifier.

• queryJson: JSON LexicalSearchQuery (see Remarks).

Returns: JSON array of KvResult rows, or null on error.

string HybridQuery(string storeId, string queryJson)

Hybrid (vector + lexical) retrieval with weighted-sum reranking.

Each branch returns up to effective_per_branch_limit candidates; scores are min-max normalised independently and combined as vector_weight * vec_norm + lexical_weight * lex_norm. The FTS column must have an FTS index.

Request JSON schema:

{
  "embeddings": [0.0, 0.1, 0.2],
  "text": "dragon attack",
  "fts_column": "key_text",
  "filter": null,
  "limit": 5,
  "per_branch_limit": 0,
  "vector_weight": 0.5,
  "lexical_weight": 0.5
}

per_branch_limit: 0 defaults to 2 * limit.

Response JSON schema:

{
  "results": [{ ...KvResult... }],
  "scores":  [{"vector_score": 0.8, "lexical_score": 0.6, "merged_score": 0.7}]
}

combined_score on each result equals the corresponding merged_score.

• storeId: Store identifier.

• queryJson: JSON HybridSearchQuery (see Remarks).

Returns: JSON object with results and per-row scores, or null on error.

AtelicoLlm

LLM subsystem providing chat completions, text completions, the Responses API, and templated LM function calls. Access via AtelicoEngine.Llm.

Methods

string ChatCompletion(string requestJson)

Synchronous (blocking) chat completion following the OpenAI Chat Completions API format. Blocks the calling thread until the full response is generated.

Request JSON schema:

{
  "model": "in-memory::meta-llama/Llama-3.2-1B-Instruct-GGUF",
  "messages": [
    { "role": "system", "content": "You are a helpful NPC." },
    { "role": "user", "content": "What quests are available?" }
  ],
  "max_tokens": 256,
  "temperature": 0.7,
  "top_p": 1.0,
  "response_format": { "type": "json_schema", "json_schema": { ... } }
}

• model (string, required): Model ID in "backend::org/model" format.
• messages (array, required): Conversation messages, each with role ("system", "user", or "assistant") and content (string).
• max_tokens (int): Maximum tokens to generate (default: 256).
• temperature (float): Controls randomness; lower is more focused and repeatable, higher more varied (0 = greedy).
• top_k (int): Only consider the K most-likely next tokens.
• top_p (float): Nucleus sampling: only consider the smallest set of tokens whose probabilities sum to p.
• min_p (float): Ignore tokens less likely than this fraction of the most-likely token.
• repetition_penalty (float): Discourage repeating words/phrases.
• seed (int): Set for reproducible output (same seed + params + prompt -> same result). Omitting it does not randomise: the engine uses one fixed default seed.
• adapter_scale (float): Strength of the LoRA adapter this request runs with (from the model::adapter suffix or the model's default, see AtelicoLora): 1.0 as trained, 0.0 the base model, values between blend. Absent uses the model's default scale, else 1.0.
• All sampling parameters are optional: each model has recommended defaults, applied when a request omits them.
• priority (string): How this request is scheduled against everything else competing for the GPU — "interactive", "normal" (the default) or "background". Use "interactive" for a line the player is waiting on and "background" for a pre-generation nobody is watching; a long job hands the GPU over at token/step boundaries so a higher-priority request does not wait for it to finish. See the GPU Scheduling guide.
• response_format (object): Optional structured output constraint.
• speculative (object): Optional { proposer, k } for multi-token (speculative) decoding to speed up generation (proposer: "prompt_lookup" or "lookahead"). Greedy output is unchanged.

Response JSON schema:

{
  "id": "chatcmpl-abc123",
  "choices": [
    {
      "message": { "role": "assistant", "content": "..." },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 25, "completion_tokens": 40, "total_tokens": 65 }
}

• requestJson: JSON string containing the chat completion request (see Remarks for schema).

Returns: JSON string containing the chat completion response, or null on error (see Remarks for schema).

string TextCompletion(string requestJson)

Synchronous (blocking) text completion. Continues a raw prompt without chat formatting.

Request JSON schema:

{
  "model": "in-memory::meta-llama/Llama-3.2-1B-Instruct-GGUF",
  "prompt": "The dragon descended upon the village and",
  "max_tokens": 100,
  "temperature": 0.8
}

• model (string, required): Model ID in "backend::org/model" format.
• prompt (string, required): Text prompt to continue.
• max_tokens (int): Maximum tokens to generate.
• temperature, top_k, top_p, min_p, repetition_penalty, seed (optional): Sampling parameters (same meanings as ChatCompletion). Each model supplies recommended defaults for any you omit.

Response JSON schema:

{
  "id": "cmpl-abc123",
  "choices": [
    { "text": " breathed fire across the rooftops...", "finish_reason": "stop" }
  ],
  "usage": { "prompt_tokens": 8, "completion_tokens": 12, "total_tokens": 20 }
}

• requestJson: JSON string containing the completion request (see Remarks for schema).

Returns: JSON string containing the completion response, or null on error (see Remarks for schema).

string Respond(string requestJson)

Synchronous response request following the OpenAI Responses API format.

Request JSON schema:

{
  "model": "in-memory::meta-llama/Llama-3.2-1B-Instruct-GGUF",
  "input": "What is the capital of France?",
  "instructions": "Answer concisely.",
  "max_output_tokens": 50,
  "temperature": 0.7
}

• model (string, required): Model ID in "backend::org/model" format.
• input (string or array): User input text or message array.
• instructions (string): System instructions.
• max_output_tokens (int): Maximum tokens to generate.
• temperature, top_k, top_p, min_p, repetition_penalty, seed (optional): Sampling parameters (same meanings as ChatCompletion). Each model supplies recommended defaults for any you omit.

Response JSON schema:

{
  "id": "resp-abc123",
  "output": [
    { "type": "message", "content": "The capital of France is Paris." }
  ],
  "usage": { "prompt_tokens": 12, "completion_tokens": 8, "total_tokens": 20 }
}

• requestJson: JSON string containing the response request (see Remarks for schema).

Returns: JSON string containing the response object, or null on error (see Remarks for schema).

string Tokenize(string modelId, string text)

Encode text into token IDs using a loaded model's tokenizer.

Loads the model on demand if it isn't already resident. Useful for budgeting prompt length against the context window before submitting a request, or for pre-tokenizing prefixes that will be reused.

Response JSON schema:

{
  "model_id": "in-memory::meta-llama/Llama-3.2-3B-Instruct-Q4_K_M",
  "tokens": [128000, 9906, 11, 1917, 0]
}

• modelId: Model identifier in "backend::org/model" format.

• text: UTF-8 text to tokenize.

Returns: JSON string with the token sequence, or null on error.

string Detokenize(string requestJson)

Decode a token ID sequence back into text using a loaded model's tokenizer.

Request JSON schema:

{
  "model_id": "in-memory::meta-llama/Llama-3.2-3B-Instruct-Q4_K_M",
  "tokens": [128000, 9906, 11, 1917, 0],
  "skip_special_tokens": true
}

skip_special_tokens defaults to true when omitted; set it to false to keep tokens like <|begin_of_text|> in the output.

Response JSON schema:

{ "text": "Hello, world!" }

• requestJson: JSON detokenize request (see Remarks for schema).

Returns: JSON string with the decoded text, or null on error.

string ModelCapabilities(string modelId)

Inspect a loaded model's static capabilities (vocab size, context window, dtype, active backend).

Loads the model on demand. Useful for telemetry, budgeting, or dispatching to the right code path based on the active dtype/device.

Response JSON schema:

{
  "model_id": "in-memory::meta-llama/Llama-3.2-3B-Instruct-Q4_K_M",
  "vocab_size": 128256,
  "max_position_embeddings": 131072,
  "dtype": "Q4_K_M",
  "device": "Cuda(0)",
  "backend_kind": "metal",
  "thinking_style": "unsupported",
  "native_tool_format": "gemma4",
  "recommended_sampling": { "temperature": 0.1, "top_p": 0.95, "repetition_penalty": 1.1, "top_k": 64 }
}

• device (string): Active device, e.g. "Cpu", "Cuda(0)", "Metal".
• thinking_style (string): "unsupported", "thinktags", or "native".
• native_tool_format (string, optional): Native tool-call format (e.g. "gemma4") when the model supports it; omitted otherwise.
• recommended_sampling (object, optional): Suggested sampling parameters; each of temperature, top_p, repetition_penalty, top_k may be omitted.

For a typed, parsed result use ModelCapabilitiesTyped.

• modelId: Model identifier in "backend::org/model" format.

Returns: JSON string with the ModelCapabilities, or null on error.

AtelicoModelCapabilities ModelCapabilitiesTyped(string modelId)

Inspect a loaded model's static capabilities as a typed object.

Calls ModelCapabilities and deserializes the JSON. All fields are optional: missing strings parse as null/empty, and missing recommended-sampling values are left at their -1 sentinel (see AtelicoRecommendedSampling). Loads the model on demand.

• modelId: Model identifier in "backend::org/model" format.

Returns: A parsed AtelicoModelCapabilities, or null on error.

AtelicoModelCapabilities

AtelicoRecommendedSampling

AtelicoLora

Per-model default LoRA adapter and strength. Access via AtelicoEngine.Lora.

A request picks its adapter with the model id ("in-memory::llama-3.2-1b::my-adapter") and its strength with the adapter_scale request field. These methods set what a model uses when a request says neither.

Methods

bool Load(string modelId, string adapterPath)

Make an adapter the default for a model: requests without a ::adapter suffix run with it from the next request on.

• modelId: Base model ID, in "backend::org/model" format.

• adapterPath: An asset-store adapter id ("my-studio/rpg-dialogue-lora"), or a local adapter directory written as a path — absolute (e.g. under Application.streamingAssetsPath) or starting with "./".

Returns: True once it is the default; false if the adapter cannot be resolved (see AtelicoNative.atelico_last_error).

bool Unload(string modelId)

Clear a model's default adapter and strength: requests without a suffix run on the base weights again.

• modelId: Base model ID.

Returns: True on success.

bool SetScale(string modelId, float scale)

Set the strength a model's adapter runs at, for requests without their own adapter_scale: 1.0 as trained, 0.0 the base model, values between blend.

• modelId: Base model ID.

• scale: Finite and non-negative; above 1.0 exaggerates the adapter.

Returns: True on success; false for an invalid scale.

string ListActive()

List the models that have a default adapter, as JSON.

Returns: A JSON array of {"model_id": ..., "adapter": ...} objects; "[]" when no model has one, or on error.