Embedding Models

Embedding models turn text into vectors for semantic search. They power the /v1/embeddings endpoint, the embed SDK calls, and the vectors you store and query in the KV store.

Embedding models are encoders (BERT-family), not chat models. The engine routes them automatically: loading an embedding model id through model_load or embed resolves to the same shared encoder instance, loaded once and reused across requests.

The model you pick determines the vector dimension your KV store must use (embed_dim): MiniLM, bge-small, and granite-small are 384-dim; bge-base and granite-english are 768-dim (see the table below).

Built-in models

These ids work out of the box — by short name or full HuggingFace repo id:

Short name	HF repo id	Dim	Params	Notes
all-MiniLM-L6-v2	sentence-transformers/all-MiniLM-L6-v2	384	23M	Default; fast, ~90 MB
bge-small-en-v1.5	BAAI/bge-small-en-v1.5	384	33M	Higher quality English
bge-base-en-v1.5	BAAI/bge-base-en-v1.5	768	109M	Higher quality, larger
granite-embedding-small-english-r2	ibm-granite/granite-embedding-small-english-r2	384	47M	ModernBERT, 8K context
granite-embedding-english-r2	ibm-granite/granite-embedding-english-r2	768	149M	ModernBERT, 8K context
granite-embedding-97m-multilingual-r2	ibm-granite/granite-embedding-97m-multilingual-r2	384	97M	Multilingual
granite-embedding-311m-multilingual-r2	ibm-granite/granite-embedding-311m-multilingual-r2	768	311M	Multilingual

Prefix with the backend as usual, e.g. in-memory::sentence-transformers/all-MiniLM-L6-v2.

Any other BERT-family model

Models outside the built-in list (gte, e5, arctic, nomic — any sentence-transformers checkpoint with model_type: "bert" or "modernbert" in its config.json) load generically: the engine detects the backbone from config.json and the pooling mode from the sentence-transformers 1_Pooling/config.json (mean pooling if absent). Embeddings are always L2-normalized.

A loadable snapshot needs at least:

config.json
tokenizer.json
model.safetensors        (float weights)  — OR —  model.gguf  (quantized)
1_Pooling/config.json    (optional — pooling mode, defaults to mean)

Quantized (GGUF) embedding models

BERT embedders can ship as quantized GGUF instead of float safetensors. An 8-bit MiniLM is ~24 MB on disk and in RAM, versus ~90 MB for the float snapshot — worthwhile for download size and for on-device/iOS memory.

Drop a model.gguf (llama.cpp bert architecture, e.g. the Q8_0 build of MiniLM) in place of model.safetensors, keeping the same config.json and tokenizer.json next to it. The engine detects the GGUF automatically, reads the linear projections as quantized tensors (embeddings and layer norms are dequantized to F32), and serves it through the same embeddings API — no model id change. When both model.safetensors and model.gguf are present, the float weights win.

Notes:

• Only the BERT backbone has a quantized graph today; ModernBERT (Granite r2) must use safetensors.
• config.json is still required (it's ~600 bytes; the size win is entirely in the weights file).

Cache layout & shipping offline

The engine resolves embedding models in this order — no network is touched if a local copy exists:

1. <cache>/embeddings/<model-id>/ — canonical location (the asset downloader places them here)
2. <cache>/models/<model-id>/ — general model cache
3. HuggingFace download (materialized into <cache>/embeddings/ for next time)

To ship a game that works offline, place the snapshot under either directory of your configured model directory, e.g.:

<ModelDirectory>/embeddings/sentence-transformers/all-MiniLM-L6-v2/
├── config.json
├── tokenizer.json
├── model.safetensors        # or model.gguf for the quantized build
└── 1_Pooling/config.json

Publishing to an object store works the same for both formats — atelico-asset-publisher publish-model <id> materializes the snapshot (safetensors or GGUF) under embeddings/<id>/ and uploads the tree.

Usage: embedding text

embed turns one or more strings into vectors. Warm-loading the model first is optional — it loads lazily on the first embed call — but pre-loading moves the one-time cost off your first request.

Python

import atelico, json
engine = atelico.Engine()

# Optional warm-load (otherwise the first embed() pays the load cost)
engine.model_load("in-memory::sentence-transformers/all-MiniLM-L6-v2")

resp = json.loads(engine.embed(json.dumps({
    "model": "in-memory::sentence-transformers/all-MiniLM-L6-v2",
    "input": ["the cat sat on the mat", "a feline rests on the rug"],
})))
vectors = [d["embedding"] for d in resp["data"]]   # two 384-dim vectors

Unity (C#)

engine.LoadModel("in-memory::sentence-transformers/all-MiniLM-L6-v2");

// Raw vectors:
string r = engine.Embeddings.Embed(JsonSerializer.Serialize(new {
    model = "in-memory::sentence-transformers/all-MiniLM-L6-v2",
    input = new[] { "the cat sat on the mat" },
}));

// Or a direct similarity score between two texts:
float sim = engine.Embeddings.Similarity(
    "in-memory::sentence-transformers/all-MiniLM-L6-v2",
    "the cat sat on the mat", "a feline rests on the rug");

Unreal (C++)

atelico_model_load(Engine, "in-memory::sentence-transformers/all-MiniLM-L6-v2");

const FString Resp = Atelico->Embed(TEXT(R"({
    "model": "in-memory::sentence-transformers/all-MiniLM-L6-v2",
    "input": ["the cat sat on the mat"]
})"));

// Or a direct similarity score:
float Sim = Atelico->Similarity(
    TEXT("in-memory::sentence-transformers/all-MiniLM-L6-v2"),
    TEXT("the cat sat on the mat"), TEXT("a feline rests on the rug"));

Godot (GDScript)

@onready var core: AtelicoCoreNode = $AtelicoCore

func _ready() -> void:
    core.model_load("in-memory::sentence-transformers/all-MiniLM-L6-v2")
    var resp := JSON.parse_string(core.embed(JSON.stringify({
        "model": "in-memory::sentence-transformers/all-MiniLM-L6-v2",
        "input": "the cat sat on the mat",
    })))
    var vector: Array = resp.data[0].embedding

HTTP

curl http://localhost:11434/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "in-memory::sentence-transformers/all-MiniLM-L6-v2",
    "input": ["the cat sat on the mat"]
  }'

Using embeddings for search

Embeddings on their own are just vectors. To build NPC memory, RAG, or lore lookups you store and query them:

• Vector Search & Memory (KV Store) — create a store, insert your vectors, and retrieve the nearest ones. Set the store's embed_dim to this model's dimension (384 or 768 above).
• Hybrid & Lexical Search — add a BM25 full-text index and blend keyword + vector relevance.

Errors you might see

• '<id>' is not a loadable embedding model: model_type '<x>' is not a supported encoder — the snapshot's config.json names an architecture the embedder can't run. Supported: bert, modernbert.
• failed to load embedding model '<id>': ... — the id is neither a built-in alias nor a resolvable snapshot. The message lists the built-in models and the expected cache layout. Unknown ids are an error — the engine never silently substitutes a different embedding model, because vectors from different models are not comparable.
• GGUF architecture 'bert' is an embedding encoder, not a chat model — you tried to load a quantized embedder as a chat model. Quantized embedders are supported, but through the embeddings API: name it as an embedding_model / embed target, not a chat model.