Speech-to-Text

Atelico transcribes audio on-device with OpenAI Whisper — from the tiny English-only build all the way up to large-v3 and large-v3-turbo, including quantized GGUF variants for memory-constrained deployments. As with the rest of the engine, there are no API keys and no network calls: the audio never leaves the machine.

STT is wired into the in-memory:: backend and exposed over the OpenAI-compatible route POST /v1/audio/transcriptions, plus every native SDK.

Quick start

Over HTTP, upload an audio file as multipart form data:

curl http://localhost:11434/v1/audio/transcriptions \
  -H "Content-Type: multipart/form-data" \
  -F model=in-memory::whisper \
  -F file=@speech.wav

Response:

{
  "text": "Hello from Atelico.",
  "language": "en",
  "duration": 1.4
}

in-memory::whisper resolves to the default base.en build. Other ids select specific Whisper sizes — see Choosing a model.

Audio input

How you pass audio depends on the surface:

• HTTP — upload a file (WAV, MP3, FLAC, …) as the file part; the server decodes and resamples it for you.
• C FFI / Python — pass a base64-encoded WAV file in an audio_b64 field. The binding decodes the RIFF header to recover the sample rate automatically, so you don't pass a separate sample_rate.
• Rust SDK — pass raw f32 PCM samples directly via audio_samples: Vec<f32> plus the sample_rate. This skips WAV encode/decode entirely, which is the cheapest path for live-mic capture where you already have float frames in hand.

Whisper operates internally at 16 kHz mono; audio at other rates is resampled before inference.

The request

The fields below come from AudioTranscriptionRequest in atelico-audio/src/audio_types.rs. Note that audio_samples and sample_rate are marked #[serde(skip)] — they are populated programmatically (or decoded from the uploaded file / audio_b64), not parsed from the JSON body.

Field	Type	Default	Description
model	string	required	Model id, e.g. in-memory::whisper
audio_samples	Vec<f32>	—	Raw PCM samples (Rust SDK; set programmatically)
sample_rate	u32	—	Sample rate of audio_samples
language	string?	auto-detect	ISO 639-1 code (en, ja, …)
response_format	string	json	json, text, or verbose_json
temperature	f64	0.0	Decoder sampling temperature (0 = greedy)
timestamp_granularities	string[]	[]	Any of segment, word

The response

The response deserializes from AudioTranscriptionResponse:

Field	Type	Description
text	string	The transcribed text
language	string?	Detected or specified language
duration	f32?	Audio duration in seconds
segments	array?	Segment-level timestamps (verbose_json)
words	array?	Word-level timestamps (when word granularity is requested)

Each segments entry is { start, end, text }; each words entry is { start, end, word } (all times in seconds).

Languages and auto-detection

English-only builds (*.en) always decode English. Multilingual builds (whisper-large-v3, large-v3-turbo, distil-large-v3, and the non-.en small/base/tiny variants) insert a language token into the decoder prompt: if you set language to an ISO 639-1 code that token is forced, otherwise the model auto-detects and reports the result in the language response field.

Timestamps

Pass response_format=verbose_json to receive segments with segment-level timing. To additionally get word-level timing, request timestamp_granularities[]=word. Word timestamps are useful for caption karaoke, subtitle alignment, or syncing on-screen events to spoken words.

Choosing a model

After the in-memory:: prefix, the id selects the Whisper build:

model value	Variant	Languages
whisper (default)	base.en	English
whisper-tiny, whisper-tiny.en	tiny / tiny.en	multi / English
whisper-base, whisper-base.en	base / base.en	multi / English
whisper-small, whisper-small.en	small / small.en	multi / English
whisper-medium, whisper-medium.en	medium / medium.en	multi / English
whisper-large-v3	large-v3	multi
whisper-large-v3-turbo	large-v3-turbo	multi
distil-large-v3	distil-large-v3	multi

Smaller builds are faster and lighter; larger builds are more accurate, especially on accented or noisy audio. large-v3-turbo is the best accuracy-per-millisecond pick for multilingual work, and quantized GGUF builds (Q5_0) cut the memory footprint substantially — the HTTP route auto-selects a quantized variant when the requested id resolves to one in the asset store.

SDK usage (in-process)

Rust SDK

The Rust SDK takes f32 PCM samples directly — no WAV round-trip:

use atelico_sdk::{Engine, AudioTranscriptionRequest};

let engine = Engine::new()?;

// Decode a WAV off disk to (samples, sample_rate), or use live-mic frames.
let (samples, sample_rate) =
    atelico_audio::processing::wav_io::read_wav_file(std::path::Path::new("speech.wav"))?;

let request = AudioTranscriptionRequest {
    model: "in-memory::whisper".into(),
    audio_samples: samples,
    sample_rate,
    ..Default::default()
};
let result = engine.audio().transcribe_sync(request)?;
println!("{}", result.text);
if let Some(lang) = result.language {
    println!("language: {lang}");
}

transcribe_sync is the blocking call; transcribe is its async sibling, and transcribe_async hands back a one-shot StreamHandle<AudioTranscriptionResponse> for polling from a game loop.

Python

The Python binding takes a base64-encoded WAV file in audio_b64:

import base64, json
from atelico import Engine

engine = Engine()

wav_b64 = base64.b64encode(open("speech.wav", "rb").read()).decode()
result = json.loads(engine.audio_transcribe(json.dumps({
    "model": "in-memory::whisper",
    "audio_b64": wav_b64,
    "language": "en",            # optional; omit to auto-detect
})))
print(result["text"])

C / FFI

// Blocking STT — audio_b64 is a base64-encoded WAV file.
const char* request =
    "{\"model\":\"in-memory::whisper\",\"audio_b64\":\"UklGRn...\",\"language\":\"en\"}";
const char* response_json = NULL;
atelico_audio_transcribe(engine, request, &response_json);
// response_json: {"text":"Hello from Atelico.","language":"en","duration":1.4}

The output pointer is owned by the FFI layer's thread-local return buffer — copy it before the next API call on the same thread. The language field is optional; omit it to let multilingual models auto-detect.

Streaming / live-mic transcription

Keep the microphone open and let the engine decide when the player spoke. A transcription stream takes audio at any sample rate and in any chunk size, finds each utterance with voice-activity detection, transcribes it through the engine's cached Whisper (the same model transcribe uses, at interactive priority — no second copy is loaded), and reports events as they are decided:

Event	When
speech_started { start_time }	the player began talking
speech_ended { start_time, end_time }	they stopped; the final text follows
transcription { text, is_final, start_time, end_time }	provisional text while they talk (opt-in), then the final text

Times are seconds since the stream opened. A failed transcription is reported as one error and the stream keeps listening.

Stream configuration (JSON in the bindings):

Field	Default	Meaning
model	required	Whisper model id, as for transcribe ("in-memory::whisper")
endpointing	"dialogue"	"command" (0.5 s of silence ends an utterance, 10 s cap), "dialogue" (0.8 s, 30 s), "dictation" (1.5 s, 60 s)
energy_threshold	0.02	RMS level that counts as speech; raise for noisy rooms, lower for quiet mics
language	auto	ISO 639-1 hint
partial_interval_ms	off	re-transcribe the utterance so far every N ms of speech, as is_final: false

Godot: AtelicoSpeechListenerNode

The node owns the microphone plumbing (a muted capture bus, the microphone stream, downmixing) and emits signals. Enable Project Settings › Audio › Driver › Enable Input first — without it Godot delivers silence.

@onready var listener: AtelicoSpeechListenerNode = $SpeechListener

func _ready():
    listener.engine = $AtelicoEngine          # an initialised AtelicoEngineNode
    listener.endpointing = "command"
    listener.speech_started.connect(func(_t): dragon.perk_up())
    listener.transcription_received.connect(_on_heard)
    listener.start_listening()

func _on_heard(text: String, is_final: bool, _start: float, _end: float):
    if is_final:
        dragon.hear(text)

Set use_microphone = false and call feed_audio(samples, sample_rate) to supply audio yourself. stop_listening() transcribes the utterance in progress, then emits listening_stopped.

Rust SDK

use atelico_sdk::{Endpointing, TranscriptionEvent, TranscriptionStreamConfig};

let mut stream = engine.audio().transcribe_stream(TranscriptionStreamConfig {
    model: "in-memory::whisper".into(),
    endpointing: Endpointing::Command,
    ..Default::default()
})?;
stream.feed(&mic_chunk, 48_000)?;                 // from the audio callback
while let Some(event) = stream.poll()? {          // from the game loop
    if let TranscriptionEvent::Transcription { text, is_final: true, .. } = event {
        println!("heard: {text}");
    }
}
stream.finish()?;                                 // closes the last utterance
# Ok::<_, atelico_sdk::SdkError>(())

Other bindings

Every binding exposes the same stream: start it with a JSON config, feed mono audio at any rate, poll events (JSON strings tagged by "type"), finish, and free.

Binding	Start	Feed / finish	Poll
Python	engine.audio_transcribe_stream(config_json)	stream.feed(samples, rate), stream.finish()	stream.poll() → JSON or None; stream.done
Unity	engine.Audio.StartTranscriptionStream(configJson), or the AtelicoSpeechListener component	Feed(samples, rate), Finish()	TryPoll(out json) → Event / Empty / Done / Error
Unreal	TranscriptionStreamStart(ConfigJson) → int64	TranscriptionStreamFeed(Id, Samples, Rate), TranscriptionStreamFinish(Id)	TranscriptionStreamPoll(Id, OutJson) → EAtelicoTranscriptionPoll
C FFI	atelico_transcribe_stream_start(engine, config_json, &id)	atelico_transcribe_stream_feed(engine, id, samples, len, rate), …_finish	atelico_transcribe_stream_poll(engine, id, &json); …_free
Lua	engine:audio():transcribe_stream(config)	stream:feed(samples, rate), stream:finish()	stream:poll()

stream = engine.audio_transcribe_stream(json.dumps({"model": "in-memory::whisper", "endpointing": "command"}))
stream.feed(samples, 48000)                 # from the audio callback thread
while (event := stream.poll()) is not None: # from the main loop
    event = json.loads(event)
    if event["type"] == "transcription" and event["is_final"]:
        print(event["text"])

// Unity: add AtelicoSpeechListener to a GameObject, then
listener.TranscriptionReceived += (text, isFinal, start, end) => { if (isFinal) Debug.Log(text); };
listener.StartListening("in-memory::whisper");

The model-owning atelico_audio::stt::whisper::StreamingTranscriber remains for standalone use outside the engine; both share the same segmentation (atelico_audio::processing::segmenter::SpeechSegmenter).

See also

• Text-to-Speech — Kokoro synthesis, the inverse path
• Models — full supported-model catalogue
• Server Configuration — STT-related env vars

Scheduling priority

Transcription defaults to interactive — it is a player talking, and the game is waiting on the words. A bulk offline transcription job should say "priority": "background" and get out of the way of everything on screen.

See GPU Scheduling.