Skip to main content

How do I use a realtime voice, text-to-speech or speech-to-text model?

Start a realtime session on the model page or through the API and WebSocket, and see what to send, what comes back and how it's billed.

Realtime models keep a live session open: you stream your voice, or send text, and hear speech or read transcripts while it happens, instead of waiting for one finished file. There are three kinds: voice conversation (for example openai/gpt-realtime), text-to-speech (for example OpenMOSS/MOSS-TTS-Realtime) and speech-to-text (for example mistralai/Voxtral-Mini-4B-Realtime-2602).

On the website

  • Open the model's page, set its inputs and click Run. When your browser asks, allow the microphone. Text-to-speech doesn't need it.

  • The session panel shows Listening... or AI is speaking..., a timer and the running Cost, with a live transcript below.

  • Mute turns your microphone off and on. Interrupt, shown while the AI is speaking, stops its voice so you can talk. End Session finishes the session.

  • Text-to-speech plays the speech and ends when it's done. Speech-to-text shows your words as one running text.

  • If you block the microphone, the panel shows Session ended. Allow it in your browser's site settings and run again.

  • Closing the tab ends the session.

With the API

  1. Run the model with POST https://api.wiro.ai/v1/Run/{owner}/{model}. Its inputs, such as the voice, instructions and audio settings, differ per model: check them with Tool/Detail or on the model page's API Integration Guide tab, which also has WebSocket code for that model (see Run parameters). The response has a socketaccesstoken.

  2. Connect to wss://socket.wiro.ai/v1 and send {"type": "task_info", "tasktoken": "YOUR_SOCKETACCESSTOKEN"}.

  3. Wait for the task_stream_ready event.

  4. Send your microphone audio as binary frames: your token, a | character, then raw PCM audio, 16-bit little-endian, mono, at 24 kHz (see Audio Format). Chunks of 200 ms (9,600 bytes) work well.

  5. Play the binary frames you receive: everything after the first | is audio in the same format. task_stream_end marks the end of the AI's turn. Transcripts arrive as task_output text starting with TRANSCRIPT_USER: (you) or TRANSCRIPT_AI: (the AI).

  6. To finish, send {"type": "task_session_end", "tasktoken": "YOUR_SOCKETACCESSTOKEN"}, then wait for task_postprocess_end before you close the connection.

// Node.js with the ws package: send one audio chunk
ws.send(Buffer.concat([Buffer.from(token + "|"), pcmChunk]));

// A received audio frame: drop everything up to the first "|"
const pcm = data.subarray(data.indexOf(0x7c) + 1);

To cut the AI off, stop playing its audio and send your voice; some models stop on their own when they hear you speak. The website's Interrupt button works this way.

Text-to-speech and speech-to-text

  • Text-to-speech: send the text in the Run request (the input's name depends on the model). You send no audio and receive only audio frames, with no transcripts. The session usually ends on its own once the audio is done; to stop earlier, send task_session_end.

  • Speech-to-text: you send audio only and receive task_output text starting with TRANSCRIPT_USER: as you speak. No audio comes back. Send 24 kHz audio even if the model works at another rate; Wiro converts it.

What doesn't work with realtime models

They run only through the normal Run URL and the WebSocket. /sync rejects them with The synchronous wrapper is not available for this model., and the LLM Gateway doesn't offer them.

How a session is billed

  • You're charged per turn while the session runs, not once at the end. Each charge arrives in a task_cost event: turnCost for that turn and cumulativeCost for the session so far. On the website, this is the running Cost.

  • Wiro checks the $0.50 minimum balance only when a session starts. After that, unless the session ends another way first, it stops right after the charged turn that uses up your available balance (in a team workspace, or with a team project's API key, the team wallet's balance). That turn is charged in full, and so is any last usage reported while the session closes, so your balance can end slightly below zero.

  • Finished turns stay charged, even if the session later ends with an error or is stopped. See Am I charged if my task fails or I cancel it?

  • For when and why a session ends on its own, see when a realtime session ends.

More

Did this answer your question?

Related Articles