Skip to main content

Does the LLM Gateway support streaming and tool calling?

Yes: standard OpenAI function tools work and "stream": true is supported, though streamed events currently arrive only after the model finishes its reply.

Yes. The LLM Gateway supports "stream": true and standard OpenAI function tools, but in the current release the streamed events are sent after the model has finished its reply, so text won't appear token by token.

  • Who runs the tools: your editor or agent, not Wiro. The model asks for a tool call, and your tool runs it and sends the result back in its next request.

  • Streamed tool calls arrive in delta.tool_calls; join their argument fragments by the call's index.

  • Details vary by model: https://llm.wiro.ai/v1/models/{owner}/{model} reports a model's tool support and token limits. Use those limits if your tool asks for them.

  • Cline: pick "OpenAI Compatible" (not "OpenAI") and tick "Computer Use" to enable tool calling.

For setup steps, see How do I use the Wiro LLM Gateway? and the Connect an agent guides in our docs.

Related Articles
Did this answer your question?