Why the audio does not go through your server

The browser holds a WebRTC peer connection straight to the model: the microphone track goes out, the voice comes back as a remote track, and a data channel carries the JSON events. Routing that audio through a Blazor Server circuit instead would cost a base64 encode, a SignalR hop and a buffer in each direction — together a few hundred milliseconds. That is the difference between a conversation and a walkie-talkie, and it is barge-in that suffers most: interrupting only feels right when it is instant.

Your server stays in charge anyway. It mints a short-lived ek_… token with the entire session baked into it — instructions, voice, model and the tool list — so the browser can read none of it and change none of it. Your API key never leaves the server.

The tools are the ones you already have

A function call arrives on the data channel, crosses into .NET, and runs there — the same AIFunction instances your text agent uses, in the same circuit, under the same signed-in user. A manipulated browser cannot invent a tool: the runner executes only what is in the list you handed it. That also means your human-in-the-loop dialogs still work mid-conversation — the model asks aloud and the confirm dialog appears.

// Program.cs — the voice runner comes with the rest of the agents package.
builder.Services.AddDrylComponents().AddDrylAgents();

// Your service — the same tool list the text agent gets.
public sealed class AssistantService(DrylVoiceRunner voice, MyOptions options)
{
    private DrylVoiceRun? _voice;

    public DrylVoiceRun Voice => _voice ??= voice.Create(new DrylVoiceOptions
    {
        ApiKey = options.ApiKey,
        Instructions = SystemPrompt + SpokenAddendum,
        Tools = _tools,
    });
}

Wiring it to the dock

Hold the run in a service, not in the page — like a canvas run, it should survive a re-render and a navigation. Hand it to the dock and the head grows a microphone; while a session is live the dock becomes a voice panel, and the transcript keeps flowing into your Log.

<DrylCanvasDock Run="Agent.Canvas"
                OnSend="SendAsync"
                Voice="Agent.Voice"
                VoiceLabel="Talk to the assistant">
    <Log>@* your own DrylMessage elements *@</Log>
</DrylCanvasDock>

One conversation, two mouths

SeedHistory is replayed into every new session, so the voice knows what was typed. When the session ends, Transcript holds what was said — hand it back to your text conversation and the assistant stays one assistant instead of becoming two that have never met.

// Going in: what has been typed so far.
Agent.Voice.SeedHistory = _history;

// Coming out: react to the session ending, then absorb what was said.
private void OnVoiceChanged()
{
    if (_lastPhase is not VoicePhase.Idle && Agent.Voice.Phase is VoicePhase.Idle)
        _history.AddRange(Agent.Voice.Transcript);

    _lastPhase = Agent.Voice.Phase;
}

Configuration is code

There is no settings UI here, on purpose. Voice, persona, model and turn detection are decisions an app makes once, in DrylVoiceOptions — not preferences a user hunts for in a panel. Note that the API locks the voice once a session has emitted audio: changing Voice means starting a new session.

new DrylVoiceOptions
{
    ApiKey            = key,                       // stays on the server
    Model             = "gpt-realtime-2.1",        // or -2, or -2.1-mini
    Instructions      = prompt,                    // role, personality, tone, language
    Voice             = "marin",                   // marin and cedar are the best two
    Speed             = 1.0,                       // 0.25–1.5
    TurnDetection     = VoiceTurnDetection.SemanticVad,
    NoiseReduction    = VoiceNoiseReduction.NearField,
    ReasoningEffort   = "low",                     // 2.1 family only
    TranscriptionModel = "gpt-4o-transcribe",      // null = no transcript at all
    Language          = "de",
    Tools             = tools,
    IdleTimeout       = TimeSpan.FromMinutes(2),
    MaxDuration       = TimeSpan.FromMinutes(30),
}

Work that outlives a single turn

A realtime session has no agent loop of its own. What the protocol continues by itself is a turn that carried a tool call: the results go back, the next response follows. A turn that was only speech ends there — and nothing ever starts another one. So an assistant working through a plan that says "let me go and check" and then stops talking is not thinking. It is finished, and it is waiting to be poked.

ShouldContinue is the way out. It is asked after every turn that ended without a tool call, and returning true prompts the model to carry on by itself. Wire it to whatever tells you there is work left. It is null by default, so a plain conversation still hands the floor back after each answer.

// Asked after any turn the model ended without calling a tool.
// True: it is sent back to work. False: it goes back to listening.
Agent.Voice.ShouldContinue = () => ValueTask.FromResult(
    _tasks.Items.Any(t => t.Status != TaskStatus.Done));

// The backstop. Only turns that achieved nothing count against it.
Agent.Voice.MaxAutoContinuations = 6;

MaxAutoContinuations (six by default) is the backstop against a predicate that never goes false. Only fruitless turns count against it: running a tool is progress and resets the budget, as does the user speaking. It caps how often the model may be nudged while achieving nothing — not how long it may work.

What it costs, and when it stops

An open session bills per minute whether or not anyone is talking, so it closes itself: IdleTimeout (two minutes of silence by default) and MaxDuration (30 minutes; the API's own ceiling is 60). Both are yours to set. gpt-realtime-2.1-mini costs roughly a third of the full model and is one line away.