Universal.OpenAI.Realtime.Client 1.0.3

Universal.OpenAI.Realtime.Client

A .NET client for OpenAI's Realtime API WebSocket event protocol (wss://api.openai.com/v1/realtime) — typed client/server events, conversation items, and a ClientWebSocket-based session wrapper for voice and text conversations, function calling, and SIP call sidebands.

Session configuration, function tool schemas, and audio format types are shared with Universal.OpenAI.Client's Universal.OpenAI.Client.V1.Realtime namespace — use that package's RealtimeClient for the REST side (accepting/rejecting SIP calls, minting ephemeral client secrets) and this package for the live WebSocket connection.

Quick Start

using Universal.OpenAI.Realtime.Client;
using Universal.OpenAI.Client.V1.Realtime;

using RealtimeClient client = await RealtimeClient.ConnectAsync(apiKey, Models.GptRealtime2);

await client.SendAsync(new SessionUpdateEvent(new RealtimeSessionConfiguration
{
    Instructions = "You are a helpful assistant.",
    OutputModalities = new[] { RealtimeOutputModalities.Text }
}));
await client.SendAsync(new ConversationItemCreateEvent(new UserMessageItem("Say hello.")));
await client.SendAsync(new ResponseCreateEvent());

await foreach (RealtimeEvent evt in client.ReceiveEventsAsync())
{
    if (evt is ResponseOutputTextDeltaEvent delta)
    {
        Console.Write(delta.Delta);
    }
    if (evt is ResponseDoneEvent)
    {
        break;
    }
}

Features

Connecting

Three ways to open a connection, depending on how the session is being used:

// Direct, using your own API key (trusted server-side/console clients)
using RealtimeClient client = await RealtimeClient.ConnectAsync(apiKey, Models.GptRealtime2);

// As the WebSocket sideband for an already-accepted SIP call
using RealtimeClient client = await RealtimeClient.ConnectForCallAsync(apiKey, callId);

// Using an ephemeral client secret (untrusted/browser clients) minted via
// Universal.OpenAI.Client.V1.Realtime.RealtimeClient.CreateClientSecretAsync
using RealtimeClient client = await RealtimeClient.ConnectWithClientSecretAsync(clientSecretValue, Models.GptRealtime2);

Sending and Receiving Events

SendAsync takes any RealtimeClientEvent (SessionUpdateEvent, ConversationItemCreateEvent, ResponseCreateEvent, InputAudioBufferAppendEvent, etc.). ReceiveEventsAsync streams the full, typed server event set (SessionCreatedEvent, ResponseOutputTextDeltaEvent, ResponseFunctionCallArgumentsDoneEvent, RealtimeErrorEvent, and 35+ others) as an IAsyncEnumerable<RealtimeEvent>, dispatched on the wire type discriminator — pattern-match on the concrete type you care about:

await foreach (RealtimeEvent evt in client.ReceiveEventsAsync())
{
    switch (evt)
    {
        case ResponseOutputAudioDeltaEvent audio:
            PlaybackQueue.Enqueue(Convert.FromBase64String(audio.Delta));
            break;
        case ResponseFunctionCallArgumentsDoneEvent call:
            string result = await RunTool(call.Name, call.Arguments);
            await client.SendAsync(new ConversationItemCreateEvent(new FunctionCallOutputItem(call.CallId, result)));
            await client.SendAsync(new ResponseCreateEvent());
            break;
        case RealtimeErrorEvent error:
            Console.WriteLine($"Error: {error.Error.Message}");
            break;
    }
}

Function Calling

Define tools on the session (reusing Universal.OpenAI.Client.V1.Responses.Tool), force a specific tool when needed via tool_choice, and round-trip the result as a FunctionCallOutputItem:

await client.SendAsync(new SessionUpdateEvent(new RealtimeSessionConfiguration
{
    Tools = new[]
    {
        new Tool
        {
            Type = ToolTypes.Function,
            Name = "get_current_time",
            Description = "Returns the current time.",
            Parameters = new { type = "object", properties = new { }, required = Array.Empty<string>() }
        }
    }
}));

Conversation Items

ConversationItem covers messages (SystemMessageItem/UserMessageItem/AssistantMessageItem), function calls (FunctionCallItem/FunctionCallOutputItem), MCP tool interactions (McpToolCallItem, McpApprovalRequestItem, McpApprovalResponseItem, McpListToolsItem), and item references — each dispatched on its own type/role discriminators, mirroring how Universal.OpenAI.Client models the Responses API's polymorphic output items.

Disposal

RealtimeClient implements IDisposable, and CloseAsync initiates a clean WebSocket close handshake before disposal:

using RealtimeClient client = await RealtimeClient.ConnectAsync(apiKey, Models.GptRealtime2);
// ...
await client.CloseAsync();
// Client will be automatically disposed