Skip to main content

Make transcription and dictation resilient to network interruptions

Handling unstable networks is made simpler thanks to the stateless nature of the websockets.

In a real-time streaming configuration, as soon as you send audio you get results back. This means that if you lose your connection with Nabla servers, you can resume later with practically no loss, as long as you keep a buffer between the audio source (e.g., microphone) and Nabla servers.

Maintaining a buffer of audio chunks​

Since audio keeps flowing from its source regardless of network interruptions, you need to locally buffer the audio you are streaming to the websocket.

  • Real-time streaming: the buffer remains practically empty, as it's filled from the audio source at 1× real time and emptied at the same rate.
  • Network interruption: the buffer keeps filling but is not emptied, since emptying happens when Nabla servers acknowledge your audio. Make sure the buffer can hold the full encounter's audio, even though interruptions should remain short.
  • Catching up: once the connection is back, empty the buffer at the fastest rate allowed by Nabla servers, which is capped by the acknowledgements described below.
Connection lostwebsocket closedConnection backnew websocketBuffer drainedReal-time streamingfills at 1×empties at 1×Network interruptionfills at 1×empties at 0×Catching upfills at 1×empties faster than 1×Back to real timefills at 1×empties at 1×audio in buffer
Size of the client audio buffer before, during and after a network interruption.

Leveraging acknowledgements to empty the buffer​

Acknowledgements tell you which audio chunks you can delete from your buffer. Enabling them has these implications:

  • Enforced sequentiality: each audio chunk must have a seq_id. Within a stream, ids increase by 1 from one audio chunk to the next: seq_id(chunk(n+1)) = seq_id(chunk(n)) + 1. This is a strict requirement (the websocket closes with an error otherwise), because out-of-order audio yields bad or irrelevant transcription. The first seq_id can be any integer, but we recommend starting at zero for each new encounter and keeping ids sequential across websockets, for easier debugging of buffer consistency.
  • Cumulative acknowledgements: an acknowledgement frame with "ack_id": 42 covers all audio chunks with a seq_id less than or equal to 42. Nabla servers may batch the acknowledgement of several chunks into one frame: acknowledgement is not one-to-one.
  • Cooperative handling of backpressure: don't stream faster than Nabla servers acknowledge receipt. More than 10 seconds of audio sent but not received yet results in a buffer overflow error.

Handling a network interruption​

When the websocket closes unexpectedly, open a new one (see Error handling for which close codes to retry and the backoff), send the same configuration, then resend the audio chunks still in your buffer in seq_id order, with their original seq_id.

Transcript item offsets (start_offset_ms, end_offset_ms) are relative to the start of the audio streamed on the current websocket: when merging transcripts across websockets, shift them by the duration of the audio that precedes the first chunk sent on the new websocket.

Acknowledgements and reconnection​

Acknowledgements worked differently on API versions 2026-10-21 and earlier: if you still use one of them, switch to its tab.

Enabling acknowledgements​

Send enable_acks: true in the configuration (first frame) of the transcription websocket:

{
"type": "CONFIG",
"encoding": "PCM_S16LE",
"sample_rate": 16000,
"speech_locales": ["ENGLISH_US"],
"streams": [{ "id": "single_stream", "speaker_type": "UNSPECIFIED" }],

"enable_acks": true
}

Nabla servers then send two kinds of acknowledgements:

  • Receipt acknowledgement, once the audio has been received:

    { "type": "AUDIO_CHUNK_ACK", "stream_id": "single_stream", "ack_id": 42 }

    Use it to pace your streaming: only audio sent and not received yet counts toward the 10-second limit.

  • Transcription acknowledgement, once the audio has been fully transcribed:

    { "type": "TRANSCRIPTION_ACK", "stream_id": "single_stream", "ack_id": 42 }

    Use it to delete audio chunks from your buffer.

For a given seq_id, the receipt acknowledgement always comes before the transcription acknowledgement. Your buffer is a first-in-first-out queue where each audio chunk goes through these states:

sent to NablaAUDIO_CHUNK_ACKTRANSCRIPTION_ACKqueuedcaptured, not sentsentnot received yetreceivednot transcribed yetdeletedremoved from bufferre-sent on a new websocket after an interruption

Reconnecting​

  • Resend every buffered audio chunk, received ones included: the audio Nabla had received but not transcribed yet was lost with the websocket.
  • Discard draft transcript items (is_final: false): their audio is still in your buffer, so the new websocket transcribes it and sends new transcript items for it. Keeping the drafts would duplicate text.

Step through an interruption:

Client audio bufferWebsocket open
1
2
queuedsentreceiveddeleted
Client transcript
No transcript item yet
Step 1 of 8: Audio is captured
Microphone → buffer: chunks 1 and 2

Each audio chunk gets the next seq_id and enters the buffer as queued. Audio capture never stops: two new chunks enter the buffer at every step.

Ending the stream​

After the end frame { "type": "END" }, the server sends the remaining final transcript items and transcription acknowledgements, then closes the websocket normally (close code 1000): all your audio has been transcribed. If the websocket closes in any other way, handle it as a network interruption.

Handling edge-cases​

User ends encounter during catch-up period​

Transcription network resilience example catching up period

Example prompt for when ending an encounter while still catching-up

Given that users can end their encounter at any moment, you might in some case need a special handling. For instance, if they end the encounter right after an interruption then the buffer is still not emptied yet (still in the catching-up period) so the transcription is still not fully done.

In this case, we recommend you stop capturing audio (to stop filling the buffer) and prompt the user with the extra-waiting time there will be to catch-up, while keeping it possible to skip and generate the note right away.

Also, don't forcefully close the websocket once all audio has been streamed: send the end frame and wait for the server to close it, as described in "Ending the stream" above.

Dictation​

The dictation websocket has a single acknowledgement kind, enabled with enable_audio_chunk_ack, on every API version.



Don't hesitate to contact our support team if you have any questions or feedback.