Audio stutter when trying to playback raw audio data

[AnasMations]

Hello, we are adding a facial animation model to Retell's voice agent so we need to pass in the raw audio data stream to our model as PCM16. We first convert that from Float32 to Uint8. The audio doesn't work and it starts stuttering badly, It used to work perfectly with the old websockets based implementation.

This is how it sounds 👇
https://github.com/user-attachments/assets/4fdb7359-9d1f-4c47-accc-2ec54df489d0

Thanks!

[gudj0]

Any update on this from the Retell team?

Our client is expecting uint8. We added the convertFloat32ToUnsigned8 from your docs: https://docs.retellai.com/knowledge/audio-basics

Still experiencing massive stutter.

Example snippet from our code:

    retellWebClient.on('audio', (audio: Float32Array) => {
        const processedAudio = convertFloat32ToUnsigned8(audio);
        simliClientRef.current.sendAudioData(processedAudio);
      } 
    });

[levihatchmatter]

I'm not sure if this will help with your particular issue but I ran into something similar. You could try wrapping the handler with useCallback.

const sendAudio = useCallback((audio: Float32Array) => {
        const processedAudio = convertFloat32ToUnsigned8(audio);
        simliClientRef.current.sendAudioData(processedAudio);
      }, [] );

retellWebClient.on('audio', sendAudio);