FG
🔌 APIs & SDKsTwilio

Realtime Google Speech Transcription

Freshabout 21 hours ago
Mar 14, 20260 views
Confidence Score52%
52%

Problem

I tried Twilio Speech recognition and was not so happy with the accuracy of conversion to text. I wanted to use Google Speech API for transcription and I was following this article: https://medium.com/@mheavers/better-twilio-transcriptions-with-the-google-web-speech-api-eb24274c5e3 There, they record the speech and then send the details to Google Speech API. Is there any way to do it realtime without hanging up the call. Something like a replacement for Twilio Speech Recognition. Thanks in advance.

Unverified for your environment

Select your OS to check compatibility.

1 Fix

Canonical Fix
Unverified Fix
New Fix – Awaiting Verification

Implement Real-time Google Speech Transcription with Twilio

Medium Risk

Twilio's built-in speech recognition may not provide the desired accuracy for transcription. To achieve better results, integrating Google Speech-to-Text API for real-time transcription can be a solution. The challenge is to capture audio in real-time during a Twilio call without disconnecting the call.

Awaiting Verification

Be the first to verify this fix

  1. 1

    Set up Google Cloud Speech-to-Text

    Create a Google Cloud project and enable the Speech-to-Text API. Generate API credentials (API key or service account) to authenticate requests.

  2. 2

    Capture Audio Stream from Twilio

    Use Twilio's <Record> verb to capture audio during the call. Configure it to stream the audio to your server in real-time using WebSockets or a similar technology.

    xml
    <Response><Record action='https://your-server.com/audio' method='POST' /></Response>
  3. 3

    Stream Audio to Google Speech-to-Text

    On your server, receive the audio stream and send it to the Google Speech-to-Text API in real-time. Use the streaming recognition feature of the API to transcribe audio as it is received.

    javascript
    const speech = require('@google-cloud/speech');
const client = new speech.SpeechClient();

const request = {
  config: {
    encoding: 'LINEAR16',
    sampleRateHertz: 16000,
    languageCode: 'en-US',
  },
  interimResults: true,
};

const recognizeStream = client
  .streamingRecognize(request)
  .on('data', data => process.stdout.write(data.results[0].alternatives[0].transcript + '\n'));

// Pipe the audio stream to Google Speech API
stream.pipe(recognizeStream);
  4. 4

    Send Transcription Back to Twilio

    Once you receive the transcription from Google Speech-to-Text, send the results back to Twilio using Twilio's API to update the call or send an SMS with the transcription.

    javascript
    const twilio = require('twilio');
const client = new twilio('TWILIO_ACCOUNT_SID', 'TWILIO_AUTH_TOKEN');

client.messages.create({
  body: transcription,
  from: 'YOUR_TWILIO_NUMBER',
  to: 'RECIPIENT_NUMBER'
});
  5. 5

    Test and Optimize

    Conduct tests to ensure that the audio is being captured, sent to Google, and returned to Twilio correctly. Optimize audio quality and API settings based on results.

Validation

Confirm the fix by making a test call through Twilio, ensuring that the audio is captured and transcribed in real-time. Check that the transcription is sent back to the specified recipient without any call interruptions.

Sign in to verify this fix

Environment

Submitted by

AC

Alex Chen

2450 rep

Tags

twiliosmsapitype:-question