Protocol Documentation

Undefined audio

ULAW 8000 HZ, 1 byte per sample

ALAW 8000 HZ, 1 byte per sample

PCM 8000 HZ, 2 bytes per sample

PCM 16000 HZ, 2 bytes per sample

PCM 22050 HZ, 2 bytes per sample

Used to indicate that no audio resource should be allocated

Undefined VAD event type

VAD begins processing audio

Barge-in occurred, audio that will be process by the ASR starts here. This notification might be useful to stop prompt playback for example

End-of-speech occurred, no further audio will be processed by VAD for the specified interaction. If the setting InteractionASR_VoiceActivityDetection.AUTO_FINALIZE_ON_EOS is true, the ASR will immediately finish processing audio at this point

VAD timed out waiting for audio barge-in (start-of-speech). The audio manager will no longer process audio for this interaction.

VAD timed out waiting for audio barge-out (end-of-speech). The audio manager will no longer process audio for this interaction.

SessionCreate Creates a new session and returns its ID and session related messages through response streamed callback messages The returned session_id (uuid) can be used when making other requests for the session. Also, optionally creates a new audio resource using the specified audio_format parameters. Only one audio resource can be created per session. Audio will be added to this audio resource via gRPC (AudioPush or AudioStream) Typically audio would either be streamed in with AudioStream or sent in blocks using AudioPush. Both methods push audio into the same internal audio resource for processing

SessionClose Closes the specified session. Once closed, a session can no longer be referenced for requests and should be assumed to be no longer valid.

AudioStream Sends a stream of binary audio data into the specified audio resource. Note that this may be called before an interaction exists, which allows audio to be added before creating interactions that will process the audio.

AudioPush Sends a block of binary audio data into the specified audio resource. Note that this may be called before an interaction exists, which allows audio to be added before creating interactions that will process the audio. Please consider the gRPC maximum message size limits

AudioPull Returns a block of audio data from an audio resource. A begin point in milliseconds and maximum length can be specified to return a segment of the audio data. By default, all audio data within the resource is returned. Please note that due to GRPC maximum message length limitations, API clients may want to retrieve audio in more manageable chunk sizes. Using the default of always returning the entire audio buffer may not be advisable in all situations. Please consider the gRPC maximum message size limits

SessionSetSettings Applies configuration changes to specified session settings.

SessionGetSettings Returns a JSON encoded string containing the requested session settings.

InteractionCreateASR Creates a new ASR interaction for the specified session. This type of object is required to access ASR functionality. Use the returned interaction_id in subsequent ASR requests.

InteractionCreateTTS Creates a new TTS interaction for the specified session. This type of object is required to access TTS functionality. Use the returned interaction_id in subsequent TTS requests.

InteractionCreateGrammarLoad Requests a grammar be loaded within the specified session/interaction. The returned interaction_id may be referenced in subsequent ASR requests.

InteractionCreateGrammarParse Create a new grammar parse interaction for the specified session. A grammar parse interaction allows sending text directly, to be parsed by the active grammars. Essentially this is the same as an ASR interaction, but the speech to text functionality is skipped. The raw text is passed in directly instead of having the ASR engine supply the text from the audio. The text is parsed with the active grammars in the same way as an ASR interaction. The returned interaction_id may be used to determine status of this request as well as to access results, when processing is completed.

InteractionSetSettings Adds or modifies the specified settings to the specified interaction. Settings not mentioned in this call will remain unaffected.

InteractionGetSettings Return a JSON encoded string containing the current settings for the specified interaction.

InteractionBeginProcessing Begins processing the specified interaction. Typically, any interaction settings that are needed should be set before calling InteractionBeginProcessing. Calling this function triggers backend services to begin processing the audio or text being inputted

InteractionFinalizeProcessing Used to force VAD complete when VAD is used, or after VAD speech begin. Takes all available resource audio and triggers an ASR decode. This is optional most of the time, when the default auto-decode setting is used. This can also be used when performing DTMF or Text type interactions Results for the interaction may be available during subsequent calls to InteractionRequestResults

InteractionRequestResults Returns an interaction's results as a JSON encoded string. Note that an empty JSON object may be returned if no results are currently available

InteractionCancel Cancels the specified interaction. Any active processing related to the interaction is stopped.

InteractionClose Closes the specified interaction.