Skip to content

Track deferred ElevenLabs webhook transcription and conversational voice APIs #2440

Description

@lforst

Sub-issue of #2293 tracking the remaining conversational execution surfaces from that issue, plus asynchronous transcription mode. These APIs are intentionally excluded from the current request/response instrumentation.

Deferred surfaces:

  • client.speechToText.convert({ ..., webhook: true }): the request returns an acknowledgement and the transcription arrives at a separately configured webhook.
  • client.speechEngine.attach(...) and SpeechEngine.Server: inbound WebSocket session handling rather than an application-initiated model request returning its result.
  • SpeechEngine.Session transcript callbacks and session.sendResponse(...): the application receives inbound transcripts and sends response text to ElevenLabs; sending text does not return the synthesized audio to that application process. Session lifecycle/listener methods are also uninstrumented.
  • Conversational voice connection/run APIs mentioned in [bot] Instrument ElevenLabs (749,855 weekly downloads) #2293: verify the exact package and execution entrypoints before implementation. Browser voice connections are provided by the separate @elevenlabs/client SDK; client.conversationalAi REST authentication and management endpoints should not be treated as model-generation calls merely because they relate to agents.

Future work needs an explicit tracing contract for asynchronous jobs or inbound conversational sessions: correlation across requests/processes, span ownership and lifetime, cancellation/disconnect behavior, and how final outputs are captured. Do not represent job acknowledgements or outbound response text as completed model generations.

The current implementation deliberately removes Speech Engine instrumentation from both manual and automatic paths and skips webhook transcription. Any future coverage should include realistic lifecycle e2e tests and preserve the request/response integration's existing trace contract.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions