Voice conversations as a feature preview, through Unmute - #326
Merged
Conversation
Settings gains a Feature previews section, and its first entry is voice conversations: a deployment switches it on and names the URL of an Unmute deployment, and a Voice button appears above the chat that opens Unmute's interface in an overlay with the microphone delegated to it. Unmute (Kyutai, MIT) wraps any OpenAI-compatible text model with streaming speech recognition, semantic turn-taking, and synthesis, and the model it wraps here is the Studio itself: the RAG endpoint now serves studio-voice, the full assistant with its tools and grounding under a spoken-answer style, one or two plain sentences, no markdown or paths, a few words before a slow tool call. deploy/workflow-unmute.yaml stands the stack up on a GPU node from the upstream compose with the bundled LLM disabled and the backend pointed at the Studio, published as a platform session. The transcript stays with Unmute in this slice; recording voice turns as Studio conversations comes when the turns flow through the Studio's own client. Two tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Settings gains a Feature previews section; its first entry is voice conversations. A deployment switches it on and names the URL of an Unmute deployment (
voiceEnabled,voiceUrl, orVOICE_ENABLED=1/VOICE_URL),/api/configreportsfeatures.voice, and a Voice button appears above the chat that opens Unmute's interface in an overlay (allow="microphone; autoplay", with an open-in-new-tab fallback).Unmute (Kyutai, MIT) wraps any OpenAI-compatible text model with streaming STT, semantic turn-taking, and TTS. The model it wraps here is the Studio itself: the RAG endpoint now serves
studio-voice[/<gateway-model>], the full assistant (tools, grounding, personas) with a spoken-answer style appended to the system prompt: one or two plain sentences, no markdown, lists, or paths, a few words before a slow tool call. It is advertised alongsidestudio-agentin/v1/models.deploy/workflow-unmute.yamlstands the stack up on a GPU node from the upstream compose: clones the repo, overlays a compose file that points the backend at the Studio (KYUTAI_LLM_URL/MODEL/API_KEY), disables the bundledllmservice, publishes traefik on the endpoint agent's port, and serves it as a platform session. Unmute needs a 16 GB GPU for STT plus TTS; the Studio's model runs wherever it already does.Known limit of this slice: the transcript stays with Unmute; recording voice turns as Studio conversations comes when the turns flow through the Studio's own client. Help guide updated; two tests; 153 server and 9 web pass.