- 💡 Summary
Add native download presets for FUTO's Audio Context Fine-Tuning (ACFT) Whisper models directly inside the LeanType Voice Plugin model selection menu.
- 🛑 The Problem
While the plugin currently supports pre-packaged OpenAI stock models (Base, Tiny, and Small), they introduce notable friction into the on-device voice input experience:
-
Hallucinations & Infinite Repetition: Stock OpenAI models frequently hallucinate, enter endless repetition loops, or output severe text stutters when processing short, variable-length conversational voice snippets—an issue especially pronounced with the stock Tiny model.
-
Higher Latency: Stock models lack optimization for dynamic conversational context windows, leading to slower text output and trailing processing delays.
-
Typing Fluidity: These output errors and inference delays directly disrupt UI snappiness and the overall flow of dictation.
-
Onboarding Friction: While users can manually side-load .bin files via the "Custom GGML Model" tab, forcing them to source, convert, and map files externally degrades the out-of-the-box experience.
- ✨ Proposed Solution
Introduce dedicated UI download presets (e.g., FUTO Base (ACFT), FUTO Tiny (ACFT), and FUTO Small (ACFT)) within the Whisper Models popup. Both LeanType and FUTO champion non-commercial, privacy-respecting free software principles, making this an ideal collaborative addition to the offline keyboard ecosystem.
Model Selection Note: FUTO lists English-only variants at the top of their page (https://keyboard.futo.tech/voice-input-models), while the multilingual versions sit further down. Targeting these multilingual builds is recommended to ensure the plugin seamlessly handles international users out of the box.
This feature will provide a seamless transition path for users migrating over from FUTO apps or anyone looking to completely avoid the friction of manually importing and mapping external model weights.
- ⚙️ Technical Justification & Resources
FUTO's ACFT models are specifically tailored to maintain high accuracy and rapid inference execution when processing short, variable-length local audio streams without triggering loops or stutters. When paired with the whisper.cpp engine backend, these optimized weights streamline the execution pipeline and significantly reduce on-device processing overhead.
Maintainers can leverage these official resources to construct the automated download endpoints and verify model structures:
- 🔀 Alternatives Considered
-
Status Quo: Sticking solely to stock OpenAI models, which forces privacy-conscious users to deal with frustrating repetition bugs and higher transcription latency on short clips.
-
Manual Setup Only: Leaving users to manually configure external .bin files via the advanced settings tab, limiting widespread adoption of robust, high-performance local weights.
I know not too long ago you did "Migrated offline voice input to Whisper AI with compact Q5_1 quantized models and in-toolbar waveform visualizer", but my suggestion would help for the community 🙏
- 💡 Summary
Add native download presets for FUTO's Audio Context Fine-Tuning (ACFT) Whisper models directly inside the LeanType Voice Plugin model selection menu.
- 🛑 The Problem
While the plugin currently supports pre-packaged OpenAI stock models (Base, Tiny, and Small), they introduce notable friction into the on-device voice input experience:
Hallucinations & Infinite Repetition: Stock OpenAI models frequently hallucinate, enter endless repetition loops, or output severe text stutters when processing short, variable-length conversational voice snippets—an issue especially pronounced with the stock Tiny model.
Higher Latency: Stock models lack optimization for dynamic conversational context windows, leading to slower text output and trailing processing delays.
Typing Fluidity: These output errors and inference delays directly disrupt UI snappiness and the overall flow of dictation.
Onboarding Friction: While users can manually side-load .bin files via the "Custom GGML Model" tab, forcing them to source, convert, and map files externally degrades the out-of-the-box experience.
- ✨ Proposed Solution
Introduce dedicated UI download presets (e.g., FUTO Base (ACFT), FUTO Tiny (ACFT), and FUTO Small (ACFT)) within the Whisper Models popup. Both LeanType and FUTO champion non-commercial, privacy-respecting free software principles, making this an ideal collaborative addition to the offline keyboard ecosystem.
Model Selection Note: FUTO lists English-only variants at the top of their page (https://keyboard.futo.tech/voice-input-models), while the multilingual versions sit further down. Targeting these multilingual builds is recommended to ensure the plugin seamlessly handles international users out of the box.
This feature will provide a seamless transition path for users migrating over from FUTO apps or anyone looking to completely avoid the friction of manually importing and mapping external model weights.
- ⚙️ Technical Justification & Resources
FUTO's ACFT models are specifically tailored to maintain high accuracy and rapid inference execution when processing short, variable-length local audio streams without triggering loops or stutters. When paired with the whisper.cpp engine backend, these optimized weights streamline the execution pipeline and significantly reduce on-device processing overhead.
Maintainers can leverage these official resources to construct the automated download endpoints and verify model structures:
Official Model Documentation: https://keyboard.futo.tech/voice-input-models
Fine-Tuning Source Repository: https://github.com/futo-org/whisper-acft
Model Weights Collection: https://huggingface.co/collections/futo-org/whisper-acft
- 🔀 Alternatives Considered
Status Quo: Sticking solely to stock OpenAI models, which forces privacy-conscious users to deal with frustrating repetition bugs and higher transcription latency on short clips.
Manual Setup Only: Leaving users to manually configure external .bin files via the advanced settings tab, limiting widespread adoption of robust, high-performance local weights.
I know not too long ago you did "Migrated offline voice input to Whisper AI with compact Q5_1 quantized models and in-toolbar waveform visualizer", but my suggestion would help for the community 🙏