Open source C/C++ engine for high-performance local LLM inference and on-device AI.
-
Updated
Jul 14, 2026 - C++
#
model-runtime
Here are 7 public repositories matching this topic...
Data-only launcher contracts and runtime-readiness surfaces for PCCX™ model integration.
python cli workflow runtime descriptor launcher inference compatibility diagnostics gemma readiness edge-ai kv260 llm model-runtime ai-accelerator evidence-gated pccx pccx-v002
-
Updated
May 31, 2026 - Python
Portable, provider-neutral model invocation contracts, adapters, and conformance tools
-
Updated
Aug 31, 2026 - TypeScript
A self-contained native runtime and model format for local AI video generation.
-
Updated
Sep 1, 2026
GPT-OSS B20 Local Execution. Lightweight local environment for running it with Python 3.12 and CUDA acceleration. - Run GPT-OSS B20 entirely offline - Optimize text generation with GPU - Enable fast, secure inference on consumer hardware.
text-generation performance-optimization gpu-optimization edge-ai inference-acceleration secure-inference model-runtime minimal-setup llm-inference open-source-llm local-execution offline-inference privacy-preserving-ai consumer-gpu gpt-oss-b20 lightweight-environment
-
Updated
Aug 13, 2025 - Python
A complete inference runtime for open-weight large language models, enabling efficient execution through streaming weights, quantization, and memory-aware scheduling
machine-learning deep-learning numpy inference quantization inference-engine memory-mapping inference-optimization kv-cache ai-inference cpu-inference large-language-models llm model-runtime safetensors llm-inference gguf streaming-inference gguf-runner streaming-inference-engine
-
Updated
Aug 14, 2026 - Python
Yollama | "Your-Ollama! Take control with true data privacy, plus, real offline Freedom for you and your LLM AI model inferencing!
ai mistral inference-engine inference-api model-runner ai-model ai-tools llm model-runtime llamacpp llama-cpp llm-inference llm-tools mistralai mistral-ai ai-coding deepseek-r1 deepseek-ai deepseek-v4 mistral-small-4-119b
-
Updated
Aug 27, 2026 - Go
Add this topic to your repo
To associate your repository with the model-runtime topic, visit your repo's landing page and select "manage topics."