A decision library for running local LLMs on multi-GPU machines without NVLink: parallelism, communication, quantization, speculative decoding, KV cache. Every conclusion carries its evidence grade and reproducible conditions.
benchmarking self-hosted nvidia performance-tuning quantization multi-gpu pcie model-parallelism inference-optimization nvlink kv-cache tensor-parallelism llm llmops vllm local-llm speculative-decoding llm-deployment gpu-serving decision-guide
-
Updated
Oct 16, 2026 - Python