Open source C/C++ engine for high-performance local LLM inference and on-device AI.
-
Updated
Mar 14, 2026 - C++
#
model-runtime
Here are 2 public repositories matching this topic...
GPT-OSS B20 Local Execution. Lightweight local environment for running it with Python 3.12 and CUDA acceleration. - Run GPT-OSS B20 entirely offline - Optimize text generation with GPU - Enable fast, secure inference on consumer hardware.
text-generation performance-optimization gpu-optimization edge-ai inference-acceleration secure-inference model-runtime minimal-setup llm-inference open-source-llm local-execution offline-inference privacy-preserving-ai consumer-gpu gpt-oss-b20 lightweight-environment
-
Updated
Aug 13, 2025 - Python
Improve this page
Add a description, image, and links to the model-runtime topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the model-runtime topic, visit your repo's landing page and select "manage topics."