dogear

enter for all results · esc to close

openinfer-project/openinfer

github.comtool

LLM inference engine in pure Rust + CUDA with no PyTorch and no Python runtime — OpenAI-compatible API, paged KV cache, CUDA Graph, serving models from Qwen3 to the trillion-parameter Kimi-K2.

visit github.com ↗
0
sign in to dogear this
from
Rust
added
2026-10-10
likes
0

similar

  1. EricLBuehler/mistral.rs github.com
    tool category #rust link ↗

    [mistralrs] - Fast, flexible LLM inference engine supporting multimodal models, quantization (GGUF/GPTQ/ISQ), and OpenAI-compatible API

  2. rust-ml/linfa github.com
    tool category #rust link ↗

    Machine learning framework.

  3. LocalAI github.com
    tool category #go link ↗

    Open Source OpenAI alternative, self-host AI models.

  4. go-ml-benchmarks github.com
    tool category #go link ↗

    benchmarks for machine learning inference in Go.

  5. bitnet.cpp github.com
    tool category #generative-ai link ↗

    Official inference framework for 1-bit LLMs, by Microsoft. #opensource

  6. blackportal-ai/delta github.com
    tool category #rust link ↗

    Δ An Open-Source Machine Learning Framework in Rust.

Rust › Libraries > Artificial Intelligence > Machine learning: “LLM inference engine in pure Rust + CUDA with no PyTorch and no Python runtime — OpenAI-compatible API, paged KV cache, CUDA Graph, serving models from Qwen3 to the trillion-parameter Kimi-K2.”