Mattbusel/tokio-prompt-orchestrator

github.comtool

[tokio-prompt-orchestrator] - Tokio pipeline for serving LLM requests with dedup, circuit breakers, retries, rate limits and a dead-letter queue in front of Anthropic, OpenAI, llama.cpp or vLLM.

from

Rust

added

2026-10-12

likes

0

similar

1. ▲

   tokio-rs/prost github.com

   tool link ↗
   
2. ▲

   tqwewe/kameo github.com

   Fault-tolerant Async Actors Built on Tokio

   tool link ↗
   tool
3. ▲

   tokyo-tojo-json github.com

   tool link ↗
   
4. ▲

   tokio-rs/tokio github.com

   A runtime for writing reliable, asynchronous, and slim applications with the Rust programming language.

   tool topic programmingtopic books & writing link ↗
   tool
5. ▲

   tokio-rs/axum github.com

   Ergonomic and modular web framework built with Tokio, Tower, and Hyper

   tool link ↗
   tool
6. ▲

   cdrs-tokio crates.io

   ] - production-ready async Apache Cassandra driverclient

   site link ↗
   

Rust › Libraries > Artificial Intelligence > Tooling: “[tokio-prompt-orchestrator] - Tokio pipeline for serving LLM requests with dedup, circuit breakers, retries, rate limits and a dead-letter queue in front of Anthropic, OpenAI, llama.cpp or vLLM.”