-
toon-format
Token-Oriented Object Notation (TOON) - a token-efficient JSON alternative for LLM prompts
-
llm
unifying multiple LLM backends
-
tool-parser
Tool/function call parser for LLM model outputs
-
llmfit
Right-size LLM models to your system hardware. Interactive TUI and CLI to match models against available RAM, CPU, and GPU.
-
liter-llm
Universal LLM API client — 142+ providers, streaming, tool calling. Rust-powered, type-safe, compiled.
-
sqz-cli
Universal LLM context compressor — squeeze tokens from prompts, code, JSON, logs, and conversations
-
gobby-squeeze
YAML-configurable output compressor for LLM token optimization
-
llm-tokenizer
LLM tokenizer library with caching and chat template support
-
wikidesk-server
MCP server that wraps LLM-wiki into a shared research service for AI coding agents
-
llm_models_spider
Auto-updated registry of LLM model capabilities (vision, audio, etc.)
-
adk-model
LLM model integrations for Rust Agent Development Kit (ADK-Rust) (Gemini, OpenAI, Claude, DeepSeek, etc.)
-
dynamo-llm
Dynamo LLM Library
-
ba
task tracking for LLM sessions
-
mistralrs
Fast, flexible LLM inference
-
noether-cli
Noether CLI: ACLI-compliant command-line interface for stage management, composition graph execution, and LLM-powered compose
-
zeph-llm
LLM provider abstraction with Ollama, Claude, OpenAI, and Candle backends
-
ast-outline
Fast, AST-based structural outline for source files. Built for LLM coding agents and humans.
-
ruvllm
LLM serving runtime with Ruvector integration - Paged attention, KV cache, and SONA learning
-
udiffx
Parse and apply LLM-optimized unified diff + XML file changes
-
llm-connector
Next-generation Rust library for LLM protocol abstraction with native multi-modal support. Supports 12+ providers (OpenAI, Anthropic, Google, Aliyun, Zhipu, Ollama, Tencent, Volcengine…
-
yomo
A QUIC-based runtime for AI-LLM tool routing and serverless execution
-
aichat
All-in-one LLM CLI Tool
-
wgpu-llm-cli
Terminal-based chat interface for the wgpu LLM inference engine
-
meerkat
Modular, high-performance agent harness for LLM-powered applications
-
sqz-engine
Adaptive multi-pass LLM context compression engine — content-aware pipeline with AST parsing, token counting, session persistence, and budget tracking
-
edgequake-llm
Multi-provider LLM abstraction library with caching, rate limiting, and cost tracking
-
error-toon
Compress verbose browser errors for LLM consumption. Save 70-90% tokens.
-
gsqz
YAML-configurable output compressor for LLM token optimization
-
dsct
LLM-friendly packet dissector CLI
-
adk-agent
Agent implementations for Rust Agent Development Kit (ADK-Rust, LLM, Custom, Workflow agents)
-
writestead
LLM Wiki
-
kalosm-sample
A common interface for token sampling and helpers for structered llm sampling
-
noether-engine
Noether composition engine: Lagrange graph AST, type checker, planner, executor, semantic index, LLM-backed composition agent
-
ai-agents-llm
LLM providers for AI Agents framework
-
git-prism
Agent-optimized git data MCP server — structured change manifests and full file snapshots for LLM agents
-
runok
Command execution permission framework for LLM agents
-
llama-mcp-server
Local LLM inference MCP server powered by llama.cpp
-
elif-core
Core architecture foundation for the elif.rs LLM-friendly web framework
-
soul-core
Async agentic runtime for Rust — steerable agent loops, context management, multi-provider LLM abstraction, virtual filesystem, WASM-ready
-
llm-kit-provider
Provider interface and traits for the LLM Kit - defines the contract for implementing AI model providers
-
magi-core
LLM-agnostic multi-perspective analysis system inspired by MAGI
-
ifran
Local LLM inference, training, and fleet management platform
-
hoosh
AI inference gateway — multi-provider LLM routing, local model serving, speech-to-text, and token budget management
-
adaptive-card-mcp
MCP server exposing Adaptive Cards v1.6 tools (validate, optimize, transform, analyze) over stdio for any LLM client
-
sqz-mcp
MCP server for sqz — expose LLM context compression over Model Context Protocol (stdio/SSE)
-
tokf
Config-driven CLI tool that compresses command output before it reaches an LLM context
-
branchforge
Graph-first Rust runtime for durable LLM agents
-
valta-cli
CLI for valta — JSON repair and validation for LLM outputs
-
noos
Reliability layer for Rust LLM agents: scope drift, cost circuit breaks, and procedural correction memory as event-driven Decisions
-
rustia-rs
Rust version of typia.io for type-safe JSON validation and LLM JSON parsing
-
tirea-extension-observability
LLM inference and tool-call telemetry aligned with OpenTelemetry GenAI conventions
-
llm-transpile
High-performance LLM context bridge — token-optimized document transpiler
-
typia
Rust version of typia.io for type-safe JSON validation and LLM JSON parsing
-
multi-llm
Unified multi-provider LLM client with support for OpenAI, Anthropic, Ollama, and LMStudio
-
toon-rust
Token-Oriented Object Notation (TOON) - JSON for LLM prompts at half the tokens. Rust implementation.
-
liter-llm-bindings-core
Shared utilities for liter-llm language bindings — case conversion, config parsing, error formatting, runtime management
-
cognate-cli
CLI tool for interacting with LLM providers via Cognate
-
miyabi-llm
LLM abstraction layer for Miyabi - GPT-OSS-20B integration
-
langchain-rust
LangChain for Rust, the easiest way to write LLM-based programs in Rust
-
llm_hunter
forensic research of LLM gguf files and more
-
qubit-metadata
Type-safe extensible metadata model for the Qubit LLM SDK
-
noether-grid-worker
RESEARCH — noether-grid worker: advertises LLM capacity, runs graphs on request
-
agent-sdk
Rust Agent SDK for building LLM agents
-
dumbo-rs
Turn any codebase into LLM-ready context : supports monorepos, multi-project runs, and git diffs
-
context-builder
CLI tool to aggregate directory contents into a single markdown file optimized for LLM consumption
-
cupel
Context window management pipeline for LLM applications
-
cli-pdf-extract
Fast Rust CLI wrapper around pdf_oxide for LLM-friendly PDF extraction
-
erinra
Memory MCP server for LLM coding assistants
-
peon-runtime
A runtime-agnostic agent execution engine with zero-trust context injection, multimodal messaging, and pluggable LLM providers
-
god-gragh
A graph-based LLM white-box optimization toolbox: topology validation, Lie group orthogonalization, tensor ring compression
-
swink-agent
Core scaffolding for running LLM-powered agentic loops
-
sochdb
LLM-optimized database with native vector search
-
toondb
LLM-optimized database with native vector search
-
mistralrs-quant
Fast, flexible LLM inference
-
codecat
「 Merge Code Repository into a Single File | Respects
.gitignore| Ideal for LLM Code Analysis 」 -
smg
High-performance model-routing gateway for large-scale LLM deployments
-
llmnop
A command-line tool for benchmarking the performance of LLM inference endpoints
-
siumai
A unified LLM interface library for Rust
-
aether-llm
Multi-provider LLM abstraction layer for the Aether AI agent framework
-
rtk-lite-cc
Lightweight CLI proxy for Claude Code — minimizes LLM token consumption by filtering command outputs
-
llama-rs
A high-performance Rust implementation of llama.cpp - LLM inference engine with full GGUF support
-
tower-llm
A Tower-based framework for building LLM & agent workflows in Rust
-
reson-agentic
Agents are just functions - production-grade LLM agent framework
-
wonk
Structure-aware code search CLI for LLM coding agents
-
ruvllm-wasm
WASM bindings for RuvLLM - browser-compatible LLM inference runtime with WebGPU acceleration
-
synoema-types
Synoema — programming language optimized for LLM code generation
-
loki-ai
An all-in-one, batteries included LLM CLI Tool
-
lkr-cli
CLI for LLM Key Ring — manage LLM API keys via macOS Keychain
-
lnmp-llb
LNMP-LLM Bridge Layer - Optimization layer for LLM prompt visibility and token efficiency
-
cli-denoiser
CLI proxy that strips terminal noise for LLM agents. Zero false positives.
-
fzp
Fuzzy Processor - parallel LLM inference pipe filter
-
hermes-llm
LLM training from scratch using Candle
-
llm-stack
Core traits, types, and tools for the llm-stack SDK
-
ought-agent
Provider-agnostic agent loop driving an Llm against a ToolSet
-
iron_runtime
Agent runtime with LLM request routing and translation
-
swiftide-query
Fast, streaming indexing, query, and agentic LLM applications in Rust
-
tiycore
Unified LLM API and stateful Agent runtime in Rust
-
llmkit
Production-grade LLM client - 100+ providers, 11,000+ models. Pure Rust.
-
llm-utl
Convert code repositories into LLM-friendly prompts with smart chunking and filtering
-
swarm-engine-llm
LLM integration backends for SwarmEngine
-
ferrum-interfaces
Core trait contracts for the Ferrum LLM inference engine
-
debugger-cli
LLM-friendly debugger CLI using the Debug Adapter Protocol
-
cllient
A comprehensive Rust client for LLM APIs with unified interface and model management
-
tiy-core
Unified LLM API and stateful Agent runtime in Rust
-
oris-mutation-evaluator
Mutation quality evaluator with static analysis and LLM critic
-
llm_runtime
Abstractions and primitives for building agents and runtimes on top of llm_adapter
-
chace
CHamal's AutoComplete Engine - An LLM based code completion engine
-
engram-agent
Reusable LLM agent loop with tool-calling and lifecycle hooks
-
liter-llm-cli
CLI for liter-llm — start an OpenAI-compatible proxy server or MCP tool server
-
seqpacker
High-performance sequence packing for LLM training
-
ferrum-engine
Model orchestration engine for Ferrum LLM inference
-
smooai-smooth-operator
Smooth Operator — Rust-native AI agent framework with built-in checkpointing, tool system, and LLM client
-
oasis-sim
Round-based social simulation with LLM agents (feeds, votes, run_state.json I/O)
-
flyllm
unifying LLM backends as an abstraction layer with load balancing
-
pctx
Generate LLM-ready context from your codebase
-
rustia-llm
Rustia-powered LLM tool-calling adapter for aisdk
-
laminae-cortex
Self-improving learning loop for LLM applications — tracks user edits, extracts preferences, builds reusable instructions
-
llm-kit-openai-compatible
OpenAI-compatible provider implementation for the LLM Kit - supports OpenAI, Azure OpenAI, and compatible APIs
-
llm-tui-rs
Terminal UI for LLM chat with multi-provider support (Ollama, Claude, Bedrock)
-
llm-voice-bridge
Lightweight pipeline: text → LLM → VOICEVOX → WAV audio
-
ucp-llm
LLM-focused utilities for the Unified Content Protocol
-
yggdra
Airgapped agentic TUI for local LLM inference with tool execution
-
inference-lab
High-performance LLM inference simulator for analyzing serving systems
-
tirea-agent-loop
LLM inference engine, tool dispatch, and streaming execution loop for tirea
-
struct-llm
Lightweight, WASM-compatible library for structured LLM outputs using tool-based approach
-
mistralrs-paged-attn
Fast, flexible LLM inference
-
aof-llm
Multi-provider LLM abstraction layer
-
brainos-cortex
LLM provider abstraction, context assembly, and action dispatch for Brain OS
-
chatpack-cli
CLI tool for parsing and converting chat exports into LLM-friendly formats
-
rig-cat
LLM agent framework built on comp-cat-rs: typed effects, no async, categorical foundations
-
roboticus-llm
LLM client pipeline with circuit breaker, ML model router, semantic cache, and multi-format translation
-
limit-llm
Multi-provider LLM client for Rust with streaming support. Supports Anthropic Claude, OpenAI, and z.ai.
-
sgr-agent
SGR LLM client + agent framework — structured output, function calling, agent loop, 3 agent variants
-
llmtrace
Transparent proxy server for LLM API calls
-
prompty
asset class and format for LLM prompts
-
sqz-wasm
Browser WASM build of sqz — LLM context compression for browser extensions
-
schoolmarm
GBNF grammar-constrained decoding for LLM inference, ported from llama.cpp
-
tersify
Universal LLM context compressor — pipe anything, get token-optimized output
-
ask_llm
request to whatever llm is the best these days, without hardcoding model/provider
-
llmux
Hook-driven LLM model multiplexer with pluggable switch policy
-
rsmap
Generate multi-layered, LLM-friendly index files for Rust codebases
-
a3s-power
A3S Power — Privacy-preserving LLM inference for TEE environments
-
mistralrs-cli
Command-line interface for mistral.rs LLM inference
-
rsrvr
Store all your LLM Interactions
-
ferrum-scheduler
Request scheduling for Ferrum LLM inference engine
-
ferrum-types
Shared type definitions for the Ferrum LLM inference engine
-
backdisco
Discover backend origins from CDN frontends using LLM-assisted pattern analysis and brute force enumeration
-
truthlens
AI hallucination detector — formally verified trust scoring for LLM outputs
-
typia-llm
Typia-powered LLM tool-calling adapter for aisdk
-
mojentic
An LLM integration framework for Rust
-
noether-grid-protocol
RESEARCH — shared serde types for noether-grid (intra-company LLM pooling)
-
cosmoflow
type-safe workflow engine for Rust, inspired by PocketFlow and optimized for LLM applications
-
memvid-ask-model
LLM inference module for Memvid Q&A with local and cloud model support
-
bare-metal-kernels
Metal GPU kernels for LLM inference on Apple Silicon — 85+ optimized compute shaders
-
neith
Graph-based context orchestrator for LLM agent conversations
-
talu
Safe, idiomatic Rust SDK for talu LLM inference
-
onetool
Sandboxed Lua REPL for LLM tool use
-
openinference-semantic-conventions
OpenInference semantic conventions for LLM observability in Rust
-
web2llm
Fetch web pages and convert to clean Markdown for LLM pipelines
-
golem-ai-llm
working with LLM APIs on Golem Cloud
-
ferrum-testkit
Testing utilities for Ferrum LLM inference engine
-
llm-pipeline
Reusable node payloads for LLM workflows: prompt templating, Ollama calls, defensive parsing, streaming, and sequential chaining
-
swiftide-agents
Fast, streaming indexing, query, and agentic LLM applications in Rust
-
swiftide-langfuse
Fast, streaming indexing, query, and agentic LLM applications in Rust
-
moesniper
Escape-proof precision file editor for LLM agents. Hex-encoded content, line-range splicing, atomic writes.
-
llm-quota
CLI tool to inspect and report LLM usage quota information
-
llmy
All-in-one LLM utilities
-
cognate-llm
A modular, extensible LLM framework for Rust with multi-provider support, type-safe tools, and RAG capabilities
-
nous-judge
Async LLM-as-judge evaluators for Nous — plan quality, adherence, task completion
-
alchemy-llm
Unified LLM API abstraction layer supporting 10+ providers through a consistent streaming interface
-
llm-cost-dashboard
Real-time terminal dashboard for LLM token spend - cost/request, projected monthly bills, per-model breakdown
-
vloom
Fast, privacy-focused CLI for recording windows and generating LLM-optimized videos
-
shopify-approver-rig-agent
RIG-based agentic workflow for LLM orchestration with GLM/Claude
-
llm-orchestrator-audit
Tamper-proof audit logging system for LLM workflows with hash chain integrity
-
astrid-llm
LLM provider abstraction with streaming support for Astrid
-
kotoba-llm
Unified multi-vendor LLM client abstraction, supporting providers such as OpenAI, Anthropic, Google Gemini, etc
-
agent-orchestrator-sdk
Rust SDK for orchestrating LLM-powered agents, shared task execution, and teammate coordination
-
llm-extract
Extract structured data from LLM responses — fence strip, JSON repair, fuzzy repair, typed deserialization
-
llm-kit-anthropic
Anthropic provider for LLM Kit - Complete Claude integration with streaming, tools, thinking, and citations
-
mistralrs-server-core
Fast, flexible LLM inference
-
cargo-prompt
Recursively minify and concatenate source code into a markdown document for llm prompting
-
token-count
Count tokens for LLM models using exact tokenization
-
meerkat-client
LLM provider abstraction for Meerkat
-
tibet-oomllama
OomLlama — Sovereign LLM runtime with .oom format, Q2/Q4/Q8 quantization, and lazy-loading inference
-
attuned-core
Core types and traits for Attuned - human state representation for LLM systems
-
infernum-server
HTTP API server for local LLM inference
-
menta
Minimal Rust library for non-UI LLM and AI primitives
-
blazen-llm
LLM provider abstraction layer for the Blazen workflow engine
-
llm_providers
A unified source of truth for LLM providers, models, pricing, and capabilities
-
charter
Fast structural context generator for Rust codebases, optimized for LLM consumption
-
mistralrs-audio
Fast, flexible LLM inference
-
liter-llm-proxy
OpenAI-compatible LLM proxy server — model routing, virtual keys, rate limiting, cost tracking
-
llm-orchestrator-state
State persistence and recovery for LLM workflow orchestrator
-
valta-core
Core JSON repair and validation library for LLM outputs
-
fuzzy-parser
Fuzzy JSON repair for LLM-generated DSL
-
sema-llm
LLM provider integrations (Anthropic, OpenAI) for the Sema programming language
-
self-llm
Unified chat API for multiple LLM providers
-
llmg-core
Core types and traits for LLMG - LLM Gateway
-
gamecode-mcp2
Minimal, auditable Model Context Protocol server for safe LLM-to-system interaction
-
tokemon
Unified LLM token usage tracking across all providers
-
llm-cost-ops
Core library for cost operations on LLM deployments
-
mesh-llm-client
Low-level Rust client implementation for Mesh LLM embedded integrations
-
nb-mcp-server
MCP server wrapping the nb CLI for LLM-friendly note-taking
-
llm-kit-azure
Azure OpenAI provider for LLM Kit
-
turbine-llm
Unified Rust interface for multiple LLM providers with growing model support
-
onde-mistralrs
Fast, flexible LLM inference
-
mirror
unifying multiple LLM backends
-
sugars_llm
LLM integration and AI agent builder utilities
-
llm-text
processing text for LLM consumption
-
deemuk
Compress any text before it enters your LLM. Less tokens, same meaning.
-
llm-latency-lens-providers
Provider adapters for LLM Latency Lens
-
ferrum-cli
CLI for Ferrum — a Rust-native LLM inference engine
-
npcrs
Rust core for the NPC system — agent kernel, jinx executor, LLM client
-
agent-io
SDK for building AI agents with multi-provider LLM support
-
armyknife-llm-redteam
LLM red-teaming security scanner — nmap for LLMs
-
mecha10-nodes-llm-command
Natural language command parsing via LLM APIs (OpenAI, Claude, Ollama)
-
onde-mistralrs-quant
Fast, flexible LLM inference
-
pmetal-models
LLM model architectures for PMetal
-
llmtrace-proxy
Transparent proxy server for LLM API calls
-
cortexai-providers
LLM provider integrations for Cortex: OpenRouter, OpenAI, Anthropic and more
-
nexus-orchestrator
Distributed LLM model serving orchestrator - unified API gateway for heterogeneous inference backends
-
llm_client
easiest Rust interface for local LLMs
-
llm-kit-xai
xAI (Grok) provider implementation for the LLM Kit - supports chat, image generation, and agentic tools
-
ralphloop
A CLI tool for creating and running Ralphloops with LLM integration
-
rosetta-aisp-llm
LLM fallback for AISP conversion using Claude SDK - extends rosetta-aisp with AI-powered conversion
-
rake-sandbox
Secure LLM agent sandbox — mount files, analyse with Claude/OpenAI/Ollama/llama.cpp, WASM-isolated
-
llm-incident-manager
Enterprise-grade incident management system for LLM operations
-
neuromance
controlling and orchestrating LLM interactions
-
jamjet-models
JamJet model adapter layer — unified interface for LLM providers
-
llm-toolkit
A low-level, unopinionated Rust toolkit for the LLM last mile problem
-
ironclad-llm
LLM client pipeline with circuit breaker, ML model router, semantic cache, and multi-format translation
-
babel
Provide Rust enums for Groq, SambaNova, Openrouter's llm model names
-
compression-prompt
Fast statistical compression for LLM prompts - 50% token reduction with 91% quality retention
-
mentedb-extraction
LLM-powered memory extraction engine for MenteDB
-
gatekpr-rig-agent
RIG-based agentic workflow for LLM orchestration with GLM/Claude
-
llm-here-core
Detection + dispatch logic for LLM CLIs and API providers — the library half of llm-here
-
infernum-paimon
LLM Studio - Teaches arts, sciences, and gives good familiars
-
lsp-llm
Opt-in LLM advisor for axon-lsp, gated behind the
llmCargo feature. Never on the critical path: deterministic capabilities (diagnostics, hover, completion) work without this crate. -
llm-relay
Shared types, format conversion, and HTTP client for Anthropic and OpenAI LLM APIs
-
bare-metal-gguf
GGUF binary format parser for bare-metal LLM inference — zero-copy mmap, all quantization types
-
fig2json
CLI tool to convert Figma .fig files to LLM-friendly JSON format
-
llm-kit-core
Core functionality for the LLM Kit - unified interface for building AI-powered applications
-
orchard-rs
Rust client for Orchard - high-performance LLM inference on Apple Silicon
-
trimcp
MCP proxy that reduces LLM token costs by 60–90% through compression and caching
-
llm-kit-groq
Groq provider implementation for the LLM Kit - supports chat and transcription models
-
saorsa-ai
Unified multi-provider LLM API
-
meritocrab-llm
LLM evaluator trait and implementations for the Meritocrab reputation system
-
nuro-llm
LLM provider abstractions and implementations for Nuro
-
llm-security
Comprehensive LLM security layer to prevent prompt injection and manipulation attacks
-
mistralrs-vision
Fast, flexible LLM inference
-
onde-mistralrs-paged-attn
Fast, flexible LLM inference
-
llm-kit-openai
OpenAI provider implementation for the LLM Kit
-
lc-cli
LLM Client - A fast Rust-based LLM CLI tool with provider management and chat sessions
-
llm-kit-huggingface
Hugging Face provider for LLM Kit
-
llm-cascade
Resilient cascading LLM inference with automatic failover across multiple providers
-
llm-orchestrator-secrets
Secret management for LLM Orchestrator with Vault, AWS Secrets Manager, and environment variable support
-
rig-openapi-tools
Turn any OpenAPI spec into LLM-callable tools for rig
-
dkdc-lm-cli
CLI for dkdc-lm: local LLM inference management
-
astmap
— code structure index with transitive impact analysis for LLM coding tools
-
legalis-llm
LLM integration layer for Legalis-RS
-
bare-metal-reference
Numerical validation harness for bare-metal LLM inference kernels
-
ferrum-sampler
Sampling strategies for Ferrum LLM inference engine
-
vex-llm
LLM provider integrations for VEX
-
llm-daemon
LLM as a daemon
-
serde_mask
Mask sensitive data during serde serialization for LLM ingestion
-
llm-registry-core
Core domain types and models for the LLM Registry - A secure, production-ready registry for Large Language Models
-
prompt-sentinel
A high-performance CLI tool for LLM prompt regression testing
Try searching with DuckDuckGo or on crates.io.