Features
What embedcache ships
Local models, a persistent cache, a REST surface, and optional semantic chunking — in one small Rust binary with nothing to provision.
Local embeddings + cache
The foundation: run the model on-box and never recompute the same vector.
Local inference
Embeddings are generated on the box you already own via the FastEmbed crate and the ONNX runtime. No calls to OpenAI, Cohere, or Voyage. No per-token bill, no rate limits.
Content-hash cache
Every vector is stored in SQLite keyed by the content hash plus the model identifier. The same input under the same model is never embedded twice.
22+ bundled models
BGE small / base / large, MiniLM, Nomic, and multilingual E5 ship from the FastEmbed catalogue. Pick per request; enable a subset with ENABLED_MODELS.
Serving surface
Use it as a Rust crate or a self-hosted REST service with interactive docs.
Library or service
Depend on the embedcache crate directly in a Rust app, or run the standalone REST service in front of any language stack.
REST API
POST /v1/embed for text arrays, POST /v1/process to fetch-chunk-embed a URL, GET /v1/params to list models and chunkers.
Four docs UIs
Swagger, ReDoc, RapiDoc, and Scalar are mounted out of the box at /swagger, /redoc, /rapidoc, and /scalar for exploring the API interactively.
Chunking
Whitespace by default; optional LLM-driven semantic boundaries.
Whitespace chunking
The default "words" chunker splits on whitespace — always available, fast, and requires no LLM. Good enough for most ingest pipelines.
Optional LLM chunking
llm-concept and llm-introspection use Ollama or OpenAI to find semantic boundaries instead of whitespace splits, for callers who want cleaner chunks.
Operations
CPU-first, env-configured, single-file state, open source.
CPU-first
The primary supported path is CPU via the ONNX runtime. No GPU is required to run any bundled model.
Env-var configuration
SERVER_HOST, SERVER_PORT, DB_PATH, ENABLED_MODELS, and the LLM_* variables configure the service from the environment or a .env file.
Single-file state
The SQLite cache file is the only state the service owns. Back it up, ship it between environments, or delete it to start cold.
Open source
embedcache is published on crates.io and developed in the open under GPL-3.0. Read the code, file issues, send patches.