Architecture
How embedcache works
One request path, one cache key, one file of state. Here is what happens between a string coming in and a vector coming out.
/v1/embed or the crateread vector from SQLite
run FastEmbed (ONNX, CPU) → write to SQLite
The cache key
Every vector is stored under a key derived from two things: a hash of the input content and the
model identifier that produced it. This matters because the same text embedded with
BGESmallENV15 and with BGEBaseENV15 are different vectors — the model is
part of the identity. Change either the content or the model and you get a fresh embedding; keep
both the same and you get a cache read.
Local inference with FastEmbed
On a cache miss, embedcache runs the model through the FastEmbed crate on the ONNX runtime. This is a CPU-first path — no GPU is required for any bundled model. Because weights are pinned ONNX files, the same model id produces the same vector over time, which is exactly what makes caching sound.
SQLite as the only state
The cache lives in a single SQLite file (DB_PATH, default cache.db). It
survives restarts, moves between environments as one file, and can be deleted to start cold. There
is no external cache server, no Redis, and no cloud dependency. embedcache is not itself a vector
database — once you have the vectors, index and search them in Qdrant, pgvector, LanceDB, or similar.
Chunking on the way in
The /v1/process endpoint fetches a URL, splits it into chunks, embeds each chunk, and
caches the results. The default words chunker splits on whitespace and needs no LLM.
If you configure an LLM_PROVIDER, the llm-concept and
llm-introspection chunkers use Ollama or OpenAI to find semantic boundaries instead.
Two ways to embed it
As a Rust library, you depend on the crate and call it in-process — no network hop at all. As a
REST service, any language can POST to /v1/embed and /v1/process, with
Swagger, ReDoc, RapiDoc, and Scalar available for exploring the API. Either way the cache is shared
through the same SQLite file.