embedcache

FAQ

Frequently asked questions

What embedcache is, what it is not, and the practical details that come up most often.

Is embedcache a caching proxy for OpenAI or Cohere embeddings?
No. embedcache replaces the hosted embedder rather than memoizing it. It runs models locally via FastEmbed. If you specifically want to cache calls to a hosted API, that is the LangChain CacheBackedEmbeddings pattern, not this project.
Is embedcache a vector database?
No. embedcache generates and caches embeddings; it does not index or search them. Pair it with a vector database such as Qdrant, pgvector, LanceDB, or Milvus for similarity search.
Do I need a GPU?
No. embedcache is CPU-first — the primary supported path is CPU via the ONNX runtime. Every bundled model runs without a GPU.
What models are available?
22+ ONNX models from the FastEmbed catalogue, including BGE small/base/large, MiniLM, Nomic, and multilingual E5. Call GET /v1/params to list the models in your build.
How is the cache keyed?
By a hash of the input content plus the model identifier. The same text under a different model is a different vector and a different cache entry, so mixing models is safe.
Where is the cache stored?
In a single SQLite file at DB_PATH (default cache.db). It survives restarts, moves between environments as one file, and can be deleted to start cold.
Can I use it as a library instead of a service?
Yes. Depend on the embedcache crate and call it in-process — no server and no network hop. The REST service is for callers in other languages.
What is the license?
embedcache is released under GPL-3.0 and developed in the open on GitHub. It is published on crates.io.
What is LLM chunking for?
The /v1/process endpoint chunks a document before embedding. The default words chunker splits on whitespace. Optional llm-concept and llm-introspection chunkers use Ollama or OpenAI to find semantic boundaries instead.