A caching layer that owns the embedder

Why this exists

Most RAG pipelines call a hosted embedding endpoint — OpenAI's text-embedding-3-small, Cohere's embed-v3, Voyage, and the like. Two frustrations recur:

• Stable inputs get billed repeatedly. The same document chunks pass through the same model and produce the same vector — over and over — because the cache was never wired up.
• Rate limits stall backfills. Embedding a large corpus against a hosted API is governed by someone else's quota, with throttling and retries baked in.

embedcache picks a different starting point: run the model on the box you already own, store the vector keyed by the content hash and the model identifier, and serve subsequent requests from SQLite.

What's in the box

• A fastembed-backed embedder with 22+ models — BGE small / base / large, MiniLM, Nomic, multilingual E5, and the rest of the FastEmbed catalogue.
• A SQLite cache that survives restarts and is the only piece of state the service owns.
• A REST surface (/v1/embed, /v1/process, /v1/params) with Swagger, ReDoc, RapiDoc, and Scalar docs pre-mounted.
• Optional LLM-driven chunking (llm-concept, llm-introspection) for callers who want semantic boundaries instead of whitespace splits.

What it deliberately isn't

It isn't a vector database — bring your own Qdrant, pgvector, LanceDB, or Milvus. It isn't a caching proxy in front of OpenAI — if you need that shape, you want LangChain's CacheBackedEmbeddings wrapped around your existing API client. And it isn't a managed service: you build it, you run it, you keep the cache file.

Who built it

Skelf Research is the lab that ships this project. The code is GPL-3.0; contributions are welcome via the GitHub repo. For commercial questions or larger collaborations, reach out.