1. Install

embedcache is published on crates.io. Install the standalone service with cargo:

cargo install embedcache

To use it as a library instead, add the crate to your Cargo.toml:

cargo add embedcache

2. Run the service

Start the server. On first run it downloads the default model weights and creates the cache file:

embedcache
# listening on http://127.0.0.1:8081
# cache: ./cache.db (sqlite)

Configuration comes from the environment or a .env file. The common variables:

SERVER_HOST — bind address (default 127.0.0.1)
SERVER_PORT — port (default 8081)
DB_PATH — cache file path (default cache.db)
ENABLED_MODELS — comma-separated model list, e.g. BGESmallENV15,AllMiniLML6V2

3. Embed some text

POST an array of strings and a model name to /v1/embed:

curl -X POST localhost:8081/v1/embed \
  -H "content-type: application/json" \
  -d '{"model":"BGESmallENV15","text":["hello world"]}'

You get back the vectors, the model used, and a cache indicator per input.

4. Watch the cache hit

Send the same request again. Because the vector is keyed by the content hash and model id, the second call is a cache read from SQLite rather than a fresh inference. Nothing leaves your machine, and no hosted API is billed.

Process a whole URL

To fetch, chunk, embed, and cache a document in one call, use /v1/process. The default words chunker splits on whitespace; if you configure an LLM_PROVIDER (Ollama or OpenAI) you can select llm-concept or llm-introspection for semantic boundaries.

curl -X POST localhost:8081/v1/process \
  -d '{"url":"https://example.com/page","model":"BGESmallENV15"}'

Explore the API

Interactive documentation is mounted at /swagger, /redoc, /rapidoc, and /scalar. Call GET /v1/params to list the models and chunkers available in your build. For the full reference, see the official docs.

From install to first cached vector