RagMeUp.Client 0.4.0

RAG engine

Document RAG over Postgres + pgvector. The engine owns ingestion, embedding, and reranking; the app owns filtered retrieval as direct SQL on tables it owns.

service/   Python FastAPI engine (`ragmeup` package): /embed, /documents, /rerank
client/    C# read-path library (RagMeUp.Client): rank, build passages, rerank, read spans
db/        schema.sql — documents + chunks tables, HNSW + GIN indexes

Each half carries its own Azure DevOps pipeline (*/azure-pipelines.yml) and releases independently — the client to nuget.org, the engine as a container image. See CI/CD.

Database

psql "$DATABASE_URL" -f db/schema.sql   # needs pgvector >= 0.7.0

Service

Models live on a volume mounted at /models, not baked into the image, so code changes rebuild and push only small layers. A one-shot seed populates the volume.

cd service
DATABASE_URL=postgres://… docker compose up --build

Compose runs model-seed first (downloads weights to the models volume), then starts engine offline against it. On OpenShift, the equivalent is a PVC at /models seeded by an init Job (or initContainer) running python -m ragmeup.download_models.

Local dev without containers: pip install -r requirements.txt then DATABASE_URL=… uvicorn ragmeup.main:app (from service/). Models download from the hub on first run.

Image flavours

One Dockerfile, two bases, selected by FLAVOR — the shared tail (deps, app, entrypoint) is written once so the flavours cannot drift:

docker build .                          # cuda, the default — GPU host (~8 GB)
docker build --build-arg FLAVOR=cpu .   # cpu — CI and a GPU-less VPS (~2.4 GB)

The CPU flavour installs torch from PyTorch's CPU wheel index; a plain pip install torch would pull the CUDA build and ~2.5 GB of nvidia-* wheels that a CPU host can never use.

Configuring a container

Everything is environment variables — the image has no config file. Only DATABASE_URL is required; every other knob has a working default, and the model paths are already set in the image.

docker run -d --name rag-engine \
  -e DATABASE_URL=postgres://user:pass@db:5432/ragdb \
  -e RAG_TORCH_THREADS=4 \
  -v models:/models -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 \
  -p 127.0.0.1:8000:8000 \
  ghcr.io/jgauffin/ragmeup-engine:latest

Two of these are not really optional in production:

• Publish on loopback, not 0.0.0.0. No endpoint is authenticated and /documents writes with a privileged connection that bypasses your RLS by design. The engine is a trusted internal service — see deployment: trust boundary & scaling.
• Set RAG_TORCH_THREADS. Unset, torch claims every logical core, which over-subscribes SMT pairs (measured: 8 → 16 threads at 0.98x, i.e. slower), and concurrent /rerank calls then fight for the whole machine.

For a Linux VPS there are two ready-made deployments. service/deploy/vps/ is the recommended one — Podman quadlets with unattended, health-gated auto-update, and the engine on an internal network with no published port at all; see hosting the engine on a Linux VPS. service/docker-compose.vps.yml is the compose equivalent for a host without Podman: it pulls the published image, reserves no GPU, binds loopback, and reads the knobs above from a .env file.

The container also serves GET /healthz — cheap, dependency-free, and 200 once the models are resident. Both deployments gate their health check on it.

Client

The app builds an NpgsqlDataSource with UseVector() and points HttpClient.BaseAddress at the engine. Filters live in the app's own SQL, which produces the candidate chunk-id set passed to Rank.

var q        = await rag.Embed(query);            // engine /embed
long[] ids   = await app.FilteredCandidates(...); // app's WHERE: RLS, geo, joins
var matches  = await rag.Rank(conn, q, query, ids, k: 50); // same `query` as embedded
var passages = rag.BuildPassages(matches);
var top      = await rag.Rerank(query, passages, topK: 5);
var spans    = await rag.ReadSpans(conn, top);    // clean original text for /answer

Methods drop the Async suffix — the whole client is async, so it adds nothing.

Tests

Both sides isolate the expensive boundaries (the GPU models, the database) so the logic around them is tested directly, and a thin layer of integration tests exercises the real SQL against Postgres+pgvector.

A throwaway pgvector instance for the integration tests:

docker run -d --name rag-test-pg -e POSTGRES_PASSWORD=test -e POSTGRES_DB=ragtest \
  -p 55432:5432 pgvector/pgvector:pg16

Service (service/) — unit tests need no model and no DB (the model boundary is mocked); the /documents integration test needs the DB above.

cd service
python -m venv .venv && .venv/Scripts/pip install -r requirements-dev.txt
.venv/Scripts/python -m pytest                       # unit only (integration auto-skips)
RAG_TEST_DATABASE_URL=postgres://postgres:test@localhost:55432/ragtest \
  .venv/Scripts/python -m pytest                     # + the /documents integration test

Client (client/) — unit tests are pure (no DB/HTTP). The integration tests use RAG_TEST_DATABASE_URL if set, otherwise spin a pgvector container via Testcontainers; they auto-skip when neither is available.

cd client
dotnet test tests/RagMeUp.Client.Tests               # unit
RAG_TEST_DATABASE_URL=postgres://… \
  dotnet test tests/RagMeUp.Client.IntegrationTests  # rank/read-path vs real pgvector

Both suites honour the same RAG_TEST_DATABASE_URL, so one local pgvector serves both. The integration tests reset their tables on start, so point it at a scratch database, not one with data you care about.

Retrieval quality (client/tests/RagMeUp.Client.EvalTests) — a labelled corpus in eval/ measured through the real read path. Needs a running engine with real weights as well as the database, and skips without one. Reports recall / MRR / nDCG rather than asserting a bar; the assertions are comparative, proving the harness can see a deliberate sabotage. See measuring retrieval quality.

RAG_EVAL_ENGINE_URL=http://127.0.0.1:8000 RAG_TEST_DATABASE_URL=postgres://… \
  dotnet test tests/RagMeUp.Client.EvalTests

The engine's own DATABASE_URL must point at that same database — it ingests, the harness reads. Do not run this concurrently with the integration suite against one database: both truncate the corpus tables. Set RAG_EVAL_DATABASE_URL to separate them.