RagMeUp.Client 0.2.0

RAG engine

Document RAG over Postgres + pgvector. The engine owns ingestion, embedding, and reranking; the app owns filtered retrieval as direct SQL on tables it owns.

service/   Python FastAPI engine (`ragmeup` package): /embed, /documents, /rerank
client/    C# read-path library (RagMeUp.Client): rank, build passages, rerank, read spans
db/        schema.sql — documents + chunks tables, HNSW + GIN indexes

Each half carries its own Azure DevOps pipeline (*/azure-pipelines.yml) and releases independently — the client to nuget.org, the engine as a container image. See CI/CD.

Database

psql "$DATABASE_URL" -f db/schema.sql   # needs pgvector >= 0.7.0

Service

Models live on a volume mounted at /models, not baked into the image, so code changes rebuild and push only small layers. A one-shot seed populates the volume.

cd service
DATABASE_URL=postgres://… docker compose up --build

Compose runs model-seed first (downloads weights to the models volume), then starts engine offline against it. On OpenShift, the equivalent is a PVC at /models seeded by an init Job (or initContainer) running python -m ragmeup.download_models.

Local dev without containers: pip install -r requirements.txt then DATABASE_URL=… uvicorn ragmeup.main:app (from service/). Models download from the hub on first run.

Image flavours

One Dockerfile, two bases, selected by FLAVOR — the shared tail (deps, app, entrypoint) is written once so the flavours cannot drift:

docker build .                          # cuda, the default — GPU host (~8 GB)
docker build --build-arg FLAVOR=cpu .   # cpu — CI and a GPU-less VPS (~2.4 GB)

The CPU flavour installs torch from PyTorch's CPU wheel index; a plain pip install torch would pull the CUDA build and ~2.5 GB of nvidia-* wheels that a CPU host can never use.

Configuring a container

Everything is environment variables — the image has no config file. Only DATABASE_URL is required; every other knob has a working default, and the model paths are already set in the image.

docker run -d --name rag-engine \
  -e DATABASE_URL=postgres://user:pass@db:5432/ragdb \
  -e RAG_TORCH_THREADS=4 \
  -v models:/models -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 \
  -p 127.0.0.1:8000:8000 \
  ghcr.io/jgauffin/ragmeup-engine:latest

Two of these are not really optional in production:

• Publish on loopback, not 0.0.0.0. No endpoint is authenticated and /documents writes with a privileged connection that bypasses your RLS by design. The engine is a trusted internal service — see deployment: trust boundary & scaling.
• Set RAG_TORCH_THREADS. Unset, torch claims every logical core, which over-subscribes SMT pairs (measured: 8 → 16 threads at 0.98x, i.e. slower), and concurrent /rerank calls then fight for the whole machine.

service/docker-compose.vps.yml is a ready-made version of all of this for a Linux VPS: it pulls the published image, reserves no GPU, binds loopback, and reads the knobs above from a .env file.

Client

The app builds an NpgsqlDataSource with UseVector() and points HttpClient.BaseAddress at the engine. Filters live in the app's own SQL, which produces the candidate chunk-id set passed to Rank.

var q        = await rag.Embed(query);            // engine /embed
long[] ids   = await app.FilteredCandidates(...); // app's WHERE: RLS, geo, joins
var matches  = await rag.Rank(conn, q, query, ids, k: 50); // same `query` as embedded
var passages = rag.BuildPassages(matches);
var top      = await rag.Rerank(query, passages, topK: 5);
var spans    = await rag.ReadSpans(conn, top);    // clean original text for /answer

Methods drop the Async suffix — the whole client is async, so it adds nothing.

Tests

Both sides isolate the expensive boundaries (the GPU models, the database) so the logic around them is tested directly, and a thin layer of integration tests exercises the real SQL against Postgres+pgvector.

A throwaway pgvector instance for the integration tests:

docker run -d --name rag-test-pg -e POSTGRES_PASSWORD=test -e POSTGRES_DB=ragtest \
  -p 55432:5432 pgvector/pgvector:pg16

Service (service/) — unit tests need no model and no DB (the model boundary is mocked); the /documents integration test needs the DB above.

cd service
python -m venv .venv && .venv/Scripts/pip install -r requirements-dev.txt
.venv/Scripts/python -m pytest                       # unit only (integration auto-skips)
RAG_TEST_DATABASE_URL=postgres://postgres:test@localhost:55432/ragtest \
  .venv/Scripts/python -m pytest                     # + the /documents integration test

Client (client/) — unit tests are pure (no DB/HTTP). The integration tests use RAG_TEST_DATABASE_URL if set, otherwise spin a pgvector container via Testcontainers; they auto-skip when neither is available.

cd client
dotnet test tests/RagMeUp.Client.Tests               # unit
RAG_TEST_DATABASE_URL=postgres://… \
  dotnet test tests/RagMeUp.Client.IntegrationTests  # rank/read-path vs real pgvector

Both suites honour the same RAG_TEST_DATABASE_URL, so one local pgvector serves both. The integration tests reset their tables on start, so point it at a scratch database, not one with data you care about.