Search plugin deployment (post-P12)¶

Since the P12 pivot (#4598), the Python in-process search daemon is gone. The Rust nexus-search-plugin cdylib, hosted by nexusd-cluster and reached over gRPC, is the sole search backend for the Nexus server. A server deployment that does not run the plugin host boots with search disabled — the boot probe fail-softs (logs a warning, /api/v2/search/* returns 503) rather than failing the boot.

This page is the deployment contract: which processes you run, how they are wired, and what changed for corpora indexed under the pre-P12 stack.

Required processes¶

Both images are published from the same commit with the same tag set (edge on develop/main, stable/latest/semver on version tags, plus the commit SHA), so a deployable pairing is pinnable by digest. Publishing is done by .github/workflows/docker-publish.yml (#4613); the e2e-edge smoke job runs the exact published plugin-host image, not an in-CI build.

The plugin-host image is reproducible locally from a nexus + nexus-vfs checkout (the nexus-vfs rev is pinned in the workspace Cargo.toml):

docker build \
  -f dockerfiles/Dockerfile.nexusd-cluster-plugins \
  --build-context nexus-vfs-src=../nexus-vfs \
  --secret id=plugin_signing_key,env=PLUGIN_SIGNING_PRIVKEY \
  -t nexusd-cluster:latest .

Wiring¶

1. Start the plugin host with the plugin dir enabled:

docker run -d --name nexus-cluster \
  -p 2126:2126 \
  -v nexus-workspace:/workspace \
  ghcr.io/nexi-lab/nexusd-cluster:stable \
  --bind-addr 0.0.0.0:2126 \
  --data-dir /app/data \
  --plugin-dir /plugins

(--no-tls --insecure-no-auth are available for trusted loopback / compose topologies; production should keep TLS + auth on.)

1. Point the server at it:

-e NEXUS_SEARCH_PLUGIN_TARGET=<cluster-host>:2126

The server's boot probe (server/lifespan/search.py) health-checks the target with a 5 s timeout. Unreachable ⇒ search disabled for the whole server lifetime with a loud warning; fix the wiring and restart the server.

1. Shared filesystem: on Index / Refresh (directory walks) the plugin reads files by absolute path through its own VFS/filesystem — the server and the plugin host must see the same bytes at the same paths (shared volume in Docker, same host mount, or the kernel VFS in federated topologies). IndexDocuments (what files/write index: true, search/refresh and search/index use) carries the text inline and needs no shared filesystem.

Write-to-searchable contract (#4736)¶

A write does not index. The server relays nothing to the plugin on a write: the plugin's NotifyFileChange carries no text, so a create / update event would be a pure ack (it answers skipped and mutates nothing), and the server's VFS hooks only send delete for deleted and renamed paths. The plugin has no background poller or scheduler: text enters the index only through an explicit indexing call. A document written with a bare POST /api/v2/files/write is therefore never searchable, however long you wait — pair the write with one of the calls below. The same holds for overwrites: writing new bytes to an already-indexed path leaves the previous text searchable until the path is re-indexed, so pair every write of an indexed file with index: true or a refresh.

All indexing calls are synchronous: the response returns after the plugin has committed, so there is no "queued" state to poll for.

Every indexing call indexes into the caller's token zone — the same zone GET /api/v2/search/query reads (the ?zone= override on files/write wins when given) — so what you index is what your queries hit. GET /api/v2/search/stats is scoped the same way and echoes the zone as zone_id.

index: true on a write and search/refresh decode the written bytes as UTF-8. A binary payload is reported skipped with a reason instead of being silently acknowledged — pass index: {"text": "<extracted text>"} to index it. Empty or whitespace-only content is skipped too (409 on refresh).

When index is requested and the search plugin is unreachable, the write fails with 503 before anything is written, so a write+index request never half-succeeds. A plugin failure after the write landed is reported as index.status = "error" (HTTP 200: the write itself succeeded); re-drive indexing with search/refresh or search/index.

Confirming a document is served¶

Every indexing call returns the plugin's index_seq: a plugin-wide, persisted, monotonic sequence stamped after the commit that made the batch visible. GET /api/v2/search/stats reports:

Stall signature — the #4725 class, index acknowledged but nothing served: pending > 0 or indexing_in_progress > 0 for longer than one indexing call should take, while last_successful_index_at does not advance. Alert on that instead of running a synthetic write probe.

The sequence lives in <data>/plugins/search/index_seq.json and survives plugin-host restarts, so a seq captured before a restart still compares correctly afterwards.

Environment reference (plugin host)¶

Set these on the nexusd-cluster process — the plugin reads process env directly.

And on the server:

Transport security: plaintext is only accepted to same-machine targets — loopback addresses and Docker's host.docker.internal alias (which by construction resolves to the machine running the container). A cross-host deployment must either terminate TLS on the plugin host (pair with NEXUS_SEARCH_PLUGIN_TLS* on the server) or explicitly set NEXUS_SEARCH_PLUGIN_ALLOW_INSECURE=true and accept that anyone on the network path can read queries and index content. There is NO blanket default for the opt-in: changing NEXUS_SEARCH_PLUGIN_TARGET to a remote host fails closed (search disabled with a loud boot warning) until you choose TLS or plaintext deliberately.

Plugin signing¶

The kernel refuses unsigned plugins — PluginLoader verifies an Ed25519 detached signature (<plugin>.so.sig next to the dylib) against compile-time trusted keys. The published image ships the signed pair; if you build your own, sign with scripts/sign_plugin.py and a key the kernel trusts (CI uses the PLUGIN_SIGNING_PRIVKEY secret). An unsigned or foreign-signed plugin is skipped at --plugin-dir scan time and search stays down.

Embedding backends and model provisioning¶

Keyword (BM25/tantivy) search works out of the box. The semantic / hybrid lane needs an embedder, one of:

Local ONNX (default)¶

The plugin loads multilingual-e5-small (384-dim) from NEXUS_SEARCH_MODEL_DIR. Provision the directory with the layout a huggingface-cli download intfloat/multilingual-e5-small produces:

<model-dir>/
  model.onnx
  tokenizer.json
  config.json
  special_tokens_map.json
  tokenizer_config.json

and set ORT_DYLIB_PATH to a matching ONNX Runtime library (1.19+ for the pinned ort 2.0 rc). Neither the model nor the ORT dylib is baked into the published image (size + licensing); mount them and set the two env vars. Missing model/dylib degrades gracefully: semantic queries return a typed "unavailable" error, keyword search keeps working.

Remote / API (#4614)¶

Set NEXUS_SEARCH_EMBED_API_URL + _MODEL + _DIM (+ _API_KEY) and the plugin embeds via any OpenAI-compatible endpoint instead of local ONNX — no model files or ORT dylib needed. This is the migration path for deployments whose corpus quality was tuned on API embeddings (e.g. 1536-dim models) under the pre-P12 stack. A configured remote endpoint wins over the local model; a partially configured one (URL without model/dim) fails loudly rather than silently falling back.

Migrating a pre-P12 corpus¶

The pre-P12 stack embedded via API models into pgvector; the post-P12 plugin owns its own tantivy (FTS) + HNSW (ANN) indices under <data>/plugins/search/<zone>/. There is no index migration — a corpus indexed under the previous stack must be fully reindexed after the upgrade (nexus reindex --target search, then nexus search index <path> or your indexing pipeline).

Choose the embedding backend before reindexing:

• Local mE5-small (384-dim): zero external dependencies, but a different (generally weaker) vector space than 1536-dim API models — re-validate retrieval quality and fusion tuning (alpha, etc.) on your own eval set.
• Remote/API embedder pointed at your previous model: preserves the vector space your tuning was measured on. The ANN index is still rebuilt (different storage engine), but embedding quality is unchanged.

The ANN directory tag encodes the embedder (ann-<tag>-v<n>, e.g. ann-mE5-small-v1-v2 or ann-api-text-embedding-3-small-1536-v2), so switching embedders later re-tags and rebuilds alongside the old directory instead of corrupting it.