WeKnora: grounded answers, scoped memory and deliberate operations
WeKnora memory recall: fix the candidate pool before tuning relevance scores
Read scoped vector ranking, candidate growth and reciprocal rank fusion through a concrete low-importance memory regression.
What you will learn
- A relevant item cannot win if it never becomes a candidate
- Broader search must preserve both scope and candidate identity
- Fuse ranks, then respect the output budget
Before you start
- Basic HTTP and container concepts
- Understanding of documents, passages and model providers
Separate ingestion, retrieval, answer support and memory scope, then design an evidence-based acceptance exercise.
Key takeaways
- Candidate selection can impose a hidden recall ceiling.
- Expanding candidates must preserve tenant and lifecycle filters.
- Rank fusion avoids adding incomparable raw scores.
A relevant item cannot win if it never becomes a candidate
The inspected repository method searches the subject’s vector set instead of only vectors belonging to an importance-first window. Its SQL filters tenant, subject, embedding model, dimensions, active status, expiry and optional kinds before distance ordering. A relevance stage cannot rescue a memory excluded before it sees the query.
Imagine one highly important but weakly related note and a low-importance exact match. Cutting to the first note and then ranking by similarity makes the second permanently unreachable. Our synthetic regression demonstrates this ordering problem with assigned scores; it does not measure real embeddings or reproduce the database engine.
Broader search must preserve both scope and candidate identity
The repository uses database distance calculation when its searchable vector column is available; otherwise it scores transferred vectors in process, with a 5000-row defensive cap. This is not an unrestricted cross-tenant search. The result query retains lifecycle and scope filters, and returned items are loaded through the scoped repository.
mergeVectorHits appends previously absent semantic matches to the candidate pool and returns ranking indexes into that shared pool. It skips excluded IDs and copies before the first append, avoiding modification of a caller-owned backing slice. Searching beyond the old window without growing the fusion pool would lose the benefit.
Fuse ranks, then respect the output budget
The service combines lexical and vector rankings using reciprocal rank fusion. At this revision the implementation adds 1/(60+rank), with rank starting at zero, and keeps stable ordering for equal scores. It does not add raw lexical counts directly to cosine similarity; those scales are not comparable.
Twelve tests cover our teaching model’s scope filters, old-window counterexample, threshold, pool growth, exclusions, stable fusion and input preservation. They are not upstream Go, SQL, halfvec or integration tests. Also note that an older lexical-file comment says recall is lexical-only; the current vector path and feature guide show why comments alone cannot define current behavior.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Trace the pool before calculating relevance.
- 2
Verify scope and lifecycle filters before ranking.
- 3
Keep semantic-only hits in the fusion pool.
- 4
Test a low-importance relevant item and forbidden-scope decoys.
Copy-ready example
const importantWindow = [{id: "popular", importance: 10, similarity: 0.55}];
const omittedMatch = {id: "relevant", importance: 1, similarity: 0.98};
// Synthetic illustration: scoring importantWindow cannot recover omittedMatch.
const fullScope = [...importantWindow, omittedMatch];
const ranked = fullScope.toSorted((a, b) => b.similarity - a.similarity);
console.log(ranked[0].id); // relevant — not a real embedding benchmarkFrequently asked questions
Would a better embedding model fix an excluded candidate?
No. A candidate omitted before scoring cannot be recovered by a better score.
Were the twelve tests run against PostgreSQL?
No. They execute a synthetic JavaScript teaching model, not upstream Go or a database.
Sources
- WeKnora / internal/application/repository/memory_vector.goSource checked 2026-09-14
- WeKnora / internal/application/service/memory/vector.goSource checked 2026-09-14
- WeKnora / internal/application/service/memory/recall_trace.goSource checked 2026-09-14
- WeKnora / internal/application/service/memory/search.goSource checked 2026-09-14
- WeKnora / internal/application/service/memory/lexical.goSource checked 2026-09-14