feat/rag-v1 #52

Merged
joshtronic merged 3 commits from feat/rag-v1 into master 2026-05-21 18:17:04 +00:00
Owner
No description provided.
feat: RAG v1 -- journal retrieval injected into discretionary user message
All checks were successful
Lint / check-sync (push) Successful in 4s
201e7a844a
First cut at the RAG layer. Scope: discretionary website ticks only.
Other tick types (issue work, PR review, maintenance) are untouched
for v1.

What it does:

- bin/setup-rag.sh -- idempotent Python venv setup. Creates a venv
  at $IGOR_STATE_DIR/rag-venv on first run; installs requirements.txt
  into it. Tracks an install stamp so subsequent calls are fast no-ops
  unless requirements.txt has actually changed (sha256 compare).
- bin/rag.py -- the actual tool. `build` walks brain/journal/, parses
  entries by `## ` headers, embeds them locally with fastembed
  (BAAI/bge-small-en-v1.5, 384 dims, ONNX, no GPU needed), and stores
  them in Redis via RedisVL. `query <text>` embeds a query and returns
  top-K relevant entries as markdown.
- Tick.sh: in the discretionary block, after the in-flight context is
  built and before the user message is assembled, run setup-rag.sh,
  rebuild the index, query with W_IN_FLIGHT as the query text, capture
  the result, and inject under "RELEVANT PAST CONTEXT" in the user
  message. All wrapped in error handling -- if any step fails, the
  tick proceeds with no RAG context rather than blocking.

Prerequisites added to docs/setup.md:

- python3 + python3-venv (the venv is auto-managed; no manual pip)
- Redis Stack (not vanilla Redis -- needs RediSearch for vector index)

Stale "fever-dream venue" reference dropped from the discretionary
user message while in the file (had been pointing at a now-deleted
identity.md section).

Future work: add RAG to other tick types, expose a Claude-callable
query helper (agent-recall.sh) for mid-tick on-demand retrieval,
embed memories/ in addition to journal entries.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
chore: install.sh pre-flights RAG prereqs + auto-bootstraps venv
All checks were successful
Lint / check-sync (push) Successful in 5s
b6040c47a1
The point of "bin/install.sh" is clone-and-go. Adding RAG broke that
slightly: new prereqs (python3-venv, redis-stack-server) and the
Python venv setup were undocumented and had to be done manually.

Changes:

- install.sh pre-flight now checks python3, python3-venv module, and
  redis-cli on PATH. Bails with apt-installable package suggestions.
- New pre-flight: redis-cli MODULE LIST must show "search" -- catches
  the common error of running vanilla redis-server instead of
  redis-stack-server.
- install.sh now runs bin/setup-rag.sh as part of install, so the
  Python venv exists and deps are installed BEFORE the first tick.
  Idempotent (no-op if requirements.txt is unchanged).
- README.md prereq section beefed up to call out the new requirements
  explicitly and point at docs/setup.md.
- docs/setup.md rewritten in the same shape; adds a one-liner check
  ("redis-cli MODULE LIST | grep -i search") so it's easy to verify
  Redis Stack is the active server.
docs: redis-stack-server is dead; modern path is redis-server 8.x
All checks were successful
Lint / check-sync (push) Successful in 4s
Lint / check-sync (pull_request) Successful in 5s
922d258d58
The setup docs and install.sh error messages I wrote referenced
"redis-stack-server" -- which no longer exists as a package for
current Debian codenames. Redis merged the search/vector modules
into base Redis 8+, so the modern path is just installing
redis-server from Redis's official apt repo (not Debian's older
redis-server package).

Hit this in practice tonight: apt couldn't find redis-stack-server.
The fix is conceptually the same -- add packages.redis.io to apt,
purge any Debian redis-server first, install redis-server from
Redis's repo, verify with redis-cli MODULE LIST | grep -i search.

Patches docs/setup.md, README.md, and the install.sh error message
to reflect the actual landscape.
joshtronic deleted branch feat/rag-v1 2026-05-21 18:17:04 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
joshtronic/igor!52
No description provided.