feat: sqlite-backed embedding cache -- 2min -> <5sec rebuilds #60

Merged

joshtronic merged 1 commit from feat/rag-embedding-cache into master

2026-05-21 19:54:26 +00:00

joshtronic commented

2026-05-21 19:54:04 +00:00

Owner

New sqlite cache at $IGOR_STATE_DIR/rag-embeddings.sqlite:

CREATE TABLE embeddings (
content_hash TEXT PRIMARY KEY, -- sha256(model_name|text)
embedding BLOB NOT NULL,
created_at TEXT NOT NULL
)

The model name is included in the hash, so switching embedders
auto-invalidates the cache (old hashes become orphans that just
never match again).

Flow on each tick:

Collect all source rows (journal + memory + commit), same as before
Hash each row's text + model -> content_hash
SELECT cached embeddings by hash
Embed only the misses (new journal entries, new commits, edited
memories)
INSERT new embeddings into cache
Push all rows (cached + new) into Redis (still flush + rebuild)

Self-healing intact:

Redis: still fully ephemeral, flushed every tick
sqlite cache: durable but trivially reconstructible -- delete the
file and the next tick rebuilds from journal/memories/git
Both layers can be nuked independently without breaking anything

Expected performance after merge:

First tick: same ~2min (rebuilds the cache from cold)
Steady-state ticks: <5sec (only a few new entries to embed,
usually 0-3 per tick)
Log line "rag: cache hit N/M, need to embed K new" tells the story

Added IGOR_RAG_CACHE_PATH knob in .env.example for visibility.

Co-Authored-By: Claude Opus 4.7 noreply@anthropic.com

Full re-embed every tick (860 entries / ~2min) was a tax that didn't earn its keep. Embeddings are deterministic derived data -- same model + same text always produces the same vector -- so caching them is safe as long as we invalidate when either changes. New sqlite cache at $IGOR_STATE_DIR/rag-embeddings.sqlite: CREATE TABLE embeddings ( content_hash TEXT PRIMARY KEY, -- sha256(model_name|text) embedding BLOB NOT NULL, created_at TEXT NOT NULL ) The model name is included in the hash, so switching embedders auto-invalidates the cache (old hashes become orphans that just never match again). Flow on each tick: 1. Collect all source rows (journal + memory + commit), same as before 2. Hash each row's text + model -> content_hash 3. SELECT cached embeddings by hash 4. Embed only the misses (new journal entries, new commits, edited memories) 5. INSERT new embeddings into cache 6. Push all rows (cached + new) into Redis (still flush + rebuild) Self-healing intact: - Redis: still fully ephemeral, flushed every tick - sqlite cache: durable but trivially reconstructible -- delete the file and the next tick rebuilds from journal/memories/git - Both layers can be nuked independently without breaking anything Expected performance after merge: - First tick: same ~2min (rebuilds the cache from cold) - Steady-state ticks: <5sec (only a few new entries to embed, usually 0-3 per tick) - Log line "rag: cache hit N/M, need to embed K new" tells the story Added IGOR_RAG_CACHE_PATH knob in .env.example for visibility. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

joshtronic added 1 commit

2026-05-21 19:54:04 +00:00

feat: sqlite-backed embedding cache -- 2min -> <5sec rebuilds

Lint / check-sync (push) Successful in 4s

Details

Lint / check-sync (pull_request) Successful in 4s

Details

b8608d954c

Full re-embed every tick (860 entries / ~2min) was a tax that
didn't earn its keep. Embeddings are deterministic derived data --
same model + same text always produces the same vector -- so
caching them is safe as long as we invalidate when either changes.

New sqlite cache at $IGOR_STATE_DIR/rag-embeddings.sqlite:

  CREATE TABLE embeddings (
      content_hash TEXT PRIMARY KEY,  -- sha256(model_name|text)
      embedding BLOB NOT NULL,
      created_at TEXT NOT NULL
  )

The model name is included in the hash, so switching embedders
auto-invalidates the cache (old hashes become orphans that just
never match again).

Flow on each tick:
  1. Collect all source rows (journal + memory + commit), same as before
  2. Hash each row's text + model -> content_hash
  3. SELECT cached embeddings by hash
  4. Embed only the misses (new journal entries, new commits, edited
     memories)
  5. INSERT new embeddings into cache
  6. Push all rows (cached + new) into Redis (still flush + rebuild)

Self-healing intact:
  - Redis: still fully ephemeral, flushed every tick
  - sqlite cache: durable but trivially reconstructible -- delete the
    file and the next tick rebuilds from journal/memories/git
  - Both layers can be nuked independently without breaking anything

Expected performance after merge:
  - First tick: same ~2min (rebuilds the cache from cold)
  - Steady-state ticks: <5sec (only a few new entries to embed,
    usually 0-3 per tick)
  - Log line "rag: cache hit N/M, need to embed K new" tells the story

Added IGOR_RAG_CACHE_PATH knob in .env.example for visibility.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

joshtronic merged commit 8e3c54878d into master

2026-05-21 19:54:26 +00:00

joshtronic deleted branch feat/rag-embedding-cache

2026-05-21 19:54:26 +00:00

joshtronic referenced this pull request from a commit

2026-05-21 19:54:27 +00:00

Merge pull request 'feat: sqlite-backed embedding cache -- 2min -> <5sec rebuilds' (#60) from feat/rag-embedding-cache into master