fix: performance optimizations for large dataset

- Add generated columns and indexes for correctness and metadata fields
- Optimize get_all() with keyset pagination
- Add get_pending_for_review() for targeted queries
- Update cron tasks to use optimized queries instead of full table scans
- This fixes timeouts in review_brain and verify_pending_external (300s timeout)

Fixes #35: Second-Brain in Takt bringen, Dedup, Pendings, Graph und Performance
This commit is contained in:
2026-06-04 12:25:11 +02:00
parent 6abe4d36e8
commit 8783bb2db5
11 changed files with 203 additions and 18 deletions

View File

@@ -18,7 +18,8 @@ BRAIN_DIR = Path("/root/.openclaw/workspace/second-brain")
DB_PATH = BRAIN_DIR / "data" / "brain.sqlite"
def get_db_stats():
conn = sqlite3.connect(str(DB_PATH))
conn = sqlite3.connect(str(DB_PATH), timeout=30)
conn.execute("PRAGMA busy_timeout=30000")
conn.row_factory = sqlite3.Row
c = conn.cursor()
total = c.execute("SELECT COUNT(*) FROM engrams").fetchone()[0]