17 Commits

Author SHA1 Message Date
20098a3253 Merge pull request 'fix: graph touch UX + colors/legend (closes #23)' (#24) from fix/graph-touch-colors into master 2026-05-29 11:56:00 +02:00
fa2ba11b66 fix: improve graph touch controls and legend 2026-05-29 11:54:00 +02:00
7dfd9c4228 Merge pull request 'feat: verdict/evidence verification model (closes #17)' (#18) from feat/verdict-evidence-issue-17 into master 2026-05-29 11:35:09 +02:00
6d99c520e6 feat: add verdict/evidence verification model 2026-05-29 11:30:24 +02:00
f10a5b9f19 docs: add dashboard UI/UX design plan 2026-05-29 10:50:23 +02:00
6232f25cc9 fix(fastapi): remove duplicate confirm/reject routes
- api_confirm and api_reject were defined twice on same paths
- FastAPI only registers first definition, causing silent 404s
- Kept api_confirm_engram and api_reject_engram (use _update_correctness)
- Removed duplicate direct DB implementations
- Fixes dashboard confirm/reject buttons not working
2026-05-27 18:36:03 +02:00
6b0cf5889f fix(store): escape FTS5 special characters in search_text()
- FTS5 crashes on dots (IP addresses) and hyphens (dates)
- Add regex to strip non-alphanumeric chars before FTS5 MATCH
- Fixes: fts5 syntax error near '.' and no such column: 05

Files changed: src/store.py
2026-05-27 17:54:51 +02:00
021fd0e328 feat(dashboard): pending queue + confirm/reject endpoints 2026-05-27 01:17:32 +02:00
d52e3a7f74 feat(ingest): transcript direct to DB 2026-05-27 01:14:42 +02:00
1635ee8b03 fix(verify): scan all engrams 2026-05-27 01:11:59 +02:00
f8ac0af869 chore(timers): ingest memory every 5 min 2026-05-27 01:11:17 +02:00
9dd5e49e2a chore(timers): process pending every 5 min 2026-05-27 01:08:08 +02:00
b158b19208 feat(ingest): tail session transcript into memory 2026-05-27 01:04:47 +02:00
095e6a33f8 feat(dashboard): realtime status + graph render 2026-05-27 00:22:14 +02:00
e5061b317f feat(dashboard): add status+graph views 2026-05-27 00:11:44 +02:00
ec8870ea40 feat(verify): add pending external verifier 2026-05-27 00:05:51 +02:00
8f47151a48 docs(systemd): fix paths + add dashboard smoke + vendor units 2026-05-26 23:47:07 +02:00
39 changed files with 2377 additions and 76 deletions

View File

@@ -2,19 +2,27 @@
This is the operational quick-reference for the shipped systemd timers/services and the optional FastAPI dashboard backend. This is the operational quick-reference for the shipped systemd timers/services and the optional FastAPI dashboard backend.
Repository root (on host): `/root/.openclaw/workspace` Repository root (on host): `/root/.openclaw/workspace/second-brain`
## Systemd units (cron jobs) ## Systemd units (cron jobs)
Unit files are shipped in `systemd/` (repo root). Install them into `/etc/systemd/system/` (symlink or copy), then reload: Unit files are shipped in `systemd/` (this repo). Install them into `/etc/systemd/system/` (symlink or copy), then reload:
```bash ```bash
sudo ln -sf /root/.openclaw/workspace/systemd/openclaw-secondbrain-*.service /etc/systemd/system/ sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-secondbrain-*.service /etc/systemd/system/
sudo ln -sf /root/.openclaw/workspace/systemd/openclaw-secondbrain-*.timer /etc/systemd/system/ sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-secondbrain-*.timer /etc/systemd/system/
sudo ln -sf /root/.openclaw/workspace/systemd/openclaw-memory-archive.* /etc/systemd/system/ sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-memory-archive.* /etc/systemd/system/
sudo systemctl daemon-reload sudo systemctl daemon-reload
``` ```
Optional (verification hardening):
```bash
sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-secondbrain-verify-pending.* /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now openclaw-secondbrain-verify-pending.timer
```
Enable timers: Enable timers:
```bash ```bash
@@ -61,7 +69,7 @@ python3 -m pip install -r second-brain/requirements-dashboard.txt
SECOND_BRAIN_WORKSPACE="/root/.openclaw/workspace/second-brain" python3 second-brain/fastapi_app.py SECOND_BRAIN_WORKSPACE="/root/.openclaw/workspace/second-brain" python3 second-brain/fastapi_app.py
``` ```
Default port is `8501` (same as Streamlit default). Do not run both on the same port. Default port is `8501` (same as Streamlit default). You can override via `SECOND_BRAIN_PORT` (or `PORT`) when starting manually.
Endpoint smoke tests: Endpoint smoke tests:

View File

@@ -0,0 +1,160 @@
#!/usr/bin/env python3
"""
Ingest OpenClaw session transcript JSONL directly into the Second-Brain DB.
State is tracked with byte offsets per transcript file.
Sources are configured via workspace/memory/session_sources.json.
"""
from __future__ import annotations
import json
import os
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, Iterable, List
from uuid import NAMESPACE_URL, uuid5
WORKSPACE = Path("/root/.openclaw/workspace")
MEMORY_DIR = WORKSPACE / "memory"
SOURCES_PATH = MEMORY_DIR / "session_sources.json"
STATE_PATH = MEMORY_DIR / "session_db_ingest_state.json"
BRAIN_DIR = WORKSPACE / "second-brain"
DB_PATH = Path(os.environ.get("BRAIN_DB", str(BRAIN_DIR / "data" / "brain.sqlite")))
import sys
sys.path.insert(0, str(BRAIN_DIR))
from src.engram import Engram, Grounding # type: ignore
from src.store import EngramStore # type: ignore
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _load_json(path: Path, default: Any) -> Any:
try:
if not path.exists():
return default
return json.loads(path.read_text(encoding="utf-8"))
except Exception:
return default
def _save_json(path: Path, payload: Any) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(payload, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
def _extract_text(content: Any) -> str:
if isinstance(content, str):
return content.strip()
if isinstance(content, list):
parts: List[str] = []
for c in content:
if isinstance(c, dict) and c.get("type") == "text" and isinstance(c.get("text"), str):
parts.append(c["text"])
elif isinstance(c, str):
parts.append(c)
return "\n".join(p.strip() for p in parts if p and p.strip()).strip()
if isinstance(content, dict) and isinstance(content.get("text"), str):
return content["text"].strip()
return str(content).strip()
@dataclass
class Source:
label: str
transcript_path: Path
def _load_sources() -> List[Source]:
payload = _load_json(SOURCES_PATH, {"sources": []})
sources: List[Source] = []
for item in payload.get("sources", []) if isinstance(payload, dict) else []:
if not isinstance(item, dict):
continue
label = str(item.get("label") or "session")
p = item.get("path")
if not isinstance(p, str) or not p:
continue
sources.append(Source(label=label, transcript_path=Path(p)))
return sources
def _iter_new_lines(path: Path, *, start_offset: int) -> Iterable[tuple[int, str]]:
with open(path, "rb") as f:
f.seek(max(0, int(start_offset)))
while True:
raw = f.readline()
if not raw:
break
line = raw.decode("utf-8", errors="ignore").strip()
if not line:
continue
yield (f.tell(), line)
def run() -> Dict[str, Any]:
if not DB_PATH.exists():
return {"success": False, "time": _now(), "error": f"db missing: {DB_PATH}"}
sources = _load_sources()
state = _load_json(STATE_PATH, {"offsets": {}})
offsets: Dict[str, int] = state.get("offsets", {}) if isinstance(state, dict) else {}
store = EngramStore(str(DB_PATH))
out = {"success": True, "time": _now(), "sources": len(sources), "messages_saved": 0, "messages_skipped": 0, "errors": []}
for src in sources:
try:
if not src.transcript_path.exists():
continue
key = str(src.transcript_path)
start = int(offsets.get(key, 0))
for new_off, line in _iter_new_lines(src.transcript_path, start_offset=start):
offsets[key] = new_off
obj = json.loads(line)
if not isinstance(obj, dict) or obj.get("type") != "message":
continue
msg = obj.get("message") if isinstance(obj.get("message"), dict) else {}
role = str(msg.get("role") or "unknown")
content = _extract_text(msg.get("content"))
if len(content.strip()) < 5:
continue
mid = str(obj.get("id") or msg.get("id") or msg.get("messageId") or msg.get("message_id") or "")
if not mid:
mid = str(uuid5(NAMESPACE_URL, f"openclaw-transcript:{src.label}:{role}:{content[:200]}"))
eid = str(uuid5(NAMESPACE_URL, f"openclaw-transcript:{src.label}:{mid}"))
if store.get(eid):
out["messages_skipped"] += 1
continue
eg = Engram.create(
content=f"[transcript:{src.label}] [{role}] [{mid}]\n\n{content}"[:4000],
source="session",
tags=["session", "openclaw", "transcript", f"role:{role}"],
session_id=src.label,
confidence=0.55,
grounding=Grounding.ASSUMPTION,
)
eg.id = uuid5(NAMESPACE_URL, eid)
eg.metadata["source"] = "session"
eg.metadata["session_id"] = src.label
eg.metadata["role"] = role
eg.metadata["message_id"] = mid
store.save(eg)
out["messages_saved"] += 1
except Exception as e:
out["success"] = False
out["errors"].append(f"{src.transcript_path}: {e}")
_save_json(STATE_PATH, {"offsets": offsets, "updated_at": out["time"]})
return out
if __name__ == "__main__":
print(json.dumps(run(), ensure_ascii=False, indent=2))

View File

@@ -0,0 +1,213 @@
#!/usr/bin/env python3
"""
Tail OpenClaw session transcript JSONL files and append new messages into
workspace/memory/YYYY-MM-DD.md so the existing ingest_memory pipeline can pick
them up.
Why: when chat_autosave hooks are missed/aborted, the "memory/*.md -> DB" ingest
doesn't see the latest conversation. This bridges transcript -> memory.
Safety: read-only access to transcript files; state stored in workspace/memory/.
"""
from __future__ import annotations
import json
import os
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, Iterable, List, Optional
try:
from zoneinfo import ZoneInfo # py3.9+
except Exception: # pragma: no cover
ZoneInfo = None # type: ignore
WORKSPACE = Path("/root/.openclaw/workspace")
MEMORY_DIR = WORKSPACE / "memory"
SOURCES_PATH = MEMORY_DIR / "session_sources.json"
STATE_PATH = MEMORY_DIR / "session_ingest_state.json"
def _local_tz():
tz = os.environ.get("TZ") or "Europe/Berlin"
if ZoneInfo:
try:
return ZoneInfo(tz)
except Exception:
return timezone.utc
return timezone.utc
def _load_json(path: Path, default: Any) -> Any:
try:
if not path.exists():
return default
return json.loads(path.read_text(encoding="utf-8"))
except Exception:
return default
def _save_json(path: Path, payload: Any) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(payload, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
def _iso_to_dt(ts: str) -> Optional[datetime]:
try:
if ts.endswith("Z"):
ts = ts[:-1] + "+00:00"
return datetime.fromisoformat(ts)
except Exception:
return None
def _ms_to_dt(ms: int) -> datetime:
return datetime.fromtimestamp(ms / 1000, tz=timezone.utc)
def _extract_text(content: Any) -> str:
if isinstance(content, str):
return content.strip()
if isinstance(content, list):
parts: List[str] = []
for c in content:
if isinstance(c, dict) and c.get("type") == "text" and isinstance(c.get("text"), str):
parts.append(c["text"])
elif isinstance(c, str):
parts.append(c)
return "\n".join(p.strip() for p in parts if p and p.strip()).strip()
if isinstance(content, dict):
if isinstance(content.get("text"), str):
return content["text"].strip()
return str(content).strip()
@dataclass
class Source:
label: str
transcript_path: Path
def _load_sources() -> List[Source]:
"""
Sources file format:
{
"sources": [
{ "label": "telegram:263887248", "path": "/root/.openclaw/agents/main/sessions/<id>.jsonl" }
]
}
"""
payload = _load_json(SOURCES_PATH, {"sources": []})
sources: List[Source] = []
for item in payload.get("sources", []) if isinstance(payload, dict) else []:
if not isinstance(item, dict):
continue
label = str(item.get("label") or "session")
p = item.get("path")
if not isinstance(p, str) or not p:
continue
sources.append(Source(label=label, transcript_path=Path(p)))
return sources
def _memory_path_for(dt: datetime) -> Path:
tz = _local_tz()
local = dt.astimezone(tz)
MEMORY_DIR.mkdir(parents=True, exist_ok=True)
return MEMORY_DIR / f"{local.date().isoformat()}.md"
def _append_memory(dt: datetime, label: str, role: str, text: str) -> None:
if not text.strip():
return
tz = _local_tz()
local = dt.astimezone(tz)
mem = _memory_path_for(dt)
if not mem.exists():
mem.write_text(f"# {local.date().isoformat()}\n\n", encoding="utf-8")
header = f"## {local.strftime('%H:%M:%S')} - {label} ({role})"
body = text.strip()
mem.write_text(mem.read_text(encoding="utf-8") + f"{header}\n\n{body}\n\n", encoding="utf-8")
def _iter_new_lines(path: Path, *, start_offset: int) -> Iterable[tuple[int, str]]:
with open(path, "rb") as f:
f.seek(max(0, int(start_offset)))
while True:
raw = f.readline()
if not raw:
break
try:
line = raw.decode("utf-8", errors="ignore").strip()
except Exception:
line = ""
if not line:
continue
yield (f.tell(), line)
def run() -> Dict[str, Any]:
sources = _load_sources()
state = _load_json(STATE_PATH, {"offsets": {}})
offsets: Dict[str, int] = state.get("offsets", {}) if isinstance(state, dict) else {}
out = {
"success": True,
"time": datetime.now(timezone.utc).isoformat(),
"sources": len(sources),
"messages_appended": 0,
"errors": [],
}
for src in sources:
try:
if not src.transcript_path.exists():
continue
key = str(src.transcript_path)
start = int(offsets.get(key, 0))
for new_off, line in _iter_new_lines(src.transcript_path, start_offset=start):
try:
obj = json.loads(line)
except Exception:
offsets[key] = new_off
continue
if not isinstance(obj, dict) or obj.get("type") != "message":
offsets[key] = new_off
continue
msg = obj.get("message") if isinstance(obj.get("message"), dict) else {}
role = str(msg.get("role") or "unknown")
content = msg.get("content")
text = _extract_text(content)
dt = None
if isinstance(msg.get("timestamp"), (int, float)):
dt = _ms_to_dt(int(msg["timestamp"]))
elif isinstance(obj.get("timestamp"), str):
dt = _iso_to_dt(obj["timestamp"])
if dt is None:
dt = datetime.now(timezone.utc)
if len(text.strip()) < 5:
offsets[key] = new_off
continue
_append_memory(dt, src.label, role, text)
out["messages_appended"] += 1
offsets[key] = new_off
except Exception as e:
out["success"] = False
out["errors"].append(f"{src.transcript_path}: {e}")
_save_json(STATE_PATH, {"offsets": offsets, "updated_at": out["time"]})
return out
if __name__ == "__main__":
res = run()
print(json.dumps(res, ensure_ascii=False, indent=2))

View File

@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""
Verify pending (unconfirmed) engrams using lightweight external checks.
Policy (conservative):
- `openclaw-memory` is treated as internal ground-truth and is auto-confirmed
by the review job (see `cron_tasks/review_brain.py` in the workspace runtime).
- For `source=web`, confirm if the grounded URL responds with HTTP 2xx, reject on
4xx/5xx, and keep pending on timeouts/unknown.
- Reject obvious low-signal placeholders (e.g. session summary stubs).
"""
import json
import os
import sys
from pathlib import Path
from datetime import datetime, timezone
from typing import Any, Optional
WORKSPACE = Path(os.environ.get("SECOND_BRAIN_WORKSPACE", "/root/.openclaw/workspace/second-brain"))
DB_PATH = Path(os.environ.get("BRAIN_DB", str(WORKSPACE / "data" / "brain.sqlite"))).resolve()
sys.path.insert(0, str(WORKSPACE))
from src.store import EngramStore
from src.engram import ReviewEntry
OUTPUT_FILE = os.environ.get("CRON_OUTPUT_FILE", "/tmp/verify_pending_external.json")
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _get_url(meta: dict[str, Any]) -> Optional[str]:
url = meta.get("url")
if isinstance(url, str) and url.startswith(("http://", "https://")):
return url
grounding = meta.get("grounding")
if isinstance(grounding, dict):
g_url = grounding.get("url")
if isinstance(g_url, str) and g_url.startswith(("http://", "https://")):
return g_url
return None
def _http_status(url: str, timeout_s: float = 6.0) -> Optional[int]:
try:
import urllib.request
req = urllib.request.Request(
url,
method="GET",
headers={"User-Agent": "openclaw-secondbrain/verify_pending_external"},
)
with urllib.request.urlopen(req, timeout=timeout_s) as resp:
return int(getattr(resp, "status", 200))
except Exception:
return None
def main() -> int:
if not DB_PATH.exists():
out = {"success": False, "error": f"db missing: {DB_PATH}", "time": _now()}
Path(OUTPUT_FILE).write_text(json.dumps(out, indent=2))
print(out["error"])
return 1
store = EngramStore(str(DB_PATH))
all_egs = []
offset = 0
while True:
batch = store.get_all(limit=2000, offset=offset)
if not batch:
break
all_egs.extend(batch)
offset += len(batch)
pending = [
eg
for eg in all_egs
if (not eg.correctness.is_final())
]
confirmed = 0
rejected = 0
still_pending = 0
checked = 0
for eg in pending:
checked += 1
src = eg.metadata.get("source")
content = (eg.content or "").strip()
if src == "session" and (
content.startswith("Session Summary (sess_") or content.startswith("Please remember ")
):
eg.correctness.reject("verify-pending", "Auto-reject: session placeholder")
store.save(eg)
rejected += 1
continue
if src == "web":
url = _get_url(eg.metadata)
if not url:
still_pending += 1
continue
status = _http_status(url)
if status is None:
still_pending += 1
continue
if 200 <= status < 300:
eg.correctness.confirm("verify-pending", f"Auto-confirm: web url ok ({status}) {url}")
store.save(eg)
confirmed += 1
else:
eg.correctness.reject("verify-pending", f"Auto-reject: web url status={status} {url}")
store.save(eg)
rejected += 1
continue
still_pending += 1
out = {
"success": True,
"time": _now(),
"total": len(all_egs),
"pending_before": len(pending),
"checked": checked,
"confirmed": confirmed,
"rejected": rejected,
"still_pending": still_pending,
}
Path(OUTPUT_FILE).write_text(json.dumps(out, indent=2))
print(
f"VERIFY: pending_before={out['pending_before']} confirmed={confirmed} rejected={rejected} still_pending={still_pending}"
)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -1,6 +1,6 @@
# Second-Brain 2.0 — Release Checklist (Grundversion + FastAPI) # Second-Brain 2.0 — Release Checklist (Grundversion + FastAPI)
Scope: OpenClaw workspace at `/root/.openclaw/workspace` with `second-brain/` and the shipped `systemd/` unit files. Scope: OpenClaw workspace at `/root/.openclaw/workspace` with `second-brain/` and the shipped `second-brain/systemd/` unit files.
This checklist is copy/paste friendly. Run as a user with `sudo` (systemd commands need it). This checklist is copy/paste friendly. Run as a user with `sudo` (systemd commands need it).
@@ -24,13 +24,21 @@ ls -la /etc/systemd/system/openclaw-memory-archive.* 2>/dev/null || true
If missing, install them (symlink is fine; copy is fine too): If missing, install them (symlink is fine; copy is fine too):
```bash ```bash
sudo ln -sf /root/.openclaw/workspace/systemd/openclaw-secondbrain-*.service /etc/systemd/system/ sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-secondbrain-*.service /etc/systemd/system/
sudo ln -sf /root/.openclaw/workspace/systemd/openclaw-secondbrain-*.timer /etc/systemd/system/ sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-secondbrain-*.timer /etc/systemd/system/
sudo ln -sf /root/.openclaw/workspace/systemd/openclaw-memory-archive.* /etc/systemd/system/ sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-memory-archive.* /etc/systemd/system/
sudo ln -sf /root/.openclaw/workspace/systemd/openclaw-secondbrain-dashboard.service /etc/systemd/system/ sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-secondbrain-dashboard.service /etc/systemd/system/
sudo systemctl daemon-reload sudo systemctl daemon-reload
``` ```
Optional (verification hardening):
```bash
sudo ln -sf /root/.openclaw/workspace/second-brain/systemd/openclaw-secondbrain-verify-pending.* /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now openclaw-secondbrain-verify-pending.timer
```
### Enable timers ### Enable timers
```bash ```bash

View File

@@ -0,0 +1,215 @@
# Second-Brain Dashboard UI/UX Design Plan (mobile-first)
Basis: `second-brain/templates/dashboard.html`, `second-brain/static/style.css` (aktueller Stand) + neue Graph-Controls aus `/tmp/second-brain-staging/`.
## 1) Konkreter Design-Plan
### Ziele (UX)
- **Schnelles Triaging unterwegs:** Pending/Errors sofort sichtbar, 1-Hand-Bedienung.
- **Konsistentes Design-System:** einheitliche Buttons/Inputs/Panels statt Einzellösungen.
- **Graph als Diagnose-Tool:** klare Controls, Legende, nachvollziehbares Feedback (Loading/Empty/Errors).
### Farbschema (Dark, high-contrast, "indigo + emerald")
- **Background:** sehr dunkel (nahe #0f1117) für weniger Blendung.
- **Surface (Cards/Panels):** abgestufte Flächen (Surface-1, Surface-2) für Hierarchie.
- **Primary:** Indigo/Blue für interaktive Elemente und Highlights (bisher #6c8af5 bleibt als Basis).
- **Success:** Emerald für Confirm/OK (bisher #3a7d3a → etwas heller/satter).
- **Danger:** Red für Errors/Reject.
- **Warning:** Amber für Pending/Match/Attention.
- **Text:** fast-weiß, sekundär gedimmt.
### Typografie
- Systemfont-Stack (schnell, gut lesbar): `ui-sans-serif, system-ui, -apple-system, Segoe UI, Roboto, Arial`.
- Skala (mobile-first):
- `text-xs` 12px (Labels, Meta)
- `text-sm` 1314px (secondary)
- `text-md` 1516px (Body)
- `text-lg` 1820px (Titles)
- **Ziffern:** optional `font-variant-numeric: tabular-nums;` für Stats.
### Spacing/Rhythm
- 4px Grid; Standard-Gaps: 8/12/16.
- Container-Padding: 12px (mobile), 1620px (>=768px).
- Border-radius: 1216px (Cards/Modals), 10px (Inputs/Buttons), 999px (Pills).
### Komponenten (konkret)
#### A) Tabs
- Tabs bleiben als 3 Buttons, aber:
- **Active State** deutlicher (background + border + subtle glow).
- **Touch target** min. 44px Höhe.
- Sticky bleibt, aber mit leichter **Backdrop-Blurring** (wenn möglich) oder solid surface.
#### B) Search
- Search-Row wird als **kompakte Toolbar** gestaltet:
- Input + Filter in einer Zeile.
- Optional Quick-Chips darunter: `All / Pending / Confirmed / Errors` (klickbar) als Alternative zum Select.
- Clear-Button (×) im Input (per CSS `::-webkit-search-cancel-button` oder eigener Button) für Mobile.
#### C) Cards
- Karte als 3 Zonen: Header (badges/tags/date), Body (content), Footer (actions).
- Status wird stärker codiert:
- left-border + kleine **Status-Pill** (OK/Pending/Error) mit eindeutiger Farbe.
- Body: bessere Lesbarkeit via `line-height: 1.55` und max-height/clamp optional.
#### D) Modal
- Modal als **Bottom Sheet** auf Mobile (>=50vh) + klassisches Center-Modal auf Desktop.
- Close-Button größer + Tap-Area.
- Inhalt in Tabs/Sections (History/Meta/Content) optional später.
#### E) Graph + Controls (aus Staging)
- Controls als **Control Bar** oberhalb Canvas:
- Primary: Physics toggle.
- Secondary: Fit, Reset, Reload.
- Text labels kurz (z.B. `Physics` statt `Physics: off`, state als Badge).
- Canvas passt sich an Viewport an:
- `width: min(100%, 560px)`; Height: `min(65vh, 560px)` (CSS statt fester HTML Attribute, wenn möglich).
- Legende als einklappbares Panel (`Details`/`summary`) oder leichtes Panel unter Canvas.
#### F) Status Panels
- Status-View nutzt vorhandene `.panel`/`.kv-*`:
- Gruppen: System, Storage, Jobs, Insights, Pending Queue.
- Jede Gruppe als Panel mit klarer Title Row.
- Kritische Werte (Errors/Pending/Queue) farblich markieren.
## 2) CSS-Variablen (Theme-Tokens) + Mapping
### Token-Vorschlag (`:root`)
```css
:root {
/* typography */
--font-sans: ui-sans-serif, system-ui, -apple-system, Segoe UI, Roboto, Arial, sans-serif;
/* colors */
--bg: #0f1117;
--surface-1: #14151d;
--surface-2: #1a1b26;
--border: #2a2d3a;
--text: #e8e8ee;
--text-muted: #8b90a3;
--text-dim: #5c6276;
--primary: #6c8af5; /* existing */
--primary-2: #8aa1ff;
--success: #2fbf71;
--danger: #ef4444;
--warning: #f7d154; /* existing-ish */
/* shadows */
--shadow-1: 0 1px 0 rgba(0,0,0,.25), 0 8px 24px rgba(0,0,0,.35);
/* radius */
--r-sm: 10px;
--r-md: 14px;
--r-lg: 16px;
/* spacing */
--s-1: 4px;
--s-2: 8px;
--s-3: 12px;
--s-4: 16px;
--s-5: 20px;
/* control sizes */
--tap: 44px;
}
```
### Mapping auf existierende Selektoren
- `body`/`.app`
- `background: var(--bg)` statt `#141419`
- `font-family: var(--font-sans)`
- `color: var(--text)`
- `.stats-bar`, `.tabs-bar`, `.search-box`
- `background: var(--surface-1)` (Stats ggf. Gradient bleibt, aber auf Tokens)
- `border-bottom: 1px solid var(--border)`
- `.panel`, `.card`, `.modal-content`, `.graph-legend`
- `background: var(--surface-2)`
- `border: 1px solid var(--border)`
- `border-radius: var(--r-md)`
- `.stat-num`, `#pageNum`, `.tab-btn.active`, `#searchInput:focus`
- `color/border-color: var(--primary)`
- `.muted`, `.stat-label`, `.kv-key`, `.date`
- `color: var(--text-muted)` bzw. `var(--text-dim)`
- Buttons
- vereinheitlichen über `.btn` + Modifier: `.btn.primary`, `.btn.danger`, `.btn.success`, `.btn.ghost`
- `min-height: var(--tap)` für Touch
## 3) 510 priorisierte UI-Änderungen (mit Begründung)
1) **Design Tokens (CSS vars) einführen** → reduziert Farbmix, erleichtert spätere Themes/Anpassungen.
2) **Einheitliche Button-Komponente (`.btn`)** (inkl. `:active`, `:disabled`, `min-height`) → bessere Touch-UX, konsistente Interaktion.
3) **Graph-Controls + Legende aus Staging in den Main-Template-Stand ziehen** → Graph wird tatsächlich bedienbar/selbsterklärend.
4) **Responsive Graph-Canvas (CSS gesteuert)** statt fixer `width/height` → bessere Nutzung auf Phones, weniger Scroll.
5) **Search als Toolbar + Clear-Action** → schnelleres Filtern unterwegs, weniger Friktion.
6) **Modal als Bottom-Sheet auf Mobile** → angenehmer für längeren Content + History, weniger „winziges Fenster“.
7) **Status/Health Werte farblich akzentuieren** (pending/errors/warn) → schnelleres Erkennen von Problemen.
8) **Cards: Status-Pill + typografische Lesbarkeit** (line-height, spacing) → weniger „Textblock“, bessere Scanbarkeit.
9) **Accessibility-Basics**: Focus-Rings, Kontrast, `prefers-reduced-motion` → weniger „invisible focus“ und bessere Bedienbarkeit.
10) **Top-level Layout Max-Width für Desktop** (z.B. 560720px) → verhindert „zu breite“ Zeilen.
## 4) Optionale Patch-Vorschläge (Diff-Snippets, NICHT anwenden)
> Hinweis: Snippets sind bewusst klein gehalten. Gesamt < 120 Zeilen.
### Snippet A — Tokens + Button-System (style.css)
```diff
--- a/second-brain/static/style.css
+++ b/second-brain/static/style.css
@@
+:root {
+ --font-sans: ui-sans-serif, system-ui, -apple-system, Segoe UI, Roboto, Arial, sans-serif;
+ --bg:#0f1117; --surface-1:#14151d; --surface-2:#1a1b26; --border:#2a2d3a;
+ --text:#e8e8ee; --text-muted:#8b90a3; --text-dim:#5c6276;
+ --primary:#6c8af5; --success:#2fbf71; --danger:#ef4444; --warning:#f7d154;
+ --r-sm:10px; --r-md:14px; --r-lg:16px;
+ --s-2:8px; --s-3:12px; --s-4:16px;
+ --tap:44px;
+}
+
+body { font-family: var(--font-sans); background: var(--bg); color: var(--text); }
+
+.btn{
+ min-height: var(--tap);
+ background: #1e1e28;
+ border: 1px solid var(--border);
+ border-radius: var(--r-sm);
+ padding: 8px 12px;
+ color: #cfd3ff;
+ font-weight: 700;
+}
+.btn.primary{ border-color: var(--primary); box-shadow: 0 0 0 1px rgba(108,138,245,0.18) inset; }
+.btn.success{ background: rgba(47,191,113,.18); border-color: rgba(47,191,113,.35); }
+.btn.danger{ background: rgba(239,68,68,.16); border-color: rgba(239,68,68,.35); }
+.btn:active{ transform: scale(.98); }
+.btn:disabled{ opacity: .45; }
```
### Snippet B — Graph Controls + Legend übernehmen (dashboard.html)
```diff
--- a/second-brain/templates/dashboard.html
+++ b/second-brain/templates/dashboard.html
@@
- <div class="graph" id="graph" style="display:none;">
- <canvas id="graphCanvas" width="440" height="520"></canvas>
- <div class="muted small" id="graphHint">Lade Graph…</div>
- </div>
+ <div class="graph" id="graph" style="display:none;">
+ <div class="graph-controls">
+ <button class="btn primary" id="btnGraphPhysics" onclick="toggleGraphPhysics()">Physics</button>
+ <button class="btn" onclick="resetGraphView()">Reset</button>
+ <button class="btn" onclick="fitGraphView()">Fit</button>
+ <button class="btn" onclick="reloadGraph()">Reload</button>
+ </div>
+ <canvas id="graphCanvas" width="440" height="520"></canvas>
+ <div class="graph-hint muted small" id="graphHint">Lade Graph…</div>
+ <div class="graph-legend">
+ <div><strong>Graph</strong>: Zoom (Wheel/Pinch), Pan (Drag). Klick auf Engram öffnet Details, Klick auf Tag setzt Suche.</div>
+ <div class="legend-row"><span class="legend-dot engram"></span> Engram</div>
+ <div class="legend-row"><span class="legend-dot tag"></span> Tag</div>
+ <div class="legend-row"><span class="legend-dot match"></span> Match</div>
+ </div>
+ </div>
```

View File

@@ -11,11 +11,14 @@ Goals:
import json import json
import os import os
import sqlite3 import sqlite3
import subprocess
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from uuid import uuid4
from urllib.parse import urlparse
from fastapi import FastAPI, Form, Query, Request from fastapi import FastAPI, Form, Query, Request
from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse, StreamingResponse
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
# ─── Config ────────────────────────────────────────────────────────────────── # ─── Config ──────────────────────────────────────────────────────────────────
@@ -51,11 +54,22 @@ def get_db():
def parse_engram(row: sqlite3.Row) -> dict: def parse_engram(row: sqlite3.Row) -> dict:
meta = json.loads(row["metadata_json"] or "{}") meta = json.loads(row["metadata_json"] or "{}")
correctness = json.loads(row["correctness_json"] or "{}") correctness = json.loads(row["correctness_json"] or "{}")
verdict = correctness.get("verdict")
if not isinstance(verdict, str) or not verdict:
# Back-compat inference for older rows
if correctness.get("confirmed", False):
verdict = "confirmed_true"
elif int(correctness.get("rejections", 0) or 0) > 0:
verdict = "confirmed_false"
else:
verdict = "unknown"
return { return {
"id": row["id"], "id": row["id"],
"content": row["content"], "content": row["content"],
"confidence": meta.get("confidence", 0.0), "confidence": meta.get("confidence", 0.0),
"confirmed": correctness.get("confirmed", False), "confirmed": correctness.get("confirmed", False),
"verdict": verdict,
"evidence": correctness.get("evidence", []),
"confirmations": correctness.get("confirmations", 0), "confirmations": correctness.get("confirmations", 0),
"rejections": correctness.get("rejections", 0), "rejections": correctness.get("rejections", 0),
"tags": meta.get("tags", []), "tags": meta.get("tags", []),
@@ -69,6 +83,158 @@ def parse_engram(row: sqlite3.Row) -> dict:
} }
def _now_iso() -> str:
return datetime.now(timezone.utc).isoformat()
def _update_correctness(engram_id: str, *, action: str, reason: str | None = None) -> dict:
"""
Update correctness_json for an engram. action: confirm|reject
"""
conn = get_db()
c = conn.cursor()
row = c.execute("SELECT correctness_json FROM engrams WHERE id = ?", (engram_id,)).fetchone()
if not row:
conn.close()
raise FileNotFoundError(f"Engram not found: {engram_id}")
corr = json.loads(row["correctness_json"] or "{}")
corr.setdefault("verdict", None)
corr.setdefault("evidence", [])
corr.setdefault("confirmed", False)
corr.setdefault("confirmations", 0)
corr.setdefault("rejections", 0)
corr.setdefault("review_history", [])
corr["last_reviewed"] = _now_iso()
entry = {
"by": "dashboard",
"action": action,
"at": corr["last_reviewed"],
"note": (reason or "").strip(),
}
try:
corr["review_history"].append(entry)
except Exception:
corr["review_history"] = [entry]
if action == "confirm":
corr["verdict"] = "confirmed_true"
corr["confirmed"] = True
corr["confirmations"] = int(corr.get("confirmations", 0) or 0) + 1
elif action == "reject":
corr["verdict"] = "confirmed_false"
corr["rejections"] = int(corr.get("rejections", 0) or 0) + 1
corr["confirmed"] = False
# Store minimal evidence for dashboard-driven actions.
try:
ev = corr.get("evidence")
if not isinstance(ev, list):
ev = []
ev.append(
{
"kind": "human",
"by": "dashboard",
"at": corr["last_reviewed"],
"action": action,
}
)
corr["evidence"] = ev[-50:] # cap growth
except Exception:
pass
c.execute(
"UPDATE engrams SET correctness_json = ?, modified_at = ? WHERE id = ?",
(json.dumps(corr, ensure_ascii=False), corr["last_reviewed"], engram_id),
)
conn.commit()
conn.close()
return {"ok": True}
def _bump_access(engram_id: str) -> dict:
conn = get_db()
c = conn.cursor()
row = c.execute("SELECT metadata_json FROM engrams WHERE id = ?", (engram_id,)).fetchone()
if not row:
conn.close()
raise FileNotFoundError(f"Engram not found: {engram_id}")
meta = json.loads(row["metadata_json"] or "{}")
meta["access_count"] = int(meta.get("access_count", 0) or 0) + 1
meta["last_accessed"] = _now_iso()
c.execute(
"UPDATE engrams SET metadata_json = ?, modified_at = ? WHERE id = ?",
(json.dumps(meta, ensure_ascii=False), meta["last_accessed"], engram_id),
)
conn.commit()
conn.close()
return {"ok": True}
def _safe_json_extract_tags(meta_json: str) -> list[str]:
try:
d = json.loads(meta_json or "{}")
tags = d.get("tags") or []
return [t for t in tags if isinstance(t, str)]
except Exception:
return []
def _host_from_meta(meta_json: str) -> str | None:
try:
d = json.loads(meta_json or "{}")
grounding = d.get("grounding")
url = d.get("url")
if isinstance(grounding, dict) and isinstance(grounding.get("url"), str):
url = grounding.get("url")
if not isinstance(url, str):
return None
parsed = urlparse(url)
return parsed.hostname
except Exception:
return None
def _systemd_unit_state(unit: str) -> dict:
"""
Best-effort systemd status snapshot for a known unit.
Never raises; returns minimal fields.
"""
try:
out = subprocess.check_output(
["systemctl", "show", unit, "--no-page", "--property=ActiveState,SubState,Result,ExecMainStatus,ExecMainCode,ExecMainStartTimestamp,ExecMainExitTimestamp"],
text=True,
stderr=subprocess.STDOUT,
timeout=2,
)
kv = {}
for line in out.splitlines():
if "=" in line:
k, v = line.split("=", 1)
kv[k] = v
return {
"unit": unit,
"active": kv.get("ActiveState"),
"sub": kv.get("SubState"),
"result": kv.get("Result"),
"exit_status": kv.get("ExecMainStatus"),
"start_ts": kv.get("ExecMainStartTimestamp"),
"exit_ts": kv.get("ExecMainExitTimestamp"),
}
except Exception as e:
return {"unit": unit, "error": str(e)}
def _dir_size_bytes(path: Path) -> int:
total = 0
try:
for p in path.rglob("*"):
try:
if p.is_file():
total += p.stat().st_size
except Exception:
pass
except Exception:
pass
return total
# ─── API Endpoints ─────────────────────────────────────────────────────────── # ─── API Endpoints ───────────────────────────────────────────────────────────
@app.get("/healthz", response_class=PlainTextResponse) @app.get("/healthz", response_class=PlainTextResponse)
@@ -83,6 +249,291 @@ def api_config():
"db_path": str(DB_PATH), "db_path": str(DB_PATH),
} }
@app.get("/api/db_info")
def api_db_info():
if not DB_PATH.exists():
raise FileNotFoundError(f"DB not found: {DB_PATH}")
st = DB_PATH.stat()
return {
"db_path": str(DB_PATH),
"size_bytes": st.st_size,
"mtime": datetime.fromtimestamp(st.st_mtime, tz=timezone.utc).isoformat(),
}
@app.get("/api/storage_stats")
def api_storage_stats():
conn = get_db()
c = conn.cursor()
total = c.execute("SELECT COUNT(*) FROM engrams").fetchone()[0]
confirmed_true = c.execute(
"""
SELECT COUNT(*) FROM engrams
WHERE (
json_extract(correctness_json, '$.verdict') = 'confirmed_true'
OR (json_extract(correctness_json, '$.verdict') IS NULL AND json_extract(correctness_json, '$.confirmed') = 1)
)
"""
).fetchone()[0]
confirmed_false = c.execute(
"""
SELECT COUNT(*) FROM engrams
WHERE (
json_extract(correctness_json, '$.verdict') = 'confirmed_false'
OR (json_extract(correctness_json, '$.verdict') IS NULL
AND json_extract(correctness_json, '$.confirmed') = 0
AND COALESCE(json_extract(correctness_json, '$.rejections'), 0) > 0)
)
"""
).fetchone()[0]
sources = {
r[0]: r[1]
for r in c.execute(
"SELECT json_extract(metadata_json, '$.source') AS src, COUNT(*) FROM engrams GROUP BY src ORDER BY COUNT(*) DESC"
).fetchall()
if r[0] is not None
}
conn.close()
chroma_dir = WORKSPACE / "data" / "chroma"
emb_cache_dir = WORKSPACE / "data" / "embedding_cache"
vec_state_path = WORKSPACE / "data" / "vector_index_state.json"
vec_state = {}
if vec_state_path.exists():
try:
vec_state = json.loads(vec_state_path.read_text())
except Exception:
vec_state = {}
obsidian_cfg_path = WORKSPACE / "data" / "obsidian_config.json"
obsidian_cfg = None
if obsidian_cfg_path.exists():
try:
obsidian_cfg = json.loads(obsidian_cfg_path.read_text())
except Exception:
obsidian_cfg = {"raw": obsidian_cfg_path.read_text()[:2000]}
backup_files = sorted((WORKSPACE / "data").glob("backup_*.jsonl"))
return {
"sql": {
"total_engrams": total,
"confirmed": confirmed_true,
"rejected": confirmed_false,
"pending": total - confirmed_true - confirmed_false,
"by_source": sources,
},
"vector": {
"chroma_dir": str(chroma_dir),
"chroma_size_bytes": _dir_size_bytes(chroma_dir) if chroma_dir.exists() else 0,
"embedding_cache_dir": str(emb_cache_dir),
"embedding_cache_files": len(list(emb_cache_dir.glob("*.json"))) if emb_cache_dir.exists() else 0,
"vector_state": vec_state,
},
"obsidian": {
"config_path": str(obsidian_cfg_path),
"configured": bool(obsidian_cfg),
"config": obsidian_cfg,
},
"backups": {
"count": len(backup_files),
"latest": str(backup_files[-1]) if backup_files else None,
},
}
@app.get("/api/jobs")
def api_jobs():
# Known units that influence "freshness" of the brain.
units = [
"openclaw-secondbrain-ingest-memory.service",
"openclaw-secondbrain-index-vectors.service",
"openclaw-secondbrain-review.service",
"openclaw-secondbrain-heartbeat.service",
"openclaw-secondbrain-verify-pending.service",
]
return {"items": [_systemd_unit_state(u) for u in units]}
@app.get("/api/insights")
def api_insights(limit: int = Query(8, ge=1, le=50)):
conn = get_db()
c = conn.cursor()
rows = c.execute(
"SELECT id, metadata_json, correctness_json, created_at, modified_at FROM engrams ORDER BY created_at DESC LIMIT 2000"
).fetchall()
total = c.execute("SELECT COUNT(*) FROM engrams").fetchone()[0]
confirmed_true = c.execute(
"""
SELECT COUNT(*) FROM engrams
WHERE (
json_extract(correctness_json, '$.verdict') = 'confirmed_true'
OR (json_extract(correctness_json, '$.verdict') IS NULL AND json_extract(correctness_json, '$.confirmed') = 1)
)
"""
).fetchone()[0]
confirmed_false = c.execute(
"""
SELECT COUNT(*) FROM engrams
WHERE (
json_extract(correctness_json, '$.verdict') = 'confirmed_false'
OR (json_extract(correctness_json, '$.verdict') IS NULL
AND json_extract(correctness_json, '$.confirmed') = 0
AND COALESCE(json_extract(correctness_json, '$.rejections'), 0) > 0)
)
"""
).fetchone()[0]
pending = total - confirmed_true - confirmed_false
tag_counts: dict[str, int] = {}
source_counts: dict[str, int] = {}
host_counts: dict[str, int] = {}
active: list[dict] = []
forgotten: list[dict] = []
for r in rows:
meta = json.loads(r["metadata_json"] or "{}")
src = meta.get("source", "unknown")
source_counts[src] = source_counts.get(src, 0) + 1
for t in (meta.get("tags") or []):
if isinstance(t, str):
tag_counts[t] = tag_counts.get(t, 0) + 1
host = _host_from_meta(r["metadata_json"])
if host:
host_counts[host] = host_counts.get(host, 0) + 1
access_count = int(meta.get("access_count", 0) or 0)
created = meta.get("created", r["created_at"])
if access_count >= 5 and len(active) < limit:
active.append(
{
"id": r["id"],
"access_count": access_count,
"source": src,
"created": created,
}
)
if access_count == 0 and len(forgotten) < limit:
forgotten.append(
{
"id": r["id"],
"access_count": access_count,
"source": src,
"created": created,
}
)
def top_k(d: dict[str, int]) -> list[dict]:
return [
{"key": k, "count": v}
for k, v in sorted(d.items(), key=lambda kv: kv[1], reverse=True)[:limit]
]
conn.close()
return {
"total": total,
"confirmed": confirmed,
"pending": pending,
"top_tags": top_k(tag_counts),
"top_sources": top_k(source_counts),
"top_hosts": top_k(host_counts),
"active_engrams": active,
"forgotten_engrams": forgotten,
}
@app.get("/api/graph")
def api_graph(limit_nodes: int = Query(200, ge=50, le=1000)):
"""
Returns a lightweight graph view:
- Nodes: engrams + tag:<tag> + host:<hostname>
- Edges: engram->tag and engram->host plus explicit engrams_links edges.
"""
conn = get_db()
c = conn.cursor()
rows = c.execute("SELECT id, metadata_json FROM engrams ORDER BY created_at DESC LIMIT 1000").fetchall()
link_rows = c.execute("SELECT from_id, to_id FROM engrams_links ORDER BY rowid DESC LIMIT 2000").fetchall()
conn.close()
nodes: dict[str, dict] = {}
edges: list[dict] = []
def add_node(nid: str, kind: str, label: str | None = None, weight: float | None = None):
if nid not in nodes:
nodes[nid] = {"id": nid, "kind": kind}
if label is not None:
nodes[nid]["label"] = label
if weight is not None:
nodes[nid]["weight"] = weight
for r in rows:
eid = r["id"]
add_node(eid, "engram", label=eid[:8])
for t in _safe_json_extract_tags(r["metadata_json"]):
tid = f"tag:{t}"
add_node(tid, "tag", label=t)
edges.append({"from": eid, "to": tid, "kind": "has_tag"})
host = _host_from_meta(r["metadata_json"])
if host:
hid = f"host:{host}"
add_node(hid, "host", label=host)
edges.append({"from": eid, "to": hid, "kind": "grounded_at"})
for fr, to in link_rows:
add_node(fr, "engram")
add_node(to, "engram")
edges.append({"from": fr, "to": to, "kind": "link"})
# Trim nodes to keep payload bounded (prefer engrams and connected tags/hosts)
if len(nodes) > limit_nodes:
# Keep a balanced subset: many engrams plus the most-connected non-engrams.
kept: dict[str, dict] = {}
engram_budget = int(limit_nodes * 0.7)
# 1) Keep newest engrams first (they appear first in `rows` loop insertion order)
for r in rows:
eid = r["id"]
if eid in nodes:
kept[eid] = nodes[eid]
if len(kept) >= engram_budget:
break
# 2) Rank remaining nodes by degree within current edge set
degree: dict[str, int] = {}
for e in edges:
degree[e["from"]] = degree.get(e["from"], 0) + 1
degree[e["to"]] = degree.get(e["to"], 0) + 1
remaining = [nid for nid in nodes.keys() if nid not in kept]
remaining.sort(key=lambda nid: degree.get(nid, 0), reverse=True)
for nid in remaining:
kept[nid] = nodes[nid]
if len(kept) >= limit_nodes:
break
nodes = kept
edges = [e for e in edges if e["from"] in nodes and e["to"] in nodes]
return {"nodes": list(nodes.values()), "edges": edges}
@app.get("/api/events")
def api_events():
"""
Server-Sent Events stream for lightweight real-time UI refresh.
"""
import time
def gen():
while True:
payload = {
"ts": datetime.now(timezone.utc).isoformat(),
"stats": api_stats(),
"storage": api_storage_stats(),
"jobs": api_jobs(),
"insights": api_insights(limit=8),
}
yield f"data: {json.dumps(payload, ensure_ascii=False)}\n\n"
time.sleep(5)
return StreamingResponse(gen(), media_type="text/event-stream")
@app.exception_handler(FileNotFoundError) @app.exception_handler(FileNotFoundError)
def handle_file_not_found(request: Request, exc: FileNotFoundError): def handle_file_not_found(request: Request, exc: FileNotFoundError):
@@ -105,10 +556,27 @@ def api_stats():
conn = get_db() conn = get_db()
c = conn.cursor() c = conn.cursor()
total = c.execute("SELECT COUNT(*) FROM engrams").fetchone()[0] total = c.execute("SELECT COUNT(*) FROM engrams").fetchone()[0]
confirmed = c.execute( confirmed_true = c.execute(
"SELECT COUNT(*) FROM engrams WHERE json_extract(correctness_json, '$.confirmed') = 1" """
SELECT COUNT(*) FROM engrams
WHERE (
json_extract(correctness_json, '$.verdict') = 'confirmed_true'
OR (json_extract(correctness_json, '$.verdict') IS NULL AND json_extract(correctness_json, '$.confirmed') = 1)
)
"""
).fetchone()[0] ).fetchone()[0]
pending = total - confirmed confirmed_false = c.execute(
"""
SELECT COUNT(*) FROM engrams
WHERE (
json_extract(correctness_json, '$.verdict') = 'confirmed_false'
OR (json_extract(correctness_json, '$.verdict') IS NULL
AND json_extract(correctness_json, '$.confirmed') = 0
AND COALESCE(json_extract(correctness_json, '$.rejections'), 0) > 0)
)
"""
).fetchone()[0]
pending = total - confirmed_true - confirmed_false
errors = c.execute( errors = c.execute(
"SELECT COUNT(*) FROM engrams WHERE json_extract(metadata_json, '$.tags') LIKE '%error%'" "SELECT COUNT(*) FROM engrams WHERE json_extract(metadata_json, '$.tags') LIKE '%error%'"
).fetchone()[0] ).fetchone()[0]
@@ -118,7 +586,8 @@ def api_stats():
conn.close() conn.close()
return { return {
"total": total, "total": total,
"confirmed": confirmed, "confirmed": confirmed_true,
"rejected": confirmed_false,
"pending": pending, "pending": pending,
"errors": errors, "errors": errors,
"avg_confidence": round(avg_conf, 2), "avg_confidence": round(avg_conf, 2),
@@ -131,6 +600,7 @@ def api_engrams(
offset: int = Query(0, ge=0), offset: int = Query(0, ge=0),
tag: str = Query(None), tag: str = Query(None),
confirmed: bool = Query(None), confirmed: bool = Query(None),
verdict: str = Query(None),
search: str = Query(None), search: str = Query(None),
min_confidence: float = Query(0.0), min_confidence: float = Query(0.0),
): ):
@@ -144,9 +614,30 @@ def api_engrams(
params.append(f'%"{tag}"%') params.append(f'%"{tag}"%')
if confirmed is not None: if confirmed is not None:
if confirmed:
# confirmed == statement is true (verdict confirmed_true)
where_clauses.append( where_clauses.append(
f"json_extract(correctness_json, '$.confirmed') = {int(confirmed)}" "("
"json_extract(correctness_json, '$.verdict') = 'confirmed_true' "
"OR (json_extract(correctness_json, '$.verdict') IS NULL AND json_extract(correctness_json, '$.confirmed') = 1)"
")"
) )
else:
# pending/unresolved (unknown/probable) but exclude confirmed_false.
where_clauses.append(
"("
"json_extract(correctness_json, '$.verdict') IN ('unknown','probable_true','probable_false') "
"OR (json_extract(correctness_json, '$.verdict') IS NULL "
" AND json_extract(correctness_json, '$.confirmed') = 0 "
" AND COALESCE(json_extract(correctness_json, '$.rejections'), 0) = 0)"
")"
)
if verdict:
v = verdict.strip()
if v in ("unknown", "probable_true", "probable_false", "confirmed_true", "confirmed_false"):
where_clauses.append("json_extract(correctness_json, '$.verdict') = ?")
params.append(v)
if search: if search:
# Use FTS # Use FTS
@@ -204,70 +695,107 @@ def api_engram_detail(engram_id: str):
return result return result
@app.post("/api/engrams/{engram_id}/confirm") @app.get("/api/pending")
def api_confirm(engram_id: str, reason: str = Form("")): def api_pending(
limit: int = Query(20, ge=1, le=200),
offset: int = Query(0, ge=0),
source: str | None = Query(None),
):
conn = get_db() conn = get_db()
c = conn.cursor() c = conn.cursor()
row = c.execute( where = ["json_extract(correctness_json, '$.confirmed') = 0"]
"SELECT correctness_json FROM engrams WHERE id = ?", (engram_id,) params: list = []
).fetchone() if source:
if not row: where.append("json_extract(metadata_json, '$.source') = ?")
params.append(source)
rows = c.execute(
f"""
SELECT * FROM engrams
WHERE {' AND '.join(where)}
ORDER BY created_at DESC
LIMIT ? OFFSET ?
""",
params + [limit, offset],
).fetchall()
items = [parse_engram(r) for r in rows]
conn.close() conn.close()
return JSONResponse({"error": "Not found"}, status_code=404) return {"items": items, "limit": limit, "offset": offset}
correctness = json.loads(row["correctness_json"] or "{}")
correctness["confirmed"] = True
correctness["confirmations"] = correctness.get("confirmations", 0) + 1
correctness["last_reviewed"] = datetime.now(timezone.utc).isoformat()
review_history = correctness.get("review_history", [])
review_history.append({
"by": "web",
"action": "confirm",
"at": datetime.now(timezone.utc).isoformat(),
"note": reason or "confirmed via dashboard",
})
correctness["review_history"] = review_history
@app.post("/api/engrams")
def api_create_engram(content: str = Form(...), tags: str = Form("")):
content = (content or "").strip()
if not content:
return JSONResponse({"error": "content required"}, status_code=400)
tag_list = [t.strip() for t in (tags or "").split(",") if t.strip()]
now = _now_iso()
engram_id = str(uuid4())
meta = {
"source": "user",
"confidence": 0.7,
"created": now,
"modified": now,
"access_count": 0,
"last_accessed": now,
"tags": tag_list,
"session_id": None,
"agent_id": None,
"grounding": 0,
}
corr = {
"confirmed": False,
"confirmations": 0,
"rejections": 0,
"last_reviewed": None,
"review_history": [],
}
conn = get_db()
c = conn.cursor()
c.execute( c.execute(
"UPDATE engrams SET correctness_json = ?, modified_at = ? WHERE id = ?", """
(json.dumps(correctness), datetime.now(timezone.utc).isoformat(), engram_id), INSERT INTO engrams (id, content, metadata_json, correctness_json, links_json, hierarchy_json, embedding_json, created_at, modified_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
engram_id,
content,
json.dumps(meta, ensure_ascii=False),
json.dumps(corr, ensure_ascii=False),
"[]",
"{}",
None,
now,
now,
),
) )
conn.commit() conn.commit()
conn.close() conn.close()
return {"success": True, "engram_id": engram_id} return {"id": engram_id}
@app.post("/api/engrams/{engram_id}/confirm")
def api_confirm_engram(engram_id: str, reason: str = Form("")):
try:
return _update_correctness(engram_id, action="confirm", reason=reason)
except FileNotFoundError as e:
return JSONResponse({"error": str(e)}, status_code=404)
@app.post("/api/engrams/{engram_id}/reject") @app.post("/api/engrams/{engram_id}/reject")
def api_reject(engram_id: str, reason: str = Form("")): def api_reject_engram(engram_id: str, reason: str = Form("")):
conn = get_db() try:
c = conn.cursor() return _update_correctness(engram_id, action="reject", reason=reason)
row = c.execute( except FileNotFoundError as e:
"SELECT correctness_json FROM engrams WHERE id = ?", (engram_id,) return JSONResponse({"error": str(e)}, status_code=404)
).fetchone()
if not row:
conn.close()
return JSONResponse({"error": "Not found"}, status_code=404)
correctness = json.loads(row["correctness_json"] or "{}")
correctness["confirmed"] = False
correctness["rejections"] = correctness.get("rejections", 0) + 1
correctness["last_reviewed"] = datetime.now(timezone.utc).isoformat()
review_history = correctness.get("review_history", [])
review_history.append({
"by": "web",
"action": "reject",
"at": datetime.now(timezone.utc).isoformat(),
"note": reason or "rejected via dashboard",
})
correctness["review_history"] = review_history
c.execute( @app.post("/api/engrams/{engram_id}/refresh")
"UPDATE engrams SET correctness_json = ?, modified_at = ? WHERE id = ?", def api_refresh_engram(engram_id: str):
(json.dumps(correctness), datetime.now(timezone.utc).isoformat(), engram_id), try:
) return _bump_access(engram_id)
conn.commit() except FileNotFoundError as e:
conn.close() return JSONResponse({"error": str(e)}, status_code=404)
return {"success": True, "engram_id": engram_id}
@app.post("/api/engrams/{engram_id}/refresh") @app.post("/api/engrams/{engram_id}/refresh")
@@ -320,6 +848,8 @@ def api_create_engram(content: str = Form(...), tags: str = Form(""), source: st
"hash": "", "hash": "",
} }
correctness = { correctness = {
"verdict": "unknown",
"evidence": [],
"confirmed": False, "confirmed": False,
"confirmations": 0, "confirmations": 0,
"rejections": 0, "rejections": 0,
@@ -347,7 +877,12 @@ def api_pending(limit: int = Query(20, ge=1, le=100), offset: int = Query(0, ge=
rows = c.execute( rows = c.execute(
""" """
SELECT * FROM engrams SELECT * FROM engrams
WHERE json_extract(correctness_json, '$.confirmed') = 0 WHERE (
json_extract(correctness_json, '$.verdict') IN ('unknown','probable_true','probable_false')
OR (json_extract(correctness_json, '$.verdict') IS NULL
AND json_extract(correctness_json, '$.confirmed') = 0
AND COALESCE(json_extract(correctness_json, '$.rejections'), 0) = 0)
)
ORDER BY created_at DESC ORDER BY created_at DESC
LIMIT ? OFFSET ? LIMIT ? OFFSET ?
""", """,

20
scripts/smoke_dashboard.sh Executable file
View File

@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${SECOND_BRAIN_HOST:-127.0.0.1}"
PORT="${SECOND_BRAIN_PORT:-${PORT:-8501}}"
BASE_URL="http://${HOST}:${PORT}"
if command -v systemctl >/dev/null 2>&1; then
if ! systemctl is-active --quiet openclaw-secondbrain-dashboard.service; then
echo "ERROR: openclaw-secondbrain-dashboard.service is not active" >&2
exit 1
fi
fi
curl -fsS "${BASE_URL}/healthz" >/dev/null
stats_json="$(curl -fsS "${BASE_URL}/api/stats")"
python3 -c 'import json,sys; json.load(sys.stdin)' <<<"$stats_json"
echo "OK: dashboard smoke test passed (${BASE_URL})"

View File

@@ -40,26 +40,60 @@ class ReviewEntry:
@dataclass @dataclass
class Correctness: class Correctness:
"""Verfolgt die Korrektheit eines Engramms über Zeit.""" """Verfolgt die Korrektheit eines Engramms über Zeit."""
# verdict model (not only binary confirm/reject)
# Values:
# - unknown
# - probable_true / probable_false
# - confirmed_true / confirmed_false
verdict: str = "unknown"
evidence: List[Dict[str, Any]] = field(default_factory=list)
confirmed: bool = False confirmed: bool = False
confirmations: int = 0 confirmations: int = 0
rejections: int = 0 rejections: int = 0
last_reviewed: Optional[str] = None last_reviewed: Optional[str] = None
review_history: List[ReviewEntry] = field(default_factory=list) review_history: List[ReviewEntry] = field(default_factory=list)
def is_final(self) -> bool:
return self.verdict in ("confirmed_true", "confirmed_false")
def set_verdict(self, by: str, verdict: str, note: str = "", evidence: Optional[List[Dict[str, Any]]] = None) -> None:
verdict = (verdict or "").strip()
if verdict not in ("unknown", "probable_true", "probable_false", "confirmed_true", "confirmed_false"):
verdict = "unknown"
self.verdict = verdict
# Keep backward-compatible boolean in sync:
# historically, confirmed=True meant "this statement is correct".
self.confirmed = verdict == "confirmed_true"
self.last_reviewed = _now()
if evidence:
try:
self.evidence.extend([e for e in evidence if isinstance(e, dict)])
except Exception:
pass
self.review_history.append(ReviewEntry(by, "set_verdict", self.last_reviewed, f"{verdict}: {note}".strip()))
def confirm(self, by: str, note: str = "") -> None: def confirm(self, by: str, note: str = "") -> None:
self.confirmations += 1 self.confirmations += 1
self.confirmed = True self.set_verdict(by, "confirmed_true", note)
self.last_reviewed = _now() # Preserve historic action tag too
self.review_history.append(ReviewEntry(by, "confirm", self.last_reviewed, note)) self.review_history.append(ReviewEntry(by, "confirm", self.last_reviewed, note))
def reject(self, by: str, note: str = "") -> None: def reject(self, by: str, note: str = "") -> None:
self.rejections += 1 self.rejections += 1
self.confirmed = False self.set_verdict(by, "confirmed_false", note)
self.last_reviewed = _now()
self.review_history.append(ReviewEntry(by, "reject", self.last_reviewed, note)) self.review_history.append(ReviewEntry(by, "reject", self.last_reviewed, note))
def score(self) -> float: def score(self) -> float:
"""Confidence-Score aus Korrekturhistorie.""" """Confidence-Score aus Korrekturhistorie."""
# verdict-first scoring (explicit, non-binary)
if self.verdict == "confirmed_true":
return 1.0
if self.verdict == "confirmed_false":
return 0.0
if self.verdict == "probable_true":
return 0.75
if self.verdict == "probable_false":
return 0.25
total = self.confirmations + self.rejections total = self.confirmations + self.rejections
if total == 0: if total == 0:
return 0.5 # Unbestimmt return 0.5 # Unbestimmt
@@ -74,6 +108,8 @@ class Correctness:
else: else:
review_history.append(entry.to_dict()) review_history.append(entry.to_dict())
return { return {
"verdict": self.verdict,
"evidence": self.evidence,
"confirmed": self.confirmed, "confirmed": self.confirmed,
"confirmations": self.confirmations, "confirmations": self.confirmations,
"rejections": self.rejections, "rejections": self.rejections,
@@ -84,11 +120,30 @@ class Correctness:
@classmethod @classmethod
def from_dict(cls, d: dict) -> "Correctness": def from_dict(cls, d: dict) -> "Correctness":
c = cls() c = cls()
verdict = d.get("verdict")
if isinstance(verdict, str) and verdict.strip():
c.verdict = verdict.strip()
c.confirmed = d.get("confirmed", False) c.confirmed = d.get("confirmed", False)
c.confirmations = d.get("confirmations", 0) c.confirmations = d.get("confirmations", 0)
c.rejections = d.get("rejections", 0) c.rejections = d.get("rejections", 0)
c.last_reviewed = d.get("last_reviewed") c.last_reviewed = d.get("last_reviewed")
ev = d.get("evidence", [])
if isinstance(ev, list):
c.evidence = [e for e in ev if isinstance(e, dict)]
c.review_history = [ReviewEntry.from_dict(r) for r in d.get("review_history", [])] c.review_history = [ReviewEntry.from_dict(r) for r in d.get("review_history", [])]
# Backfill verdict if missing/invalid.
if c.verdict not in ("unknown", "probable_true", "probable_false", "confirmed_true", "confirmed_false"):
if c.confirmed:
c.verdict = "confirmed_true"
elif c.rejections > 0:
c.verdict = "confirmed_false"
else:
c.verdict = "unknown"
# Ensure boolean stays consistent for older mixed data.
if c.verdict == "confirmed_true":
c.confirmed = True
elif c.verdict == "confirmed_false":
c.confirmed = False
return c return c

View File

@@ -6,6 +6,7 @@ Keine externen Abhängigkeiten außer sqlite3 (stdlib).
import json import json
import sqlite3 import sqlite3
import os import os
import re
from pathlib import Path from pathlib import Path
from typing import List, Optional, Dict, Any from typing import List, Optional, Dict, Any
from uuid import UUID from uuid import UUID
@@ -158,7 +159,12 @@ class EngramStore:
def search_text(self, query: str, limit: int = 10) -> List[Engram]: def search_text(self, query: str, limit: int = 10) -> List[Engram]:
"""Full-Text-Suche über Engramm-Inhalt via SQLite FTS5 (OR-Verknüpfung).""" """Full-Text-Suche über Engramm-Inhalt via SQLite FTS5 (OR-Verknüpfung)."""
# FTS5-Syntax: Wörter mit OR verbinden für bessere Ergebnisse # FTS5-Syntax: Wörter mit OR verbinden für bessere Ergebnisse
words = [w.strip() for w in query.replace("'", "''").split() if w.strip()] words = []
for word in query.split():
# Nur alphanumerische Zeichen als FTS5-Tokens akzeptieren
clean_word = re.sub(r'[^a-zA-Z0-9]+', '', word)
if clean_word:
words.append(clean_word)
safe_query = " OR ".join(words) if len(words) > 1 else (words[0] if words else "*") safe_query = " OR ".join(words) if len(words) > 1 else (words[0] if words else "*")
sql = """ sql = """
SELECT e.* FROM engrams e SELECT e.* FROM engrams e

View File

@@ -27,6 +27,31 @@ body {
top: 0; top: 0;
z-index: 50; z-index: 50;
} }
.tabs-bar{
display:flex;
gap:8px;
padding:8px 12px 10px;
background:#141419;
border-bottom:1px solid #252530;
position: sticky;
top: 52px;
z-index: 45;
}
.tabs-bar .tab-btn{
flex:1;
background:#1e1e28;
border:1px solid #2a2a3a;
border-radius: 12px;
padding:10px 10px;
color:#cfd3ff;
font-weight:700;
font-size:0.82rem;
}
.tabs-bar .tab-btn.active{
border-color:#6c8af5;
box-shadow:0 0 0 1px rgba(108,138,245,0.22) inset;
}
.stat { .stat {
text-align: center; text-align: center;
min-width: 60px; min-width: 60px;
@@ -54,6 +79,122 @@ body {
padding: 10px 12px; padding: 10px 12px;
background: #141419; background: #141419;
} }
/* tab buttons styled via .tabs-bar */
/* ─── Panels (Graph/Status) ──────────────────────────────────────────────── */
.panel {
margin: 8px 12px;
background: #1a1a24;
border: 1px solid #252533;
border-radius: 14px;
padding: 10px 12px;
}
.panel-title {
font-size: 0.8rem;
text-transform: uppercase;
letter-spacing: 0.5px;
color: #9aa3d9;
margin-bottom: 8px;
}
.kv-row {
display: flex;
gap: 10px;
padding: 6px 0;
border-bottom: 1px solid #20202a;
}
.kv-row:last-child { border-bottom: none; }
.kv-key {
width: 110px;
color: #888899;
font-size: 0.78rem;
}
.kv-val {
flex: 1;
color: #e8e8ee;
font-size: 0.85rem;
word-break: break-word;
}
.pill {
display: inline-block;
margin: 2px 4px 2px 0;
padding: 2px 8px;
border-radius: 999px;
background: #2a2a3a;
color: #8a9aff;
font-size: 0.72rem;
}
.verdict-pill{
display:inline-block;
margin: 2px 6px 2px 0;
padding: 2px 8px;
border-radius: 999px;
font-size: 0.72rem;
font-weight: 800;
letter-spacing: 0.4px;
border: 1px solid #2a2a3a;
background: #1e1e28;
color: #cfd3ff;
}
.verdict-pill.v-true{ border-color:#2f6b3f; color:#aaf0b6; }
.verdict-pill.v-false{ border-color:#7a2c2c; color:#ffb3b3; }
.verdict-pill.v-prob-true{ border-color:#6c8af5; color:#cfd9ff; }
.verdict-pill.v-prob-false{ border-color:#b08a2a; color:#ffe2a3; }
.verdict-pill.v-unknown{ border-color:#3a3a55; color:#b9b9c9; }
.muted {
color: #888899;
font-size: 0.8rem;
margin-top: 6px;
}
.small { font-size: 0.75rem; }
/* Graph canvas */
#graphCanvas{
display:block;
margin: 8px auto 0;
background:#12121a;
border:1px solid #252533;
border-radius: 14px;
touch-action: none;
}
.graph-controls{
display:flex;
gap:8px;
padding: 10px 12px 0;
align-items:center;
flex-wrap: wrap;
}
.graph-controls .btn{
background:#1e1e28;
border:1px solid #2a2a3a;
border-radius: 10px;
padding: 8px 10px;
color:#cfd3ff;
font-weight:700;
font-size:0.82rem;
}
.graph-controls .btn.primary{
border-color:#6c8af5;
box-shadow:0 0 0 1px rgba(108,138,245,0.18) inset;
}
.graph-legend{
margin: 8px 12px 0;
padding: 10px 12px;
background:#1a1a24;
border:1px solid #252533;
border-radius: 14px;
color:#b9b9c9;
font-size:0.8rem;
line-height:1.4;
}
.legend-row{ display:flex; align-items:center; gap:8px; margin-top:6px; }
.legend-dot{ width:10px; height:10px; border-radius:50%; display:inline-block; }
.legend-dot.engram{ background:#6c8af5; }
.legend-dot.tag{ background:#8a9aff; }
.legend-dot.match{ background:#f7d154; }
.graph-hint{ padding: 4px 12px 10px; }
#searchInput { #searchInput {
flex: 1; flex: 1;
background: #1e1e28; background: #1e1e28;

View File

@@ -0,0 +1,9 @@
[Unit]
Description=OpenClaw archive memory/*.md older than 7 days
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py archive_memory_md'

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw archive memory/*.md daily
[Timer]
OnCalendar=*-*-* 03:30:00
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain backup_secondbrain
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py backup_secondbrain'

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw Second-Brain backup_secondbrain (daily 02:00)
[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,15 @@
[Unit]
Description=OpenClaw Second-Brain Dashboard (FastAPI)
After=network.target
[Service]
Type=simple
WorkingDirectory=/root/.openclaw/workspace/second-brain
Environment=SECOND_BRAIN_WORKSPACE=/root/.openclaw/workspace/second-brain
Environment=SECOND_BRAIN_PORT=8501
ExecStart=/root/.openclaw/workspace/second-brain/.venv/bin/uvicorn fastapi_app:app --host 0.0.0.0 --port 8501
Restart=always
RestartSec=2
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain export_obsidian
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py export_obsidian'

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw Second-Brain export_obsidian (hourly)
[Timer]
OnCalendar=hourly
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain heartbeat_secondbrain
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py heartbeat_secondbrain'

View File

@@ -0,0 +1,11 @@
[Unit]
Description=OpenClaw Second-Brain heartbeat_secondbrain (every 6h)
[Timer]
OnCalendar=*-*-* 00,06,12,18:00:00
RandomizedDelaySec=300
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw Second-Brain index_vectors
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
Environment=HF_HOME=/root/.openclaw/workspace/second-brain/data/hf_cache
Environment=SENTENCE_TRANSFORMERS_HOME=/root/.openclaw/workspace/second-brain/data/st_cache
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py index_vectors'

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw Second-Brain index_vectors (every 30 min)
[Timer]
OnCalendar=*:0/30
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain ingest_memory
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py ingest_memory'

View File

@@ -0,0 +1,9 @@
[Unit]
Description=OpenClaw Second-Brain ingest_memory (every 5 min)
[Timer]
OnCalendar=*:0/5
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain ingest_obsidian
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py ingest_obsidian'

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw Second-Brain ingest_obsidian (every 15 min)
[Timer]
OnCalendar=*:0/15
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,9 @@
[Unit]
Description=OpenClaw Second-Brain ingest transcript -> DB
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py ingest_transcript_to_db'

View File

@@ -0,0 +1,12 @@
[Unit]
Description=OpenClaw Second-Brain ingest transcript -> DB (every 5 min)
[Timer]
OnBootSec=90s
OnUnitActiveSec=300s
Unit=openclaw-secondbrain-ingest-transcript-to-db.service
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain ingest transcript -> memory/*.md
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/second-brain/cron_tasks/ingest_transcript_to_memory.py'

View File

@@ -0,0 +1,11 @@
[Unit]
Description=OpenClaw Second-Brain ingest transcript -> memory (every 1 min)
[Timer]
OnBootSec=30s
OnUnitActiveSec=60s
Unit=openclaw-secondbrain-ingest-transcript-to-memory.service
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain failure notify (%i)
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc '/root/.openclaw/workspace/notify-telegram.sh "❌ Second-Brain job failed: %i. Check: journalctl -u %i -n 50 --no-pager"'

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw Second-Brain proactive_search_wrapper
Wants=network-online.target
After=network-online.target
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py proactive_search_wrapper'

View File

@@ -0,0 +1,11 @@
[Unit]
Description=OpenClaw Second-Brain proactive_search_wrapper (every 4h)
[Timer]
OnCalendar=*-*-* 01,05,09,13,17,21:10:00
RandomizedDelaySec=600
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,8 @@
[Unit]
Description=OpenClaw Second-Brain review_brain
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py review_brain'

View File

@@ -0,0 +1,9 @@
[Unit]
Description=OpenClaw Second-Brain review_brain (every 5 min)
[Timer]
OnCalendar=*:0/5
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -0,0 +1,13 @@
[Unit]
Description=OpenClaw Second-Brain task (%i)
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py %i
Nice=10
IOSchedulingClass=best-effort
IOSchedulingPriority=6

View File

@@ -0,0 +1,9 @@
[Unit]
Description=OpenClaw Second-Brain verify_pending_external
OnFailure=openclaw-secondbrain-notify@%n.service
[Service]
Type=oneshot
WorkingDirectory=/root/.openclaw/workspace
ExecStart=/bin/bash -lc 'flock -n /tmp/%n.lock /usr/bin/python3 /root/.openclaw/workspace/openclaw_cron_wrapper.py verify_pending_external'

View File

@@ -0,0 +1,10 @@
[Unit]
Description=OpenClaw Second-Brain periodic verify_pending_external
[Timer]
OnBootSec=2min
OnUnitActiveSec=5min
Persistent=true
[Install]
WantedBy=timers.target

View File

@@ -16,6 +16,12 @@
<div class="stat"><span class="stat-num" id="statErrors">-</span><span class="stat-label">Err</span></div> <div class="stat"><span class="stat-num" id="statErrors">-</span><span class="stat-label">Err</span></div>
</header> </header>
<div class="tabs-bar">
<button class="tab-btn active" id="tabCards" onclick="setView('cards')">Cards</button>
<button class="tab-btn" id="tabGraph" onclick="setView('graph')">Graph</button>
<button class="tab-btn" id="tabStatus" onclick="setView('status')">Status</button>
</div>
<!-- Search --> <!-- Search -->
<div class="search-box"> <div class="search-box">
<input type="text" id="searchInput" placeholder="🔍 Suche..." /> <input type="text" id="searchInput" placeholder="🔍 Suche..." />
@@ -23,6 +29,7 @@
<option value="all">Alle</option> <option value="all">Alle</option>
<option value="pending">Pending</option> <option value="pending">Pending</option>
<option value="confirmed">Confirmed</option> <option value="confirmed">Confirmed</option>
<option value="rejected">Rejected</option>
<option value="errors">Errors</option> <option value="errors">Errors</option>
</select> </select>
</div> </div>
@@ -37,6 +44,27 @@
<!-- Cards --> <!-- Cards -->
<div class="cards" id="cards"></div> <div class="cards" id="cards"></div>
<!-- Graph -->
<div class="graph" id="graph" style="display:none;">
<div class="graph-controls">
<button class="btn primary" id="btnGraphPhysics" onclick="toggleGraphPhysics()">Physics: off</button>
<button class="btn" onclick="resetGraphView()">Reset view</button>
<button class="btn" onclick="fitGraphView()">Fit</button>
<button class="btn" onclick="reloadGraph()">Reload</button>
</div>
<canvas id="graphCanvas" width="440" height="520"></canvas>
<div class="graph-hint muted small" id="graphHint">Lade Graph…</div>
<div class="graph-legend">
<div><strong>Graph</strong>: Zoom per Pinch (2 Finger), Pan per Drag (1 Finger). Tap auf Engram öffnet Details, Tap auf Tag setzt Suche.</div>
<div class="legend-row"><span class="legend-dot engram"></span> Engram</div>
<div class="legend-row"><span class="legend-dot tag"></span> Tag</div>
<div class="legend-row"><span class="legend-dot match"></span> Match (Suche)</div>
</div>
</div>
<!-- Status -->
<div class="status" id="status" style="display:none;"></div>
<!-- Pagination --> <!-- Pagination -->
<div class="pagination" id="pagination"> <div class="pagination" id="pagination">
<button id="btnPrev" onclick="prevPage()"></button> <button id="btnPrev" onclick="prevPage()"></button>
@@ -67,6 +95,8 @@ let state = {
filter: 'all', filter: 'all',
search: '', search: '',
autoRefresh: true, autoRefresh: true,
view: 'cards',
lastEvent: null,
}; };
// ─── Fetch ────────────────────────────────────────────────────────────────── // ─── Fetch ──────────────────────────────────────────────────────────────────
@@ -84,11 +114,36 @@ async function loadStats() {
document.getElementById('statErrors').textContent = s.errors; document.getElementById('statErrors').textContent = s.errors;
} }
function updateStatsFromEvent(ev) {
if (!ev || !ev.stats) return;
const s = ev.stats;
document.getElementById('statTotal').textContent = s.total;
document.getElementById('statConfirmed').textContent = s.confirmed;
document.getElementById('statPending').textContent = s.pending;
document.getElementById('statErrors').textContent = s.errors;
}
function setView(view) {
state.view = view;
document.getElementById('tabCards').classList.toggle('active', view === 'cards');
document.getElementById('tabGraph').classList.toggle('active', view === 'graph');
document.getElementById('tabStatus').classList.toggle('active', view === 'status');
document.getElementById('cards').style.display = view === 'cards' ? '' : 'none';
document.getElementById('pagination').style.display = view === 'cards' ? '' : 'none';
document.getElementById('graph').style.display = view === 'graph' ? '' : 'none';
document.getElementById('status').style.display = view === 'status' ? '' : 'none';
if (view === 'graph') loadGraph();
if (view === 'status') loadStatus();
}
async function loadCards() { async function loadCards() {
let url = `/api/engrams?limit=${state.limit}&offset=${state.offset}`; let url = `/api/engrams?limit=${state.limit}&offset=${state.offset}`;
if (state.search) url += `&search=${encodeURIComponent(state.search)}`; if (state.search) url += `&search=${encodeURIComponent(state.search)}`;
if (state.filter === 'confirmed') url += '&confirmed=1'; if (state.filter === 'confirmed') url += '&confirmed=1';
if (state.filter === 'pending') url += '&confirmed=0'; if (state.filter === 'pending') url += '&confirmed=0';
if (state.filter === 'rejected') url += '&verdict=confirmed_false';
if (state.filter === 'errors') url += '&tag=error'; if (state.filter === 'errors') url += '&tag=error';
const data = await api(url); const data = await api(url);
@@ -99,12 +154,480 @@ async function loadCards() {
document.getElementById('btnNext').disabled = data.items.length < state.limit; document.getElementById('btnNext').disabled = data.items.length < state.limit;
} }
async function loadStatus() {
const [cfg, db, jobs, ins, stor] = await Promise.all([
api('/api/config'),
api('/api/db_info'),
api('/api/jobs'),
api('/api/insights?limit=8'),
api('/api/storage_stats'),
]);
const pend = await api('/api/pending?limit=20&offset=0');
const el = document.getElementById('status');
const jobsHtml = (jobs.items || []).map(j => `
<div class="kv-row">
<div class="kv-key">${j.unit}</div>
<div class="kv-val">${j.error ? ('ERR: ' + j.error) : (j.active + '/' + j.sub)}</div>
</div>
`).join('');
const topTags = (ins.top_tags || []).map(t => `<span class="pill">${t.key} (${t.count})</span>`).join(' ');
const topHosts = (ins.top_hosts || []).map(t => `<span class="pill">${t.key} (${t.count})</span>`).join(' ');
const bySource = Object.entries((stor.sql && stor.sql.by_source) ? stor.sql.by_source : {})
.slice(0, 8)
.map(([k,v]) => `<span class="pill">${k}: ${v}</span>`)
.join(' ');
const pendItems = (pend.items || []);
const pendHtml = pendItems.map(p => `
<div class="kv-row" onclick="showDetail('${p.id}')">
<div class="kv-key">${(p.source||'').slice(0,12)}</div>
<div class="kv-val">
<span class="pill">${p.id.substring(0,8)}</span>
${escapeHtml((p.content||'').substring(0,120))}${(p.content||'').length>120?'…':''}
<div class="actions" style="margin-top:6px" onclick="event.stopPropagation()">
<button class="btn-ok" onclick="confirm('${p.id}', event)">✅</button>
<button class="btn-no" onclick="reject('${p.id}', event)">❌</button>
</div>
</div>
</div>
`).join('');
el.innerHTML = `
<div class="panel">
<div class="panel-title">Config</div>
<div class="kv-row"><div class="kv-key">workspace</div><div class="kv-val">${cfg.workspace}</div></div>
<div class="kv-row"><div class="kv-key">db</div><div class="kv-val">${db.db_path}</div></div>
<div class="kv-row"><div class="kv-key">db mtime</div><div class="kv-val">${new Date(db.mtime).toLocaleString()}</div></div>
</div>
<div class="panel">
<div class="panel-title">Storage</div>
<div class="kv-row"><div class="kv-key">SQL</div><div class="kv-val">${stor.sql.total_engrams} engrams (ok ${stor.sql.confirmed}, pending ${stor.sql.pending})</div></div>
<div class="kv-row"><div class="kv-key">Vector</div><div class="kv-val">chroma ${(stor.vector.chroma_size_bytes/1024/1024).toFixed(1)} MB, cache ${stor.vector.embedding_cache_files} files</div></div>
<div class="kv-row"><div class="kv-key">Obsidian</div><div class="kv-val">${stor.obsidian.configured ? 'configured' : 'not configured'}</div></div>
<div class="kv-row"><div class="kv-key">By source</div><div class="kv-val">${bySource || '-'}</div></div>
</div>
<div class="panel">
<div class="panel-title">Jobs</div>
${jobsHtml || '<div class="muted">Keine Daten</div>'}
</div>
<div class="panel">
<div class="panel-title">Insights</div>
<div class="kv-row"><div class="kv-key">pending</div><div class="kv-val">${ins.pending}</div></div>
<div class="kv-row"><div class="kv-key">top tags</div><div class="kv-val">${topTags || '-'}</div></div>
<div class="kv-row"><div class="kv-key">top hosts</div><div class="kv-val">${topHosts || '-'}</div></div>
</div>
<div class="panel">
<div class="panel-title">Pending Queue (latest)</div>
${pendHtml || '<div class="muted">Keine Pendings</div>'}
</div>
`;
}
async function loadGraph() {
const g = await api('/api/graph?limit_nodes=200');
renderGraph(g.nodes || [], g.edges || []);
}
function reloadGraph() { loadGraph(); }
// ─── Graph Renderer (Canvas) ────────────────────────────────────────────────
let graphState = {
nodes: [],
edges: [],
sim: [],
links: [],
nodeById: new Map(),
simById: new Map(),
degree: new Map(),
physicsOn: false,
draggingId: null,
panning: false,
lastX: 0,
lastY: 0,
panX: 0,
panY: 0,
zoom: 1,
raf: null,
search: '',
pointers: new Map(),
pinchStartDist: null,
pinchStartZoom: null,
pinchStartPan: null,
};
function _graphCanvas() { return document.getElementById('graphCanvas'); }
function _graphCtx() { return _graphCanvas().getContext('2d'); }
function _graphNodeRadius(n) {
const d = graphState.degree.get(n.id) || 0;
const base = n.kind === 'tag' ? 4 : 6;
return Math.max(3, Math.min(14, base + Math.sqrt(d)));
}
function _graphNodeFill(n) {
if (n.kind === 'tag') return '#8a9aff';
return '#6c8af5';
}
function _graphMatches(n, term) {
const t = (term || '').trim().toLowerCase();
if (!t) return false;
const id = (n.id || '').toLowerCase();
const label = (n.label || '').toLowerCase();
return id.includes(t) || label.includes(t);
}
function _graphWorldFromScreen(cx, cy) {
return {
x: (cx - graphState.panX) / graphState.zoom,
y: (cy - graphState.panY) / graphState.zoom,
};
}
function _graphHitTest(wx, wy) {
for (let i = graphState.sim.length - 1; i >= 0; i--) {
const n = graphState.sim[i];
const r = _graphNodeRadius(n) + 2;
const dx = wx - n.x;
const dy = wy - n.y;
if ((dx*dx + dy*dy) <= r*r) return n;
}
return null;
}
function _graphInitInteractions() {
const canvas = _graphCanvas();
if (canvas._graphBound) return;
canvas._graphBound = true;
const toCanvasXY = (ev) => {
const rect = canvas.getBoundingClientRect();
return { cx: ev.clientX - rect.left, cy: ev.clientY - rect.top };
};
const pinchMidpoint = () => {
const pts = Array.from(graphState.pointers.values());
if (pts.length < 2) return null;
return { cx: (pts[0].cx + pts[1].cx) / 2, cy: (pts[0].cy + pts[1].cy) / 2 };
};
const pinchDistance = () => {
const pts = Array.from(graphState.pointers.values());
if (pts.length < 2) return null;
const dx = pts[0].cx - pts[1].cx;
const dy = pts[0].cy - pts[1].cy;
return Math.sqrt(dx*dx + dy*dy);
};
canvas.addEventListener('pointerdown', (ev) => {
canvas.setPointerCapture(ev.pointerId);
const pos = toCanvasXY(ev);
graphState.pointers.set(ev.pointerId, pos);
// When 2 pointers: start pinch
if (graphState.pointers.size === 2) {
graphState.panning = false;
graphState.draggingId = null;
graphState.pinchStartDist = pinchDistance();
graphState.pinchStartZoom = graphState.zoom;
graphState.pinchStartPan = { panX: graphState.panX, panY: graphState.panY };
_graphDraw();
return;
}
// Single pointer: pan or drag node
const { cx, cy } = pos;
const w = _graphWorldFromScreen(cx, cy);
const hit = _graphHitTest(w.x, w.y);
graphState.lastX = cx; graphState.lastY = cy;
if (hit) graphState.draggingId = hit.id;
else graphState.panning = true;
});
canvas.addEventListener('pointermove', (ev) => {
if (!graphState.pointers.has(ev.pointerId)) return;
const pos = toCanvasXY(ev);
graphState.pointers.set(ev.pointerId, pos);
// Pinch zoom (2 pointers)
if (graphState.pointers.size >= 2 && graphState.pinchStartDist) {
const dist = pinchDistance();
const mid = pinchMidpoint();
if (!dist || !mid) return;
const zoomOld = graphState.zoom;
const wx = (mid.cx - graphState.panX) / zoomOld;
const wy = (mid.cy - graphState.panY) / zoomOld;
const factor = dist / graphState.pinchStartDist;
const next = Math.max(0.35, Math.min(3.0, graphState.pinchStartZoom * factor));
graphState.zoom = next;
graphState.panX = mid.cx - wx * graphState.zoom;
graphState.panY = mid.cy - wy * graphState.zoom;
_graphDraw();
return;
}
// Single pointer
const { cx, cy } = pos;
const dx = cx - graphState.lastX;
const dy = cy - graphState.lastY;
graphState.lastX = cx; graphState.lastY = cy;
if (graphState.draggingId) {
const w = _graphWorldFromScreen(cx, cy);
const n = graphState.simById.get(graphState.draggingId);
if (n) { n.x = w.x; n.y = w.y; n.vx = 0; n.vy = 0; }
_graphDraw();
return;
}
if (graphState.panning) {
graphState.panX += dx;
graphState.panY += dy;
_graphDraw();
}
});
const endPointer = (ev) => {
if (graphState.pointers.has(ev.pointerId)) graphState.pointers.delete(ev.pointerId);
try { canvas.releasePointerCapture(ev.pointerId); } catch {}
if (graphState.pointers.size < 2) {
graphState.pinchStartDist = null;
graphState.pinchStartZoom = null;
graphState.pinchStartPan = null;
}
graphState.draggingId = null;
graphState.panning = false;
};
canvas.addEventListener('pointerup', endPointer);
canvas.addEventListener('pointercancel', endPointer);
}
function renderGraph(nodes, edges) {
const canvas = _graphCanvas();
const hint = document.getElementById('graphHint');
const ctx = _graphCtx();
const w = canvas.parentElement.clientWidth - 24;
canvas.width = Math.max(320, Math.min(520, w));
canvas.height = 520;
graphState.nodes = nodes || [];
graphState.edges = edges || [];
graphState.nodeById = new Map(graphState.nodes.map(n => [n.id, n]));
graphState.degree = new Map();
for (const e of graphState.edges) {
graphState.degree.set(e.from, (graphState.degree.get(e.from) || 0) + 1);
graphState.degree.set(e.to, (graphState.degree.get(e.to) || 0) + 1);
}
if (!graphState.nodes.length || !graphState.edges.length) {
hint.textContent = 'Graph: keine Kanten (noch keine Links/Tags im Sample).';
ctx.clearRect(0,0,canvas.width,canvas.height);
return;
}
hint.textContent = `nodes=${graphState.nodes.length} edges=${graphState.edges.length}`;
graphState.sim = graphState.nodes.map(n => ({
id: n.id,
kind: n.kind,
label: n.label || n.id,
x: (Math.random() - 0.5) * canvas.width,
y: (Math.random() - 0.5) * canvas.height,
vx: 0, vy: 0,
}));
graphState.simById = new Map(graphState.sim.map(n => [n.id, n]));
graphState.links = graphState.edges
.map(e => ({a: graphState.simById.get(e.from), b: graphState.simById.get(e.to), kind: e.kind}))
.filter(l => l.a && l.b);
graphState.panX = canvas.width / 2;
graphState.panY = canvas.height / 2;
graphState.zoom = 1;
graphState.search = state.search || '';
_graphInitInteractions();
for (let i = 0; i < 120; i++) _graphStepPhysics(0.9);
_graphDraw();
}
function _graphStepPhysics(alpha = 1.0) {
const canvas = _graphCanvas();
const repulsion = 180;
const damping = 0.86;
const target = 80;
const springK = 0.018;
const sim = graphState.sim;
for (let i = 0; i < sim.length; i++) {
for (let j = i + 1; j < sim.length; j++) {
const a = sim[i], b = sim[j];
const dx = a.x - b.x, dy = a.y - b.y;
const d2 = dx*dx + dy*dy + 0.02;
const f = (repulsion / d2) * alpha;
a.vx += dx*f; a.vy += dy*f;
b.vx -= dx*f; b.vy -= dy*f;
}
}
for (const l of graphState.links) {
const a = l.a, b = l.b;
const dx = b.x - a.x, dy = b.y - a.y;
const dist = Math.sqrt(dx*dx + dy*dy) || 1;
const f = (dist - target) * springK * alpha;
const fx = (dx/dist) * f, fy = (dy/dist) * f;
a.vx += fx; a.vy += fy;
b.vx -= fx; b.vy -= fy;
}
for (const n of sim) {
n.vx *= damping; n.vy *= damping;
n.x += n.vx; n.y += n.vy;
const pad = 10;
const bx = canvas.width / graphState.zoom;
const by = canvas.height / graphState.zoom;
n.x = Math.max(-bx/2 + pad, Math.min(bx/2 - pad, n.x));
n.y = Math.max(-by/2 + pad, Math.min(by/2 - pad, n.y));
}
}
function _graphDraw() {
const canvas = _graphCanvas();
const ctx = _graphCtx();
const hint = document.getElementById('graphHint');
ctx.clearRect(0,0,canvas.width,canvas.height);
ctx.save();
ctx.translate(graphState.panX, graphState.panY);
ctx.scale(graphState.zoom, graphState.zoom);
ctx.globalAlpha = 0.45;
ctx.strokeStyle = '#3a3a55';
ctx.lineWidth = 1 / graphState.zoom;
for (const l of graphState.links) {
ctx.beginPath();
ctx.moveTo(l.a.x, l.a.y);
ctx.lineTo(l.b.x, l.b.y);
ctx.stroke();
}
ctx.globalAlpha = 1.0;
const term = (graphState.search || '').trim();
let matches = 0;
for (const n of graphState.sim) {
const r = _graphNodeRadius(n);
const isMatch = _graphMatches(n, term);
if (isMatch) matches++;
if (isMatch) {
ctx.beginPath();
ctx.strokeStyle = '#f7d154';
ctx.lineWidth = 3 / graphState.zoom;
ctx.arc(n.x, n.y, r + 3, 0, Math.PI*2);
ctx.stroke();
}
ctx.beginPath();
ctx.fillStyle = _graphNodeFill(n);
ctx.arc(n.x, n.y, r, 0, Math.PI*2);
ctx.fill();
}
ctx.restore();
hint.textContent = `nodes=${graphState.nodes.length} edges=${graphState.edges.length}` + (term ? ` | match=${matches}` : '');
}
function _graphLoop() {
if (!graphState.physicsOn) return;
_graphStepPhysics(1.0);
_graphDraw();
graphState.raf = requestAnimationFrame(_graphLoop);
}
function toggleGraphPhysics() {
graphState.physicsOn = !graphState.physicsOn;
const b = document.getElementById('btnGraphPhysics');
b.textContent = `Physics: ${graphState.physicsOn ? 'on' : 'off'}`;
b.classList.toggle('primary', graphState.physicsOn);
if (graphState.physicsOn) {
if (graphState.raf) cancelAnimationFrame(graphState.raf);
graphState.raf = requestAnimationFrame(_graphLoop);
} else {
if (graphState.raf) cancelAnimationFrame(graphState.raf);
graphState.raf = null;
_graphDraw();
}
}
function resetGraphView() {
const canvas = _graphCanvas();
graphState.panX = canvas.width / 2;
graphState.panY = canvas.height / 2;
graphState.zoom = 1;
_graphDraw();
}
function fitGraphView() {
const canvas = _graphCanvas();
if (!graphState.sim.length) return;
let minX = Infinity, minY = Infinity, maxX = -Infinity, maxY = -Infinity;
for (const n of graphState.sim) {
minX = Math.min(minX, n.x); minY = Math.min(minY, n.y);
maxX = Math.max(maxX, n.x); maxY = Math.max(maxY, n.y);
}
const w = Math.max(1, maxX - minX);
const h = Math.max(1, maxY - minY);
const zx = (canvas.width - 40) / w;
const zy = (canvas.height - 40) / h;
graphState.zoom = Math.max(0.35, Math.min(2.5, Math.min(zx, zy)));
graphState.panX = canvas.width / 2;
graphState.panY = canvas.height / 2;
_graphDraw();
}
function graphApplySearch(term) {
graphState.search = term || '';
_graphDraw();
}
// Real-time updates via SSE
function startEvents() {
try {
const es = new EventSource('/api/events');
es.onmessage = (msg) => {
try {
state.lastEvent = JSON.parse(msg.data);
updateStatsFromEvent(state.lastEvent);
if (state.view === 'status') {
// refresh status panels without heavy re-render: just rerun loadStatus occasionally
loadStatus();
}
if (state.view === 'graph') {
// fetch graph less often (every ~15s)
const t = Date.now();
if (!state._lastGraphFetch || (t - state._lastGraphFetch) > 15000) {
state._lastGraphFetch = t;
loadGraph();
}
}
} catch (e) {}
};
es.onerror = () => {
// keep UI usable even if SSE drops
};
} catch (e) {}
}
startEvents();
function renderCards() { function renderCards() {
const el = document.getElementById('cards'); const el = document.getElementById('cards');
el.innerHTML = state.items.map(item => ` el.innerHTML = state.items.map(item => `
<div class="card ${item.confirmed ? 'confirmed' : ''} ${item.rejections > 0 ? 'rejected' : ''}" data-id="${item.id}"> <div class="card ${item.confirmed ? 'confirmed' : ''} ${item.rejections > 0 ? 'rejected' : ''}" data-id="${item.id}">
<div class="card-header"> <div class="card-header">
<span class="conf-badge" style="background:hsl(${item.confidence*120},70%,40%)">${Math.round(item.confidence*100)}%</span> <span class="conf-badge" style="background:hsl(${item.confidence*120},70%,40%)">${Math.round(item.confidence*100)}%</span>
${renderVerdictPill(item)}
<span class="tags">${item.tags.map(t => '<span class="tag">'+t+'</span>').join('')}</span> <span class="tags">${item.tags.map(t => '<span class="tag">'+t+'</span>').join('')}</span>
<span class="date">${fmtDate(item.created)}</span> <span class="date">${fmtDate(item.created)}</span>
</div> </div>
@@ -123,6 +646,19 @@ function renderCards() {
`).join(''); `).join('');
} }
function renderVerdictPill(item) {
const v = (item.verdict || '').toString();
if (!v) return '';
let cls = 'v-unknown';
let label = v;
if (v === 'confirmed_true') { cls = 'v-true'; label = 'TRUE'; }
else if (v === 'confirmed_false') { cls = 'v-false'; label = 'FALSE'; }
else if (v === 'probable_true') { cls = 'v-prob-true'; label = 'LIKELY'; }
else if (v === 'probable_false') { cls = 'v-prob-false'; label = 'UNLIKELY'; }
else if (v === 'unknown') { cls = 'v-unknown'; label = 'UNKNOWN'; }
return `<span class="verdict-pill ${cls}">${label}</span>`;
}
function fmtDate(iso) { function fmtDate(iso) {
const d = new Date(iso); const d = new Date(iso);
return `${d.getDate().toString().padStart(2,'0')}.${(d.getMonth()+1).toString().padStart(2,'0')} ${d.getHours().toString().padStart(2,'0')}:${d.getMinutes().toString().padStart(2,'0')}`; return `${d.getDate().toString().padStart(2,'0')}.${(d.getMonth()+1).toString().padStart(2,'0')} ${d.getHours().toString().padStart(2,'0')}:${d.getMinutes().toString().padStart(2,'0')}`;
@@ -220,6 +756,7 @@ document.getElementById('searchInput').addEventListener('input', (e) => {
state.search = e.target.value; state.search = e.target.value;
state.offset = 0; state.offset = 0;
loadCards(); loadCards();
if (state.view === 'graph') graphApplySearch(state.search);
}); });
document.getElementById('filterSelect').addEventListener('change', (e) => { document.getElementById('filterSelect').addEventListener('change', (e) => {