Sleeper Cognition Stack

Map of how `~/chat/`, `~/thoughts/`, `~/github/sfl/`, and `~/sfl-hook/` interact, the seams between them, and the fixes worth shipping


The Sleeper server hosts a personal "cognition stack" — but in practice it's three half-integrated stacks pretending to be one. A polling pipeline (sleeper-articles + thoughts-sync) pulls SFL ideas, extracts them, and writes them to disk. A reactive pipeline (sfl-hook) receives a webhook, runs claude as a subprocess, and posts a reply back to SFL. A retrieval pipeline (chat-backend + wiki-mcp) surfaces wiki and memory inside conversations. The four directories at the heart of all this — ~/chat/, ~/thoughts/, ~/github/sfl/, and ~/sfl-hook/ — grew organically over months, and the seams between them are where every interesting bug lives.

This article maps the topology, walks three end-to-end information flows, and ranks the seams worth addressing. All claims are pinned to repo commits and live process state captured at 2026-05-10T13:44Z. Sources at the bottom are file:line citations to the underlying code.

Scope

In scope: the four directories above plus the implicit fifth component — the SFL Worker on Cloudflare (sfl-api.aiwdm.workers.dev) which is the only writer of SFL ideas. Out of scope: tasks/memory CLI internals, aiwdm, wavespeed, almanac-design, the council-of-experts mechanics themselves, and Cloudflare Pages deploys for phareim.no.

Topology

                          ┌───────────────────────────┐
                          │  SFL Worker (Cloudflare)  │
                          │  sfl-api.aiwdm.workers.dev│
                          │  D1 + R2 — only writer    │
                          └─────────────┬─────────────┘
                ┌────────────────────────┼────────────────────────┐
                │                        │                        │
        POST /sfl-hook          GET /api/ideas (5min)     GET /api/ideas (5min)
        webhook on user msg     full-scan, no `since`     full-scan, no `since`
                │                        │                        │
                ▼                        ▼                        ▼
   ┌──────────────────────┐   ┌────────────────────┐   ┌────────────────────────┐
   │  ~/sfl-hook          │   │ sleeper-articles   │   │  thoughts-sync         │
   │  port 4000           │   │ port 3003 (PM2)    │   │  PM2 (no port)         │
   │  ✗ NOT in PM2        │   │ SQLite + FTS5      │   │  state.json + raw/     │
   │  user-systemd unit   │   │ Venice + memory    │   │  depends on articles   │
   │  spawn('claude','-p',│   │  bridge            │   │  full-scan SFL too     │
   │   ...,'--dangerously-│   └────┬──────┬────────┘   └────┬───────────────────┘
   │   skip-permissions') │        │      │                 │
   └─────────┬────────────┘        │      └─POST /memory─┐  │
             │                     ▼                     ▼  ▼
             │              ┌─────────────┐         ┌────────────────┐
             ▼              │ iOS Read    │         │ sleeper-memory │
   POST /api/messages       │   tab       │         │  port 3004 (PM2)│
   (sender:'sleeper')       │ (articles   │         └────────────────┘
             ▲              │  REST API)  │                  ▲
             │              └─────────────┘   ┌──────────────┘
             │                                │
             │              ┌──────────────── │ ──────────────────┐
             │              │  ~/thoughts/    │                   │
             │              │   raw/  ──manual fold──>  wiki/     │
             │              │   .processed-raw-files (append-only)│
             │              │   sync/wiki-index.json (built)      │
             │              │   thoughts-autocommit (PM2, inotify)│
             │              └────┬─────────────┬──────────────────┘
             │                   │             │
             │       ┌───────────┘             │ disk read (mtime cache)
             │       ▼                         ▼
             │ ┌──────────────┐       ┌─────────────────────────┐
             │ │ wiki-mcp     │       │ ~/chat/backend          │
             │ │ port 3007    │       │ services/retrieval.ts   │
             │ │ PM2          │       │ services/wikiVectors.ts │
             │ │ used by:     │       │ does NOT use wiki-mcp   │
             │ │  iOS app,    │       │ does NOT call port 3007 │
             │ │  Claude.ai   │       └─────────────────────────┘
             │ └──────────────┘
             │
             └─SSE /events─> SFL chat UI

Pin block (used to anchor every claim that follows):

Component Commit / state
~/chat f8e0bf2
~/github/sfl ba824ae
~/sfl-hook 46bcb1d (single commit)
~/thoughts 47f7c84
PM2 services online sleeper-articles (uptime 42h), thoughts-sync (43h), thoughts-autocommit (25d), wiki-mcp (25d), sleeper-memory, sleeper-chat
sfl-hook NOT in PM2 — managed by user-systemd (~/.config/systemd/user/sfl-hook.service, enabled, Restart=always, EnvironmentFile=%h/.config/sfl/hook.env). PID 27318 since 2026-03-08T12:42Z
System-level systemd units for sfl* none

Top 5 things worth addressing

Update 2026-05-25: Four of these five rough edges were resolved or materially mitigated in the sleeper-stack-2026-05-25-maintenance-run. The text below is preserved as the original survey because it explains the trade-offs the fixes made. The current state of each item is annotated inline as "Resolved 2026-05-25" or "Mitigated 2026-05-25".

Ranked by blast radius if left alone. The descriptive material lives below the fixes because the description exists to justify the fixes, not the other way round. Sleeper-tasks IDs are filed for tracking.

1. sfl-hook has a latent fail-open in its secret check, behind a permission-skipping claude subprocess (sleeper-tasks #22) — Resolved 2026-05-25

The SDK migration shipped (@anthropic-ai/sdk with cached system block, no more claude -p); loadConfig fails closed on missing secret or API key; nginx now restricts the path to Cloudflare egress IPs with an exact-match /sfl-hook/health carve-out; per-request structured logs land in data/requests.jsonl with 10 MB × 5 rotation. Both "smallest fix" and "biggest fix" from this section were taken.

What. A 153-line server.js running on port 4000, proxied at sleeper.phareim.no/sfl-hook and /events. On POST /sfl-hook, it validates X-SFL-Secret and async-spawns claude -p <body> --dangerously-skip-permissions (server.js:41) with a 120s hard timeout, no concurrency cap, no rate limit, no application-level log file (the user journal at journalctl --user -u sfl-hook captures stdout/stderr from the unit). The service is supervised — managed by user-systemd via ~/.config/systemd/user/sfl-hook.service with Restart=always and EnvironmentFile=%h/.config/sfl/hook.env, enabled and active since 2026-03-08T12:42Z. The supervision story is fine; the input-validation story is not.

Why. The webhook secret check at server.js:104-110 reads if (config.SFL_WEBHOOK_SECRET && secret !== config.SFL_WEBHOOK_SECRET) — it fails open when SFL_WEBHOOK_SECRET is empty. Today the secret is sourced from ~/.config/sfl/hook.env by the systemd unit, so the live process has it in env. But loadConfig itself reads only ~/.config/sfl/config.json + process.env; the JSON does not contain the secret (only SFL_API_URL + SFL_API_KEY). Any path that runs node server.js outside the systemd unit — or any future edit that removes the line from hook.env, or any startup that races the env-file load — drops the guard, and any unauthenticated POST to /sfl-hook then runs Claude with --dangerously-skip-permissions as petter (whose sudo is passwordless).

Blast radius. Arbitrary code execution on Sleeper as petter if the secret line is ever empty for any reason. Separately, the --dangerously-skip-permissions flag means Claude has no second line of defence against prompt injection in webhook bodies.

Smallest fix today (~10 min).

Biggest fix. Replace the claude CLI subprocess with a direct Anthropic SDK call (@anthropic-ai/sdk with prompt caching). This eliminates --dangerously-skip-permissions entirely (no tool use unless explicitly wired), gives structured errors, removes the spawn-per-request cost, and frees the unit to add a bounded queue with idempotency keyed on the SFL message id. Add a /sfl-hook/health endpoint for the nginx-logs monitor.

2. Two parallel SFL pollers, two state stores, no coordination (sleeper-tasks #23) — Resolved 2026-05-25

sleeper-articles now fires POST http://localhost:3013/sync/ingest (Bearer-authed via ARTICLES_API_KEY) at every status='ready' transition, and thoughts-sync listens on loopback 3013 with a GET /sync/health. The 5-minute cron stays as the safety net; per-event latency dropped from up to 5 minutes to seconds. The smallest-fix direction was taken — push pipeline first, with the cron preserved as the self-healing fallback.

What. Both sleeper-articles (articles/src/services/poller.ts:initPoller) and thoughts-sync (thoughts/sync/src/index.ts:main) run cron.schedule('*/5 * * * *', ...) with no phase offset, calling GET /api/ideas?type=page&limit=100 against sfl-prod on the same wall-clock five-minute boundary. Both full-scan: SFL's cursor is older-than pagination, not a since filter, so neither poller can move to incremental without a worker-side change. Articles tracks state in SQLite (articles.db.json_extract(doc,'$.sfl_id')); sync tracks state in ~/thoughts/sync/data/state.json. The sync_state.sfl_cursor row in articles is vestigial.

Why. thoughts-sync is structurally downstream of sleeper-articlessync/src/articles.ts:buildSflIdIndex calls articles HTTP GET /articles/?source=sfl&status=ready to obtain the extracted Markdown — but the polls are independent loops with no coordination. If articles is down or behind, sync silently skips ideas (missing++ in pm2 logs thoughts-sync) and the wiki stops getting sources.

Blast radius. Silent wiki staleness. Visible only by tailing pm2 logs thoughts-sync for missing-article= counts. No metric, no alarm, no dashboard.

Smallest fix today. Drop the SFL poll in thoughts-sync entirely. Sync already only consumes what articles has marked status='ready' — the SFL poll is dead weight. Replace the upstream call inside sync/src/index.ts:syncPages with GET /articles/?source=sfl&status=ready&limit=100 (which buildSflIdIndex already does — just promote it to the source of truth). Same loop, one upstream call instead of two, no rate-budget contention, no cron-phase puzzle.

Biggest fix. Push instead of pull. Have sleeper-articles emit a "ready" event (in-process pubsub or a tiny webhook) and let sync subscribe. One state store, no polling at all on the sync side, latency floor drops to seconds.

3. X-post extraction duplicated — different call AND different render (sleeper-tasks #24) — Open

Not addressed in the 2026-05-25 run. The push pipeline from fix #2 makes the deeper fix easier (sync now consumes whatever articles has marked ready, so dropping sync's X path is mechanical), but the actual deletion of ~/thoughts/sync/src/x.ts is still pending. Track sleeper-tasks #24.

What. ~/chat/articles/src/services/x-extractor.ts (173 lines) + x-client.ts (287 lines) on one side; ~/thoughts/sync/src/x.ts (102 lines) on the other. diff -u | wc -l = 268; zero shared symbol names. Articles hits https://api.x.com/2/tweets/<id> with axios, asks for note_tweet, lang, public_metrics, entities, attachments, media.fields, user.fields=verified, and renders byline → italic UTC timestamp → URL-unwrapped body → ![alt] photos / <video> for video → quoted/replied-to blockquote → metrics line. Sync hits https://api.twitter.com/2/tweets/<id> (different hostname) with fetch, asks for created_at, text, referenced_tweets only, and renders **Name** (@handle) → raw created_at → text → --- → quote → [View on X] footer. Same SFL bookmark, same X bearer (X_BEARER_TOKEN env, single token), two API calls per cycle, two distinct Markdown documents stored in two places.

Why. No commit references the other module; the two extractors share zero symbol names. They were written independently against the same API, at different times, and never reconciled.

Blast radius. Doubled rate-limit consumption (X v2 free tier is small). The iOS Read tab shows likes/RTs/replies/impressions and expanded URLs; the wiki version of the same post has none of that. Bug fixes in one extractor don't propagate to the other.

Smallest fix today. Drop sync's X path: detect x.com|twitter.com/*/status/<id> in sync/src/index.ts:syncPages and read the already-extracted Markdown from articles/<id> like every other URL. One bearer call per X bookmark, one render shape, immediately consistent.

Biggest fix. Same as fix #2's biggest fix — once sync subscribes to articles' "ready" event, the two extractors collapse naturally.

4. wiki-mcp is half-wired (sleeper-tasks #25) — Resolved 2026-05-25

The wiring contract is now explicit in ~/thoughts/sync/wiki-index.schema.md: local consumers (chat-backend retrieval + wikiVectors) read wiki-index.json off disk; remote consumers (iOS, Claude.ai) go through wiki-mcp at port 3007. Schema drift is caught by sync/check-schema.cjs, which runs after build-wiki-index.cjs in the autocommit chain and refuses the commit on any required-key absence or non-empty health.dead_links. The producer additionally emits health.field_coverage to surface structural drift in the index itself. The "biggest fix (a) branch" — disk reads with shared schema + CI check — was the path taken.

What. Two consumers of ~/thoughts/wiki/, two unrelated access paths. iOS app and Claude.ai use wiki-mcp (port 3007, MCP over StreamableHTTP at https://sleeper.phareim.no/wiki-mcp/mcp). Chat backend does not — a grep for wiki-index.json|wikiVectors|@modelcontextprotocol|3007|wiki-mcp against ~/chat/backend/src returns three hits, all in-process file reads (retrieval.ts:8, retrieval.ts:35, wikiVectors.ts:6). No MCP client, no localhost:3007. The backend reads /home/petter/thoughts/sync/wiki-index.json directly via retrieval.ts:loadWikiIndex (mtime-cached) and wikiVectors.ts:ensureFresh (which rebuilds data/wiki-vector-index.bin). On the schema side, the same on-disk JSON is parsed by three independent code paths — retrieval.ts:WikiArticle, wikiVectors.ts:WikiArticleRef, and mcp/server.js:loadIndex — each picking a different subset of fields. The producer (~/thoughts/sync/build-wiki-index.cjs) emits sources[], related[], line_count that none of the three consumers read.

Why. The chat backend is local to the same disk as the producer; an HTTP hop to MCP would be a regression. So the MCP exists for remote clients only, and the schema has two implicit owners (the JSON producer and the three private parsers).

Blast radius. Schema-drift bugs that bite only one consumer at a time — the hardest class to catch in dev. A field renamed in ~/thoughts/sync/build-wiki-index.cjs will pass type-check on the backend (because both backend interfaces are partial) but silently drop fields from one consumer and not the others. category exists in retrieval.ts:WikiArticle but is missing from wikiVectors.ts:WikiArticleRef.

Smallest fix today. Extract the WikiIndex shape into a tiny shared package (or a single types file re-exported by all four consumers — backend retrieval, backend wikiVectors, wiki-mcp server, sync producer). No runtime change; just one source of truth for the schema.

Biggest fix. Decide the contract explicitly: either (a) backend keeps disk reads with a shared schema and a CI check that the producer's JSON matches; or (b) backend moves to MCP-only with a thin localhost:3007 client and a disk fallback for restart-cold paths. Both are small projects; the larger cost is committing to one.

5. .processed-raw-files is append-only; "content-changed re-process" is unimplemented (sleeper-tasks #26) — Resolved 2026-05-25

.processed-raw-files.sha now carries one <sha256> <entry> line per entry, with # missing-source markers for entries whose source file has been deleted upstream. ~/thoughts/sync/needs-refold.cjs runs first in the autocommit chain and emits sync/needs-refold.json listing new / changed (with prev_sha/current_sha) / missing_source entries plus totals; build-wiki-index.cjs injects those totals as health.refold_queue so the fold backlog is visible at-a-glance through every consumer of wiki-index.json. Both "smallest" and "biggest" fixes from this section were taken.

What. ~/thoughts/CLAUDE.md:12 promises "Re-process a file only if its content has changed since it was last recorded." The implementation in ~/thoughts/.claude/skills/wiki-maintenance/SKILL.md does not implement this — its "Processing Loop" is "process any file not in that list" with no hash, mtime, sha, checksum, or reprocess logic. Meanwhile thoughts-sync will overwrite raw/<sfl_id>-<slug>.md if SFL idea or articles updated_at advances. The basename is already in the ledger, so wiki-maintenance never picks the file up again.

Why. Aspiration in docs, never wired in code.

Blast radius. Silent disagreement between ~/thoughts/raw/<id>.md (latest) and ~/thoughts/wiki/<slug>.md (snapshot from first fold). The wiki is authoritative for retrieval (via wiki-mcp and sync/wiki-index.json), so a corrected SFL idea or a re-extracted article never reaches readers.

Smallest fix today (~30 min). Add a sidecar .processed-raw-files.sha, one <sha256> <basename> per line. Step 2 of the loop becomes "re-list a file when recorded sha != current sha." One-time backfill: cd ~/thoughts && for f in raw/*.md; do echo "$(sha256sum "$f" | awk '{print $1}') $(basename "$f")"; done > .processed-raw-files.sha.

Biggest fix. Promote ledger management from "convention enforced by a skill" to a script in ~/thoughts/sync/ that runs at autocommit time and emits a list of "needs re-fold" files. Could even file a sleeper-tasks ticket per re-fold candidate for review.

There's a sixth honourable mention — the thoughts-autocommit push-retry gap, already filed as sleeper-tasks #20. auto-commit.sh:commit_and_push runs if git push origin main 2>&1; then echo "pushed"; else echo "push failed"; fi and only retries when the next inotify event fires. On a quiet day, that can be hours of GitHub being behind. Fix when you tackle this list anyway; the smallest patch is a 30s until git push origin main; do sleep 30; done retry loop with a max-attempts cap.

The components

SFL Worker (Cloudflare) — sfl-prod

The only writer of SFL ideas. Lives at sfl-api.aiwdm.workers.dev with sfl.hareim.no as its web client. D1 (ideas table, messages table) + R2 (uploads). Hono-based Worker. From Sleeper's POV this is the durable origin — every poller, every webhook starts here. Source at ~/github/sfl/api/. Best-effort webhook firing in api/src/routes/messages.js:fireWebhook POSTs to env.WEBHOOK_URL for chat messages; currently set to https://sleeper.phareim.no/sfl-hook.

~/github/sfl/sfl-repo

The source monorepo. package.json declares "name": "sfl" (private) with workspaces: ["api", "web", "chrome-extension", "cli"]. pnpm-workspace.yaml lists only api/web/chrome-extension (a workspace-config drift worth fixing on its own). Mostly inert from Sleeper's POV — Worker code, Svelte web client, Chrome extension, CLI package. Relevant when changing the API contract that ports 3003, sync, and sfl-hook all depend on. What works here: clean separation between Worker / web / clients; the CLI is published globally via ~/.npm-global/bin/sfl → cli/bin/sfl.js.

~/chat/articlessleeper-articles (PM2, port 3003)

Polls sfl-prod every 5 min for type=page ideas, classifies as article|video|post, extracts via Mozilla Readability + turndown (or oEmbed for video, X API v2 for posts), runs the Venice processor for summary/tags/key_points/entities, and fires the memory-bridge POST to sleeper-memory. Stores in SQLite + FTS5 at ~/chat/articles/data/articles.db. Serves the iOS Read tab via REST. What works here: the pipeline is clean — poller → classifier → extractor → processor → memory-bridge — and well-documented in ~/chat/CLAUDE.md.

~/thoughts/thoughts-sync + thoughts-autocommit + wiki-mcp (all PM2)

Three siblings sharing a git-tracked tree. thoughts-sync polls SFL on the same */5 * * * * boundary as articles and writes ~/thoughts/raw/<sfl_id>-<slug>.md files (depending on articles having extracted bodies first — see fix #2). thoughts-autocommit does inotify-debounced git add -A && commit && push with a 30s window, and rebuilds sync/wiki-index.json whenever wiki/ or INDEX.md changes. wiki-mcp (port 3007, MCP server, ~/thoughts/mcp/server.js) exposes three tools — list_articles, get_article, search_wiki (which shells rg --json over wiki/) — over StreamableHTTP at https://sleeper.phareim.no/wiki-mcp/mcp. Manual fold from raw/ to wiki/ is a Claude task (running the wiki-maintenance skill). What works here: thoughts-autocommit is a small sensible inotify+debounce script; wiki-mcp is small, well-scoped, and stateless.

~/sfl-hook/ — port 4000, supervised by user-systemd (not PM2)

A 153-line server.js plus a stale nginx-sleeper.conf template (the live config is /etc/nginx/sites-enabled/sleeper). On POST /sfl-hook, validates X-SFL-Secret (with the latent fail-open from fix #1), responds 200 OK immediately, then async-spawns claude -p <body> --dangerously-skip-permissions with a 120s hard timeout. Posts the reply back to sfl-prod/api/messages with sender: 'sleeper' (which suppresses the webhook-refire on the Worker side). Broadcasts a data: { ts, user, reply } event on GET /events SSE for live chat UIs. The only on-disk artifact is /tmp/sfl-events.jsonl, which is volatile. What works here: the happy path; the loop-prevention via sender: 'sleeper' is clean; the SFL Worker's best-effort fireWebhook correctly swallows errors so user-message persistence is never blocked.

Information-flow walkthroughs

Trace A — bookmark → wiki

# Hop Where What changes
1 iOS Share Extension → SFL sfl-prod (D1 + R2) New row in D1 ideas, type=page, url, title
2 sleeper-articles poll (cron */5 * * * *) articles/src/services/poller.ts:syncOncesfl.ts:fetchNewIdeas Full-scan /api/ideas?type=page&limit=100; dedup via getArticleBySflId + getArticleByUrl; insert articles row, status='pending', `kind='article
3 Extraction half extractor.tsprocessor.ts (Venice) → memory-bridge.ts Row flips to status='ready', doc.content_md, doc.summary, doc.key_points. Memory bridge POSTs to sleeper-memory:3004 /extract-document, fire-and-forget
4 thoughts-sync poll (cron */5 * * * *, same boundary, no offset) sync/src/index.ts:syncPagesarticles.ts:buildSflIdIndex Re-fetches the same SFL /api/ideas?type=page list, then articles HTTP GET /articles/?source=sfl&status=ready&limit=100 paginated; build Map<sfl_id, ArticleListItem>; lookup → getArticle(stub.id) for doc.content_md
5 Writer sync/src/writer.ts Writes ~/thoughts/raw/<sfl_id>-<slug>.md with frontmatter (sfl_id, url, tags, sfl_created_at, sfl_summary); records progress in ~/thoughts/sync/data/state.json keyed by sfl_id
6 Manual fold Claude/Petter running wiki-maintenance skill Read raw/<file>, fold into wiki/<slug>.md, append basename to .processed-raw-files, update INDEX.md. No daemon does this
7 thoughts-autocommit ~/thoughts/sync/auto-commit.sh (PM2, inotify, 30s debounce) git add -A && git commit && git push origin main (no retry — see honourable-mention fix); rebuilds sync/wiki-index.json if wiki/ or INDEX.md changed
8 wiki-mcp port 3007 (PM2) Per-process re-read of sync/wiki-index.json on next request; consumed by Claude.ai + iOS — not by ~/chat/backend
9 iOS Read tab frontend/SleeperChat/.../Read/ReaderView.swift Reads articles via REST, not the wiki — they're separate surfaces

Worst-case latency. Floor: 5min poll + 5min sync + (manual fold) + 30s debounce + push. Ceiling: unbounded — gated on the manual fold. If articles takes longer than the sync poller's 5-min start, sync misses the row and waits another 5 min, so effectively two intervals = 10 min before raw/ lands.

Trace B — same bookmark, X-post variant

When the URL is https://x.com/<user>/status/<id>, SFL stores it as type=page and both pollers fire on the same boundary. Both decide it's an X URL, both GET /2/tweets/<id> against the same X_BEARER_TOKEN — but to different hostnames (articles → api.x.com, sync → api.twitter.com), with different fields, different HTTP clients (axios vs fetch), different error models (typed XApiError with retryAfter vs raw throw new Error), and different render functions. iOS Read shows the rich render (likes/RTs/replies/impressions, <video>, alt text). The wiki ultimately reads the poor render (text + --- + quote + [View on X]). A claim-supporting metric visible on iOS is not present in the wiki source of the same post. See top-fix #3 for the cleanup.

Trace C — sfl-hook chat round-trip

[user types in SFL chat UI]
     │
     ▼
POST sfl-prod /api/messages   { body, sender: 'user' }
     │  (Hono Worker)
     │  inserts row, then if env.WEBHOOK_URL set:
     │  c.executionCtx.waitUntil(fireWebhook(...))
     ▼
POST https://sleeper.phareim.no/sfl-hook
     headers: X-SFL-Secret: <env.WEBHOOK_SECRET>
     │
     ▼
nginx → proxy_pass http://localhost:4000
     │
     ▼
node ~/sfl-hook/server.js  (PID 27318, supervised by user-systemd)
     │  validates X-SFL-Secret against config.SFL_WEBHOOK_SECRET
     │   (latent fail-open if config returns empty — see fix #1)
     │  responds 200 OK immediately (server.js:125)
     │  async:
     │    spawn('claude','-p',body,'--dangerously-skip-permissions')
     │    setTimeout(..., 120_000): proc.kill() on expiry
     ▼
POST sfl-prod /api/messages   { body: replyText, sender: 'sleeper' }
     │  (Worker shortcut suppresses re-fire)
     ▼
append /tmp/sfl-events.jsonl  +  broadcastEvent(...)
     ▼
GET https://sleeper.phareim.no/events  (SSE) → SFL chat UI

The webhook ACK is fast (immediate 200 before claude starts) so the SFL Worker is never blocked. End-to-end reply latency is bounded by the claude CLI invocation; on 120s timeout, proc.kill() fires, the promise rejects, and console.error writes to stdout/stderr captured by the user journal — but no surfacing back to the chat UI, no retry, no DLQ. The live nginx blocks (verbatim from sudo nginx -T):

location /sfl-hook {
    proxy_pass http://localhost:4000;
    proxy_set_header X-SFL-Secret $http_x_sfl_secret;
    proxy_set_header Host $host;
}
location /events {
    proxy_pass http://localhost:4000;
    proxy_buffering off;
    proxy_cache off;
    proxy_set_header Connection '';
    proxy_set_header Host $host;
    proxy_http_version 1.1;
    chunked_transfer_encoding on;
}

The repo template ~/sfl-hook/nginx-sleeper.conf (47 lines) and the live config (/etc/nginx/sites-enabled/sleeper, 121 lines) happen to agree on these two location blocks today, which is what masks the divergence. Anyone editing the repo template expecting changes to take effect will be surprised — the live config is the source of truth.

Trace D (footnote) — chat retrieval bypasses wiki-mcp

When a chat turn arrives, ~/chat/backend/src/services/context.ts runs the V4 router and calls retrieval.ts:retrieve with the corpora the router selected. retrieve fans out to retrieveWiki (and four siblings); retrieveWiki calls loadWikiIndexfs.readFileSync('/home/petter/thoughts/sync/wiki-index.json') — for BM25-style ranking, and searchWikiVec for vector recall, which independently re-reads the same JSON, embeds each article (title + hook + summary), and persists embeddings to backend/data/wiki-vector-index.bin. Two ranked lists merged via RRF, top hits cross-encoder reranked, slotted into the ## Relevant from your knowledge base block of the prompt. Throughout, wiki-mcp (port 3007) is never contacted: the chat backend reads the wiki via the filesystem the wiki-mcp server happens to share, not via wiki-mcp's tools. The MCP is reserved for clients that can't read the disk. See fix #4.

Inconsistencies and weaknesses (full ranked catalog)

The five fixes above each map to a top-ranked weakness. The full list, with the same blast-radius ordering:

# Inconsistency Where Blast radius
1 sfl-hook latent fail-open in secret check, behind claude --dangerously-skip-permissions; supervised by user-systemd, not PM2 ~/sfl-hook/server.js:41 (spawn), :104-110 (guard); ~/.config/systemd/user/sfl-hook.service (Restart=always) Arbitrary code execution as petter if the hook.env line is ever empty
2 Two parallel SFL pollers, two state stores, no coordination articles/poller.ts:syncOncesync/index.ts:syncPages; articles.db.json_extractstate.json Silent wiki staleness when articles down/behind; sync depends on articles
3 X-post extraction duplicated — different call AND different render articles/x-client.ts + x-extractor.tssync/x.ts (api.x.com vs api.twitter.com) 2× rate-limit; iOS and wiki diverge on the same post
4 wiki-mcp half-wired: chat backend reads disk, iOS+Claude.ai use MCP, three private parsers, no shared schema retrieval.ts, wikiVectors.ts, mcp/server.js, producer sync/build-wiki-index.cjs Schema-drift bugs that bite only one consumer at a time
5 .processed-raw-files append-only; content-changed re-process unimplemented despite the docs claiming otherwise ~/thoughts/.claude/skills/wiki-maintenance/SKILL.md step 2; ~/thoughts/CLAUDE.md:12 (broken promise) Silent disagreement between raw/ (latest) and wiki/ (snapshot)
6 thoughts-autocommit push has no retry auto-commit.sh:commit_and_push (echo "push failed") Local-only commits until next file event (sleeper-tasks #20)
7 Documentation fragmentation — no file describes the whole flow ~/CLAUDE.md omits sfl-hook; chat/CLAUDE.md never names thoughts-sync; thoughts/CLAUDE.md omits sfl-hook and wiki-mcp; the stale raw note still references /Users/petter/... mac paths Future-Claude rebuilds the wrong mental model

Naming hygiene

There are four distinct artifacts named "sfl" on Sleeper, each with a different job. Documentation should use these labels consistently:

What I'd build today (counterfactual)

If you rebuilt this from scratch today: one poller (kept where it lives now, in sleeper-articles); thoughts-sync reduced to a thin downstream that reads articles' "ready" stream instead of polling SFL directly; one shared extractor library used by both articles and sync (no duplicated X path, one hostname, one render); sfl-hook under PM2 with SFL_WEBHOOK_SECRET mandatory and nginx restricted to Cloudflare Worker IPs; the Anthropic SDK in place of claude --dangerously-skip-permissions; a single WikiIndex types file shared between ~/thoughts/sync/ (producer), wiki-mcp (server), and ~/chat/backend (the two private consumers). The shape is already 80% there — the missing 20% is consolidation, not invention.

— GRAPH
— 4 RELATED