The Loop

Issue 03 · next 1 Oct 2026

Issue 0310 September 2026Still open

Issue 03
Desk
2026.09

If you mock the shell, you aren’t testing the agent.

A test harness for an autonomous coding agent has to wait on real jobs, assert on tool trajectories, and refuse to stub the filesystem. The first version slept for thirty seconds and called that engineering. We still do not know how to cache the fixtures.

Read issue 03Follow

In this issue

  1. 03If you mock the shell, you aren’t testing the agent.10 September 2026 · 9 min · FIG. 3 · Still openA test harness for an autonomous coding agent has to wait on real jobs, assert on tool trajectories, and refuse to stub the filesystem. The first version slept for thirty seconds and called that engineering. We still do not know how to cache the fixtures.
  2. 02The UI is calm. The intelligence is under the hood.20 August 2026 · 8 min · FIG. 2 · Still openAn Android-first personal knowledge corpus. Capture is almost free; synthesis is expensive; the library on the phone and the graph on the server tell different stories. We nearly wiped a live Room database to make a demo green.
  3. 01The planner that thinks like a local.30 July 2026 · 8 min · FIG. 1 · GraduatedAn AI travel planner that has to look local while every upstream API is late, rate-limited, or politely wrong. We used to block the map on live data. Now the itinerary is a document with timestamps, and the live calls are a sprinkle. We still do not have a language for ‘this might be stale.’

The challenge spread

Teaching a model to wait

Tests want to be fast, deterministic, and isolated. An agent is none of those things. It reads a directory, notices a missing dependency, runs a real install, waits, edits a file, starts a server. The first Rooklight harness slept for thirty seconds and called that a strategy. Sometimes the install finished in five. Sometimes it finished after the test had already failed and polluted the next run.

We extended the sleep. Thirty became sixty, then two minutes. The suite took an hour. Developers stopped running it. We had built a harness nobody would touch — which is how you know the harness has failed, even if every assertion is green in CI.

The turn was to stop treating the agent as a request with a deadline. The harness polls the jobs queue. While a process is running, we wait. The moment the loop is idle, we assert — not on the wording of the reply, on the trajectory of the tools.

Plate 3Rooklight · the tool loopfold out Rooklight tool loop An agent talks to the harness, the harness acts on a real filesystem, and the harness records a trajectory of tool calls that the assertions check instead of the model’s prose.FIG. 3 · REAL I/O, POLLED WAITNOT TO SCALEAgent / LLMprose is not the testactionresultRooklight harnessintercept · log · pollread · editstdout · exitReal filesystemnot a mock of bashpollwhile jobs runningwait for the next tickno 30s sleeptrajectory logread docs/api.mdbash npm installassert actionsnot the wordingIf you mock the shell, you are testing your assumptions.

Plate 3 · folded out

Rooklight tool loop An agent talks to the harness, the harness acts on a real filesystem, and the harness records a trajectory of tool calls that the assertions check instead of the model’s prose.FIG. 3 · REAL I/O, POLLED WAITNOT TO SCALEAgent / LLMprose is not the testactionresultRooklight harnessintercept · log · pollread · editstdout · exitReal filesystemnot a mock of bashpollwhile jobs runningwait for the next tickno 30s sleeptrajectory logread docs/api.mdbash npm installassert actionsnot the wordingIf you mock the shell, you are testing your assumptions.
Detail: the wait, which is a poll rather than a sleep · shown at 2×
Subject
Rooklight — the harness we test the coding agent with, before it is a product.
What it is
A Pi coding-agent extension suite. Pre-release, not on npm
Deepfield
Local-project evidence layer; reads a span, mutates only if its hash still matches
Shell
Background jobs, recoverable logs, LeanCTX-compressed stdout
Wait
Polls the job queue and the agent loop; no fixed sleep anywhere
Assertions
On tool trajectories — which tool, which path, which flag
Still open
Fixture caching, and visual assertions that do not restore the timeouts

Continue reading issue 03

The cabinet

Plates

Architecture as we actually run it. Hover a corner to magnify; open the plate to fold it out.

Plate 1Travel Ruter · cache and stalenessfold out Travel Ruter cache and staleness A local itinerary snapshot is served in under fifty milliseconds; Mapbox, OpenRouteService and Amadeus refresh behind it; the interface marks which values are scheduled and which are live.FIG. 1 · CACHE BEFORE LIVENOT TO SCALEItinerary snapshotPostGIS · t₀ · locallocal readPlanner UIday plan · map · notesthen refreshMapboxtiles · geocodeOpenRouteroad ETAsAmadeushotels · often staleUI languageink = snapshotrust = live / lateage of the schedulet₀stale

Plate 1 · folded out

Travel Ruter cache and staleness A local itinerary snapshot is served in under fifty milliseconds; Mapbox, OpenRouteService and Amadeus refresh behind it; the interface marks which values are scheduled and which are live.FIG. 1 · CACHE BEFORE LIVENOT TO SCALEItinerary snapshotPostGIS · t₀ · locallocal readPlanner UIday plan · map · notesthen refreshMapboxtiles · geocodeOpenRouteroad ETAsAmadeushotels · often staleUI languageink = snapshotrust = live / lateage of the schedulet₀stale
Detail: the snapshot hand-off, where the plan is already on screen · shown at 2×
Subject
Travel Ruter — a planner that serves a day from a local snapshot and refreshes behind it.
Stack
FastAPI · async SQLAlchemy · PostGIS · React · Mapbox GL
Orchestrator
PydanticAI over a WebSocket, multi-model (Claude, GPT, Gemini)
Tool surface
MCP server: destinations, trips, routes, POIs, accommodations, notes, budget, scheduler
Refresh
A stored segment is re-fetched only when it is stale; the document holds the plan
Status
Graduated — the one project here with a production stack
Still open
A confidence interval for a walking time that is not a disclaimer
Plate 2Save4later · capture to graphfold out Save4later capture and sync path Capture from clipboard, share sheet and media session into a Room library, pushed by WorkManager to a FastAPI worker, enriched into SurrealDB, with a synthesis card returning to the phone.FIG. 2 · CAPTURE → ROOM → GRAPHNOT TO SCALEClipboardzero tapsShare / a11yURL bar · intentMediaSessionplaying nowRoom libraryRoom v4 → v15 · no wipeWorkManagerFastAPI workerenrich · embedsynthesiseSurrealDBgraph + vector“What for you” returns to the phoneFeed said sync enabled. Settings said failed. That line is the product.

Plate 2 · folded out

Save4later capture and sync path Capture from clipboard, share sheet and media session into a Room library, pushed by WorkManager to a FastAPI worker, enriched into SurrealDB, with a synthesis card returning to the phone.FIG. 2 · CAPTURE → ROOM → GRAPHNOT TO SCALEClipboardzero tapsShare / a11yURL bar · intentMediaSessionplaying nowRoom libraryRoom v4 → v15 · no wipeWorkManagerFastAPI workerenrich · embedsynthesiseSurrealDBgraph + vector“What for you” returns to the phoneFeed said sync enabled. Settings said failed. That line is the product.
Detail: the return path, the one that disagrees with the feed · shown at 2×
Subject
Save4later — an Android library where capture is free and synthesis is expensive.
Device
Kotlin · Views/XML · Room · WorkManager · Retrofit · Coil · DataStore
Server
FastAPI · SurrealDB 3 in Docker, with a worker for enrichment
Capture
Clipboard, share intent, URL-bar extraction, MediaSession
Schema
Room v15 · seven migrations in place, including a 4→14 jump. No destructive fallback anywhere
Status
Open — a mature prototype, not on the Play Store
Still open
One sync truth. The feed and Settings still tell different stories

Graduated

Started as a challenge. Now a tool.

Challenge 01Travel Ruter began as “make a planner that does not invent the city.” It now runs as a FastAPI + PostGIS + React stack we actually use: maps, routing, a streaming agent, a marketplace that sells digital itineraries — not bookings. The repo is still private. That is a fact, not a tease.

Save4later and Rooklight have not graduated. They are in this issue because they are still the work.

Still open

We don’t know these yet

  • Still openHow to communicate the confidence of a departure, a hotel rate, or a model’s opening hours without turning the planner into a disclaimer.

    Follow this · Issue 01

  • Still openSync that tells the same story on the feed and in Settings. The library is local; the graph is remote; they disagree more often than we like.

    Follow this · Issue 02

  • Still openA copy-on-write cache for agent fixtures, so a clean temp directory does not re-download three hundred megabytes. And visual assertions that do not bring the timeouts back.

    Follow this · Issue 03

The pass

Two people. The pass buys bench time, not a sales team.

Membership is designed. It is not open yet — we will not take money for a cadence we have not kept. Until then, the issues are free, and so is the RSS.

01 · Reader

The issues

Every public issue, the plates, the RSS. What it does not buy: build logs, the call, member pricing on tools that graduate.

02 · Pass

The bench

Monthly: every issue, every build log, graduated tools at the member price, the cohort call. This is the one we will open first.

03 · Patron

Named work

Pass, plus the right to sponsor a challenge by name. No logo wall. The colophon will say who paid for the weeks, and what we still got wrong.