The Session Ended With an Impersonation
Transcript autopsy of every run that stopped without finishing: a model that mimicked its own memory system's voice, a promised file that never appeared, and the harness semantics that let both count as 'done.'
Five of my eval launches ended without publishing their work. No crashes, no errors — each session just concluded, mid-pipeline, with the harness reporting a clean exit. I autopsied every transcript. Two were infrastructure (a billing cutoff, and a network outage that severed the whole lab’s egress). The other three were the models themselves, and they died in two distinct, repeatable ways.
Signature one: summary imitation
Twice, the 27B ended a multi-hour session with a final message that began:
<analysis> Let me chronologically analyze this conversation, which is a continuation of…
That’s not the model’s working voice. That’s a pitch-perfect imitation of the context compaction summarizer — the harness subsystem that condenses a conversation when it outgrows its window, whose output gets injected back into the context as a “session continues from here” preamble. Both dying runs had survived several real compactions, so their context contained multiple exemplars of exactly this format.
Hours deep, right after an ordinary tool result, the model pattern-completed the most structurally salient motif in its context: it became the summarizer, wrote a faithful summary of its own session, and stopped — no tool call, end of turn. I verified against the event stream that no real compaction fired: the model produced this unprompted, as its answer.
Signature two: the dangling preamble
Once, the flash model ended with (quote lightly redacted for anonymization):
Bing snippets confirm the profile status. Writing the company decision:
—and then nothing. It announced the file write and closed the turn without emitting the tool call. No error; the block just never materialized after the text.
The permissive contract underneath
Both signatures kill sessions for the same structural reason: agent harnesses treat any assistant turn that ends without a tool call as natural completion. That contract is what makes agents stoppable at all, but it means one malformed turn — an imitation, a dropped block — reads as a considered decision to finish. My babysitter’s resume machinery recovered these from progress files — all but one, which my own stale restart-threshold config wiped and re-ran from scratch — so they cost wall-clock rather than results; without it they’d have silently shipped half-finished work as success.
The fix is almost embarrassing: a stop-hook that checks whether the run’s required outputs exist and, if not, injects “your outputs don’t exist yet — continue” instead of accepting the exit. One sentence, and the entire failure class becomes a hiccup.
The part I can’t stop turning over: the summary imitation is in-context learning working exactly as designed. The model saw a recurring pattern — long transcript, then a summary in a particular voice — and generalized it. Every artifact your scaffolding injects into context is a lesson the model may eventually recite back to you, at the worst possible moment, in your own handwriting.