Raw-260k Interaction Capability Map

Status: static, read-only behavioral audit of the existing raw-260k manual, continuation, renderer, scheduler, and packet-interface artifacts. This is development evidence. It is not a public benchmark, a training result, an architecture localization, or evidence of broad or latent reasoning.

1. Bottom line

Raw 260k has one narrow, reproducible positive behavior: under the familiar Problem: ...\nWork: renderer, it often applies a supplied arithmetic operation to a visible integer state. A deterministic external scheduler, first-line integer parser, and state carrier compose that behavior into 10/20 correct development chains. The strongest family is sequential add/multiply/subtract: 15/15 gold-state atomic transitions and 5/5 model-carried chains.

The result does not reduce to one global failure label:

• Executor: strong for sequential add/multiply/subtract, mixed for general multiplication/subtraction, and weak for remainder. It is not a general arithmetic executor.
• Compiler/selector: ordinary task prompts often omit an operation, treat a non-decimal numeral as decimal-looking text, or enter an unrelated template. External schedule compilation materially changes the result.
• Updater: the external parser/carrier can recur over visible integers, but the model fails the explicit compact State/Plan packet update contract 0/2. These are different capabilities.
• Terminator/serializer: correct content often appears before unrelated continuation. All 110 Problem/Work atomic-plus-chained calls contain text after the parsed first line, and the packet halt response starts with the correct integer but does not stop.
• Parser: there are two separate parser facts. The model-facing integer parser succeeds on every scheduled development call. The original continuation evaluator had its own numbered-header bug; the hash-bound assessor corrects direct QA from 1/20 to 4/20 without regenerating a response. That discrepancy is an evaluator error, not an executor, compiler, or updater error.

The strongest justified description is therefore renderer-indexed, externally scheduled visible-state execution on a small development board. It is not autonomous model recurrence or broad reasoning.

2. Behavioral definitions

These labels describe transcript behavior. They do not assert separable neural modules.

Atomic gold-state calls use the correct input to isolate the executor. Chained local-operation correctness asks whether the model correctly applied an operation to the state it actually received, even if an earlier error made that state globally wrong. Full chain requires the final visible state to equal the gold answer.

3. Evidence custody

The local checkpoint and tokenizer were rehashed directly:

Every model-bearing artifact below records checkpoint step 260000 and the same checkpoint/tokenizer SHA-256 pair, except the older manual artifact, which records the checkpoint path and step but predates those hash fields.

The corrected assessor binds the unchanged transcript, case payload, and generation script (d4bdb45be00d161f1948c7f9669f7ced3201d33a199c8301c7b64634b34ac801). The 256-case file has 256 board rows and zero response fields. It is not a scheduled transcript and contributes no model score to this map.

4. Manual interaction map

The seven-case manual artifact scores 1/7 initial, 0/7 independent review, 1/7 supplied verified fact, and 0/7 state reuse. No compact response begins with the exact requested state= prefix. The sole initial answer-level success is logic; its text starts No but immediately gives the false rationale that a zol is not a mar, so even that success is not a stable faithful derivation.

This manual board alone would support a negative verdict. The controlled arithmetic artifacts explain why that verdict is too coarse for one narrow family.

5. Whole-task prompt form

The corrected 20-case confirmation uses five fixed-seed cases in each family. Intermediates means every required intermediate value is present in the assessed first answer segment. Worked completion includes two solved examples, so it is not a pure format-only intervention.

The byte-frozen v1 transcript embedded direct 1/20. Its parser failed to terminate on numbered headers such as Question 2: and included later text in the answer segment. The separate v2 assessor finds direct 4/20; no response was regenerated. The model still has one genuine strict termination miss: direct sequential case 3 contains the correct complete chain and final 55, then performs an unrequested division before the first new header.

Observed continuation beyond the corrected first answer segment is 18/20 direct, 3/20 bare, and 20/20 worked. Segment parsing prevents most of that run-on text from changing the score, but the behavior is still a model termination defect.

6. Scheduled renderer map

6.1 Failed Next state renderer

The first scheduler prompt used:

Current state: <integer>
Requested operation: <operation>
Apply exactly this one operation. Return only the next integer.
Next state:

The external integer parser succeeds 55/55, but the model emits input_state + 1 on 43/55 calls. Only 1/55 outputs is accidentally correct for the locally requested operation, every first transition is wrong (0/20), and every chain is wrong (0/20). All 55 responses contain extra text after the first line. For example, state 32 with multiply by 12 starts with 33.

This is a renderer/format failure, not a parser failure. Because the prompt strongly elicits lexical integer succession, it is also not clean evidence that the arithmetic executor is absent.

6.2 Three-renderer atomic and recurrence matrix

Each renderer makes 110 calls: 55 independent calls from gold states plus 55 calls in chains carrying the parser-extracted model state. There are no demonstrations, retries, repairs, search, or verifier feedback.

Problem/Work has four chained transitions that correctly apply the local operation to an already wrong carried state: local-operation correctness is 44/55 while gold-state correctness is 40/55. This is evidence that visible state carry is behaviorally active, but also shows how executor errors compound under recurrence.

The Problem/Work operation breakdown is:

The family-level behavior map is more diagnostic than the aggregate:

The parser is an essential external resource. It chooses the last integer on the first nonempty line and supplies that integer to the next prompt. The model does not autonomously select the next operation, retain the plan, or stop at the state boundary.

7. Does the model read the carried state?

The renderer-interchange artifact holds the operation fixed and crosses the visible state against the state implied by retained source text. In all six crossed add/multiply/subtract cells, the candidate computed from the displayed state has higher summed log probability than the source-implied candidate:

• displayed/local wins: 6/6
• source-implied wins: 0/6
• minimum absolute summed-log-probability margin: 0.793863810133189
• candidate-sequence evaluations: 18
• generated tokens: 0

This rejects pure source-answer replay for those six Problem/Work cells. It supports causal use of the displayed integer, not a latent program, a general parser, or model-internal recurrence.

8. Explicit packet compiler, updater, and halt

The five-call residual-packet probe asks for a compact visible object with a current State and remaining Plan.

This resolves an important ambiguity. Successful externally carried recurrence does not imply that raw Shohin can author or update the recurrent packet. The external controller currently owns parsing, state replacement, plan advancement, operation selection, and call scheduling.

A second twelve-call diagnostic decomposes that updater into independent subskills using three fresh examples in each cell:

All twelve calls reach max_new. The most informative near miss starts from observed state 37, correctly executes the two remaining operations to reach 92, and then applies the already-consumed subtraction. The missing behavior is therefore not simply arithmetic. It is a consume-and-transport queue-state transducer: bind the observation as state, delete one instruction, preserve the residual order, and stop at the packet boundary. Combined strict joint accuracy is 0/6.

9. Error attribution by capability

10. Claim boundary and next evidence

The Problem/Work renderer was selected after the 20-case development transcripts were observed. The format matrix and renderer interchange are post-hoc diagnostics on the same cases. They establish a useful hypothesis and a local mechanism boundary, not an independent confirmation.

The immutable 256-case fresh board is suitable for confirmation, but the local artifact is only a board manifest with no responses. Until a completed, hash-bound transcript and independent assessment exist, this map authorizes no fresh-board score, internalization result, broad reasoning claim, or R12 novelty claim.

The static replay command is:

python3 train/analyze_raw260k_interactions.py

It verifies all eleven file hashes, recomputes the statistics above, and makes no model calls or file writes.