# Jev Arena: methodology, experiments and current implementation

**Updated:** 21 September 2026, America/Mexico_City.  
**Implementation reviewed:** `3d392a0`, branch `codex/arena-simple-battle`.  
**Scope:** the decision laboratory, its curated experimental artifacts, and the private playable demo. This report consolidates existing evidence; preparing it did not run new model evaluations.

## 1. Current conclusion

The retained playable system combines **V4 retrieval, a brief of Jev's own team, factual battle memory, a Jev shortlist of up to four actions, simulations against explicitly hypothetical opponent sets, and a final Jev decision using a readable consequence brief**. Jev is the only model in this runtime. The experimental Luna summaries are preserved separately.

Three results answer different questions and must remain separate:

| Question | Relevant result | What it establishes |
| --- | --- | --- |
| Which context-only baseline did we retain? | V4 original: **305/376, 81.12%** | Preferred context-only reference on the known development corpus, without a second review |
| What was the strongest historical result with open set information? | Top-four simulation with brief V2: **371/376, 98.67%** | Agreement with the original accepted-action labels under that experiment's information and simulation conditions |
| Does inferred-set simulation help under the new hidden-information policy? | Paired closed-information control **251/376, 66.76%** → candidate **295/376, 78.46%** | Better agreement with the original reference after hiding opponent sheets; it does not revalidate those labels as optimal under uncertainty |

The latest version was retained for private play after reviewing the hidden-set benchmark. The user's expectation that it will transfer well to ordinary battles is a **product hypothesis**, not an observed win-rate estimate. The prototype has functional battle validation, but no controlled competitive win-rate study.

Sources: [V4 baseline](../experiments/baselines/README.md), [brief V2 experiment](../experiments/brief-v2-ab/README.md), [closed-information comparison](../experiments/hidden-sets-paired-01/README.md), [private demo](private-demo.md).

## 2. Research question and division of responsibilities

The original question was how to make a small decision model choose useful **joint actions in doubles Pokémon**, using a wiki and the current battle state. A joint action includes both active Pokémon's moves or switches, their targets, required passes, and an optional legal Mega Evolution. Choosing a move name alone is insufficient.

We progressively separated responsibilities:

| Component | Responsibility |
| --- | --- |
| Pinned Showdown engine and deterministic code | Legality, native menus and exact commands; rule execution, damage, turn order and hypothetical outcomes |
| Wiki | Versioned definitions, conditional mechanics, dependencies and sourced build candidates |
| Jev | Relevance judgments, bounded choices, own-action shortlisting and ranking plausible opponent builds |
| Code-based context assembly | Copying selected facts/rules, applying budgets, preserving uncertainty, and formatting evidence |
| Controller | Freshness checks, sending one valid command, handling rejection, waiting and completion |
| Evaluator | Reading reference answers **after** decisions are frozen; scoring and diagnosing failures |

Jev returns typed judgments rather than a generated explanation of its reasoning. We inspect what it received, selected and assigned probability to; statements about why it chose something are hypotheses unless isolated experimentally. The project does not hardcode “use Quick Attack in this benchmark case” or make an answer key available to the player.

The live TypeSafe documentation was reviewed from its [index](https://docs.typesafe.ai/llms.txt), especially [Choice](https://docs.typesafe.ai/primitives/choice) and [Confidence](https://docs.typesafe.ai/confidence). Choice supports up to 255 options. Its probability distribution and confidence describe a bounded judgment; neither is automatically a calibrated probability of winning a Pokémon battle.

## 3. The knowledge substrate

### 3.1 Frozen environment

The wiki and engine target:

- Format: `gen9championsvgc2026regmc`.
- Showdown revision: `2ddfa0476f8207e12e204b1c69f7c7683b17633c`.
- Wiki snapshot: 19 September 2026, version 0.7.
- Jev version in the reported experiments: `jev-1.13.0`.
- Installed TypeSafe JavaScript SDK: `@typesafe-ai/sdk` 0.6.0.

Pinning makes experiments interpretable; it does not establish exhaustive equivalence to every retail-game behavior. Champions training points must not be interpreted as conventional EVs, and holding a Mega Stone does not make the starting form already Mega.

### 3.2 Wiki construction and retrieval

The [wiki](../../docs/wiki/README.md) contains 1,364 pages: 382 Pokémon/form profiles, 511 move entries including Struggle, 216 abilities, 166 items, 18 types, 49 mechanics, 20 interactions and two project guides.

Human-readable articles, model-facing cards, a retrieval index, structured entities and a relationship graph are separate artifacts. Stable identifiers let code load selected material and its dependencies. A file path or URL alone does not provide its contents to Jev.

The methodology distinguishes:

- **Identity and mechanics:** types, move effects, priority, targets, abilities, item behavior and conditional exceptions.
- **Candidate builds and priors:** plausible sets derived from recorded sources, never automatically the opponent's actual set.
- **Provenance and maintenance:** source URLs, hashes, validation and review records retained outside the decision text.
- **Observed state:** current HP, PP, position, effects and reveals, supplied by the battle rather than static articles.

A `can_learn` relation does not mean a Pokémon knows or can currently use that move. A source frequency is not an opponent-action probability. Dependencies retain their conditions; including a rule does not assert that its condition is active.

Build coverage is narrower than article coverage: 184 profiles have complete build candidates and 198 retain unspecified builds/stat bounds. That distinction later became an important simulation bottleneck. See [competitive build records](../../docs/wiki/data/competitive-builds.json), [model context contract](../../docs/wiki/model-context.md) and [population/validation method](../../docs/wiki/population-and-validation.md).

## 4. Evolution of the experimental input

### 4.1 From a hand-written position to actual actions

The initial prototype used an explicit JSON position, including the Rillaboom/Incineroar versus Whimsicott/Salamence example. It established the pipeline:

**state → indispensable definitions → Jev retrieval → selected wiki content → Jev joint action**.

This was an inspectable context prototype, not yet evidence of competitive ability. Its supplied options and hypothetical sets were not equivalent to a fully observed native engine request. The [initial trial](../TRIAL-2026-09-19.md) and [original baseline](tested-baseline.md) retain that scope.

Early scenario catalogues used explanatory situations and conceptual multiple-choice answers. They were replaced as the main evaluation contract because recognizing a rule in prose is different from executing the correct Showdown action.

### 4.2 Log-only control and independent context treatment

The native benchmark uses:

```text
state:
  extraContext: null or the treatment under test
  showdownLog: the authorized player's protocol log

questions:
  fixed decision instructions
  original legal joint actions, with stable IDs and command mapping
```

C0 means **no added wiki/context**, not “no instructions or choices.” The authorized log includes the player's request and, in the historical corpus, open team sheets. Those sheet disclosures were part of that experiment's information policy.

For controlled context comparisons, the case log, instruction, menu, action descriptions, option order and model are fixed. Only `state.extraContext` changes. Some experiments also explicitly manipulate field placement, decision architecture, menu size or simulated evidence; they are identified as such rather than called pure context ablations.

The log remains the source record. A shorter current-state capsule supplements it. This preserves evidence that a projection might omit and allows later diagnosis of stale HP, lost status or missing history.

See [native benchmark usage](../benchmarks/micro-scenarios-v0.3/USAGE.md) and [mechanical validation](../benchmarks/micro-scenarios-v0.3/VALIDATION.md).

### 4.3 Corpus construction and scope

| Set | Size | Structure and role |
| --- | ---: | --- |
| Native micro-scenarios | 32 | Initial known development cases covering multiple mechanics |
| Mechanics bulk | 320 | 80 templates with four variants, across 16 families |
| Longitudinal mechanics | 24 | 12 pairs designed to distinguish histories |
| **Main comparison corpus** | **376** | Fixed denominator for V1–V4 and later full-corpus simulation comparisons |
| Separate planning exercises | 220 | Different input contract; explicitly excluded from the original V3 comparison |

The broader inventory totals 596 exercises, but that is **not** the denominator of the 376-case experiments. The original inputs are preserved in the [native corpus snapshot](../experiments/native-corpus-v3/source-snapshot.json.gz). The [bulk-generation handoff](handoffs/benchmark-bulk-generation.md) records the authoring and certification workflow.

Coverage goes beyond priority: speed, terrain, protection, immunities, redirection, spread attacks, weather, Mega, residual damage, status, damage mechanics, items, action restrictions, replacements and history. Native command validation and stored engine evidence support the reference answers under each scenario's declared assumptions. An accepted set can contain multiple commands, including different target commands with equivalent engine behavior.

The intended reference criterion is a justified action under the scenario's stated opponent responses, randomness and horizon. Engine acceptance proves a command is executable; it does not by itself prove strategic optimality. Audit claims must follow each dataset's actual certification scope.

The corpus has substantial limitations: 300/320 bulk cases contain Perish Song; always choosing the first option would score 239/320. Templates and pairs are correlated. Scenario IDs can describe an earlier version of a fixture, so diagnosis must read the current log rather than infer mechanics from the name.

## 5. Experimental controls and metrics

The recurring procedure was:

1. Specify the treatment and freeze code, wiki, inputs, menus, configuration and hashes.
2. Build contexts without supplying reference actions, case-family explanations, private engine state or future results.
3. Validate size, complete candidates, information boundaries and command mappings.
4. Run the explicitly enabled live calls; record the exact requests and raw responses.
5. Validate saved model artifacts, then open reference labels in the evaluator.
6. Report paired corrections and regressions, failures, coverage, probabilities, size, usage and timing.
7. Preserve unfavorable variants and previous versions rather than overwrite them.

Most later experiments use one response per distinct request. The first five-hint Quick Attack experiment used five repetitions per condition; subsequent work followed the user's preference to avoid repeated voting. This does not imply absolute determinism: the five-version study found differing historical distributions for an identical C0 request and retained the already agreed control, rather than selecting a favorable response.

Exact request/model matches may reuse saved responses. Reports distinguish **logical decisions**, **new API calls**, historical cache references and within-run reuse. A reused response is not a new observation and its original inference was not free. Offline/mock runs validate execution with accuracy left null; they are not measurements of Jev.

### Metric definitions

| Metric | Definition and limit |
| --- | --- |
| Accepted-action accuracy | Fraction of explicit provider choices included in the frozen accepted-command set |
| P(selected) | Provider probability of its explicitly selected action |
| Acceptable probability mass | Sum of provider probabilities over all accepted commands; requires evaluator labels |
| Best acceptable probability | Highest probability of an individual accepted command |
| Confidence | Provider distribution-concentration measure; not an independent correctness detector |
| Top-k coverage | Whether at least one accepted action survives shortlisting; an offline ceiling, not a live guarantee |
| Simulation coverage | Whether usable simulated evidence actually reaches the final model request |
| Corrections/regressions | Paired changes in accepted-action status for the same cases |
| Usage and latency | Provider tokens and explicitly scoped timing; separate retrieval, shortlist, simulation, formatting and final choice |

We retain the provider's explicit choice, even when rounded displayed probabilities suggest a different argmax. We do not renormalize accepted rounded distributions. Probabilities from different menus, tournament groups or Score rubrics are not interchangeable. A higher acceptable mass need not change the selected action.

Bytes are local size measurements; provider-reported tokens are the accounting measure. No single end-to-end cost-per-game or dollar total is established by adding these experimental tables: caches, reused stages, different game lengths and unreported Luna usage prevent that inference.

## 6. Context research: V1 through V7

### 6.1 The Quick Attack diagnostic

The example places Lucario at 1 exact HP against the last opposing Arcanine at a displayed 1% HP with increased Speed. The engine reference accepts priority attacks that secure the necessary KO; the visible percentage is not the opponent's exact HP.

The successive probes showed:

- **Five small hints:** 30 calls including repetitions. “quick attack has a priority +1” increased acceptable mass from 27.2% to 34.4%; none of the five treatments changed the selected Aura Sphere.
- **Twenty single-pass contexts:** one treatment, C19, combined current facts with explicit move priorities and order. Acceptable mass rose from 26% to 57% and Quick Attack was selected. Adding further objective/KO text in C20 reduced the mass to 45% and restored the wrong choice.
- **Actual V1/V2 retrieval:** both still chose Aura Sphere, despite containing relevant rules. Acceptable mass was 29% for that control, 34% for V1 and 32% for V2; the generated added contexts were 56,777 and 30,728 bytes.

Different C0 figures belong to different preserved runs. They must not be silently combined into one repeated-measures estimate. These probes established sensitivity to representation, not a universal optimal context length.

Evidence: [five hints](../experiments/quick-attack-hints-v1/RESULTS.md), [twenty contexts](../experiments/quick-attack-context-sweep-v2/RESULTS.md), [generated V1/V2 comparison](../experiments/quick-attack-generated-contexts-v1-v2/RESULTS.md).

### 6.2 Architecture progression

| Version | Construction | Main development finding |
| --- | --- | --- |
| V1 / legacy | Required entity/core cards plus Jev Noul retrieval of additional cards and dependencies | Broad coverage, large context |
| V2 / concise | Projected observation, deduplicated concise definitions and relevant knowledge | Smaller representation; not an automatic cure for tactical failures |
| V3 / selective | Jev selects short blocks using Noul, threshold 0.5, dependency closure and a 12,000-byte context budget | 20/32 versus C0 17/32; later 296/376 |
| V4 phase A | State plus mandatory intrinsic definitions; Noul-selected rules | Eight variants; best 18/32 |
| V4 phase B / V4B_FOCUS | Compact position capsule plus one thematic rule pack chosen by Jev Choice | 20/32 at about 2,524 added bytes; later retained as V4 original |
| V5 | Relational/action-linked facts, Noul probes/flags and pairwise preferences | Best FLAGS 21/32; small net gain, not a promoted live policy |
| V6 | Flat/compact views, pooled distributions, hierarchical choices and direct Scores | Best development score 23/32, with two tie-sensitive successes; no consistent transfer gain |
| V7 / Luna v1 | Jev V3 retrieval first, then GPT-5.6 Luna summary, then Jev decision | 18/32 versus unchanged V3 20/32 |
| Luna v2 | Same Jev-selected sources; copied state, summarized rules and a generated turn objective | 18/32 with or without objective; no net accuracy gain |

V4B_FOCUS chooses among order; protection/targets; damage/types/accuracy; status/residuals; field/weather/Mega; and switching/resources. This is **selection of information**, followed by a separate action choice. It is not an instruction to take a predetermined move.

The exploratory V4 phases included two responses rejected by probability-sum validation. Their original reports retain the errors and full denominators; the later 376-case five-version comparison completed without those response failures.

V5's pairwise preferences sometimes reinforced the same error. V6's synthetic transfer set also had a strong first-option baseline: 27/32. On the same 31 valid transfer cases, the best development Score composition achieved 23/31 versus V5's 28/31. V6 changed architecture and aggregation, so it was not a context-only comparison.

Luna remained an external layer after Jev retrieval. Its outputs were generated by `gpt-5.6-luna` through Codex sessions and then consumed as saved artifacts; a production Luna API path was not deployed. Fidelity audits found unsupported additions, omissions and state-description errors. Exact quotations matching source text did not guarantee that they supported the associated claim. Generated content was preserved for evaluation rather than semantically repaired after seeing Jev's results. Luna provider cost was unavailable, not zero.

Sources: [V3](selective-context.md), [V4](../experiments/focused-context-v4/README.md), [V5](../experiments/relational-context-v5/README.md), [V6](../experiments/architectures-v6/README.md), [Luna v1](../experiments/summary-context-v7/README.md), [Luna v2](../experiments/luna-v2/README.md).

### 6.3 Paired five-version comparison on 376 cases

| Context | Correct | Accuracy | Mean acceptable mass | Mean added context, UTF-8 bytes | Mean final-decision input tokens |
| --- | ---: | ---: | ---: | ---: | ---: |
| C0 | 258 | 68.62% | 56.56% | 0 | 3,536 |
| V1 | 285 | 75.80% | 63.35% | 65,893 | 20,405 |
| V2 | 294 | 78.19% | 65.88% | 32,689 | 14,675 |
| V3 | 296 | 78.72% | 67.91% | 11,559 | 8,025 |
| **V4 original** | **305** | **81.12%** | **67.92%** | **2,870** | **4,225** |

All 1,880 logical final decisions completed. There were 2,168 new API calls across retrieval and decision stages, with historical/exact reuse accounted for separately.

V4 versus V3 corrected 24 cases and regressed 15. Its mean acceptable mass was almost unchanged despite nine more accepted selections. V4's mean full-process input usage, including retrieval, was about 7,560 tokens per case, rather than the 4,225 final-choice-only figure.

The longitudinal subset remained weak: C0 17/24, V4 13/24. V4 chose the same Night Shade attack in every longitudinal case. This behavior is consistent with multiple hypotheses, including attack preference or menu-position sensitivity; no option-permutation experiment established the cause.

See [full comparison](../experiments/context-versions-corpus/README.md) and [structured summary](../experiments/context-versions-corpus/summary.json).

## 7. Error analysis and the review detour

The 71 V4 errors split into 48 bulk, 12 micro-scenario and 11 longitudinal failures. Important patterns were:

- Eight cases had an accepted action differing only by the Mega flag.
- Eight type-immunity failures involved attacks such as Seismic Toss into Gengar or Hex into a Normal-type opponent.
- Some selected packages omitted essential relationships, such as Quick Attack's priority.
- Other failures occurred despite the relevant rule being present, including sleep restrictions.
- Some state projections lost protection effects, history distinctions or toxic status; the original log retained information the capsule omitted.
- Forty-five cases failed in all C0–V4 conditions; ten were correct in C0 but incorrect in V4.

Errors were not simply low-confidence cases. Mean V4 confidence was 0.443 in errors versus 0.431 in successes; mean P(selected) was 54.3% versus 52.7%. Wrong choices could receive 80–81%.

We tested a one-review policy triggered by **P(selected) ≤75%**, retaining up to four actions. It reviewed 340/376 decisions and caught 64 of 71 errors, but also reviewed 276 originally correct decisions. The error rate was not higher in the reviewed group. Of the 64 reviewed errors, 61 retained an accepted action in the top four.

| Policy on the same 376 cases | Correct | Paired interpretation |
| --- | ---: | --- |
| V4 original | 305 | No review |
| KEEP: same context, reduced menu | 307 | 10 corrections, 8 regressions |
| REFOCUS: retrieve again for reduced menu | 308 | 11 corrections, 8 regressions |
| Factual review base | 303 | New observed/action facts did not beat KEEP |
| Factual base + Jev-selected rules | 307 | Tied KEEP; 24 corrections and 24 regressions against it |

REFOCUS reproduced exactly the KEEP context in 283/340 reviews. Restricting a menu can inflate option probabilities without adding understanding. The proposal to iterate until a confidence/probability threshold or five rounds was **not established as a successful policy**; the reported experiments used one review per condition. The user retained V4 original as the preferred context-only baseline.

Evidence: [probability diagnostics](../experiments/context-errors-corpus/README.md), [top-four review](../experiments/top4-review-v4/README.md), [factual review](../experiments/factual-review-v1/README.md), [71-error audit](../experiments/v4-error-patterns/README.md). Baseline tag: `baseline/v4-context-best-2026-09-20`.

## 8. Simulation as a separate architecture

### 8.1 What simulation means

The system reconstructs **synthetic possible states** from authorized observations. It executes candidate actions against possible opponent responses using the pinned engine, with independent analysis randomness. Jev then receives conditional consequences.

It does not inspect the live opponent command, real battle seed, hidden benchmark snapshot or future result. The engine is exact for the constructed state; the constructed state may still be an incomplete hypothesis about the real battle. “Wins every sampled branch” is not “guaranteed to win.”

### 8.2 SIM_V1 and SIM_V2

SIM_V1 was intentionally isolated from V4. Only 141/376 inputs prepared successfully; 234 were unsupported and one exceeded the context limit. A preselected 24-case pilot produced 17 decisions: 16 correct versus V4's 13 on those same 17. Seven unsupported cases were retained in reporting; 16/24 is the planned-set success fraction.

A Heatproof example showed the cost of a narrow hidden-stat assumption: legal HP spreads compatible with the same public percentage produced different survival outcomes. That finding motivated explicit alternative hypotheses.

SIM_V2 expanded effects, state reconstruction, Mega, counters and replacement support to all 376 original inputs without changing their menus or labels. Eight hypotheses crossed four declared opponent training profiles with two compatible HP/timer endpoints. All original actions participated. It resolved 83,192 branches; 9,880 additional branches were explicitly omitted by budget. Full case coverage was not exhaustive opponent or randomness coverage.

| Historical open-information condition | Correct / 376 | Change against the relevant comparator |
| --- | ---: | --- |
| V4 original | 305 | Context-only baseline |
| SIM_V2 | 350 | Versus V4: 58 corrections, 13 regressions |
| Frozen V4 added to SIM_V2 | 344 | Versus SIM_V2: 5 corrections, 11 regressions |

The V4 + SIM experiment changed only `state.v4Context` in frozen SIM_V2 requests. It did **not** use V4 to generate better simulations. Despite the lower benchmark result, the user chose that combination as a live development basis to explore team resources and evolving battle information.

Replacement horizons matter: SIM_V2 scored 338/355 ordinary decisions but 12/21 entry-only replacements. It simulates an incoming Pokémon and entry effects without reading or continuing a private pending-action queue. Nine of its thirteen regressions against V4 were replacements.

Sources: [SIM_V1](../experiments/sim-v1-pilot/README.md), [SIM_V2](../experiments/sim-v2-full-corpus/README.md), [V4 + SIM](../experiments/v4-plus-sim-v2/README.md).

### 8.3 V4 shortlist, shared opponent responses, final Jev choice

Live inspection exposed another issue: simulations can spend budget on implausible opponent behavior, including unhelpful direct ally attacks. The next architecture concentrated simulation around Jev's own candidate actions:

1. Build V4 and let Jev evaluate the legal own-action menu.
2. Retain up to four actions.
3. Construct a deterministic, diverse portfolio of up to four hypothetical opponent responses.
4. Test the **same response portfolio and paired analysis randomness** against each own action.
5. Format consequences; let Jev make a final choice among the retained actions.

Opponent portfolios are coverage heuristics, not calibrated predictions. Recognized beneficial ally interactions can remain available; ally targeting is not globally banned. The final request restores native menu order and does not receive the initial preference probabilities. Shortlisting can still remove the correct action.

The full-corpus experiment used frozen V4, without the demo's team-brief/memory calls. It achieved **358/376**, compared with 302 accepted first-stage selections. The final stage corrected 66 and regressed 10. The shortlist contained an accepted action in 375/376 cases. Seventeen of the eighteen final errors therefore occurred after an accepted action had survived.

In Quick Attack, the simulator showed a terminal win in 32/32 tested branches for Quick Attack and a loss in 32/32 for Aura Sphere; Jev nevertheless chose Aura Sphere. Available evidence and correct mechanical execution were insufficient when their presentation was hard to interpret.

Sources: [live top-four design](live-top4.md), [376-case experiment](../experiments/top4-full-corpus/README.md).

### 8.4 Bullet Punch probe and consequence brief V2

Replacing Quick Attack with Bullet Punch changed the final selection to the priority attack. This was a sensitivity test, **not proof of type bias**: it also changed the move name, STAB/resistance interactions, a logged event and one automatically retained finalist. Both priority attacks won their tested branches. See [controlled substitution and limits](../experiments/priority-bullet-punch/README.md).

Brief V2 addressed evidence presentation directly. Code places each action's description and command next to its outcomes, names participants, summarizes terminal results, HP, KO and effects, and preserves differences between hypotheses/responses. Joint KO patterns remain distinct from marginal HP ranges. It neither recommends an action nor sorts actions by quality.

The A/B test replaced **only `state.consequenceBrief`** in all 376 frozen final requests. Model, instructions, candidates/order, V4, log, hypotheses and 33,536 simulated branches stayed fixed.

| Measure | Brief V1 | Brief V2 |
| --- | ---: | ---: |
| Accepted selections | 358/376 | **371/376** |
| Ordinary decision horizon | 343/355 | 352/355 |
| Entry-only replacements | 15/21 | 19/21 |
| P(Quick Attack), identical final menu | 45% | 99% |
| Mean brief size | 8,840 bytes | 10,721 bytes |

V2 corrected 14 cases and regressed one while becoming about 21% longer. This is evidence for the tested **representation change**, not a universal rule that smaller prompts are better or that one sentence caused the gain.

Five errors remained, including one preselection failure that no final brief could repair. Approximately 92% of the corpus's simulated branches ended the hypothetical battle; clearer terminal summaries may be unusually valuable in this development distribution.

Source: [brief V2 experiment and exact artifacts](../experiments/brief-v2-ab/README.md).

## 9. Live team context, memory and information policy

### 9.1 Own-team brief and battle memory

The [implemented team-context layer](live-v4-sim.md) builds candidate own-team resources from known moves, items, abilities and wiki definitions. Jev selects relevance and conditional cooperation/interference links; code assembles the brief. The initial policy retains up to six resources and two links within a 2,048-byte budget, with whole-unit omissions recorded.

The brief is reused only when its team, model, policy and wiki signature remain valid. After selection, only the four participants support active plans. Consumed items, fainted Pokémon and obsolete forms cannot silently remain available resources.

Memory reconstructs factual current state from the authorized event history and latest own request. Previous memory is used to identify changes, not to carry forward simulated outcomes as facts. The model sees the state plus up to eight recent changes, while the audit retains the full change list; the complete-state size guard is explicit.

These features were functionally integrated, but their separate contribution has not been established through a controlled full-battle ablation. The major 376-case top-four and hidden-set comparisons omitted the extra demo team-brief/memory calls.

### 9.2 Historical disclosure versus current disclosure

The earliest local design disclosed complete sets and the identities of the human's selected four, but hid their order/leads and pending commands. Later playable variants used public open sheets with more restricted private-stat access. These historical policies belong to their own saved experiments.

**The current policy is revealed-only:**

| Information | Available to Jev now? |
| --- | --- |
| Its own exact team and current private request | Yes |
| Opponent's public six-species preview | Yes |
| Human's privately selected four or lead order | No |
| Opponent moves, items and abilities before revelation | No |
| Publicly announced moves, items, abilities, consumption and forms | Yes, updated from the authorized log |
| Opponent exact trained stats, nature and training points | No; simulations may posit explicit alternatives |
| Public opponent HP display | Yes, with its rounding uncertainty |
| Pending opponent command, live seed, future outcomes | No |

Human confirmation sends a readiness signal, `|jevconfirmed|`, rather than the selected identities. Jev still waits for confirmation but cannot use that private selection. Seen participants become known through public entries.

Initially, removing open sheets disabled simulation because the old reconstruction needed complete sets. That behavior is preserved at `codex/demo-v1-closed-baseline`, commit `4e6a7e0`. It is a regression reference, not the active limitation of the latest version. See [disclosure change](closed-opponent-information.md).

## 10. Current hidden-set simulation

### 10.1 Construction and limits

The current architecture identifier is `V4_TOP4_WIKI_HYPOTHESES_V1`, using consequence brief V2.

```mermaid
flowchart TD
  A[Authorized player log and latest own request] --> B[Observed state and factual battle memory]
  W[Pinned wiki] --> C[V4 rules selected by Jev]
  B --> C
  B --> T[Own-team resource brief]
  W --> T
  C --> D[Jev shortlists up to four legal joint actions]
  T --> D
  B --> H[Compatible opponent build candidates]
  W --> H
  H --> I[Jev ranks plausible builds]
  I --> J[Code constructs legal hypothetical worlds]
  D --> K[Shared opponent-response portfolios and Showdown simulations]
  J --> K
  K --> L[Code formats conditional consequence brief V2]
  L --> M[Jev chooses among finalists]
  D --> M
  M --> N[Check battle, side, rqid and observation revision]
  N --> O[Send exact native command once]
  K -->|Unsupported or complete evidence too large| F[Explicit no-simulation fallback]
  F --> M
```

Inputs to opponent inference are the authorized log, own request, public preview and pinned wiki. It cannot accept the private opponent catalogue assignment, selected four, live Battle object, pending command or seed.

Jev ranks compatible wiki builds in up to six Choice questions. Public native-move, item and ability revelations constrain candidates; a called move does not prove a native moveslot. Consumed items remain unavailable. Unknown facts remain unknown in memory even when simulated worlds assign values.

Code constructs at most six coherent worlds, includes all seen participants, and considers unseen participants from the remaining preview. It attempts roster coverage before further set variation. Six-member hypothetical teams must satisfy engine validation and clauses; exactly four participate in each simulated battle.

A fully populated move comparison has at most **4 own actions × 4 opponent responses × 6 worlds × 2 samples = 192 branches**. HP/timer endpoints and build variants are bounded hypotheses, not exhaustive integration over all legal states. Ranking probabilities are not calibrated joint-world frequencies.

The final brief explicitly labels assumed sets and hypothetical rosters. It can deduplicate repeated detailed evidence without deleting it. If a complete brief does not fit, the system declares simulation unavailable; it does not present a truncated result as complete.

See [implementation and privacy boundary](hidden-set-simulation.md).

### 10.2 Paired evaluation under hidden information

Both arms used the same projected closed-information log, newly retrieved V4 and identical first-stage shortlist. The candidate added inferred sets and simulations. Opponent sheet lines were removed, while public revelations and native own actions remained. Both arms used the same small observer correction preserving explicitly public bench faints.

| Population | No hidden-set simulation | Inferred-set simulation | Corrections / regressions |
| --- | ---: | ---: | ---: |
| All 376 cases | 251/376 | **295/376** | 55 / 11 |
| 133 cases receiving simulation evidence | 79/133 | **117/133** | 42 / 4 |
| 243 cases without delivered simulation evidence | 172/243 | 178/243 | 13 / 7 |

The overall gain is 44 accepted selections, **+11.70 percentage points**. Among cases that actually received a brief, the gain is 38. The remaining net gain of six is associated with changed fallback explanations and must not be credited to simulation.

Useful coverage was **133/376, 35.37%**, not every case where the engine ran:

- **220** lacked legal compatible wiki completions. An article is not necessarily a structured build template.
- **23** ran simulations but could not deliver the complete brief within its size budget.
- Delivered briefs contained **22,440 resolved branches**, with no failures or omissions within their declared budgets.

All four regressions with delivered simulation evidence were mid-turn Baton Pass replacements involving weather or terrain. They deserve horizon/hypothesis investigation without granting access to the private pending queue.

There were 1,654 unique API calls, 7,263,620 input tokens and 160,944 output tokens, with no failed calls. Median summed API time per case was 0.629 seconds for the control and 0.749 seconds for the candidate; these measurements excluded engine work, used three concurrent workers and included many no-simulation fallbacks. They are **not full live-turn latency**.

Most importantly, labels were inherited from the **open-information** corpus. Some accepted actions may depend on information now hidden. Thus 78.46% measures agreement with that reference, not certified optimality under partial observation, and cannot be directly compared as a performance decline from the historical 98.67%.

Sources: [results](../experiments/hidden-sets-paired-01/results.json), [analysis and diagnostics](../experiments/hidden-sets-paired-01/ANALYSIS.md).

## 11. Battle execution and user-facing validation

The controller follows **observe → prepare → consult → check freshness → send → observe updates**. Battle identity, side, `rqid` and observation revision bind each decision to its request. Reset invalidates pending work; stale or duplicate decisions are not sent. A rejection with new information requires rebuilding legal choices.

Six-to-four Team Preview offers all **360 ordered selections**. To respect Choice limits, the established preview tournament uses deterministic groups of at most 120, retaining three finalists per group before another comparison. Groups shrink when size limits require it. Every original option participates, but a tournament is an approximation to a single global ranking; cross-group probabilities are not compared directly. The action-shortlist path retains up to four and records its own grouping policy.

The private demo now provides 24 engine-validated tournament teams: 13 Worlds Masters top-cut teams, the Seniors champion and ten Baltimore teams, with their provenance preserved. Random team selection samples enabled entries; it does not randomize Jev's initial selection or turn decisions. See [team sources](../teams/worlds-2026-masters/README.md) and [demo instructions](private-demo.md).

The latest interface presents a minimal English lobby and guided battle decisions, with Help and collapsible history. It forwards actions to the pinned official client's current native controls rather than implementing a second move-legality engine.

At implementation checkpoint `3d392a0`:

- Type checking passed; 694 tests passed, none failed, one optional server test was skipped in the general suite.
- Earlier explicit server integrations tested confirmation, reset, KO, replacements and complete games; mock opponent decisions are labeled as such.
- The latest browser QA completed a real Jev game and exercised preview, targets, locked moves, switching, Mega, replacements, a sole remaining attacker, terminal result and rematch.
- Desktop and mobile layouts were inspected. Functional QA is not a strategic comparison or a user study.
- A pre-existing browser MutationObserver console notice remained; it did not block the exercised flow.

The demo listens locally on client port 8140 and server port 8060 by default. It remains a single-active-battle private preview, not a public multi-user service. Credentials stay in the adapter; private runtime traces and environment files are excluded from Git.

## 12. What the research supports

**Supported observations**

1. Retrieved context improved aggregate accepted-action accuracy over C0 in the fixed 376-case comparison.
2. V4 provided a substantially smaller context than V1–V3 while retaining the preferred context-only result.
3. More text, more calls, pairwise voting and a generative summary did not reliably improve decisions.
4. Missing context, lost state, incomplete hypotheses and failure to use present evidence are distinct failure modes.
5. A controlled reformatting of identical simulation evidence produced the clearest isolated improvement: 358 → 371 accepted selections.
6. Inferred-set simulation improved the paired closed-information result where sufficient compatible evidence could be delivered.

**Not established**

- A universal optimal context length, confidence threshold or number of candidates.
- An intrinsic reasoning improvement in Jev rather than a better composed system.
- Calibrated probabilities of battle victory or of an opponent set.
- Competitive win rate, generalization to unseen teams, or an independently measured benefit from team brief/memory.
- Exhaustive future search or certainty about private opponent information.
- Broad usefulness of every synthetic template or readiness for public deployment.

**Next methodological priorities, proposed rather than executed here**

Build a separate closed-information evaluation whose accepted actions are justified from observable evidence; split by template rather than near-duplicate row; balance action order; add nonterminal and multi-turn objectives; report performance by simulation availability and horizon; and measure complete battles against fixed opponents with matched teams/seeds. Test team brief, memory, shortlisting and inferred-set coverage as separate ablations. Preserve V4 and the previous closed baseline throughout.

Expanding rare synthetic wiki builds is a product-priority choice, not a prerequisite for retaining the present private version. However, accepting that coverage limit does not establish that ordinary battles will avoid it.

## 13. Reproduction, artifacts and preservation

Run commands from `decision-lab`. A fresh output directory is required for experiments. Do not overwrite historical traces or regenerate fixtures while reproducing frozen results.

```powershell
# Local code checks; no Jev evaluation
npm run typecheck
npm test

# Private demo, latest policy
npm run demo

# Scripted integration/UI mode; does not measure Jev
npm run demo:offline

# Hidden-set preparation or dry-run, no model-performance claim
node --import tsx scripts/benchmark-hidden-sets.ts --mode prepare --out runs/NEW_PREPARATION
node --import tsx scripts/benchmark-hidden-sets.ts --mode dry-run --out runs/NEW_DRY_RUN

# New paid evaluation: load local credentials and enable it explicitly
node --env-file-if-exists=.env --import tsx scripts/benchmark-hidden-sets.ts --mode run --out runs/NEW_LIVE_RUN --live
node --import tsx scripts/evaluate-hidden-sets.ts --run runs/NEW_LIVE_RUN --out experiments/NEW_REPORT
```

These commands document the current hidden-set workflow, not a universal replay command for every older architecture. Historical reproduction must use its recorded source snapshot, protocol, wiki, configuration and model; current integration sources can legitimately differ from the frozen experimental sources.

Curated experiment directories preserve readable reports, structured results, protocols, manifests and, for many later studies, compressed exact requests/responses and source snapshots. Compression formats differ: use each archive's README/manifest rather than assuming one schema. Local `runs/` folders contain additional private traces and are not guaranteed to exist on another clone.

| Implementation area | Entry point |
| --- | --- |
| Original schemas, retrieval and Jev adapter | [schema](../src/schema.ts), [wiki](../src/wiki.ts), [pipeline](../src/pipeline.ts), [Jev](../src/jev.ts) |
| Showdown observation, selection and execution | [observer](../src/showdown/observer.ts), [decision](../src/showdown/decision.ts), [controller](../src/showdown/controller.ts) |
| Team resources and memory | [brief](../src/team-context/brief.ts), [memory](../src/team-context/memory.ts) |
| Synthetic simulation reconstruction | [prepare](../src/sim-v1/prepare.ts), [hypotheses](../src/sim-v1/hypotheses.ts), [simulate](../src/sim-v1/simulate.ts) |
| Inferred opponent worlds | [hidden-set hypotheses](../src/hidden-sets/hypotheses.ts) |
| Shortlist, opponent portfolio and final brief | [shortlist](../src/live-top4/shortlist.ts), [opponents](../src/live-top4/opponents.ts), [brief V2](../src/live-top4/brief-v2.ts), [decision V2](../src/live-top4/decision-v2.ts) |
| Current playable entry point and controls | [local server](../src/showdown/local.ts), [guided arena](../src/showdown/arena.js) |

Version names are separate axes: context V4, SIM_V2, Luna v2 and consequence brief V2 are not successive releases of one component. The directory name `src/sim-v1/` also persists in later simulation implementations; use version identifiers and frozen source manifests to identify an experiment.

Preserved checkpoints include `tested-context-v1`, `baseline/v4-context-best-2026-09-20`, SIM_V1 on `codex/jev-sim-v1`, the historical `codex/jev-sim-coverage` and `codex/v4-sim-context` branches, and `codex/demo-v1-closed-baseline`. The current guided interface is a separate commit on top of hidden-set inference. Restoring one should use an isolated branch/worktree rather than reset or overwrite the active implementation.

This report supersedes overview statements that called the 12-team V4 + SIM build current or said hidden-set simulation was still disabled. Individual experimental reports remain historical records of what was actually run.