Pikachu has one health point left. The opposing Mewtwo is faster and almost knocked out. Pikachu can use Quick Attack, which acts before ordinary attacks, or Thunderbolt, which does not.

Any experienced Pokémon player would recognize the answer in this illustrative position: Quick Attack wins. Thunderbolt lets Mewtwo attack first and knock Pikachu out.

Yet our Jev-based Pokémon trainer kept making the wrong choice: the attack that would never get a chance to land.

That small mistake captured the challenge behind our project: could we build an agent that plays competitive Pokémon by consulting the right knowledge, looking ahead, and choosing before the clock runs out?

The game behind the experiment

In Pokémon VGC, the official competitive doubles format, each player typically brings a team of six Pokémon, selects four for a game, and has two active at a time. Both players choose their actions without seeing the other's choices first. Championship Series games allow 45 seconds to select moves, subject to a separate seven-minute player clock. Play! Pokémon tournament handbook.

A move can look good until an ability changes its target, the opponent protects, a partner takes damage from it, or a faster Pokémon acts first.

The combinatorial scale of VGC can dwarf even chess and Go. Familiar game-tree estimates put chess at roughly 10120 possible move sequences and Go at 10360. Our earlier rough estimate for VGC went beyond 10500, combining the enormous variety of teams with branching battle outcomes. This is an illustrative extrapolation of the search space, not a verified count of legal complete games. Chess estimate · Go estimate.

To make the scale concrete: using VGC-Bench’s approximate 10139 configurations per team gives about 10278 team pairings. Extending its order-of-magnitude worst-case branching estimate of 1012 outcomes per turn across an illustrative 20 turns gives 10278 × (1012)20 = 10518. That calculation assumes the same huge branching factor throughout; real battles do not maintain it. It shows why exhaustive search becomes impractical so quickly. A player must recognize which tiny fraction of the possibilities matters now. VGC-Bench, Appendix A.

A strong player needs to know the mechanics, recognize which ones matter now, and anticipate a few plausible replies. There is no time to examine every legal continuation.

An experienced player carries a reference library in their head, but does not mentally read the whole library every turn. A faster opponent brings priority and speed control to mind; an opponent that redirects attacks makes targeting more important. The position determines which knowledge matters.

We tried to build around that sequence: recognize the relevant facts → shortlist the plays → examine possible replies → choose. We would build a reference library of Pokémon knowledge, use Jev to help select relevant facts and promising plays, and use a simulator to explore their consequences.

Other agents have explored different parts of this problem. Foul Play uses a search engine to look ahead; Metamon learns from human games and agent play; and PokéChamp uses a language model within a minimax search. Those projects focus on singles, where each side has one active Pokémon. VGC-Bench studies learning approaches directly in doubles. Results in one format cannot simply be transferred to the other.

Consistently beating strong human players across diverse VGC teams remains an unsolved challenge in the research cited here. VGC-Bench achieved a win against a professional in a restricted mirror match, but that success did not establish general superiority across matchups. VGC-Bench study.

Why we chose Jev

A general-purpose LLM might arrive at a good answer. The problem for our design was getting that answer in time: feeding it an entire rulebook, asking it to reason through possible scenarios, and reviewing its conclusions would leave an uncertain amount of a 45-second turn for simulation and action. We needed several decisions within that window, which made a long, open-ended reasoning process a poor fit.

Our experiment used Jev, a model from TypeSafe designed to make quick, structured decisions. An application supplies the possible answers, and Jev selects or scores them and returns weights for those answers. TypeSafe describes a parallel output process and reports response times of roughly 70–500 milliseconds for its own workloads. TypeSafe's introduction to Jev.

The appeal was practical: we wanted enough time in a turn for several focused decisions and some simulation. Speed would help us test that design; it would not make the choices correct by itself.

Give it the knowledge—and a clear test

We assembled a wiki of 1,364 pages about Pokémon, moves, items, abilities, and mechanics. Our system retrieved relevant passages from the wiki and supplied them to Jev alongside the current battle position. We did not retrain Jev on the wiki.

At first, we gave it generous context. If an ability created an exception or a move had a special effect, we wanted Jev to see it. Yet in our early practice games, I kept winning and noticed Jev making what looked like silly mistakes—choices that missed something an experienced player would catch immediately.

But my impression that it was making bad choices was not an objective measure. We needed repeatable situations with clear reference answers, so we could compare versions instead of relying on how a few games felt. We built a benchmark of 376 designed situations covering priority, protection, speed, weather, immunities, status, and decisions shaped by earlier turns. We supplied each battle record and the actions available in that position. A test could have more than one accepted play.

The priority mistake from the opening became one of our clearest diagnostic examples. Returning to Pikachu and Mewtwo: with one health point left, Pikachu needs to act first. Quick Attack can finish Mewtwo; Thunderbolt cannot fire before Mewtwo’s ordinary attack. We tried adding a direct hint: Quick Attack has +1 priority. Our trainer gave that move more weight, but still chose the attack that would act too late. Longer explanations of the relevant rules did not reliably fix it either.

We could see the wrong choice, but we could not inspect Jev's internal reasoning. The test let us ask a narrower question: was supplying the rule enough for the model to use it when it mattered?

You can explore all 376 situations, inspect the available plays, and reveal the reference answers.

These were diagnostic exercises. Many shared templates, and we revisited them while developing the system. Their scores tell us how the versions handled these known situations; they do not estimate the agent's win rate against people or its accuracy on fresh battles.

Find the part of the library that matters

We started with broad context, then made it more selective.

One version shortened and combined overlapping explanations. Another let Jev choose small pieces of reference material. Eventually, a version we called V4 asked it to select a focused group of rules relevant to the current position.

The reasoning followed our original picture of the human player. If turn order is the immediate problem, perhaps a clear reminder about turn order is more useful than pages of facts about every Pokémon involved.

Across five ways of supplying reference material, the fourth version scores highest while using much less extra text.
Figure 1. A score counts choices matching an accepted play for the test situation. C0 has no extra wiki notes; V1–V4 use different ways to select and organize them. The right panel measures the amount of added text. Experiment details.

Without extra context, Jev matched an accepted play in 258 of 376 situations. With V4, it reached 305, while supplying about 96% less added text than our broad first version.

Yet the result was not simply “shorter is better.” We had changed what the text contained and how it was organized. Some kinds of situation improved while others worsened. Cases that depended on remembering a distinction in battle history remained a particular weakness.

What did this extra context look like? Here is an illustrative version for Pikachu and Mewtwo, with the priority rule included:

Pikachu; left; 1 HP remaining; ability Static; no item.

Mewtwo; left; 1% displayed HP; Speed +6; ability Pressure; no item.

The move description makes the relevant rule explicit:

Quick Attack: Normal, Physical, base power 40, accuracy 100%, priority +1, target normal. No additional effect.

Would a second look help?

Our next idea was familiar from human decision-making: when you are unsure, reconsider.

Jev gives weights to its options. We used those to identify decisions it did not strongly favor, retained up to four promising alternatives, and asked again with more focused information.

The hope was that a smaller decision would be easier.

But uncertainty was a poor guide to the mistakes we wanted to fix. The review rule caught 64 of V4's 71 errors, while also sending 276 already-correct decisions back for reconsideration.

It was asking almost everyone to take the exam twice.

The best of those review conditions improved the total from 305 to 308. It fixed eleven decisions and spoiled eight. Often, asking for more focused context produced the same material again.

Repeating the question was not the same as making Jev think harder. Its output is restricted to the choices we supply: it cannot hallucinate a new move or generate an invented explanation. But a valid choice can still be a bad choice. With essentially the same context, our repeated reviews often returned the same decision. To change that decision reliably, we needed to improve the evidence it was judging. How Jev returns decisions.

We needed a way to show it something new.

Show it what could happen next

We used Pokémon Showdown to simulate shortlisted plays against a selection of possible opposing responses. Jev first narrowed the options to as many as four plays. The simulator resolved the mechanics, and we summarized the resulting positions for Jev to compare.

Those simulations were hypothetical. They did not reveal the opponent's pending move or use the actual game's future random sequence. Our initial selection of opposing replies covered alternatives we considered useful; it was not a validated model of what a human opponent was likely to do.

This version chose an accepted play in 358 of 376 situations. The priority test still failed: Quick Attack won all 32 simulated possibilities, while the ordinary attack lost all 32. Jev still selected the losing move.

The rules were no longer the only issue. The simulated outcomes were present, but the way we showed them made the link between each play and its consequences hard to follow.

Make the consequences easy to follow

We rewrote the simulation summary so that each action appeared beside its result. Pokémon were named. Wins and losses came first, followed by health, knockouts, and other relevant effects. The central comparison was simple:

  • Pikachu uses Quick Attack on Mewtwo: Our side wins in all 32 tested possibilities.
  • Pikachu uses Thunderbolt on Mewtwo: Our side loses in all 32 tested possibilities.

We did not tell Jev which move to pick. We made the relationship between a move and its result easier to read.

With the same model, battle records, choices, reference context, and simulated outcomes, changing the presentation of the simulation raised the score from 358 to 371 out of 376. Fourteen choices improved and one became worse. In the priority test, Jev's weight on Quick Attack rose from 45% to 99%, and it finally chose that move.

Rewriting the simulation summary raises the score from 358 to 371 out of 376, and increases the weight assigned to Quick Attack from 45% to 99% in the diagnostic example.
Figure 2. Only the presentation of the simulation evidence changed. Fourteen decisions improved and one became worse. The Quick Attack panel shows one example; its percentage is the weight Jev assigned to that choice, not a chance of winning. Controlled comparison.

The rewritten summary was about 21% longer. Earlier, less reference text had helped; here, extra words helped because they clarified what happened. The useful variable was not word count alone. It was whether the relevant knowledge and consequences were easy to connect to the available action.

The 371 score was encouraging, but it still came from our familiar exercises. Many of the simulated situations ended in a clear hypothetical win or loss, making this style of summary especially useful. It does not mean a 98.7% win rate against players.

Keep guesses separate from observations

Our early tests gave Jev considerable information about the opposing team. Later, we chose a stricter setting for the demo: it could see the opponent's six Pokémon, but would learn their moves, items, and abilities only when they appeared in battle. It could not see which four the human had privately selected. This is similar to a human battle played without shared team lists: you see the opposing Pokémon, then discover how they are equipped as the game unfolds.

We still wanted simulations to work. A human does not stop thinking ahead just because an opponent has not revealed every move.

The wiki supplied plausible ways an opposing Pokémon could be equipped. Jev helped rank those possibilities, and the code constructed legal hypothetical teams consistent with what had been observed.

Crucially, a guess stayed a guess.

An item assumed for a simulation did not become an item the opponent was recorded as having. As the battle revealed new facts, the system updated what it knew and constrained the possibilities accordingly.

The score against the original reference answers fell from 371 of 376 situations (98.7%) with opponent details available to 295 of 376 (78.5%) with revealed information and inferred sets.

The earlier system with opponent team details available scores 371 out of 376, or 98.7 percent. The later system using revealed information and inferred opponent sets scores 295, or 78.5 percent.
Figure 3. Scores against the original reference answers for the same 376 situations. Earlier and later system configurations. Earlier results · Later results.

From test decisions to complete games

The prototype now combines a reference library, selection of relevant rules, a shortlist of plays, simulations of opposing replies, and summaries of their consequences. It also maintains a brief of its own team and a record of what has been revealed about the opponent.

The experiments taught us three practical lessons. Supplying a rule does not ensure it will influence a choice. Asking for a second judgment can repeat the first mistake. And even correct simulation results can go unused if their relationship to the actions is difficult to read.

We still need tests on new scenarios, more positions where good play pays off several turns later, complete games against consistent opponents, and measurements of the full pipeline's decision time under a tournament-sized clock. The 376 situations were useful development tools, but repeated work on them may have made their scores optimistic about generalization.

Then came the part no benchmark could settle: playing against it.

I am an average VGC player, leaning toward mediocre. And I still beat the agent almost every time. After all those improvements, full battles could still expose mistakes our exercises had not prepared it for.

But Jev did beat me once. After spending so much time watching it miss obvious plays, losing to it was a surprisingly satisfying moment. We had built something that could occasionally get the better of its own builder. Now the challenge is to turn that one satisfying win into consistent play across entire battles.

Try the simulator

Play against Jev: play.neurogabo.com

You can now play a complete VGC battle against Jev in your browser and watch how it narrows the available choices before making a move. Explore the 376 benchmark situations and the experiment results and methodology to inspect the evidence behind this article.