Generations of the Learned Tank

Watch each generation

Each clip is the net on its own defining test, on the first held-out seed it passes, so it shows what the net can do; the untrained net is shown on its first seed. Blue is the learner and its drones, red the enemy, orange the shells with their landing marked, cyan the learner's interceptors. The bar over the tank is its armour.

In battle

The gauntlet: one decision at a time

Nine short tests, each on open grass with nothing else on the map, each asking one decision. Every brain plays the same 200 seeds. The cell is the share passed; below it, the median reaction time (the first shot, the first move, or the first interceptor, as the test times it).

Green deepens with the share passed. A dash: that reaction never came.

What the gauntlet found

• Shortcuts. The map 7a net goes home 99.7% of the time when low on armour and shells (its training map), but only 50% when only armour is low (G5): it learned "few shells, go home". The map 4 net takes a guarded depot 98% of the time but an unguarded one 0% (G6): it learned the scene, not the depot. Neither shortcut showed on the training maps.
• The scripted brain's blind spot. Inside a hostile pillbox's ring at 40% armour (G8), the scripted brain fights and dies every time; every trained net does better, and the map 4 net escapes on all 200.
• No terrain yet. A depot across a strip of water (G9): the scripted brain 100%, every net 0%. No net has trained on terrain.
• Gains traded away. The map 7 net, the best in matches, is worse than the map 5 net on several single decisions (G3 a shell, G7 a pillbox, G8 its ring): what it learned for map 7 cost it there.

In a real match: living versus fighting

Seat 9 of a dealt 16-seat match (a ranger whom nobody follows) given to each net, the scripted ranger in the same seat on the same 40 seeds, first 3 minutes.

The two measures pull apart. The untrained net lives as long as the scripted ranger because it does almost nothing (11 damage dealt). Phase 1 and 3 nets fight (54 and 49) and die three times in three minutes. The survival maps moved the map 7 net toward the corner the scripted ranger holds, to 0.82 deaths, at the cost of half its damage. The scripted ranger does both: 0.33 deaths and 63 damage.

The census

A team of five nets in one team's seats against the scripted defaults, 110 seeds, 11 maps, three matches a seed; zero is the scripted brain.

The map 7 net is the first to climb off the bottom: +3.96 ± 0.53 over the map 4 net, with four times the kills a match (3.7) and two fewer deaths (16.6 of 20). It is level with a random net, so its rule (two errors above random) fails. The census asks the most of a net: the whole game, supply, capture, building and teams, none of which a net does yet.

Method: a sandbox built on the scripted brain

The learned tank did not get a world of its own. It trains inside the one the scripted brain (the "brainz": the Commander, its team board and 110 census-tuned weights) was built and measured in, and every tool made for the scripted brain became a tool for training it:

• One file, two brains. A training map is a scenario file. With its net seat swapped for a Commander it is a test of the scripted brain; with a net, a training course. Both read on identical seeds.
• The rule first. Each map's pass bar is written down before it trains, and read on the untrained net and the scripted brain first. Twice a bar was found empty that way (an untrained net "passing" by mortar; a map no brain could survive) and changed before training.
• Probe the bound. When the dodge stalled at 93%, an oracle playing the net's own nine drive actions forward in a copy of the world dodged 300 of 300: the actions sufficed, the senses did not. A shell contact took the net to 96%.
• Trace the death. A net that never survived map 7 was dying to strike drones, not shells; the drone head and an air contact took it from 0 to 44%.
• Step maps. Map 7 alone never moved; map 7a (the same without the enemy) taught "low means home" in 300 generations, and map 7 followed.

What it costs to run the tank on its AI alone

The net is cheaper to think than the scripted brain, and the world's own step is most of a match's cost either way. No GPU is involved: training is as fast as the CPUs can play episodes.

The quality of the improvements

Every gain was measured on seeds no generation trained on; every loss is in the same tables. The next stage is set by them: gauntlet items G5 and G6 become training maps against the shortcuts, map 8 and the duel get the drones, and the census is read again on the same seeds.

Run it yourself

make gauntlet                 # 9 tests x every net, 200 seeds: build/train/gauntlet.tsv
make clips                    # these GIFs: build/train/clips/
make train CHECK=nets/map7.bin MATCH=40      # one seat in 40 real matches
make train GROUND=phase1,phase3,scenarios/training/04-the-guarded-depot.toml START=nets/phase3.bin
CENSUS_NET=nets/map7.bin SEEDS=110 make census