Self-play.
This experiment trains a program generator and a byte predictor together, starting from random weights. Generated programs produce the predictor’s training data. A gradient-based reward changes which programs the generator produces as training progresses.
Model & training settings · inspect architecture or customize
Quick and Long run restore their defaults. Choose Custom to inspect or edit the current values. Editing any field marks a custom draft; the running models stay unchanged until you apply it.
WebGPU executes transformer forward passes and full parameter gradients on your GPU. Program execution, cached sampling, reward reduction and Adam updates run on the CPU. Small models can be limited by GPU dispatch overhead; larger models do more work per dispatch.
The live curriculum
WAITING FOR FIRST BATCHStart from zero to generate the first real batch.
The first lesson is usually how to predict zeros.
Inside one program
Start training to inspect a generated program.Predictions receive the true preceding bytes, never the source code. Shading marks padding.
More training diagnostics · reward, output, program length & exploration
Reward: batch mean ± standard deviation
Solid line: mean reward. Other lines: mean + standard deviation and mean − standard deviation, clipped at zero. Each point uses an actual batch.
Program output before padding
Percentage of training bytes emitted by programs. This is an amount, not a prediction score; neither direction is automatically better.
Program length: mean ± standard deviation
Instruction tokens per program, including F if present; macros count as single tokens.
Generator uncertainty & archive coverage
Entropy in bits per next instruction, averaged over fresh sampled prefixes. Uniform predictions: log₂(19) ≈ 4.25 bits.
Evaluate your own sequence
EVALUATION ONLY · NEVER USED FOR TRAININGText is encoded as UTF-8. Each byte is predicted using its preceding bytes.
What should I look for?
First seconds: both networks start random. Most programs emit very little, so the learner quickly learns padding. That is real learning, but a modest result.
Next few minutes: watch which programs earn reward, whether more of their output is actually emitted, and whether prediction of the fixed probes improves. The fixed-prior learner draws random programs from the same language and gets the same number of training bytes.
Leave it running: the generator keeps searching; mutation and replay retain useful programs. The larger model offers more room to learn at a slower pace. Save a checkpoint to continue later. There is no scripted progression or guarantee of richer discoveries.
Constant-byte and periodic probes test a little predictive structure. One short text probe is a sanity check, not a natural-language benchmark. This miniature experiment does not establish the paper’s scaling or broad transfer claims.
How closely does this follow the paper?
Based on Self-Play Pretraining with Zero Data, Cowsik et al. (2026). The generator and learner are independently initialized causal transformers. The generator samples from the paper’s 19-token Brainfuck + macro alphabet. The learner predicts all 256 byte values.
The generator is rewarded by the actual, absolute, Adam-preconditioned gradient alignment:
r = | ∇L(y; θₑ) · [ Pₑ ⊙ (θ⌊ₑ/₂⌋ − θₑ) ] |
We compute full gradients, use the exact halfway checkpoint, and train the generator with policy gradients, a uniform-prior KL penalty, replay importance correction, and reward-weighted expert iteration. Mutation rows do not enter the policy gradient. All output rows, including zero padding, train the learner.
Browser reductions: preset models use one transformer layer and 2 attention heads, with learned positions and a ReLU feed-forward layer, rather than Llama. Quick mode uses width 24, 64 output bytes, up to 48 program tokens, 12 programs per round, 256 tape cells and 4,096 primitive execution steps. Long-run mode uses width 48, 128 bytes, 96 program tokens, 24 programs and 16,384 steps. Banks are bounded. No programs are preloaded into training.
Historical weights are reconstructed by deterministic, half-speed copies of the same training process. This preserves the exact growing lookback window with logarithmic memory and under twice the training work. See the implementation & verification notes.
Difficulty ablation: replaces the alignment reward with output cross-entropy. It deliberately removes the paper’s defense against rewarding unpredictable data. A new run is required to change the objective.
Try a program · evaluation only
Run your own code against the current learner. These programs never enter either model’s training data or program bank. Running one replaces the selected program in “Inside one program” above and takes you to that result.
< > move · + − change byte · [ ] loop · . emit · , random input · F finish. Unmatched brackets are no-ops. Macros: Z clear, R move right, L triple right, N subtract left, C copy-add to two neighbors, G/H scan, W increment right & clear current, V print to zero, X set 16. Bytes wrap modulo 256. Program length follows the selected model’s limit.