A CNN pretrained on MNIST digits 0 to 8 has never seen a 9. Each dot is a test 9, green when the model currently classifies it as a 9 and red when it does not.
Placement: PCA of the pretrained model's penultimate features; UMAP of the features of a reference model trained on all ten digits from the start; or UMAP of the point's cotraining profile, the loss it has after each single pool point is trained on alone.
Click a ringed pool point to posttrain on that single example. Drag the slider to follow greedy forward selection over the pool.
Ringed dots are the pool of candidate 9s. Numbered rings are the ones greedy has picked so far. Posttrain: 500 steps, lr 0.003, 50% of each batch from the pool, rest replayed 0–8.