Predicting Memorization Before Fine-Tuning
A classifier trained only on pre-fine-tuning base-model features predicts which sequences a fine-tuned model will memorize. The "Explore held-out examples" tab verifies this against ground truth on held-out examples from a fresh fine-tuning run; the "Score your own text" tab applies the same classifier to a passage you provide.
Pick a real Pythia-1.4B fine-tuning sequence and compare the classifier's pre-fine-tuning risk score with what the fine-tuned model actually generated.
Paste a passage (at least ~100 tokens). We compute its base-model features using Pythia-1.4B and return the classifier's predicted memorization risk, and show where your text's features fall relative to memorized and not-memorized examples.
Datasets shown are public (FineWeb, PG-19, The Stack, OpenWebMath). The classifier uses only base-model features and was trained on a separate fine-tuning run.