At first glance, these models look great. Both gpt-oss-20b and gpt-oss-120b show strong heavy-tailed structure, with very few underfit layers (α > 6). That’s usually a sign of solid training.
But when you dig deeper into the alpha histograms and correlation flows, a clear pattern emerges:
There are a surprisingly large number of layers with α < 2, across both models.
This suggests widespread overfitting, especially in middle and deeper layers. And while 120b is bigger, it’s actually a bit worse, showing more noisy outliers and unstable α values.
We’re also seeing signs of correlation traps (not shown), where α dips and recovers sharply across adjacent layers — another red flag for learning inefficiency.
Bottom line:
These models look balanced at first, but their internal spectra tell a different story. The overfit layers dominate the signal — and bigger doesn’t mean cleaner.