Title: CMF-180: Historical Curriculum in Blocks for Small Language Models
Tags: model-training, measurement
---
The paper reports a preliminary experiment on whether the order in which data is presented changes what a small language model learns. A GPT-style model of about 36 million parameters was trained from scratch on CMF-180, a corpus of Greek, Latin, documentary and epigraphic texts limited to 180 AD (about 37,800 documents and 38.9 million words). Corpus, tokenizer, architecture, validation set, hyperparameters and number of steps were held constant. Only the order of the training chunks changed.

Three regimes were compared: global random shuffling, rigid chronological order, and a **block-based historical curriculum**, in which the historical phases follow one another but the chunks inside each phase are shuffled. The final validation loss was 5.10 for random, 6.12 for rigid chronology and 4.93 for the block curriculum. Rigid chronology also produced the strongest anisotropy in the embedding space and the most fragmented semantic neighbours, while the block curriculum stayed close to random on those measures and remained geometrically distinct from it.

The interpretation offered is that order matters, but not every order helps: a rigid sequence makes local exposure too homogeneous, while historical blocks give a global direction and keep local diversity. The paper calls this a metastable regime, proposes the name Metastable Historical Curriculum Learning, and links it to Information Gravity Theory as the ground on which representations settle.

The result is offered as an experimental signal, to be replicated on other model sizes, seeds and corpora.