Three inches east
On May 31, 1886, thousands of sledgehammers went to work on the railroads in the southern United States. Over two days, work crews pulled railroad spikes from 13,000 miles of the west rail. They shifted that rail three inches east and hammered it back down. Laid end to end, it would cross the United States more than four times.
When railroads were first built in the U.S., the gauge - the distance between the two rails - varied from line to line. The New York and Erie Railroad ran a 6' gauge so it could carry heavier locomotives. Down South, the South Carolina Canal and Rail Road settled on 5', and that width spread until it was what the region ran on.
Break of gauge at Gloucester, June 1846. W J Linton's plate for the Illustrated London News makes comedy of Britain's Battle of the Gauges, where Brunel's Great Western ran just over 7' and Stephenson's lines ran 4' 8". Passengers and luggage bound for Birmingham had to be moved carriage to carriage. Source: Science & Society Picture Library.Two railroads with different gauges could not connect. Everything came off at a break of gauge, the spot where one line met another of a different width. Passengers got out and boarded a second train. Freight had to be transloaded, and the labor involved showed up in the price of everything that was shipped.
Some towns made money off the breaks. Erie, Pennsylvania sat where a 6' line met a 4' 10" one, so anything going from Buffalo to Cleveland changed trains twice there. Delays were common and passengers stuck overnight ate in Erie's restaurants and slept in its hotels, while local crews earned wages hauling freight from one train to the other. In 1853, the railroads announced they would align the gauges. Erie rioted. The townspeople tore up the new track and wrecked the bridges in the Erie Gauge War.
After the Civil War, trade between North and South grew, and so did the bill for transloading at every break. In 1886, the South changed its gauge to 4' 9", near enough to what the North ran that trains could roll straight through. A freight car loaded in Atlanta could stay on the same rail all the way to Chicago. Daniel Gross at the National Bureau of Economic Research compared freight traffic before and after the change. Railroads took business from steamships and operated like a cartel, pocketing much of the savings from standardization. Machine learning has this same break of gauge, because every lab ships its models in its own format, and that was one way to get there.
Changing the wheels in motion
Spain never moved its railroads. It runs a variable gauge system that lets a train cross from one gauge to another without stopping. The older network sits on the wide Iberian gauge shared with Portugal. When Spain built its high-speed lines later, it used standard gauge so those trains could run into France.
E. de Latorre's 1906 map of Spain's railways. Broad-gauge lines are color-coded by operator, while narrow-gauge lines appear in black. Source: Biblioteca Nacional de España.Spain's Talgo trains ride on variable gauge axles. A train rolls into the gauge changer and guide rails take its weight. The wheels unlock from the axle, and a track that widens or narrows slides each wheel along the axle to its new position. A second set of guide rails locks it there, and the train rolls out onto the new gauge. Its wheels get reset underneath the passengers while the whole train keeps moving.
These are two different ways to standardize. One forces everyone to move their rails, and the other makes crossing the boundary free. Hugging Face Transformers is the library much of the open-model world loads its models through, and its dynamic weight converter took the second way across. Labs keep their formats, and the converter meets each format at the boundary. Before it existed, the ecosystem paid for every mismatch with a hand-written script.
Every lab lays its own track
No model lab picks its checkpoint format to be difficult. A checkpoint is the file a trained model ships in. Inside it are arrays of numbers called weights, and each array has a name. Some railroads picked a wider track because they wanted to run bigger locomotives, and others went narrower because it cost less to build. Labs make the same kind of choice, laying out a checkpoint to match how the model was trained and the hardware it was born on.
A checkpoint can drift from the standard in more than one place, and each choice makes sense on its own. Any two of them together will make one lab's loader useless on another lab's file.
Start with how the model gets divided. Mixtral-8x22B from Mistral is a mixture-of-experts (MoE) model. Rather than one big network, it holds eight smaller ones called experts, and a router sends each token to a couple of them. The checkpoint stores the experts the way the model is built, as eight separate sets of weights. Transformers keeps its experts fused as a single block, so it cannot read that arrangement until it gathers the eight back together.
Qwen2-57B-A14B is another MoE, this one from Qwen, carrying 57B parameters and activating 14B of them. Qwen split its experts the way Mistral did, but followed the Llama naming convention, so its weight matrices are called gate_proj, up_proj, and down_proj. The idea is the same but the names are not, so a loader built for one file cannot find anything in the other. DeepSeek-V4 diverges for its own reason. It ships attention weights under the names from its paper, wq_a, wq_b, wkv, wo_a, which is convenient for anyone who read the paper and incomprehensible to any loader accustomed to a different vocabulary.
Names aside, the numbers can be arranged in a different order. Qwen3-VL-235B-A22B ships its experts already fused into 3D tensors, then flips the last two axes. Every name can line up and the file still won't load, because the numbers are in the wrong order. Everyone else lays them out as (experts, ffn, hidden), and Qwen3-VL lays them out as (experts, hidden, ffn), because its training stack read them the other way around.
DeepSeek-V3 ships weights compressed to FP8 with 128x128 block scales since that's how it was trained. A loader expecting full precision reads noise though because the numbers only make sense next to their scales. FP8 halves the file against the usual bf16, so a checkpoint that would be 1.3TB becomes 671GB.
Working with any of these models used to mean running a one-off conversion script. The script ran once, and the converted weights are published on the Hugging Face Hub, where the ecosystem downloads its models. That copy became the Transformers format in practice. Every other tool then adapted to that shape, and the places where a tool didn't adapt are where the ecosystem fragmented. Somebody had to write each script, and these grew to more than 300 in Transformers.
The dynamic weight converter retires the one-off script, and along with it, the idea that one format has to be canonical. Arthur Zucker, the Head of Transformers, shipped it in November 2025. One file, conversion_mapping.py, accounts for 164 model types, and only 76 of them lay any track of their own. Most of the other 88 take a single line each, and that line points the model at an existing one whose format it already matches, so it borrows that conversion.
_MODEL_TO_CONVERSION_PATTERN = {
# Mixtral-style MoE
"minimax": "mixtral",
"minimax_m2": "mixtral",
# Qwen2-style MoE
"afmoe": "qwen2_moe",
"deepseek_v2": "qwen2_moe",
"deepseek_v3": "qwen2_moe",
"deepseek_v32": "qwen2_moe",
...
}The old scripts ran to more than 100,000 lines all told, and conversion_mapping.py does that work in 1,616. A script was a transloading crew standing around at each break of gauge. The converter changes the gauge in motion instead, and for most models nothing has to stop.
Nobody touches the rails
Every array in a checkpoint has a name and a shape. Loading a model changes both, renaming each array to what Transformers calls it and restacking the numbers into the shape Transformers expects. The converter does this while the file streams in, one array at a time, the way a Talgo changes gauge without stopping.
Follow one of Mixtral's expert weights as it arrives.
model.layers.7.block_sparse_moe.experts.3.w1.weight # Mixtral
model.layers.7.mlp.experts.gate_up_proj # TransformersThe converter rewrites the name first. Mixtral files this set of weights under the name block_sparse_moe, and Transformers expects to find it under mlp.
Then it works out where the renamed array belongs and which neighbors it joins. Mixtral's eight experts arrive as eight separate arrays and Transformers wants one stacked block, so this array waits until the other seven show up. The converter pulls each array off disk at the moment it needs it, and everything else streams past untouched.
Once the eighth arrives, the numbers get restacked by a short chain of small operations. MergeModulelist stacks the eight experts into a single block. Then Concatenate fuses gate and up, which Mixtral had kept apart. That is where the gauge actually changes. Each operation is small and they chain in any order, so a short list of them covers a whole model where a script used to run to hundreds of lines.
The finished block is mounted on the running model and the converter moves to the next array. The model is built in the act of loading it, with no separate conversion pass and no intermediate copy written to disk and read back in.
The journey runs backwards too. Two weeks after the converter shipped, Cyril Vallez, a core maintainer, gave every operation an inverse, so a model loaded into the Transformers layout can be saved back into its native one. A stack becomes a split. Because it works in both directions, no lab is ever forced to re-lay its rails.
The freight that follows
Standardization means any model loads anywhere, with no conversion step between formats. The change shows up first in how fast new models arrive. Switch Transformers was the first MoE in the library. Between it and the converter, Transformers added 31 more over three years, roughly one a month, each carrying its own weight-handling code. In the seven months since the converter, Transformers added 20 more, one every 11 days. Thirteen of those needed no new code at all, most just a line pointing at Mixtral or Qwen2. MoEs were becoming more popular regardless, so the converter did not start that rush. What it changed is the cost of keeping up, and a new MoE that used to take a few hundred lines now usually takes one.
DeepSeek-V3 is stored compressed in FP8. Each block of numbers travels with a small scale value that says how to expand it back to full precision, and that pairing is fragile. When the converter stacks the experts into one block, every scale has to ride along and stay matched to the exact numbers it belongs to, or the decompression comes out as noise. Keeping them aligned used to require a dedicated script. The converter slots a decompression step into the front of the same operation chain and lets the scales travel beside their weights, so the merge that follows runs the way it runs for anything else. The whole FP8 path is one line in the conversion mapping pointing DeepSeek-V3 at Qwen2's pattern, plus that decompression step. There is no DeepSeek-specific code anywhere. The same slot runs the other direction, so a full-precision checkpoint can be compressed on the way in. Quantization backends hook into that slot, and none of them carries a loader of its own.
The standard layout is also what lets a model spread across many GPUs. Stacked into one block, the experts divide evenly. DeepSeek-V3's 256 experts are divided 32 to a GPU across eight of them. Each GPU holds either a few whole experts or a thin slice of every expert. A scattered pile of separate pieces can't divide that cleanly. That same stacked block happens to be the shape the fastest math routines want, since they multiply all the experts in one batched step rather than looping over them.
What matters most here barely touches Transformers, and goes instead to everyone building around it. Without a common shape, every tool needs its own way to read every other tool's models, and the number of those paths grows faster than the number of tools. A shared standard needs one path. Unsloth and vLLM stop meeting each other halfway and meet at the same model definitions instead, so a checkpoint passes between them without being rewritten. Each tool aligns once and works with every model that adopts the standard. None of this requires running Transformers at all. That is what makes it a standard rather than a Transformers feature. A model only has to match the definitions, or import them, and it joins the ecosystem the moment it does.
The gauge still has to be laid by hand the first time. The 76 model types carrying conversion code of their own are the ones no existing rails fit, and the new format has to be mapped before anything can ride in behind it. The converter only makes every crossing after the first one free.
Just another progress bar
Unlike the Erie Gauge War, which made winners and losers, the converter makes no enemies. Nobody's rails get torn up, and no lab is forced to move. Labs keep uploading checkpoints in whatever format suits them, and the reconciling happens at the boundary, at the moment Transformers loads the model.
Crossing for free is only possible because of what the rails are made of. Moving steel takes labor and runs one direction, so a railroad that picks a different gauge is mostly stuck with it. A conversion is cheap and every one has an inverse, so a weight can cross between two formats and cross straight back. Standardizing can be frictionless only when translation is cheap and reversible.
Railroads are not the only network to have split this way. Japan's power grid has run at two frequencies since the 1890s, when Tokyo bought its first generators from Germany and Osaka bought its own from the United States. The east has run at 50Hz and the west at 60Hz ever since. Nobody re-laid that track either. Frequency converter stations sit at the boundary and translate power from one side to the other, but electrical translation is expensive, so the crossing stayed narrow. When the 2011 earthquake shut down power stations in the east, the west had electricity to spare, and the converters could pass about one gigawatt of it, roughly a tenth of what the east was missing. Each boundary gets as much standardizing as its translation can afford, and weights cost almost nothing to move, because they are only numbers, and numbers can be renamed and restacked for free.
In 1886, changing the gauge took thousands of sledgehammers and a coordinated two days across 13,000 miles of track. Today, you can load a 1T-parameter FP8 MoE checkpoint across eight GPUs, and the converter does all of it without anyone swinging a hammer or getting off the train. It is just another progress bar, and the ordinariness is the achievement. The next model like it should cross for free.
Thanks to Lysandre and Anton Vlasjuk for their thoughtful feedback.