DeepSeek-V4-Pro-0813 model upload is causing a bit of a headache for anyone trying to run a standard deployment. I tried pulling the weights for a local test, but the repository structure seems inconsistent with their previous releases, and I'm hitting a wall with the configuration files. Specifically, when trying to load the model via the Transformers library, I keep getting a runtime error that suggests a mismatch between the config JSON and the actual tensor shapes.
Here is the exact error I'm seeing in my terminal:
RuntimeError: size mismatch, m1: [1, 4096], m2: [5120, 12288] at /pytorch/aten/src/ATen/native/Linear.cpp:110
This looks like a classic dimensionality mismatch. It's as if the model weights were uploaded from a different checkpoint than the config file accompanying them. I've spent the last few hours doing a deep dive into the config.json
and comparing it to the .safetensors
files, and the hidden size parameters don't seem to align. If you're attempting a real-world deployment of this specific version, you might find that the standard from_pretrained
method just crashes.
My diagnosis so far #
After digging through the files, I suspect a few things could be happening here:
Incorrect Config Version: Theconfig.json
might be a leftover from a V3 or a different experimental branch, while the weights are actually V4 Pro.Partial Upload: Some of the shards might be corrupted or missing, leading the to misinterpret the layer boundaries.Custom Architecture: DeepSeek often tweaks their MoE (Mixture of Experts) routing, and if the current Transformers version doesn't have the specific update for the 0813 build, it'll throw a shape error during the linear layer projection.
I tried to bypass this by manually overriding the
hidden_size
in the config to match the tensor dimensions I found in the weights, but that just pushed the error further down the line to the attention heads. For anyone else trying to build an AI workflow around this, I'd recommend checking the checksums of your downloaded shards. If you're using a custom LLM agent framework, be very careful with the versioning on this specific 0813 build. It feels like a "leaked" or "preview" upload rather than a polished release.
I'm currently testing whether a specific commit of the transformers
library from the main branch fixes the logic, but so far, it's a toss-up. If anyone has managed to get this running without a RuntimeError
, I'd love to see your environment config.
Next Why Random Forest crushed Linear Regression for my IMDb score →
an AI side-hustle playbook, with plenty of directly applicable cases.