Freedream-1B-Instruct V1.1 from scratch published!

An eighth-grade developer using the handle Digilidaahz published Freedream-1B-Instruct V1.1, a 1B-parameter language model trained from scratch on a single RTX 5080 GPU with 65GB of DDR5 memory. On the lm-evaluation-harness at 0-shot, Freedream-1B-Instruct V1.1 scored 28.33% accuracy and 30.46% normalized accuracy on ARC-Challenge (standard error 1.34%), above the 25% random-guessing baseline, though the model still hallucinates on math, facts and code and has had no safety alignment, so it is released for educational and experimental use only. The release credits Hugging Face user @John6666 for finding a tokenizer bug in V1 that mapped '{' to <|unk|>, which is slated for a fix in the planned V2 version pretrained on 50B tokens.

Hello guys, I am a grade 8 student nicknamed Digilidaahz, this is an 1B model trained from scratch with a single rtx5080 and ddr5 65gb. V1.1 is an update from the previous V1 version, where generally the ability of writing and chat improved, but still hallucinations on maths, facts, and code. Special note that this model has not done any kinds of safety allignment, so it is only for educational and experiment purposes. Models Performance: Model Evaluation V1.1 Only Evaluated using the standard lm-evaluation-harness 0-shot setting . | Benchmark | Metric | Score | Notes | |---|---|---|---| | ARC-Challenge | acc | 28.33% | | | ARC-Challenge | acc norm | 30.46% | Standard Error: 1.34% | Showing that it has the ability to reason, above the 25% random guessing baseline. Special thanks to @John6666 https://discuss.huggingface.co/u/john6666 who found the tokenizer bug { mapping to <|unk| in the previous version. This bug will be fixed in the upcoming V2 50B tokens pretrain version of Freedream-1B. Thanks for reading, have a good day