Hello guys, I am a grade 8 student nicknamed Digilidaahz, this is an 1B model trained from scratch with a single rtx5080 and ddr5 65gb. V1.1 is an update from the previous V1 version, where generally the ability of writing and chat improved, but still hallucinations on maths, facts, and code. Special note that this model has not done any kinds of safety allignment, so it is only for educational and experiment purposes.
Models Performance:
Model Evaluation (V1.1 Only)
Evaluated using the standard `lm-evaluation-harness` (0-shot setting).
| Benchmark | Metric | Score | Notes |
|---|---|---|---|
| ARC-Challenge | acc | 28.33% |
| ARC-Challenge | acc_norm | 30.46% | (Standard Error: 1.34%) | Showing that it has the ability to reason, above the 25% random guessing baseline.
Special thanks to @John6666 who found the tokenizer bug ({ mapping to <|unk|>) in the previous version. This bug will be fixed in the upcoming V2 (50B tokens pretrain) version of Freedream-1B.
Thanks for reading, have a good day