Hi everyone,
I just made public a small peanut project I built on a CPU-only laptop, no GPU.
Repo: https://github.com/dskc/peanut Idea is simple. Peanut v2 is Adam with two changes:
Everything else is standard Adam.
Results, Fashion-MNIST, CNN, batch 64, lr 0.01, 3 seeds:
Experiment 01, 10 epochs:
Experiment 02, 20 epochs, long training:
Peanut does not beat Adam by a lot. The real win is stability.
AdamW collapsed on some seeds in my tests, 0.8803 on CNN, 0.8366 on MLP.
Peanut v2 never collapsed.
Limits, to be honest:
I kept the repo tiny on purpose:
I would love your help to test it further:
If it fails, that is also useful. Open an issue with seed, dataset, lr, and final accuracy, and I will add it to the README. Welcome to experiment. All feedback welcome, especially negative results.
Please pay a visit to dskc/peanut · Discussions · GitHub share your thoughts there as well
Thanks!