Peanut v2 - A more stable Adam variant, tested on CPU only. Try it on your dataset?

A developer publishing as dskc released Peanut v2, an Adam optimizer variant with two modifications, built and tested on a CPU-only laptop with no GPU. On Fashion-MNIST with a CNN, batch size 64, learning rate 0.01 and 3 seeds, Peanut v2 did not beat Adam by a large margin but never collapsed, while AdamW collapsed on some seeds at 0.8803 on the CNN and 0.8366 on the MLP. The developer is asking users to test Peanut v2 on their own datasets and to open GitHub issues with seed, dataset, learning rate and final accuracy, including negative results, for addition to the README.

Hi everyone, I just made public a small peanut project I built on a CPU-only laptop, no GPU. Repo: https://github.com/dskc/peanut https://l.meta.ai/?u=https%3A%2F%2Fgithub.com%2Fdskc%2Fpeanut&h=AUBzmfVeqgT0DDYXwOqyTMhbsX3D9gOzyZ4OzEHEu4B8FgYkOzUTf0mIfwA9xd4J-r0KJcgtihhFt1c4vTQ0HKZe7bv4GPyGEPdFqW WF8r1CRmIVZrT8TFBcrVazf4MrBsI4-Z uNlXfOh7osaPEzGugoWdd1jJINBwEQ Idea is simple. Peanut v2 is Adam with two changes: Everything else is standard Adam. Results, Fashion-MNIST, CNN, batch 64, lr 0.01, 3 seeds: Experiment 01, 10 epochs: Experiment 02, 20 epochs, long training: Peanut does not beat Adam by a lot. The real win is stability. AdamW collapsed on some seeds in my tests, 0.8803 on CNN, 0.8366 on MLP. Peanut v2 never collapsed. Limits, to be honest: I kept the repo tiny on purpose: I would love your help to test it further: If it fails, that is also useful. Open an issue with seed, dataset, lr, and final accuracy, and I will add it to the README. Welcome to experiment. All feedback welcome, especially negative results. Please pay a visit to dskc/peanut · Discussions · GitHub https://github.com/dskc/peanut/discussions share your thoughts there as well Thanks