Mini-AGI actually trains on 8GB VRAM without exploding
A project called Mini-AGI is training a dynamic continual learning model on 8GB of VRAM by combining dynamic Mixture of Experts, which adds and prunes experts during training, with batch-1 training on a single continuous…