Train 1T parameter LLM with 8 GPUs?

A new open-source project called Orbit claims to enable training of a 1-trillion-parameter large language model using just eight GPUs. The tool, hosted on GitHub by Sphere-AI-Lab, aims to dramatically reduce the hardware requirements for large-scale AI model development. If successful, this could lower the barrier to entry for training massive LLMs, a process typically requiring thousands of specialized processors.