Train 1T parameter LLM with 8 GPUs?

A new open-source project called Orbit claims to enable training of a 1-trillion-parameter large language model using just eight GPUs. The tool, hosted on GitHub by Sphere-AI-Lab, aims to dramatically reduce the hardware requirements for large-scale AI model development. If successful, this could lower the barrier to entry for training massive LLMs, a process typically requiring thousands of specialized processors.

Hacker News new | past | comments | ask | show | jobs | submit login Train 1T parameter LLM with 8 GPUs? 1 point by kendy1992 15 minutes ago | hide | past | favorite | discuss See this project https://github.com/Sphere-AI-Lab/orbit. help Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact Search: