Petals

Petals, an open-source distributed inference system, now supports Llama 3.1 models with up to 405 billion parameters, allowing users to run large language models across a volunteer network of consumer…

Petals, a project from the BigScience research workshop, lets users run large language models like Llama 3.1 (up to 405B parameters) at home by distributing model parts across a peer-to-peer network, …

A developer created Biopetals, a modified version of the Petals library that enables distributed, BitTorrent-style inference for biology-tuned Llama models across a network of computers. The project w…