|
@@ -1,6 +1,7 @@
|
|
|
<p align="center">
|
|
|
<img src="https://i.imgur.com/7eR7Pan.png" width="400"><br>
|
|
|
- Decentralized platform for running 100B+ language models<br><br>
|
|
|
+ Easy way to efficiently run 100B+ language models<br>
|
|
|
+ without high-end GPUs<br><br>
|
|
|
<a href="https://github.com/bigscience-workshop/petals/actions">
|
|
|
<img src="https://github.com/bigscience-workshop/petals/actions/workflows/run-tests.yaml/badge.svg?branch=main">
|
|
|
</a>
|
|
@@ -11,10 +12,10 @@
|
|
|
|
|
|
## Key features
|
|
|
|
|
|
-- Run inference or fine-tune large language models like [BLOOM-176B](https://huggingface.co/bigscience/bloom) by joining compute resources with people all over the Internet. No need to have high-end GPUs.
|
|
|
-- It's difficult to fit the whole BLOOM-176B into GPU memory [unless](https://twitter.com/Tim_Dettmers/status/1559892918395031552) you have multiple high-end GPUs. Instead, **Petals** allows to load and serve a small part of the model, then team up with people serving all the other parts to run inference or fine-tuning.
|
|
|
-- This way, one inference step takes ≈ 1 sec — much faster than possible with offloading. Enough for chatbots and other interactive apps.
|
|
|
-- Beyond traditional language model APIs — you can employ any fine-tuning and sampling methods by executing custom paths through the model or accessing its hidden states. This allows for the comforts of an API with the flexibility of PyTorch.
|
|
|
+- Run inference or fine-tune large language models like [BLOOM-176B](https://huggingface.co/bigscience/bloom) by joining compute resources with people all over the Internet.
|
|
|
+- **Petals** allows to load and serve a small part of the model, then team up with people serving the other parts to run inference or fine-tuning.
|
|
|
+- This way, one inference step takes ≈ 1 sec — 10x faster than possible with offloading. Enough for chatbots and other interactive apps.
|
|
|
+- Beyond classic language model APIs — you can employ any fine-tuning and sampling methods by executing custom paths through the model or accessing its hidden states. This combines the comforts of an API with the flexibility of PyTorch.
|
|
|
|
|
|
<p align="center">
|
|
|
📜 <b><a href="https://arxiv.org/pdf/2209.01188.pdf">Read paper</a></b>
|