Inicio Explorar Axuda
Rexistro Iniciar sesión
AIForce
/
train-dalle-together
réplica de https://github.com/learning-at-home/dalle-hivemind.git
5
0
Fork 0
Ficheiros Incidencias 0 Wiki

Sen descrición

Árbore: 872db56e0b
Ramas Etiquetas
train-dalle-tog...
ZIP TAR.GZ
Alexander Borzunov 872db56e0b Fix bitsandbytes version %!s(int64=3) %!d(string=hai) anos
inference acf688eac2 Use --top_p and --top_k options in run_inference.py %!s(int64=3) %!d(string=hai) anos
lib 64dee420da Upgrade to using hivemind.optim.experimental %!s(int64=3) %!d(string=hai) anos
.gitignore 72fc0bcdb7 Initial commit (ru-max branch without private code) %!s(int64=3) %!d(string=hai) anos
README.md d4140807f4 Update readme %!s(int64=3) %!d(string=hai) anos
arguments.py 19e3d2d060 Make aux peers fetch checkpoints every 2 steps by default %!s(int64=3) %!d(string=hai) anos
callback.py 4918c58cb6 Polish stdout %!s(int64=3) %!d(string=hai) anos
data.py c61c61b20d Use t5-small tokenizer %!s(int64=3) %!d(string=hai) anos
huggingface_auth.py 3e604bc1f5 fix auth %!s(int64=3) %!d(string=hai) anos
manage_scaleset.py c365b2ec9f Tweak settings for the upcoming demo (#2) %!s(int64=3) %!d(string=hai) anos
requirements.txt 872db56e0b Fix bitsandbytes version %!s(int64=3) %!d(string=hai) anos
run_aux_peer.py 748f20991c Make runners executable and improve their shebangs %!s(int64=3) %!d(string=hai) anos
run_trainer.py 748f20991c Make runners executable and improve their shebangs %!s(int64=3) %!d(string=hai) anos
run_trainer_tpu.py 748f20991c Make runners executable and improve their shebangs %!s(int64=3) %!d(string=hai) anos
task.py 09240991cc Make model uploading use access token from authorizer (#7) %!s(int64=3) %!d(string=hai) anos
utils.py f621362466 Make logging less verbose %!s(int64=3) %!d(string=hai) anos

README.md

Training DALL-E with volunteers from all over the Internet

This repository is a part of the NeurIPS 2021 demonstration "Training Transformers Together".

In this demo, we train a model similar to OpenAI DALL-E — a Transformer "language model" that generates images from text descriptions. Training happens collaboratively — volunteers from all over the Internet contribute to the training using hardware available to them. We use LAION-400M, the world's largest openly available image-text-pair dataset with 400 million samples. Our model is based on the dalle‑pytorch implementation by Phil Wang with a few tweaks to make it communication-efficient.

See details about how to join and how it works on our website.