AIForce/train-dalle-together @ 7ac2b44807953dc6739d7bce67d32e117d1a5e27

Ei kuvausta

7 Haarat

SaulLu 7ac2b44807 add `self.repo.git_pull` before `self.repo.push_to_hub` (#5)		3 vuotta sitten
lib	64dee420da Upgrade to using hivemind.optim.experimental	3 vuotta sitten
.gitignore	72fc0bcdb7 Initial commit (ru-max branch without private code)	3 vuotta sitten
README.md	d4140807f4 Update readme	3 vuotta sitten
arguments.py	27139b8a28 Upgrade to the new auth version (#3)	3 vuotta sitten
callback.py	4918c58cb6 Polish stdout	3 vuotta sitten
data.py	c61c61b20d Use t5-small tokenizer	3 vuotta sitten
huggingface_auth.py	3e604bc1f5 fix auth	3 vuotta sitten
manage_scaleset.py	c365b2ec9f Tweak settings for the upcoming demo (#2)	3 vuotta sitten
requirements.txt	a2e1dfc253 Use user access token instead of username/password (#4)	3 vuotta sitten
run_aux_peer.py	7ac2b44807 add `self.repo.git_pull` before `self.repo.push_to_hub` (#5)	3 vuotta sitten
run_trainer.py	c365b2ec9f Tweak settings for the upcoming demo (#2)	3 vuotta sitten
run_trainer_tpu.py	64dee420da Upgrade to using hivemind.optim.experimental	3 vuotta sitten
task.py	2fed6ba68e Use HF username as wandb run name	3 vuotta sitten
utils.py	f621362466 Make logging less verbose	3 vuotta sitten

Training DALL-E with volunteers from all over the Internet

This repository is a part of the NeurIPS 2021 demonstration "Training Transformers Together".

In this demo, we train a model similar to OpenAI DALL-E — a Transformer "language model" that generates images from text descriptions. Training happens collaboratively — volunteers from all over the Internet contribute to the training using hardware available to them. We use LAION-400M, the world's largest openly available image-text-pair dataset with 400 million samples. Our model is based on the dalle‑pytorch implementation by Phil Wang with a few tweaks to make it communication-efficient.

See details about how to join and how it works on our website.

README.md

Training DALL-E with volunteers from all over the Internet