Credits
We kindly thank (in random order)
- Artem Babenko and
Vladimir Aliev (yandex research) for helpful discussions
and editorial review of the paper,
- Jacob R. Steeves (bittensor) for discussions on RPC frameworks and NAT traversal and
peer-to-peer technologies.
- Dmitry Afanasiev (yandex) for his guidance on networking
and communication technologies,
- Lidi Zheng (google) and grpc-aio contributors for their awesome framework and this pr
- Brian Muller (parallel markets) for his implementations of kademlia and rpcudp
- Alexander Sherbakov (itsoft) for helpful discussions on PC and server component architecture,
- Our early adopters, contributors, and reviewers
Related projects
We also want to reference several projects that have similar ideas in mind:
- BitTensor - a decentralized deep learning ecosystem with with incentive
mechanism. Like hivemind, but peers are getting rewarded for their contribution to other peers.
(note: as of 26.08.2020 the project is in the early stages development).
- GShard - a paper by Dmitry Lepikhin et al that demonstrate the effectiveness
of huge Mixture-of-Experts models on conventional hpc hardware. Those guys train models 4 times the size of GPT-3 on thousands of TPUv3.
- Also doing research in decentralized deep learning? Let us know!