 xtinkt
|
eedfae4bd4
Merge branch 'decentralized_lr_scheduler' of https://github.com/learning-at-home/hivemind into decentralized_lr_scheduler
|
4 years ago |
|
xtinkt
|
be77054fad
sync peer in constructor id needed
|
4 years ago |
 Anton Sinitsin
|
edfc3c005b
Update hivemind/client/averaging/training.py
|
4 years ago |
|
Anton Sinitsin
|
26ac1c7299
Update hivemind/optim/averaged.py
|
4 years ago |
|
Anton Sinitsin
|
fe41e0d95d
Update hivemind/optim/averaged.py
|
4 years ago |
|
xtinkt
|
017fbe43a2
change DecentralizedState to TrainingState
|
4 years ago |
|
xtinkt
|
c27925de26
fix some comments
|
4 years ago |
|
xtinkt
|
42423cfe6b
fix some pr issues
|
4 years ago |
|
xtinkt
|
517018b136
added scheduler state loading and saveing to averager
|
4 years ago |
|
xtinkt
|
937e622b8b
fix some issues in pr
|
4 years ago |
|
xtinkt
|
15f1e9b0b5
fix some issues in pr
|
4 years ago |
|
xtinkt
|
14404e4168
fix some issues in pr
|
4 years ago |
|
xtinkt
|
97cec29e13
Added info about new params. Changed some names.
|
4 years ago |
|
xtinkt
|
de01a9cb15
add decentralized learning rate scheduler and epochs abstraction
|
4 years ago |
 Aleksandr Borzunov
|
08ee017f0f
Add nltk to ALBERT example's requirements (#251)
|
4 years ago |
 Roman Zhytar
|
e833a7efb9
Decentralized adaptive optimizers (#243)
|
4 years ago |
|
Aleksandr Borzunov
|
18add2c04b
Implement combining validators (#249)
|
4 years ago |
 Max Ryabinin
|
0a1fdb172f
Fix incorrect data types/values in RemoteSwitchMixtureOfExperts (#246)
|
4 years ago |
|
Max Ryabinin
|
dfbc401196
Add Dockerfile, refactor tests (#245)
|
4 years ago |
 justheuristic
|
ddb5389e66
Fix server hanging in certain cases when connection is lost (#247)
|
4 years ago |
|
Aleksandr Borzunov
|
a3feafa907
Add DHT schema validator (#227)
|
4 years ago |
 Michael Diskin
|
2314e7ebd5
fix metrics (#240)
|
4 years ago |
 Alexey Bukhtiyarov
|
27ea94e3f9
Add example for collaborative ALBERT training (#226)
|
4 years ago |
|
Max Ryabinin
|
62652e1717
Add Switch Transformers-like RemoteMixtureOfExperts (#228)
|
4 years ago |
|
justheuristic
|
3d6a242e30
Ensure version-consistent result rounding in load_balance_peers (#230)
|
4 years ago |
|
Roman Zhytar
|
8c3bd93e87
Statistics averaging (#229)
|
4 years ago |
 Vsevolod-pl
|
91d17a4ebc
Delta gradients transmission (#225)
|
4 years ago |
 romakail
|
ca5c7610ae
Add tool for custom user experts (#189)
|
4 years ago |
|
justheuristic
|
32b87bf3fe
Reset gradient buffers when synchronizing with peers (#222)
|
4 years ago |
|
justheuristic
|
b906ae94ed
better zero_grad behavior in CollaborativeOptimizer (#221)
|
4 years ago |
