 Aleksandr Borzunov
|
4669a9cd91
Support -p n_gpus arg
|
2 rokov pred |
|
Aleksandr Borzunov
|
19602a4f5c
Fix initial peer
|
2 rokov pred |
|
Aleksandr Borzunov
|
d3121d7f08
Revert to MAX_TOKENS_IN_BATCH = 1024 default
|
2 rokov pred |
|
Aleksandr Borzunov
|
e188610125
benchmark_forward: Use dtype=bfloat16
|
2 rokov pred |
|
Aleksandr Borzunov
|
02a406129f
benchmark_forward: Set MAX_TOKENS_IN_BATCH
|
2 rokov pred |
|
Aleksandr Borzunov
|
d98bb72ff9
Fix initial peer for inference
|
2 rokov pred |
|
Aleksandr Borzunov
|
a4d0c9e82f
Fix
|
2 rokov pred |
|
Aleksandr Borzunov
|
51d96edfa7
Fix
|
2 rokov pred |
|
Aleksandr Borzunov
|
dcf5183b69
Don't use .lm_head() in benchmark_forward.py
|
2 rokov pred |
|
Aleksandr Borzunov
|
cf87264199
Hardcode more initial peers
|
2 rokov pred |
|
Aleksandr Borzunov
|
88251e100c
MAX_TOKENS_IN_BATCH = 512
|
2 rokov pred |
|
Aleksandr Borzunov
|
52c1149751
Use dtype float32
|
2 rokov pred |
|
Aleksandr Borzunov
|
d5e27c262b
Fix initial peer
|
2 rokov pred |
|
Aleksandr Borzunov
|
4721094b84
Add benchmark_forward.py
|
2 rokov pred |
|
Aleksandr Borzunov
|
205eb2f2d8
Add initial peer, show speed
|
2 rokov pred |
|
Aleksandr Borzunov
|
84776bff73
Mark required args
|
2 rokov pred |
|
Aleksandr Borzunov
|
96ad0e0cc3
chmod +x benchmark_inference.py
|
2 rokov pred |
|
Aleksandr Borzunov
|
5092b35171
Add inference benchmark
|
2 rokov pred |
|
Aleksandr Borzunov
|
2cfd70d751
Debug mode: load empty block
|
2 rokov pred |
 Alexander Borzunov
|
675bacb592
Bump version to 1.1.5 (#312)
|
2 rokov pred |
|
Alexander Borzunov
|
e026952338
Abort speedtest if it runs too long (#316)
|
2 rokov pred |
|
Alexander Borzunov
|
6eb306a605
Raise error for unexpected .generate() kwargs (#315)
|
2 rokov pred |
|
Alexander Borzunov
|
d9e7bfc949
Divide compute throughput by average no. of used blocks (#314)
|
2 rokov pred |
|
Alexander Borzunov
|
6137b1b4b0
Replace .make_sequence(..., mode="random") with mode="max_throughput" (#313)
|
2 rokov pred |
|
Alexander Borzunov
|
0a313bf6c5
Update hivemind to 1.1.8, enable efficient bfloat16 encoding (#311)
|
2 rokov pred |
|
Alexander Borzunov
|
8f6342a861
Refactor RemoteSequenceManager (#309)
|
2 rokov pred |
|
Alexander Borzunov
|
454c193863
Fix OOMs happening in case of accelerate >= 0.16.0 (#310)
|
2 rokov pred |
|
Alexander Borzunov
|
93c4eba5d1
Bump version to 1.1.4 (#306)
|
2 rokov pred |
|
Alexander Borzunov
|
c0e0e1319d
Force transformers to use config.torch_dtype by default (#307)
|
2 rokov pred |
|
Alexander Borzunov
|
98be9ffe4c
Relax the rest of Hugging Face dependencies (#305)
|
2 rokov pred |
