Parcourir la source

If speedtest fails, assume network speed of 100 Mbit/s (#404)

The value is chosen as some safe value below average at https://health.petals.dev/

Note that if a server uses relays, the effective throughput will be further divided by 2 (see #399).
Alexander Borzunov il y a 2 ans
Parent
commit
30b94ef18b
1 fichiers modifiés avec 1 ajouts et 1 suppressions
  1. 1 1
      src/petals/server/throughput.py

+ 1 - 1
src/petals/server/throughput.py

@@ -138,7 +138,7 @@ def measure_throughput_info(
 
 
 def measure_network_rps(
-    config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 25e6
+    config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 100e6  # 100 Mbit/s
 ) -> Optional[float]:
     bits_per_request = config.hidden_size * 16  # Clients usually send 16-bit tensors for forward/backward
     try: