浏览代码

If speedtest fails, assume network speed of 100 Mbit/s (#404)

The value is chosen as some safe value below average at https://health.petals.dev/

Note that if a server uses relays, the effective throughput will be further divided by 2 (see #399).
Alexander Borzunov 2 年之前
父节点
当前提交
30b94ef18b
共有 1 个文件被更改,包括 1 次插入1 次删除
  1. 1 1
      src/petals/server/throughput.py

+ 1 - 1
src/petals/server/throughput.py

@@ -138,7 +138,7 @@ def measure_throughput_info(
 
 
 def measure_network_rps(
-    config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 25e6
+    config: PretrainedConfig, *, timeout: float = 60, default_speed: float = 100e6  # 100 Mbit/s
 ) -> Optional[float]:
     bits_per_request = config.hidden_size * 16  # Clients usually send 16-bit tensors for forward/backward
     try: