[ \tilde\nabla_k = \nabla \ell(w^(e)_k; x_k) + \alpha \cdot (w^(c)_k - w^(e)_k) ]
SimulTrain reduces latency by 78% on 4G and 71% on 5G compared to SyncSGD. FedAvg hides latency via local steps but suffers from model drift. | Method | Upload per step (KB) | Download per step (KB) | |----------------|----------------------|------------------------| | Centralized | 7,500 (video frame) | 75 (weights) | | SyncSGD | 75 (gradients) | 75 (weights) | | SimulTrain | 30 (activations) | 75 (delta weights) |
SimulTrain matches centralized accuracy within 0.5%, while FedAvg drops by ~3% due to local overfitting. Removing gradient forecast causes divergence after 500 steps (accuracy falls to 45%). Removing weight reconciliation increases staleness indefinitely, leading to 12% higher loss. 7. Discussion Why does SimulTrain work? The key is the forecast+reconciliation loop. Forecast reduces bias, reconciliation prevents catastrophic staleness. The pipeline ensures that both edge and cloud are always busy, achieving near-optimal utilization.
Simultrain Solution 【iPad】
[ \tilde\nabla_k = \nabla \ell(w^(e)_k; x_k) + \alpha \cdot (w^(c)_k - w^(e)_k) ]
SimulTrain reduces latency by 78% on 4G and 71% on 5G compared to SyncSGD. FedAvg hides latency via local steps but suffers from model drift. | Method | Upload per step (KB) | Download per step (KB) | |----------------|----------------------|------------------------| | Centralized | 7,500 (video frame) | 75 (weights) | | SyncSGD | 75 (gradients) | 75 (weights) | | SimulTrain | 30 (activations) | 75 (delta weights) | simultrain solution
SimulTrain matches centralized accuracy within 0.5%, while FedAvg drops by ~3% due to local overfitting. Removing gradient forecast causes divergence after 500 steps (accuracy falls to 45%). Removing weight reconciliation increases staleness indefinitely, leading to 12% higher loss. 7. Discussion Why does SimulTrain work? The key is the forecast+reconciliation loop. Forecast reduces bias, reconciliation prevents catastrophic staleness. The pipeline ensures that both edge and cloud are always busy, achieving near-optimal utilization. [ \tilde\nabla_k = \nabla \ell(w^(e)_k; x_k) + \alpha