Streaming DiLoCo in MaxText Trains LLMs Across Metros
Streaming DiLoCo in MaxText lets TPU slices in different datacenters train one LLM, with a 9.57x speedup over standard data parallelism in tests.
Streaming DiLoCo in MaxText lets TPU slices in different datacenters train one LLM, with a 9.57x speedup over standard data parallelism in tests.