Topology design for distributed training
Tree, torus, dragonfly — and what comes next when wavelength multiplexing and reconfigurable switches change the cost model.
Area 02
Photonic Data Center Networks
Wiring up thousands of accelerators with light.
Philosophy
Training a frontier AI model requires thousands of GPUs working together. The bottleneck is no longer compute — it's communication. Electrical interconnects between accelerators burn energy and limit bandwidth; as model parallelism deepens, the network becomes the dominant cost.
We design photonic interconnects and network topologies for warehouse-scale AI training and inference, asking how the physical layer should change when the workload is distributed transformer training, mixture-of-experts inference, or large graph neural networks. The interesting questions aren't just "more bandwidth, less energy" — they're about co-designing the network and the algorithm so the whole system gets faster and greener at once.
Open questions
- What's the optimal topology for a photonic AI training fabric?
- How do photonic interconnects change the algorithmic design of distributed training?
- Can we co-design the model and the network for energy-optimality?
- How should scheduling, all-reduce, and pipelining strategies change when bandwidth is cheap but reconfiguration is slow?
Foundational reading
Where this thread started
- Dang et al., BiGNoC: Accelerating Big Data Computing with Application-Specific Photonic Network-on-Chip Architectures, IEEE TPDS.
- Dang et al., ConvLight: A Convolutional Neural Network Accelerator with Photonic Networks.
- Dang et al., LiteCON: An All-Photonic Neuromorphic Accelerator for Energy-efficient Deep Learning.
- Dang et al., SUSTAINPHOT: Sustainable Large-Scale AI Training Using Analog Silicon Photonic Accelerators, ICCD 2025.
- Hamilton et al., research on photonic networks-on-chip and dense WDM interconnects.
[Topology diagram — pending]
Sub-themes & ongoing projects
Co-packaged optics for AI
Bringing the optical I/O into the accelerator package itself: latency, density, and the system-level energy budget.
Carbon-aware exascale AI
SUSTAINPHOT — modeling and minimizing the carbon footprint of training runs across geographically-distributed photonic fabrics.
Hardware-aware distributed training
How collective-communication primitives (all-reduce, all-to-all) should be redesigned for a fabric where bandwidth is plentiful but topology is sparse.