
This repository contains the code for the paper **Carbon-Aware Dynamic Parallelism for Distributed Training**, which studies how to adjust data, tensor, and pipeline parallelism choices during large model training based on carbon intensity.

The core idea is to treat parallelism as a scheduling decision, not just a throughput decision. When grid carbon intensity is high, the scheduler can reduce active data parallelism and retune the 3D parallelism configuration to lower emissions. When the grid is cleaner, it can scale back up to recover training progress.

## What It Includes

- Carbon-aware scheduling experiments for distributed training workloads
- Search over data, tensor, and pipeline parallelism configurations
- Comparisons against fixed and simpler carbon-aware baselines
- Evaluation code for emissions, runtime, and carbon efficiency tradeoffs

## Paper

Wafik Aboualim and Stephen Lee. **Carbon-Aware Dynamic Parallelism for Distributed Training**. ACM Energy Informatics Review, HOTCARBON 2026 proceedings, Volume 6, Issue 2, July 2026.

## Repository

[Wafik20/scheduling-carbon-aware-3d-parallelism](https://github.com/Wafik20/scheduling-carbon-aware-3d-parallelism)


<p>This repository contains the code for the paper <strong>Carbon-Aware Dynamic Parallelism for Distributed Training</strong>, which studies how to adjust data, tensor, and pipeline parallelism choices during large model training based on carbon intensity.</p>
<p>The core idea is to treat parallelism as a scheduling decision, not just a throughput decision. When grid carbon intensity is high, the scheduler can reduce active data parallelism and retune the 3D parallelism configuration to lower emissions. When the grid is cleaner, it can scale back up to recover training progress.</p>
<h2>What It Includes</h2>
<ul>
<li>Carbon-aware scheduling experiments for distributed training workloads</li>
<li>Search over data, tensor, and pipeline parallelism configurations</li>
<li>Comparisons against fixed and simpler carbon-aware baselines</li>
<li>Evaluation code for emissions, runtime, and carbon efficiency tradeoffs</li>
</ul>
<h2>Paper</h2>
<p>Wafik Aboualim and Stephen Lee. <strong>Carbon-Aware Dynamic Parallelism for Distributed Training</strong>. ACM Energy Informatics Review, HOTCARBON 2026 proceedings, Volume 6, Issue 2, July 2026.</p>
<h2>Repository</h2>
<p><a href="https://github.com/Wafik20/scheduling-carbon-aware-3d-parallelism">Wafik20/scheduling-carbon-aware-3d-parallelism</a></p>


Code for the paper "Carbon-Aware Dynamic Parallelism for Distributed Training" (HOTCARBON 2026)