DRL Traffic Signal Controls with Optimized CO2 Emissions
This IROS paper extends reward-shaped traffic signal control to multi-intersection networks with direct CO2 measurement embedded in the training objective, demonstrating that multi-objective optimization scales beyond single intersections. The work applies an eval-first methodology: explicit emissions metrics in training, reproducible SUMO simulation benchmarks in evaluation. It illustrates how real-world constraints - regulatory emissions targets - can be encoded directly into AI policy optimization rather than post-hoc filtered.
Standard DRL-based traffic signal control optimizes for throughput metrics - queue length, waiting time, average speed. These correlate with emissions but do not capture them directly: a signal timing that minimizes queue length may still produce high emissions if it creates frequent stop-start cycles.
Extension to Multi-Intersection Networks
This paper extends EcoLight (NeurIPS Workshop 2021) from single-intersection reward shaping to coordinated multi-intersection control. The key challenge: in a network, optimizing one intersection's signal timing affects upstream and downstream queue dynamics. A purely local reward signal leads to locally optimal but globally suboptimal policies.
Agent design: Each intersection is controlled by an independent DRL agent (PPO) observing local queue lengths, phase timing, and vehicle counts per lane. Agents share a policy network but maintain independent state. A coordination term in the reward penalizes patterns that optimize locally while creating congestion pressure on neighboring intersections.
Reward shaping: The reward combines a throughput term (negative queue length) with a CO2 penalty derived from per-vehicle HBEFA emissions estimates integrated into the SUMO simulator. The weighting between terms is an explicit hyperparameter, allowing operators to trade off throughput against emissions at configuration time rather than training time.
Results
Evaluated on a network of 9 intersections calibrated to a real Vancouver arterial corridor. The CO2-aware policy achieves 14% reduction in emissions compared to the throughput-only baseline, with a 4% reduction in throughput - a quantified, explicit trade-off rather than an unintended side effect.
The emissions reduction transfers to out-of-distribution traffic densities (rush hour patterns not seen during training), indicating the policy learned a general emissions-reduction principle rather than overfitting to training scenarios.