Offline Reinforcement Learning on a Real-World Power Grid Control Problem

📋 Type Project
⚡ Status finished
📅 Duration Mar 10, 2026 – Jun 10, 2026
👤 Primary supervisor Julian Oelhaf
👥 Co-supervisors Siming Bayer Andreas Maier
🎓 Student Alexander Luce Artificial Intelligence, M.Sc.

Abstract

This project investigates whether offline reinforcement learning can isolate a fault in a simulated CIGRÉ medium-voltage distribution network. A Conservative Q-Learning agent learns from simulated fault and non-fault episodes to decide when to wait and which line to trip. The study compares raw voltage and current waveforms, features derived from phasors and apparent impedance, and a combination of both. It also examines how the observation window, reward design and strength of conservative regularization affect the agent’s decisions. Combining waveforms with electrical features gave the highest per-time-step F1 score among the configurations tested.

Best Paper Award · IEEE PESS 2026

The paper that grew out of this project received the Best Paper Award at the IEEE Power & Energy Student Summit 2026 in Karlsruhe.

Offline Reinforcement Learning for Distribution-Grid Protection
Julian Oelhaf, Alexander Luce, Christian Bergler, Andreas Maier and Siming Bayer

The paper evaluates the agent on 225 held-out simulated episodes. The best configuration achieved a per-time-step F1 score of 0.974. To assess how the agent would act as a relay, the authors also examined its first trip in each episode. The default agent using both waveforms and electrical features selected the correct line on its first trip in 98.1% of 214 fault episodes. It also issued an unwanted trip in 8 of 11 non-fault episodes. The result illustrates why protection studies should report unwanted trips alongside classification scores.

Preprint (arXiv) · Conference programme