On the Role of Simulation in Reinforcement Learning Evaluation

Simulation gives us a controlled environment for testing policies under rare events, uncertain arrivals, and structural changes.

A policy is more than its training reward

A learned policy can appear effective under one training configuration while remaining fragile to changes in arrival intensity, service variability, or cost assumptions. Simulation makes those assumptions explicit and permits structured stress tests.

Evaluation should be designed

Useful evaluation includes multiple random seeds, warm-up periods, independent test replications, analytical checks where possible, and comparisons to interpretable baselines. The objective is not merely to show that learning occurred, but to understand where a policy remains dependable.