← ProjectsIndustry

Reinforcement learning for Formula 1 race strategy

Internship at the Alfa Romeo F1 Team ORLEN, improving an AI agent that recommends live race-strategy decisions during Grands Prix.

Role: Reinforcement learning intern
With: Strategy team, Sauber Motorsport AG
Tools: Python, PyTorch, TensorFlow

Context

During a Grand Prix, strategy decisions — when to pit, which tyre compound to fit, how to react to safety cars — must be made in seconds and under heavy uncertainty. From September 2022 to February 2023 I worked at the Alfa Romeo F1 Team ORLEN (Sauber Motorsport AG, Hinwil) on an AI agent supporting live race strategy.

What I did

Simulation environment. Developed the race simulation environment in which the agent is trained and evaluated.
Convergence in a stochastic setting. Studied how reinforcement learning algorithms converge when the environment is highly stochastic, as a race is.
Algorithms. Implemented F1-specific versions of state-of-the-art algorithms: DQN, PPO and SAC.
Engineering. Led the migration of the code base from TensorFlow to PyTorch.

Takeaway

A project where research-grade methods had to work inside an engineering team, on real constraints and real deadlines.