Comprehensive Research Dossier

Simon Hirländer: Theoretical Physics, Autonomous Systems, and Industrial Artificial Intelligence

---

Executive Summary

This dossier presents an exhaustive analysis of the research profile, academic trajectory, and industrial activities of Simon Hirländer. Currently serving as the Head of the "Smart Analytics & Reinforcement Learning" (SARL) team at the Intelligent Data Analytics (IDA) Lab at the University of Salzburg, Simon Hirländer occupies a unique niche at the convergence of high-energy theoretical physics and modern artificial intelligence.

Simon Hirländer’s career is characterized by a "theory-to-application" pipeline. His foundational work involves rigorous mathematical physics, specifically the modeling of electromagnetic interactions in particle accelerators using complex Green functions, which he developed during his doctoral tenure at CERN and TU Wien. This theoretical bedrock has evolved into a specialized focus on Reinforcement Learning (RL) for the control of complex, non-linear dynamical systems: in 2018/19 he designed and carried out the first deep reinforcement learning experiment at CERN, on the Low Energy Ion Ring (see Section 3.1). He is a pivotal figure in the movement towards "Autonomous Accelerators," leading international collaborations to automate the world’s largest machines, such as the Large Hadron Collider (LHC) and the AWAKE experiment.

Beyond academia, Simon Hirländer has established a robust portfolio of industrial technology transfer. He applies the safety-critical, high-precision control methods developed for proton synchrotrons to diverse industrial sectors, including dental medical technology (W&H), steel manufacturing (Danieli), and energy optimization (Copa-Data). This report details his methodological innovations, including Model-Based Reinforcement Learning (MBRL) and Deep Meta-RL, and documents his extensive footprint in teaching and community building through initiatives like the "RL Coffee" and international bootcamps.

1. Academic Identity and Professional Stature

1.1 Current Institutional Roles and Leadership

Simon Hirländer’s professional identity is anchored in dual affiliation, bridging the academic environment of Salzburg with the high-energy physics community in Geneva.

• Primary Academic Appointment: He holds a Postdoctoral position and serves as the Team Lead (Head) for "Smart Analytics & Reinforcement Learning" (SARL) within the Department of Artificial Intelligence and Human Interfaces (AIHI) at the University of Salzburg.
• IDA Lab Management Structure: The Intelligent Data Analytics (IDA) Lab is a competence center for data science. Simon Hirländer serves as a Vice Director of the laboratory and leads one of its core research units (SARL). The directorate also includes Prof. Wolfgang Trutschnig (Director) and Profs. Christian Borgelt and Roland Kwitt (Vice-Directors).
• International Scientific Appointment: He has been a Visiting Scientist at CERN since 2022 (European Organization for Nuclear Research), ensuring that his AI methodologies are continuously tested on the world’s most advanced particle accelerators.

1.2 Educational Pedagogy and Foundations

Simon Hirländer’s transition into Artificial Intelligence is distinguished by his background in rigorous theoretical physics. Unlike data scientists who approach problems purely from a statistical perspective, Hirländer approaches AI from the perspective of dynamical systems and field theory.

The continuity in his education, from Green functions in capacitors (MSc) to Green functions in particle accelerators (PhD), demonstrates a long-standing specialization in modeling electromagnetic fields, which he later learned to control using neural networks.

2. Doctoral Research: Theoretical Physics and Electromagnetism

To understand the sophistication of Simon Hirländer’s AI architectures, one must first analyze the complexity of the physical problems he solved analytically during his doctoral studies. His PhD thesis, Exact Solutions of Indirect Transverse Field Effects in Elongated Structures, addresses one of the fundamental limits of high-intensity particle accelerators: the "Indirect Space Charge Driven" (ISCD) effect.

2.1 The Problem: Indirect Space Charge

In a particle accelerator like the LHC or the Proton Synchrotron (PS), a beam of charged particles travels at near-light speeds inside a vacuum chamber (the beam screen). The beam carries a strong electromagnetic field. This field interacts with the metallic walls of the vacuum chamber, inducing image currents and charges. These induced charges, in turn, create a secondary electromagnetic field that acts back on the beam. This "indirect" interaction can cause the beam to become unstable, leading to a shift in the betatron tune (the frequency of the particle's oscillation) and potentially causing beam loss.

Simon Hirländer’s research identified that an appropriate quantitative explanation for the ISCD tune-shift, which requires constant correction during LHC operation, was missing from the standard models. His work specifically identified the electric interaction with the beam screen as the primary origin of this tune shift.

2.2 Methodological Innovation: The Riemann-Sphere Operator

Modeling these interactions usually requires computationally expensive numerical simulations (Finite Element Method). However, for real-time control and deep understanding, analytical (closed-form) solutions are superior. Simon Hirländer developed a novel mathematical framework to derive these solutions.

• Complex Green Functions: He utilized complex analysis to represent the electrostatic potentials. Green functions are impulse response functions used to solve inhomogeneous differential equations subject to boundary conditions (like the walls of the LHC beam pipe).
• The Lorentz Operator on the Riemann Sphere: A central contribution of his thesis was the derivation of a "novel Lorentz operator of the Green functions on the Riemann-sphere (RS)". By mapping the complex geometry of the accelerator chamber onto the Riemann sphere, he could simplify the calculation of the electromagnetic fields.
• Image Operators: From the Lorentz operator, he derived "image operators" for arbitrary beam distributions. This allowed for the estimation of the ISCD tune-shift in complex accelerator models without relying solely on numerical integration.

2.3 Application to the LHC and PS

The practical application of this theoretical work was immediate and critical for CERN:

• LHC Beam Screen: He proved a novel method to approximate the fundamental electrostatic field of arbitrary simply-connected domains, applying it to the "rect-elliptical" geometry of the LHC beam screen. This closed-form model provided agreement with measurements with "unprecedented accuracy" and is applicable to the future High Luminosity (HL)-LHC, where these effects will be double the current magnitude.
• Multi-Turn Extraction (MTE) in the PS: In the Proton Synchrotron, a process called Multi-Turn Extraction splits the beam into a main beam and four "islands" or beamlets. Measurements had shown an unexplained intensity dependence in the beamlet positions. Hirländer’s analytical calculations and simulations resolved this dependence as being caused by ISCD effects.

This body of work established Simon Hirländer as an expert in the underlying dynamics of the machines he would later seek to automate.

3. The Paradigm Shift: Reinforcement Learning in Particle Physics

While still completing his PhD (submitted February 2020), and as CERN Senior Fellow in the SPS Operations Group responsible for LEIR, Simon Hirländer shifted his focus from modeling these dynamics to controlling them using Machine Learning. This transition marks the beginning of the "Autonomous Accelerators" era in his career. His research argues that while analytical models (like his own) are powerful, the operational complexity of machines like the LHC, with thousands of drifting parameters, requires adaptive agents capable of learning from experience.

3.1 The First Reinforcement Learning Experiment at CERN (LEIR)

Simon Hirländer designed and carried out the first deep reinforcement learning experiment on a CERN accelerator. As machine supervisor of the Low Energy Ion Ring (LEIR), he had introduced numerical optimisation algorithms into LEIR operation in 2018, which contributed to the record Pb-ion intensity delivered by LEIR that year. The next step was reinforcement learning: he trained a Deep Q-Network (DQN) agent directly on the machine, with no prior knowledge, to steer the beam at LEIR injection.

• Set-up: action = change by ±Δ or hold of dipole BHN10 in the Linac3–LEIR transfer line; state = beam position at BPM60; reward = injected intensity at BCT10.
• Result: trained from scratch on LEIR in about 600 iterations (measured), the agent learned the task and recovered the optimum after a few moves in new situations.
• Sample efficiency: pre-training on data from a Powell optimiser run halved the required iterations; a deep double Q-learning (DDQN) variant cut them by a further factor of three. Implementation in Python (TensorFlow/Keras, OpenAI Gym interface).
• Follow-up: this work opened the RL programme at CERN, continued in 2019 with experiments at AWAKE and LINAC4 and published as Kain, Hirländer et al., “Sample-efficient reinforcement learning for CERN accelerator control”, Phys. Rev. Accel. Beams 23, 124801 (2020).

3.2 The "Sample Efficiency" Challenge

A recurring theme in Hirländer’s AI research is the problem of sample efficiency. In standard Reinforcement Learning (e.g., training an agent to play a video game), the agent can fail millions of times in a simulation. In a particle accelerator:

• Beam time is expensive: Running the LHC costs immense amounts of energy and money.
• Safety is critical: A "bad" exploration step by an AI agent could steer a high-energy beam into the wall, damaging sensitive superconducting magnets.

Therefore, Simon Hirländer’s research specifically targets RL methods that learn quickly and safely.

3.3 Model-Based vs. Model-Free RL (The FERMI FEL Study)

One of his seminal contributions is the comparative analysis of Model-Free (MFRL) and Model-Based Reinforcement Learning (MBRL) in accelerator environments.

• The FERMI FEL Study: In a key paper titled "Model-free and Bayesian Ensembling Model-based Deep Reinforcement Learning for Particle Accelerator Control Demonstrated on the Fermi FEL", Hirländer and colleagues (Niky Bruchon, et al.) investigated intensity optimization on the Free Electron Laser (FEL) at FERMI.
  • Methodology: They implemented a model-based algorithm in a "DYNA-style" architecture using an uncertainty-aware model (Bayesian ensembling). This was compared against a tailored Deep Q-Learning (Model-Free) approach.
  • Findings: The study found that the Model-Based approach demonstrated higher representational power and sample efficiency, making it superior for rapid tuning where data is scarce. However, the Model-Free method showed slightly superior asymptotic performance (it reached a better final solution given enough time).
  • Noise Robustness: Both algorithms were engineered to handle the significant noise omnipresent in accelerator diagnostics.

3.4 The AWAKE Project: Deep Meta-Reinforcement Learning

The AWAKE experiment (Advanced WAKEfield Experiment) is a frontier project accelerating electrons in plasma wakes created by protons. It is highly unstable and non-linear, making it the perfect testbed for Hirländer’s advanced RL concepts.

• Trajectory Steering: The core problem was steering the electron beam to match the proton beam's trajectory.
• Deep Meta-RL: In collaboration with S. Pochaba and C. Xu, Hirländer applied Deep Meta-Reinforcement Learning. "Meta-learning" is learning to learn. The goal was to create an agent that could adapt to "Linear Markov Decision Processes" that change over time (e.g., due to thermal drifts in magnets).
• Impact: This research demonstrated that Meta-RL agents could achieve rapid adaptation, correcting the beam trajectory in "few-shot" scenarios where traditional controllers would lag or require recalibration.
• Variational Auto-Encoders (VAE): Hirländer utilized VAEs to compress the high-dimensional input images from beam screens into a "latent space." This allowed the RL agent to make decisions based on simplified, essential features rather than raw noisy pixels, a crucial step for real-time control.

3.5 LHC Tune Feedback

In collaboration with Leander Grech and Gianluca Valentino, Hirländer applied RL to the LHC Tune Feedback system. The "tune" (oscillation frequency) is a critical parameter for stability.

• The Challenge: The LHC's tune drifts due to magnet hysteresis and decay.
• The Solution: They reimplemented the original feedback controller (QFB) and compared it against RL agents.
• Result: The RL agent's performance exceeded the classical controller-based paradigm in simulated environments (Grech et al., Frontiers in Physics 10, 929064, 2022).

4. Industrial Technology Transfer: From CERN to the Factory Floor

A distinguishing feature of Simon Hirländer’s profile is his aggressive pursuit of technology transfer. He leverages the "Safe RL" and "Control Theory" expertise developed for CERN to solve problems in Austrian and international industry. His industrial projects span medical technology, steel, energy, and logistics.

4.1 W&H Dentalwerk: Precision Control in Medical Devices

• Project Title: Regelungsoptimierung von Piezoantrieben.
• Timeline: December 1, 2025 – December 31, 2026.
• Partner: W&H Dentalwerk Bürmoos GmbH.
• Context: W&H is known for innovations like the "Primea Advanced Air" turbine.
• Research Focus: Piezoelectric drivers are used in dental scalers and surgical instruments for ultrasonic oscillation. These systems must maintain precise frequency and amplitude under varying loads (e.g., contacting a tooth vs. gum).
• The Control Application: Simon Hirländer’s team is applying advanced control strategies to create an adaptive controller for these drivers. The system assesses the non-linear response of the piezo-ceramic material and the load, adjusting the driving signal in microseconds to maintain cutting efficiency while minimizing heat generation: a classic control problem solved with modern methods.
• Additional Project: UNSAD (Unsupervised Anomaly Detection in Sterilisation Devices). This project (September 2023 – May 2024) focuses on ensuring the reliability of sterilization equipment using AI.

4.2 Danieli Automation: Heavy Industry & Steel

• Project Title: Danieli X - PLUS.
• Timeline: March 1, 2025 – October 31, 2025.
• Partner: Danieli Automation (Italy/Global).
• Context: Danieli builds steel plants, including rolling mills and blast furnaces.
• Research Focus: This "Research co-operation" aims to optimize steel production processes. Steel manufacturing involves extreme variables (temperature, viscosity of molten metal, roller speeds). Traditional automation relies on look-up tables and PID control. Hirländer’s work suggests the implementation of RL agents for process optimization, potentially to control the speed of rolling mills or the cooling curves of steel, reducing waste and energy consumption. The project involves Master's student Olga Mironova.

4.3 Copa-Data: Energy Optimization and Smart Grids

• Projects: DEOP 2.0 / 2.5: Dynamische Energieoptimierung (Dynamic Energy Optimization).
• FOCUS: Forecasting and optimization under constraints and uncertainty for sustainable industrial energy systems (2025–2028).
• Partner: Copa-Data (creator of zenon software platform).
• Research Focus: These projects address the global challenge of energy sustainability. Simon Hirländer is developing algorithms for Forecasting and Optimization under Uncertainty.
• Mechanism: Using RL and time-series forecasting, the system predicts energy demand (e.g., in a factory or smart building) and optimizes the usage of renewables or grid power. This effectively turns a factory into a "smart agent" that negotiates its energy consumption to minimize cost and carbon footprint.

4.4 Palfinger: Pattern Recognition in Machine Data

• Projects: MuMa, MuMa II and MuMa III (Mustererkennung Maschinendaten), 2021–2022, Principal Investigator for MuMa II and III.
• Partner: Palfinger AG (hydraulic lifting solutions).
• Research Focus: Pattern recognition in machine operating data.

4.5 Redlink GmbH: Large Language Models for Companies

• Project: LLMs für Unternehmen (Large Language Models for companies), FFG Innovationsscheck with Redlink GmbH as company partner.
• Timeline: May 2024 – July 2024.
• Context: Redlink specializes in Connected Data and Knowledge Graphs.
• Research Focus: Applying large language models to company knowledge and data; the results fed into a joint Redlink whitepaper.

4.6 STIWA Group: Manufacturing Automation

• Project: DS STIWA II (Data Science STIWA II).
• Timeline: November 2025 – April 2026.
• Partner: STIWA Group (High-performance automation).
• Research Focus: Led by Simon Hirländer as PI (following DS STIWA, where he was Co-I), this project focuses on assembly line optimization. PhD student Reuf Kozlica is working on "Hierarchical reinforcement learning in assembly line optimization" under this umbrella, using Hierarchical RL to manage the complex, multi-stage decisions required in high-speed manufacturing.

5. Methodological Innovations in AI

Simon Hirländer’s research is not merely the application of existing tools; it involves the development of new algorithmic variations tailored for physical systems.

• Python-Simulink Bridging: In the paper "Python-Based Reinforcement Learning on Simulink Models", Hirländer addresses a major engineering hurdle. Most industrial control systems are designed in MATLAB/Simulink, while modern RL is done in Python (PyTorch/TensorFlow). His team developed robust interfaces to allow Python agents to train on Simulink digital twins, facilitating the deployment of academic AI into legacy industrial environments.
• Probabilistic Deep Learning and Uncertainty: A core tenet of his philosophy is that AI must be "uncertainty aware." Across his work (e.g. the Bayesian-ensembling model-based RL on FERMI and Gaussian-process MPC with uncertainty-aware kernels), he develops methods to quantify the confidence of an RL agent. If an agent is "uncertain" about a state (e.g., an unknown beam configuration), it should act cautiously. This is implemented via Bayesian Neural Networks and Ensembling techniques, which are critical for safety certification in both CERN and industrial contexts.
• Multi-Objective Optimization: Real-world systems rarely have a single goal. In "Online multi-objective particle accelerator optimization of the AWAKE electron beam line", Scheinker, Hirländer et al. (AIP Advances 10, 055320, 2020) demonstrated online optimization of several conflicting beam objectives simultaneously on the AWAKE electron line.

6. The "RL4AA" Initiative and Community Leadership

Simon Hirländer is a central architect of the Reinforcement Learning for Autonomous Accelerators (RL4AA) international collaboration. This initiative was established to consolidate knowledge and share experiences among laboratories like CERN, DESY, KIT, and SLAC.

• Strategic Workshops: He has organized and hosted key events that define the research agenda for this community:
  • RL4AA'24 (Salzburg): Held at the University of Salzburg on 5–7 February 2024, this workshop brought together physicists and computer scientists to discuss the deployment of RL in control rooms.
  • RL4AA'25 (Hamburg): Co-organized for April 2025, continuing the momentum.
• The "RL Coffee": Beyond formal conferences, Hirländer organizes the "RL Coffee," a monthly meetup (first Friday of the month) that serves as a platform for researchers and industry professionals to exchange ideas. This informal yet rigorous setting fosters a continuous dialogue between theory and practice.
• Bootcamps and Education: Recognizing the skills gap in advanced AI, he runs Reinforcement Learning Bootcamps:
  • 1st Bootcamp (Sept 2024), 2nd Bootcamp (Sept 2025) and 3rd Bootcamp (2026) at the University of Salzburg.
  • Curriculum: Includes "Advanced Policy Gradients," "Actor Critics," and hands-on workshops like "Walk, Ant, Walk (PPO for Continuous Control)". These bootcamps serve to train the next generation of PhDs and engineers who will implement these technologies in the field.

7. Teaching and Mentorship Profile

Simon Hirländer’s influence extends to a new generation of data scientists through his teaching at the University of Salzburg and his supervision of diverse thesis topics.

7.1 Coursework

He teaches advanced modules that reflect his research interests:

• Advanced Reinforcement Learning and Agentic AI Systems
• Introduction into Deep Reinforcement Learning
• Advanced Topics in Reinforcement Learning
• Special Topics in Data Science
• Mathematical Foundations in Precision Medicine
• AI Bachelor Seminar

7.2 Student Supervision

His supervision portfolio highlights the breadth of his research interests:

• Olga Mironova: Master's student and project assistant in SARL, working on the Danieli steel automation project.
• Sarah Trausner: PhD student in FOCUS: Forecasting and optimization under constraints and uncertainty for sustainable industrial energy systems.
• Benjamin Halilovic: Applied Image/Signal Processing student, working on particle accelerator optimization.
• Reuf Kozlica (PhD): "Hierarchical reinforcement learning in assembly line optimization" (Industrial manufacturing focus).
• Georg Schaefer (PhD): "Reinforcement learning in cyber-physical systems."
• Juan Manuel Montoya Bayardo (PhD): "Reinforcement learning in nautical robotics" (Project NauROM).
• Raoul Kutil (PhD): "Knowledge Graphs in medicine."

8. Strategic Outlook and Future Directions

The trajectory of Simon Hirländer’s research points toward three emerging pillars:

• Sustainable Industrial AI (Green AI): Through the FOCUS grant (running until 2028), he is positioning RL as a key enabler of industrial sustainability. By optimizing energy grids and manufacturing processes, his algorithms directly contribute to carbon reduction.
• Nautical Robotics: The NauROM project indicates an expansion into maritime systems. In NauROM (with EvoLogics) his team works on occupation maps for nautical robots.
• Human-AI Symbiosis: Situated in the department of "Artificial Intelligence and Human Interfaces," his work increasingly considers how human operators interact with autonomous agents. His focus on "interpretability" (via Knowledge Graphs) and "uncertainty quantification" ensures that the AI systems of the future are trusted partners in the control room, not black boxes.

9. Conclusion

Simon Hirländer stands as an exemplar of the modern "Physicist-Engineer." His work effectively demystifies the application of Deep Reinforcement Learning in high-stakes environments. By proving the mathematical bounds of stability in his PhD (ISCD effects) and then developing the AI agents capable of navigating those bounds (Model-Based RL), he has created a cohesive research narrative.

His extensive industrial collaborations, from the precision of W&H dental tools to the massive scale of Danieli steel plants, validate the universality of his approach: that complex dynamical systems, whether subatomic or industrial, can be tamed through the intelligent, uncertainty-aware application of Reinforcement Learning.

Publication Highlights & References

• Automatic performance optimisation and first steps towards reinforcement learning at the CERN Low Energy Ion Ring (2nd ICFA ML Workshop, PSI, 2019), first deep RL experiment at CERN.
• Sample-efficient reinforcement learning for CERN accelerator control (Phys. Rev. Accel. Beams 23, 124801, 2020).
• Exact Solutions of Indirect Transverse Field Effects in Elongated Structures... (PhD Thesis, 2020).
• Model-free and Bayesian Ensembling Model-based Deep Reinforcement Learning... (arXiv:2012.09737, 2020).
• The Reinforcement Learning for Autonomous Accelerators Collaboration (IPAC, 2024).
• Deep Meta Reinforcement Learning for Rapid Adaptation... (SMPS 2024, Advances in Intelligent Systems and Computing, Springer).
• Multi-Objective Reinforcement Learning for Energy-Efficient Industrial Control (DEXA 2025 Workshops, CCIS, Springer).