Accelerating Physical Systems with Imagination Models

Abstract

Cutting-edge experimental systems comprise carefully assembled, finely tuned collections of interdependent components. As complexity grows, maintaining optimal performance manually becomes a bottleneck, especially for remotely deployed systems. Sample-efficient automation is therefore essential for scalability. While reinforcement learning promises a general solution to autonomous control, practical adoption is limited by extensive training needs or the development of accurate simulations.

In this work, I introduce a simplified model-based reinforcement learning scheme that can be pre-trained from existing datasets, featuring a completely task-agnostic algorithmic framework. It employs generative models to imagine potential outcomes before real-world execution, reducing the number of physical interactions required. I have demonstrated its efficacy in two distinct experimental challenges. In an optical resonator, it autonomously aligns and mode-matches an input laser beam using precise actuation of lenses and mirrors. Despite drift and noisy feedback, the method attains human-level efficiency in a few steps starting from arbitrary misalignment. In a silicon quantum-dot system, it tunes fourteen gate and timing parameters to optimise spin-parity initialisation and readout. Tested over several days under varied conditions, the method rapidly recalibrates the system, even across unseen dot pairs, thereby illustrating its adaptability and scalability.

Additional studies illustrate the framework's extension to sequence optimisation, virtualising interactions with a cold atomic system and exploration of its learnt representations of task-relevant features. Collectively, these results highlight the framework's potential to accelerate progress in emerging quantum and space technologies by minimising manual interventions and establishing scalable automation. Show more