Content Generation for Open-World Physics-Based Environments

Abstract

Researchers are driven by the aspiration to develop AI systems that can seamlessly operate in the real physical world, aiming to enhance human life through automation and assistance. However, such AI systems encounter an array of challenges. Foremost among these hurdles is the essential need for physical reasoning capabilities, enabling the AI to make informed decisions regarding the behaviour of objects under the influence of physics. Furthermore, the open nature of the real world means that novel and unforeseen situations frequently arise, necessitating AI to possess adaptive capabilities in order to thrive in such dynamic environments. In this research, we propose various techniques to design and generate physics-based content, which serves as an experimental and evaluative platform to facilitate addressing the above challenges faced by those AI systems.

Frequently, simulation environments are used by researchers to solve complex real-world problems as they offer controllable environments for experiments. Our study centres on Angry Birds, a physics-simulating puzzle game, to present various approaches for designing and generating tasks to facilitate the development of the aforementioned AI systems. However, generating physics-based content tailored for AI evaluation poses challenges, demanding physical reasoning and intricate design to meet evaluation requirements. Regrettably, existing physics-based content generation methods lack the specificity of producing content for evaluating AI systems in open-world physical environments.

In this thesis, firstly, we propose techniques to design and generate content for evaluating the physical reasoning capabilities of AI systems. These techniques involve generating tasks capable of deceiving AI systems in physics-based environments, crafting tailored tasks to measure the physical reasoning intelligence of agents in comparison with human intelligence, and developing methods to generate tasks based on the causal physical interactions between objects in the environment. Secondly, we discuss methods to design and generate physics-based tasks that incorporate novel situations. These works include creating testbeds with tasks to measure AI systems' performance in open-world physics-based environments and generating tasks with detectable and adaptable novelties for AI agents, allowing confident evaluation of their capabilities. Through the application of our research findings, we aim to foster the development of AI systems that can effectively and efficiently thrive in open-world physics-based environments.