Bayesian well-made surprises: theory, modelling and insight

Abstract

Good narrative surprise is highly sought after for its entertainment value and insight-like characteristics, and plot twists are the underpinning of entire genres, from long-form detective fiction to short jokes and humorous narratives. Although we understand many writing techniques behind successful surprises and the psychological mechanisms they leverage in the audience, a large part of the enjoyability of narrative surprise is highly subjective and seems to evade analysis. This itself is unsurprising, but furthermore it's unclear what aspects of surprise are formally modellable, and to what extent a formal approach can be applied to the analysis of narrative surprises. Existing formal models of narrative surprise, mainly from computational narrative, are primarily concerned with generating novel surprising narratives, rather than analysing existing ones, and have rarely been used outside of computational narrative research.

In this work, we first outline formal requirements for a model of narrative surprise that is suitable for both analytical and generative purposes, and for use by a cross-disciplinary audience. We then propose a formal model of narrative surprises, based on operationalisations of narrative theory into computer science formalisms. Our proposed model is based on Markov Logic Networks (MLNs), which encode audience knowledge and inferences throughout the narrative into first-order logic. By operationalising the qualities of ``well-madeness'' of a surprise into functions of the MLNs' outputs, our model produces estimates of those qualities.

We evaluated the performance of our model by quantitatively comparing its outputs to ratings of the same qualities collected from participants in two studies. We found that when comparing two variants of the same narrative that differ in a given quality, our model's outputs change between the two variants in a way that is consistent to the ratings of the human participants, in particular for the qualities of consistency and divergence (a kind of surprisingness). We also qualitatively evaluated the model's adherence to the requirements we previously outlined, and found that it satisfies several, being versatile and expressive.

Our work lays foundations for future research into formally modelling narrative surprises by contributing on multiple levels: a set of requirements that other formal models of surprise can be designed around and evaluated with, a prototype model of narrative surprise, a small curated corpus of short narratives with surprises, and an experimental scale for measuring key qualities of ``well-madeness'' of a surprise. Our results suggest that while the well-madeness of a narrative surprise is heavily subjective, it is nonetheless related to patterns in the audience's knowledge and inferences about the narrative, and that some of those patterns can be formally modelled and computationally analysed to further our understanding of the narrative, and of narrative surprises as a whole. Show more