«Abstract. Using Punch and Judy as a story domain, we describe an interactive puppet show, where the ﬂow and content of the story can be inﬂuenced ...»
An interactive, generative Punch and Judy show
using institutions, ASP and emotional agents
Matt Thompson1,2, Julian Padget1, and Steve Battle2
Department of Computer Science,
University of Bath, UK
Bristol & Bath Science Park, Bristol BS16 7FR
Abstract. Using Punch and Judy as a story domain, we describe an interactive
puppet show, where the ﬂow and content of the story can be inﬂuenced by the
actions of the audience. As the puppet show is acted out, the audience reacts to events by cheering or booing the characters. This affects the agents’ emotional state, potentially causing them to change their actions, altering the course of the narrative. An institutional normative model is used to constrain the narrative so that it remains consistent with the Punch and Judy canon. Through this vignette of a socio-technical system (STS), comprising human and software actors, an institutional model – derived from narrative theory – and (simplistic) technological interaction artifacts, we begin to be able to explore some of the issues that can arise in STS through the prism of the World-Institution-Technology (WIT) model.
1 Introduction Agent-based approaches for interactive narrative generation use intelligent agents to model the characters in a story. The agents respond to the interactions of a player with dialogue or actions ﬁtting the shape of a story. However, these agents have little autonomy in their actions, bound as they are to the strict requirements of their role in the narrative.
Other approaches to balancing authorial control with player or character agency include the use of director agents , reincorporation of player actions back into the narrative  and mediation to prevent narrative-breaking actions .
An institutional model can be used as a normative framework for governing the actions of agents in a story. By describing the rules of a narrative in terms of social expectations, the agents are encouraged to perform certain types of actions while still remaining free to break free of these expectations. As in society in the real world, breaking agreed norms comes with consequences, and only generally happens in exceptional circumstances. One situation where this may reasonably occur is when agents experience emotions. An agent experiencing an extreme emotion – in respect of some emotional model – such as rage or depression, may be allowed to act unusually or uncharacteristically. Allowing characters to break with the norms of the narrative enables them to An earlier version of this work was presented at the AI & Games symposium, held as part of AISB 2015, Canterbury, UK.
be ‘pushed too far’ by circumstances, with results that can add an extra dimension of richness to the telling.
There are two novel aspects to the approach we describe here: (i) the use of an institutional model to describe a narrative ‘world’ or domain, and (ii) how emotional models can give intelligent agents an alternative form of autonomy – from being limited by knowledge, reasoning capacity and time (aka bounded-rationality) and self-interest – both to act in idiosyncratic ways and to react emotionally to input from the audience.
Here we present an implementation in the form of an interactive Punch and Judy puppet show, in which the course of the story changes in accordance with the responses that come from the audience.
The structure of the system takes cues from the WIT model  which offers a pattern for analysing socio-technical systems through the interaction of three views of a system: (i) the world view (W), as human and software agents see it: in this case, the audience (human) and the actors (BDI agents) playing roles in Punch and Judy, (ii) the institutional view (I) that sets out the regulation of the system: in this case the narrative structure corresponding to Punch and Judy, captured in terms of Propp’s  story moves and roles, and (iii) the technological view (T) that identiﬁes the components (software and hardware) that enable the realization of the system, in this case, the means to capture audience input and the visualization of the performance. More importantly, WIT emphasises the role of the institution both as regulator and monitor of behaviour, which is exactly what we see in our system, since through permission and obligation it directs the actors towards the conclusion of the narrative, while also observing their actions for adherence to the narrative structure.
The puppets in the show are each realised by belief-desire-intention (BDI) agents augmented with a valence, arousal, dominance (VAD) emotional model, which we describe in section 4. The story is modelled by a set of institutional norms (section 6.1) that describe the Punch and Judy story domain in terms of Propp’s ‘story moves’  (section 2). The agents communicate with their environment using the Bath Sensor Framework , described in section 6.3. In the ﬁnal sections, we describe the animation system that functions both as the agents’ environment (section 6.3), and the means for the audience to interact with the system (section 7). The focus here is on the more technical aspects of the system and how the various components ﬁt together, while more detail on the narrative side appears in .
2 Propp’s story moves and roles
To express story events as an institution, we need some sort of formalisation for the analysis of the story – rather than an arbitrary selection of features – and so we look to narrative theory for inspiration. Instead of describing parts of the Punch and Judy story explicitly (such as ‘Punch is expected to hit the policeman in this scene’), it is desirable to describe scenes in a more
way using roles (‘The villain ﬁghts the victim in this scene’). The use of more general story fragments allows us to reuse them in multiple scenes, or even in other stories.
Narratology, and structuralism in particular, supply such generalised building blocks for stories. Russian formalism is an early movement in narrative theory that sought to formalise the elements of narrative, and Vladimir Propp was a prominent ﬁgure in this school. One outcome of this movement was Propp’s 1928 formalism derived from the study of Russian folktales, The Morphology of the Folktale , which is what we use to build a model to direct the course of the narrative. In this formalism, Propp identiﬁes recurring characters, which become roles, and motifs, which become action fragments, in Russian folklore, distilling them down to a concise syntax with which to describe stories. Propp’s functional, event-driven style translates comfortably to an institution comprised of event-based norms. However, while these action fragments ﬁt the Punch and Judy story adequately, we note that the role labels can sound rather awkward because of the apparent semantic import of the textual label.
In Propp’s formalism, characters have roles, such as hero, villain, dispatcher, false hero, and more. Characters performing a certain role are able to perform a subset of story moves, which are actions that make the narrative progress. For example, the dispatcher might send the hero on a quest, or the victim may issue an interdiction to the villain, which is then violated.
Propp deﬁnes a total of 31 distinct story functions. Each function is given a number and symbol in order to create a succinct way of describing entire stories. Examples of
such functions are:
– One of the members of a family absents himself from home: absentation.
– An interdiction is addressed to the hero: interdiction.
– The victim submits to deception and thereby unwittingly helps his enemy: complicity.
– The villain causes harm or injury to a member of the family: villainy.
Each of these functions can vary in subtle ways. For example, the villainy function can be realised as one of 19 distinct forms of villainous deed, including the villain abducts a person, the villain seizes the daylight, and the villain makes a threat of cannibalism.
These functions are enacted by characters following certain roles. Each role (or dramatis persona in Propp’s deﬁnition) has a sphere of action consisting of the functions that they are able to perform at any point in the story. Propp deﬁnes seven roles each of which has distinct spheres of action: villain, donor, helper, princess, dispatcher, hero, and false hero. In a typical story, one story function will follow another as the tale progresses in a sequential series of cause and effect. However, Propp’s formalism does also allow for simultaneous story functions.
2.1 Propp example: sausages and crocodile scene To provide some context for Punch and Judy, since it is a peculiarly British phenomenon,
although with Italian origins, we quote from Wikipedia:
Punch and Judy is a traditional, popular, and usually very violent puppet show featuring Mr Punch and his wife, Judy. The performance consists of a sequence of short scenes, each depicting an interaction between two characters, most typically Mr Punch and one other character (who usually falls victim to Mr. Punch’s club). It is often associated with traditional British seaside culture. The Punch and Judy show has roots in the 16th-century Italian commedia dell’arte.
http://en.wikipedia.org/wiki/Punch_and_Judy, retrieved 2015-05-06.
The common elements of Punch and Judy are easily described in terms of Propp’s story functions. Here we pick one scene to use as an example: the scene where Punch battles a crocodile in order to safeguard some sausages. In this scene, Joey the clown (our narrator) asks Punch to guard the sausages. Once Joey has left the stage, a crocodile appears and eats the sausages. Punch ﬁghts with the crocodile, but it escapes. Joey then
returns to ﬁnd that his sausages are gone. The corresponding story functions are:
1. Joey tells Punch to look after the sausages (interdiction).
2. Joey has some reservations, but decides to trust Punch (complicity).
3. Joey gives the sausages to Punch (provision or receipt of a magical agent).
4. Joey leaves the stage (absentation).
5. A crocodile enters the stage and eats the sausages (violation).
6. Punch ﬁghts with the crocodile (struggle).
7. Joey returns to ﬁnd that the sausages are gone (return).
Some story functions map to Punch and Judy better than others (for example, it is debatable as to whether or not the sausages can be considered a “magical agent”), but Propp’s formalism seems well suited to Punch and Judy for the most part. The advantage of using Propp for the Punch and Judy story domain is that the story function concept maps well to the idea of internal events in institutional models.
3 Institutional model An institution describes a set of ‘social’ norms describing the permitted and obliged behaviour of interacting agents. Noriega’s ‘Fish Market’ thesis  describes how an institutional model can be used to regiment the actions of agents in a ﬁsh market auction.
Several [2, 7, 5] extend this idea to build systems where institutions actively regulate the actions of agents, while still allowing them to decide what to do. We build on the work of Cliffe et al.  and Lee et al.  to adapt it for the world of narrative, using an institutional model to describe the story world of Punch and Judy in terms of Propp’s story moves and character roles.
Institutional models use concepts from deontic logic to provide obligations and permissions that act on interacting agents in an environment. By combining this approach with Propp’s concepts of roles and story moves, we describe a Propp-style formalism of Punch and Judy in terms of what agents are obliged and permitted to do at certain points in the story.
For example, in one Punch and Judy scene a policeman enters the stage and attempts to apprehend Punch. According to the rules of the Punch and Judy world, Punch has an obligation to kill the policeman by the end of the scene (as this is what the audience expects to happen, having seen other Punch and Judy shows). The policeman has an obligation to try his best to catch Punch. Both agents have permission to be on the stage during the scene. The policeman only has permission to chase Punch if he can see him (Punch is obliged to hide from him at the start of the scene).