The Relationship Between Audio and Immersion in Video Games

One of the things I attempted to demonstrate in a recent work in progress presentation was how flawed approaches to speech in video games can disrupt player immersion. I used the following videos to provide examples of how poor recreation of the human voice can intrude upon the player's experience.

Examples of Speech & Voice Issues in Video Games from Liam Donnellan on Vimeo.


Examples of In-Game Speech Issues in Sports Video Games from Liam Donnellan on Vimeo.

What I failed to communicate during the presentation is why player immersion is so important in the first place. This post intends to briefly outline the importance of immersion in games and discuss the contribution of audio to the phenomenon.

What is Immersion?

Before discussing the importance of immersion in video games, it is crucial to define the term. Garneau (2001)  summarises: 'Immersion covers the pleasures of being in a different environment than usual... by physical means or by use of one's imagination.'. Concordantly, Huiberts (2010) notes: 'Most people who have ever played a computer game are likely to have experienced certain feelings of being absorbed by it... most commonly called "immersion"...'. For the purposes of this blog post then, immersion can be defined as the psychological state a player enters while playing a game, in which they imagine themselves to be part of a game's environment.

Why is Immersion Important?

Woyach (2008) notes that immersion '[lowers] a person's need to suspend their disbelief* by removing the text, the seat, or the keyboard, placing the person in the scene itself... ...until [the game environment] is as real to us as everyday life'. In role-playing games, where stories are often set in other-worldly environments, it's clear to see why a suspension of disbelief on the player's part could be useful. In the case of sports games I would argue that a suspension of disbelief is required for the player to attach any meaning to his/her actions within the game. This would be particularly true in PDC...'s 'Career Mode' in which the player can create a custom made professional darts player and compete with the real world professionals in various competitions. 

* A semi-conscious decision in which a player puts aside their disbelief and accept a premise as being real for the duration of a given story. (Definition from MediaCollege.com)


Of interest: Results of a user survey by Huiberts (2010) show that most players (at least 72% in this case) like to be immersed into games.


The Role of Audio

Taylor (2002) notes 'Diegetic immersion requires that the game have a consistent world, so that the player is not forced from immersion by inconsistencies of the game space'. It is therefore required of a games audio to help create and maintain a consistent reality if a player is to become effectively immersed. Before attempting to aid immersion with game audio however, it is fruitful to analyse scenarios where audio can disrupt the phenomenon. Huiberts (2010) identifies three dimensions of immersion which can be disrupted by audio.

Sensory Immersion: '...concerns engagement with the sensory rewarding aspects of a game... ...often stimulate the feeling of being [in the game world]' See discussion below.

Challenge-based Immersion: '...concerns the engagement with a competitive process, problem solving, interacting with the game and competing or cooperating with others.' This is often cause by music [ video example pending] (Huiberts, 2010). 

Imaginative Immersion: '...concerns the engagement with the "imaginary world and fantasy, game characters, worlds and story line"...' This is demonstrated above in God of War, Duke Nukem and Lord of the Rings Online, as these issues specifically concern characters within an imaginary world.

In the Rugby League Live and PDC... examples above, the most prevalent disruption is of sensory immersion, specifically when the commentary systems in each game output unnatural speech and inappropriately selected speech topics respectively. These inadequacies make the player aware of the system, and result in technological listening* from the player, rather than the desired semantic listening*.

* Semantic listening and technological listening (coined by Smalley and Chion respectively) are terms used in the field of electroacoustic composition. Semantic listening is used for the purpose of gaining information about what is communicated in the sound (Chion, 1994) whereas technological listening occurs when a listener perceives the technology behind a sound rather than the sound itself (Smalley, 1997). I believe these terms are applicable to my area of research, where disruption of sensory immersion causes a (brief) switch from semantic (listening to the commentary for information) to technological listening (hearing the system behind the commentary, rather than the commentary itself).

Summary

Thus, it would appear that a game world's consistency is of key importance to aiding immersion. That is not to say a game's reality needs to be consistent with our own reality (Taylor, 2002). Demonstrating this, Nintendo franchises such as Pokémon bear little resemblance to the world we live in, yet keep young players playing for hours on end (Thomsen, 2010). Game audio can aid immersion in maintaining a game world's internal consistency. In the cases of sports simulations where the aim is to recreate the scenario of a given sport, audio also needs to be consistent with the real world, as players that are fans of the sport are likely to notice any unnatural or unbelievable output from audio systems.

References

Chion, M. (1994). AudioVision: Sound on Screen. Columbia University Press.  

Garneau, P.A. (2001). Fourteen Forms of Fun. Gamasutra. [Online] Available from: http://www.gamasutra.com/features/20011012/garneau_01.htm [Last Accessed 1/8/2011]

Huiberts, S (2010). The Role of Audio for Immersion in Computer Games. PhD thesis, Ultrecht School of the Arts, University of Portsmouth. [Online] Available from: http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf [Last Accessed 1/8/2011]

Media College (2011). Suspension of Disbelief. MediaCollege.com. [Online] Available from: http://www.mediacollege.com/glossary/s/suspension-of-disbelief.html [Last Accessed 1/8/2011]

Smalley, D. (1997). Spectromorphology: Explaining sound-shapes, Cambridge: Cambridge University Press.

Taylor, L. (2002). Video Games: Perspective, Point-of-View, and Immersion. PhD thesis, University of Florida. [Online] Available from: http://www.laurientaylor.org/research/taylor_l.pdf [Last Accessed 1/8/2011]

Thomsen, M. (2010). Designing for Immersion: Recreating Physical Games. Gamasutra. [Online] Available from: http://www.gamasutra.com/view/feature/4236/designing_for_immersion_.php [Last Accessed 1/8/2011]

Woyach, S. (2008). Immersion Through Video Games. illumin. [Online] Available from: http://illumin.usc.edu/article.php?articleID=103&page=1 [Last Accessed 1/8/2011]

Bibliography

3D Realms, Gearbox Software. (2011). Duke Nukem Forever. 2K Games.

Big Ant Studios, (2010). Rugby League Live. Tru Blue Entertainment.

Ready at Dawn, Sony Computer Entertainment Santa Monica Studio (2010). God of War: Ghost of Sparta. Sony Computer Entertainment.

Redoubt, Rebellion Developments (2010) PDC World Championship Darts Pro Tour. O-Games

Turbine, Inc. (2007). Lord of the Rings Online: Shadows of Angmar. Midway Games

Introduction to Final Project - Contextually Driven Speech

For the final project of the Sound and Music for Interactive Games course I shall be developing a contextually driven darts commentary system for a darts game I have previously made in Max/MSP.

What is meant by contextually driven speech?

Contextually driven speech uses a list of contexts (assigned with boolean variables) to calculate the most effective lines of dialogue to output. For instance, if in the dying minutes of a football game a player who has previously missed 10 shots scores a goal, the commentator might choose to say 'and finally he gets the goal he's been chasing all game!'.

The list of contexts leading to the selection of this line might include:

Goal has been scored = TRUE

Player has scored previously = FALSE

Player has >=10 misses = TRUE

Match time >85 minutes = TRUE

These 4 contexts hold the information needed for the example line of commentary to be selected.

As demonstrated by Durity and Piltz at the 41st International AES Conference in 2011, commentary systems that are triggered on a play-by-play basis are diminishing in popularity in favour of contextually driven speech. In Durity's demonstration, it was shown how the commentary in EA Canada's FIFA10 makes decisions not only on in-game contexts but also events from the real world. For example, an impressive victory by Manchester United last weekend might be acknowledged by the commentary team during a game in 'career mode'.

Why a darts game?

As noted in a previous post, the audio systems in PDC World Championship Darts 2009 ultimately output unbelievable commentary, focusing on play-by-play (per dart) hits and misses from the player and rarely (if at all) commenting on the wider context of the developing match. In the more recent PDC World Championship Darts Pro Tour, while the issue has been partly tackled with both the introduction of a second commentator and recording of more 'colour' commentary (i.e. not solely play-by-play), analysis reveals that there is still a large difference between the phrase content output of the video game commentary and that of real life commentary.

As a darts fan and game audio enthusiast, I feel that with today's technology it would be perfectly possible to create a more authentic commentary system for darts using contextually driven speech. Further, a general conceptual design of a sports commentary system would benefit other games that suffer from similar issues. Ultimately, these two products are the main goals of my project.

References



Durity, G. & Piltz, D. (2011) Production and Implementation Methodologies of Contextually Driven Speech. Electronic Arts Canada. Proceedings of the AES 41st International Conference, London, UK, 2011 February 2-4


Rebellion Developments (developers) PDC World Championship Darts 2009, Oxygen (2009)

Rebellion Developments (developers) PDC World Championship Darts Pro Tour, O-Games (2010)