Video Game Console Audio: Evolution and Future Trends

Video Game Console Audio: Evolution and Future Trends KyuSik Chang, GyuBeom Kim, TaeYong Kim Department of Image Engineering, Graduate School of Advan...
Author: Harvey Stokes
2 downloads 1 Views 783KB Size
Video Game Console Audio: Evolution and Future Trends KyuSik Chang, GyuBeom Kim, TaeYong Kim Department of Image Engineering, Graduate School of Advanced Imaging Science Multimedia, and Film, Chung-Ang University, 221 Huksuk-Dong Dongjak-Ku, 156-756 Seoul, South Korea {[email protected], [email protected], [email protected]} Abstract Rapidly advancing technologies and the increasing role of computers give the computer game boundless potential for the future. Among the various game industries, the console game industry in particular has attracted significant attention, bringing the importance of sound and music to the forefront. Game music was traditionally used for sound effects to prevent user boredom. As gaming developed, so did audio, being utilized for numerous purposes. As gaming platforms increased in power and complexity, audio, which was once limited to simple beeps, started to play a larger role, becoming Back Ground Music (BGM) having quality at par with that of films. This paper discusses the evolution of video game console audio specifications and hardware, from simple 8-bit machines to future directions in the development of audio generators for game sound & music. Keywords--- Game Sound, Music, Interactive, Audio, Game Trends

1. Introduction Rapid improvements on game platform capabilities, have produced diverse game techniques and genres, requiring game sound & music that effectively express the genre characteristics and enhance the game experience. Due to the unique characteristics and technical limitations of the platform, each game platform requires its own mode of music, with different game genres needing their own kind of music. This paper discusses the roots of game console audio, chronicles its evolution into the powerful driving force it is today, and challenges sound designers to take the next step in its growth.

2. 8-bit Machines & Chip music

predating the Atari PONG home consoles by several years (Figure 1) [16] [22] .

Figure 1 Magnavox Odyssey Ping-Pong

The Odyssey lacked sound capability, something that was corrected with the "PONG systems" several years later, which included Magnavox's own Odysseylabeled Pong consoles [12] [15] [16] [22] . 2.2 Pong: Atari, 1975 PONG is based on the sport of table tennis, and named after the sonar-blip sound generated by the circuitry when the ball is hit (Figure 2).

Figure 2 Pong for Home, Atari

2.3 Channel F: Fairchild, 1976

Figure 3 Channel F System II

The sound Channel F was played through an internal speaker, rather than the TV set, using 500 Hz, 1 kHz, and 1.5 kHz tones (Figure 3). These frequencies could be modulated quickly to produce different tones. There were major changes in the unit design; the sound was mixed into the TV signal, the unit no longer needed a speaker [12] [22] . 2.4 Atari 2600: Atari, 1977

2.1 Magnavox: Odyssey, 1972 The Magnavox Odyssey was the world’s first video game console [9] [15] . It was released in the fall of 1972, Figure 4 ATARI 2600 and Pokey Chip

Computer Graphics, Imaging and Visualisation (CGIV 2007) 0-7695-2928-3/07 $25.00 © 2007

The Pokey was a digital I/O chip found in the Atari 8-bit family of home computers and many arcade games in the 1980s (Figure 4) [24] . The Pokey was also well known for its sound effect and music generation capabilities, producing a distinctive square wave sound popular among chip tune aficionados. The design of the Pokey made it possible for games to have polyphonic music and sound effects of up to four channels. The Atari 2600, which utilized the Pokey, was capable of generating only two tones, or ‘notes’, at a time [11] . One of the sound engines developed for the Atari 8-bit family was called the Advanced Music Processor (AMP) engine [22] [23] .

machine, with the state being set up in a series of sixteen 8-bit registers. These were programmed over an 8-bit bus that was used both for addressing and data by toggling one of the external pins. The 8910 and its variants became popular chips in many arcade games, and was used on, among others, the Intellivision and Vectrex video game consoles and the MSX, Atari ST, Amstrad CPC, Oric 1 and Sinclair ZX Spectrum 128/+2/+3 home computers as well as the Mocking board sound card for the Apple II family [2] [27] . 2.8 Family Computer : Nintendo, 1983

2.5 Oddissey 2: Magnavox, 1978 One of the remarkable points of the Oddissey2 system was its brilliant speech synthesis unit, which was released as an add-on for speech, music, and sound effects enhancement [12] . The speech synthesis of the Odyssey 2 may be remembered for pioneering the fusion of board and video games. There was only one Intel 8244 custom IC chip in the system, which performed both mono audio and video functions, 24-bit shift register, clockable at 2 frequencies, noise generator [22] . 2.6 Intellivision: Mattel, 1979

Figure 5 Intellivision, Mattel

In 1983 Mattel introduced a new peripheral innovation for the time (Figure 5): Intellivoice, a voice synthesis device which produced speech when used with certain games, most of which would not work without the add-on component [4] [25] . The Intellivoice Voice Synthesis Module was an adapter for the Intellivision platform, Mattel's home gaming console that utilized voice synthesizers to generate audible speech. The Intellivoice was a large, brown cartridge that could be plugged into the Intellivision, at which point games specifically designed for the device could be inserted like a normal cartridge into the right side of the module [22] . 2.7 Vectrex: 1983

Figure 6 Vectrex, AY-8912 and Mocking board

The Vectrex generated sound and music directly or by use of the AY-8912 sound chip (Figure 6). It was a 3voice programmable sound generator (PSG) designed by General Instrument, initially for use with their 16-bit CP1610 or one of the PIC1650 series of 8-bit microcomputers. The 8910 was essentially a state

Computer Graphics, Imaging and Visualisation (CGIV 2007) 0-7695-2928-3/07 $25.00 © 2007

Figure 7 NES & FamiCom, Nintendo

The Famicom had five sound channels (Figure 7), featuring: „ 2 pulse-wave channels, variable duty, 16-level volume control, hardware pitch-bend support, supporting frequencies from 54 Hz to 28 kHz. „ 1 triangle-wave channel, fixed volume, supporting frequencies from 27 Hz to 56 kHz „ 1 white-noise channel, 16-level volume control, supporting two modes at 16 preprogrammed frequencies „ 1 delta pulse-code modulation (DPCM) channel with 6 bits of range, using 1-bit delta encoding at 16 preprogrammed sample rates from 4.2 kHz to 33.5 kHz, also capable of playing standard PCM sound by writing individual 7-bit values at timed intervals [22] [28] .

3 Synthesis & Sampling 3.1 Master System: SEGA, 1986

Figure 8 Master System, SEGA

The Japanese Master System (Figure 8) included the YM2413 FM sound chip, called OPLL, a cost-reduced sound chip manufactured by Yamaha Corporation and based on their YM3812 (OPL2). To make the chip cheaper to manufacture, many of the internal registers were removed. The result of this is that the YM2413 can only play one user-defined instrument at a time; the other 15 instrument settings are hard-coded and cannot be altered by the user. There were other cost-cutting modifications: the number of waveforms was reduced to two, and an adder is not used to mix the channels; instead, the chip's DAC (Digital to Analog Converter) plays each channel one after the other, and the output of this is usually passed through an analog filter. The

YM2413 was used as a sound expansion on the MSX and the SG-1000 Mark III [22] . 3.2 PC Engine : NEC, 1987

Figure 9 PC-Engine First Model & TurboGrafx-16

The PC Engine (PCE) (Figure 9) has three main chips in it: CPU + sound, graphics processor, and video encoder. It has 6 PSG audio channels, programmable through the HuC6280A CPU [22] . The CDROM attachments (all of them) include a new ADPCM sound chip and some extra RAM for playing sound samples. This RAM is not the same as normal PCE RAM, and was designed to be used with the new sound chip exclusively. Through smart programming, the game ‘Monster Lair’ used this RAM to store extra sprite animation, essentially bypassing the RAM limitations of the PCE and System Card. This was probably the first and only time that new sound hardware increased graphics quality. Interestingly, this affected emulation as well, as Monster Lair suddenly looked better when this sound chip was accurately emulated [9] [22] [31] . 3.3 Mega Drive: SEGA, 1988

Figure 10 Mega Drive, SEGA & YM2612 chip

The Mega Drive (Figure 10) has two sound chips, one is the main sound chip (YAMAHA YM2612), and the other is the secondary sound chip (Texas Instruments SN76489) [22] [26] . The main sound chip has six FM channels, with four operators each; channel 6 can be used for PCM data or as a regular channel. The chips are programmable oscillators of low-frequency and stereo panning [32] . The secondary sound chip has four channels had a Programmable Sound Generator. Three square wave channels and one white noise channel. The main sound chip had programmable tone, noise and attenuation. This was used for master system compatibility mode as well as to supplement FM. It has a different random noise generation compared to a real SN76489, SN76489A chip [12] [22] [32] . 3.4 Super Nintendo Entertainment System, 1990

Figure 11 SNES, Nintendo

Computer Graphics, Imaging and Visualisation (CGIV 2007) 0-7695-2928-3/07 $25.00 © 2007

The Super Nintendo Entertainment System (SNES) (Figure 11) had a sound controller chip: an 8-bit Sony SPC700 CPU for controlling the DSP chips independent of the main SNES CPU. The main CPU communicates with this sound controller through a set of four memory-mapped registers [22] [33] [34] . It featured a second order low-pass filter, one for each channel, for improved quality of low-frequency (bass) tones. A Pulse Code Modulator: 16-bit Adaptive Differential Pulse Code Modulation (ADPCM) Although the SNES is normally only able to output stereo sound, a few games (such as Jurassic Park and Super Turrican) used Dolby Pro-Logic to create surround sound embedded in the stereo sound signals [33] [34] [35] . 3.5 SEGA Saturn: 1994

Figure 12 Saturn & Custom Sound Processor

The Saturn Custom Sound Processor (SCSP) is manufactured by Yamaha and consists of several components (Figure 12). The SCSP is a multi-function game sound generator LSI that consists of a PCM sound generator and DAC (Digital to Analog Converter). The SCSP creates and processes sound mixes. It contains a 32-slot sound generator and sound effect DSP, a 16channel digital mixer and timer, and an interrupt controller [36] .

4 Pre-Recorded & Streaming Music 4.1 Streaming Music Home console systems also developed specialized streaming formats, ADPCM sources with up to 24 channels and up to 44.1 kHz sampling rate and containers for compressed audio playback [18] [22] . Games would take full advantage of this ability, sometimes with highly praised results [4] [7] . This overall freedom offered to music composers gave video game music the equal footing with other popular music it had lacked. A musician could now, with no need to learn about programming or the game architecture itself, independently produce the music to their satisfaction [15] [16] . This flexibility would be exercised as popular mainstream musicians used their talents for video games specifically [5] [8] [9] [10] [11] [12] . 4.2 Play Station and PSF: 1994

Figure 13 Playstation, Sony

A Portable Sound Format (PSF) file is a sound data file (from the Nintendo Entertainment System, and other console related sound formats) ripped directly from video games from a variety of game consoles. The format was originally used for Sony PlayStation video games (Figure 13). Generally PSF files contain a number of samples and a sequence player program. This takes far less space than the equivalent streamed format of the same song while still sounding exactly like the original song [3] [7] [38] . PSF initially stood only for "PlayStation Sound Format", but with the addition of the PSF2, SSF (Sega Saturn Sound Format), DSF (Dreamcast Sound Format), USF (Nintendo Ultra 64 Sound Format), QSF (Capcom Q-Sound Format), and GSF (Game Boy Advance Sound Format) sub formats, the more generic acronym "Portable Sound Format" was developed [7] [38] . 4.3 Nintendo64: 1996

Figure 14 Nintendo 64

The Nintendo 64's graphics and audio duties are performed by the 64-bit SGI co-processor, named the "Reality Co-Processor" (Figure 14). The RSP frequently performs audio functions. It can playback virtually any type of audio including uncompressed PCM, MP3, MIDI, and tracker music. The RSP is capable of a maximum of 100 channels of PCM at a time, but this is with 100% system utilization for audio. It has a maximum sampling rate of 48 kHz with 16-bit audio. However, there were storage limitations caused by the cartridge format’s limited audio size (and thus quality). 4.4 Dreamcast: 1998 The Dreamcast had the sound engine of Yamaha AICA Sound Processor (Figure 15): 22.5 MHz 32-Bit ARM7 RISC CPU: 45 MHz, 64 channel PCM/ADPCM sampler (4:1 compression), XG MIDI support, and 128 step DSP [7] [9] .

Figure 16 a. PS2

b. GameCube

c. Xbox

4.5.1 Sony PlayStation 2: March, 2000 The PS2 console (Figure 16 a.) made significant technological advances from its predecessor. In addition to quadrupling its graphical processing capabilities, the PS2 also doubled the original PlayStation’s 24-channel sound chip to a 48-channel system. The PS2 also expanded sound barriers by creating an internal sound processor that existed independent from graphics processors to keep from having to share its resources with visual graphics, which historically have garnered greater emphasis and devoted CPU memory. The PS2’s independent sound processor opened new doors for sound designers [15] [16] . Audio artists designing for the PS2 could implement techniques with fewer restrictions, having the opportunity to preserve high quality audio, incorporate longer loops, larger sound samples, more processing and modeling capabilities, and 48 independent voices that can be streamed simultaneously in stereo sound [18] . 4.5.2 Nintendo GameCube: September, 2001 Similar to the PS2, the GameCube (Figure 16 b.) is also capable of 48kHz quality stereo or surround sound, but has a digital signal processor (DSP) that supports more than 100 voices, and can play up to 64 real-time 3D voices at one time. The beauty of the GameCube, from a sound composer designer’s standpoint, is that there is no need for a development kit to create audio for Nintendo games. Music and sound effects can be produced in whatever method the sound artists see fit, and can be implemented and easily inserted into the game by the programmer using Nintendo’s audio tool called MusyX. Audio designers developing for GameCube titles can create and apply components of different sounds separately, and then have them played back at random, which can then be layered according to events, ultimately allowing for more interactivity. 4.5.3 Microsoft Xbox: November, 2001

Figure 15 Dreamcast

The sound chip was just as good as any of the sound cards on the market during that period. It offered 64 simultaneous audio channels at 16-bit, 44.1 kHz as well as XG MIDI support. 2MB of RAM was also available for storing both sound effects, and wave table instrument samples [3] [7] [9] [22] . 4.5 PlayStation 2, XBox & GameCube: 2000~1

Computer Graphics, Imaging and Visualisation (CGIV 2007) 0-7695-2928-3/07 $25.00 © 2007

The Xbox’s (Figure 16 c.) audio data unit is the Media Communications Processor (MCP), which includes four independent audio processors for digital audio and 3D positioning; the set-up engine, Voice Processor (VP), Global Processor (GP), and Encode Processor (EP). The processors each perform a specific function to ensure high quality digital audio. Unique to Microsoft is DirectMusic Producer, a composition tool used in conjunction with DirectX 8 audio scripting, Down Loadable Sound (DLS) and Windows Media software for creating and programming game sound [13] [14] . DirectMusic Producer is a program that can be used to generate interactive audio [5] [6] [7] . The Xbox

had a definite edge in the audio department in that it was the first console with the ability to incorporate in-game Dolby Digital surround, as well as the capability of communicating 256 simultaneous stereo voices through 64 discrete channels. The Xbox created a far superior auditory experience to that of the PlayStation 2 or GameCube.

5 Game Consoles at Present

Figure 17 a. Xbox360

b. PS3

c. Wii

5.1 Microsoft Xbox 360: November, 2005 All games made for the Xbox 360 (Figure 17 a.) are required to support at least 5.1-channel Dolby Digital surround sound. The console works with over 256 audio channels and 320 independent decompression channels using 32-bit processing for audio, with support for 48 kHz 16-bit sound, hardware codec streaming. Sound files for games are encoded using Microsoft's XMA audio format. 5.2 Sony PlayStation 3: November, 2006 Future technology does not represent any fundamental shift in video game music creation for Sony. The PlayStation 3 (Figure 16 b.) handles multiple types of surround sound technology, including Dolby TrueHD, and DTS-HD. 5.3 Nintendo Wii: November, 2006 Nintendo's Wii console (Figure 16 c.) shares many audio components with the Nintendo GameCube from the previous generation, including Dolby Pro Logic II. These features are extensions of technology already currently in use.

6. Game music as a genre Many of the discussed consoles and other early game systems featured a similar style of music which may come closest to being described as the "video game genre" in terms of musical composition, as opposed to simply "video game music" for being in a video game, this genre is increasing in popularity, being played by live orchestras and bands in concerts [30] . Some compositional features of this genre continue to influence certain music today, though game soundtracks currently tend to emulate movie soundtracks more-so than this classic genre [37] . This genre's compositional elements may have developed due to technological restraints. The genre might also have been influenced by techno-pop bands such as Yellow Magic Orchestra, which were quite popular during the period in which videogame music took its trademark sound [9] [11] [17] .

Computer Graphics, Imaging and Visualisation (CGIV 2007) 0-7695-2928-3/07 $25.00 © 2007

Songs feature a heavy amount of synchronization between instruments, in a way that would be difficult for a human to play. For example, although the tones featured in NES music can be thought of emulating a traditional four piece rock band (triangle wave used as a bass, two pulse waves analogous to two guitars, and an affected white noise channel used for drums), and although video game music was influenced by rock or pop music at the time, composers would often go out of their way to compose complex and rapid sequences of notes. This has been compared to music composition during the baroque period, where it is believed that composers compensated for instruments such as the harpsichord by focusing more on musical embellishments. Composers were also limited in terms of polyphony, or the number of notes that can be played at once. Only three notes can be played at once on the Nintendo Entertainment System. A great deal of effort was put into creating the illusion that more notes are playing. As of the late 1990s, musical groups covering these melodies have sprung up. One such group is The Minibosses, who attempt to emulate these melodies as closely as possible using real instruments [19] . Another such group is the NESkimos, who opt to explore these songs artistically, and create entirely new songs out of them. And also Brazilian heavy metal band, MegaDriver [20] [21] .

7. Current Developments Today, the game developer has many choices on how to develop game audio. More likely, changes in video game music creation will have very little to do with technology and more to do with other factors of game development as a business whole. As sales of video game music separate from the game itself become increasingly marketable(compared to Japan where game music CDs had been selling for years), business elements also wield a level of influence where it had little before [9] [11] [22] . Game sound engineers from outside the game developer's immediate employment, such as music composers, sound designers, recording & mixing engineers, voice actors and pop artists, have been contracted to produce game sound and music just as they would for a theatrical movie [8] [22] . Many other factors have a growing influence, such as demanding consumers, one source multi use(OSMU) for content, politics on some level of the development, executive input and various other elements [1] [29] .

8. Conclusion The role of game audio depends on the content, progressive manner and mood of the game, defined by its requirements for sound and music with characteristics that produce an immersive atmosphere according to the game storyline. The characteristics of game audio, which fully engage the attention of the user, have been analyzed in terms of technological limitations. The game sound market has dramatically changed from that of 10 years

past, instrumentation was simpler then by comparison. Rarely would composers have more than 9 to 11 voices of polyphony and never more than a bank of 127 possible instruments with the old General MIDI tracks. Present consoles allocate a larger amount of memory for sound generation, effectively allowing developers to create more intricate and complex sound, using a wide variety of tools. To date, most developments in enhancing games have been involved with improvements to game visuals, with audio being tragically overlooked in favor of better graphics. A game's interactive elements are enhanced through the successful mating of graphics and audio. The use of game sound and music promotes user understanding of the game by creating a realistic feel by complementing and highlighting elements of the virtual world. The ultimate goal of game music and audio is to provide the player with a rewarding game experience by increasing interactivity, improving realism and enhancing user emotions. The game sound director achieves this by analyzing user response to game music, and tailoring game audio accordingly. The only limitations in the continuing advances of game audio are present audio tools and the imagination of sound designers. The challenge is to take game audio to the next level of evolution, by conducting further research and development in interactive audio.

Acknowledgement This research was supported by the ITRC(Information Technology Research Center, MIC) program and Seoul R&BD program, Korea.

References [1]

J. Kirriemuir, “The Console Market”, Virtual Reality, volume 5, number 4, 2000 [2] Patrick Bass, “ST Sound – Hearing the AY-3-8910 chip”, Antic Vol. 4, No. 7, 1985, 16p [3] David Miles Huber, "The MIDI Manual, Second Edition', Focal Press, 1998 [4] Martin Russ, "Sound Synthesis and Sampling", Focal Press, 1996. [5] Jonathan B. Fish, “Interactive and Adaptive Audio for Home Video Game Consoles”, M.A., the School of Communication, Simon Fraser University, 2003 [6] Szuyi Yeh, Pei-Ju Lin, Hsiu-Ying Liu and Rong-Ming Chen, “The Implementation of Interactive Music System for Massive Multi-player Online Game”, Proceedings of the IEEE Fourth International Symposium on Multimedia Software Engineering (MSE’02), 2002 [7] Aarn Marks, “The Complete Guide To Game Audio”, CMP BOOKS, Kansas, 2001. [8] Alexander Brandon, Instrumentation: “The Game Composer's New Challenge”, CMP Media, 2004 [9] Stephen D. Bristow, “ The History of Video Games”, IEEE Transactions on Consumer Electronics, 1977, 6263 [10] Greg Hill, “Capturing Engine Sounds for Games”, Gamasutra, 2002.

Computer Graphics, Imaging and Visualisation (CGIV 2007) 0-7695-2928-3/07 $25.00 © 2007

[11] Eric Pidkameny, “Levels of Sound”, VGMusic.com, Video game music archive, 2002, http://www.vgmusic.com/information/vgpaper2.html [12] Glenn McDonald, “Gamespot: Brief Timeline of Game Music History”, 2001, http://www.gamespot.com/ gamespot/features/video/vg_music/index.html [13] Ross, Rob, "Interactive Music...er, Audio," Gamasutra, http://www.gamasutra.com/resource_guide/20010515/ros s_01.htm, 2001, [14] Alexander Brandon, "Interactive Music: Merging Quality with Effectiveness." Gamasutra, 27 March 1998, vol.2, http://www.gamasutra.com/features/sound_and_music/1 9980327/interactive_music_quality_intro.htm [15] Nam-Yeh Park, “A Study on the Musical Characteristics of Game Music Appearing in the Game Genre”, Music in Modern Media, The Graduate School of Ewha Woman’s University, June 2004. [16] Mina Jang, “A Characteristic Feature and the Element Analysis of the Game Music”, Department of Music, The Graduate school of Sangmyung University, December, 2004. [17] Yellow Magic Orchestra discography on a fan website : http://www.recohan.net/artists/yellow-magic-orchestra204.shtml [18] Sony : http://www.playstation.com [19] Minibosses band : http://www.minibosses.com [20] Neskimos band : http://www.neskimos.com [21] Megadriver band : http://www.megadriver.com.br/ [22] WIKIPEDIA: http://en.wikipedia.org/wiki [23] Atari SAP Music Archive: http://asma.atari.org [24] POKEY Chip data sheet : http://www.leopardcats.com/bbpokey/pokey.pdf [25] Intellivision Lives Intellivoice Games: http://www.intellivisionlives.com/bluesky/games/credits/ voice.html#space_spartans [26] John Kortink’s website, SN76489 sound chip details: http://web.inter.nl.net/users/J.Kortink/home/articles/ sn76489/index.htm wikipedia/s/sn/sn76489.html [27] Vectrexnews : http://www.vectrexnews.com/tools.html [28] Martin Nielsen (Author of NES WORLD), “The Nintendo Entertainment System” FAQ v3.0A Released : http://www.classicgaming.com/museum/faqs/nesfaq.shm l, 8th October 1997 [29] Stephen Kline, Nick Dyer-Witheford, and Greig de Peuter, “Digital Play: The Interaction of Technology, Culture, and Marketing”, McGill-Queens University Press, 2003 [30] Videogameslive.com: http://www.videogameslive.com [31] 101 PC Engine Secrets: http://nfggames.com/games/pce/ [32] Sega Mega Drive Frequently asked questions: http:// www.angelfire.com/ny/dezmoowu/Sega/megadrive.txt [33] SPC700 APU Manual: http://snesmusic.org/files/spc700_apu_manual.txt [34] A tutorial on using the SPC700 for the SENS: http://en.wikibooks.org/wiki/Super_NES_Programming/ Loading_SPC700_programs [35] Super Nintendo/Famicom F.A.Q: http://www.gamers graveyard.com/repository/snes/snesfaq.txt [36] Sega Saturn Emulation and Information, The Vintage Gaming Network: http://vg-network.com/saturn.html [37] SPC Player : http://www.zophar.net/utilities/spc.html [38] Neill Corlett’s home page, PSF Central : http://www.neillcorlett.com/psf/