An Informal History of Video Game Music

I gave a talk for the South Bay Game Dev meetup on the History of Video Game Music.

Check it out on YouTube.

An Informal History of Video Game Music


Official Talk Description:

This is an informal look at music in video games from the early days of video games, starting from the 1970’s, but particularly focused on the MIDI and tracker (MOD) decade of music (1988-1998). 

This talk will include games that are not widely discussed by today’s YouTubers, but nonetheless significant to the history of the industry. And this talk will reflect on the innovations & ambitions we have lost in today’s world in comparison.


Games mentioned:


Gun Fight (1975)

Space Invaders (1978)

Rally-X (1980)

Vanguard (1981)

Journey (1983)

Spy Hunter (1983)

Gyrus (1983)

Frogger (1981)

Galaga (1981)

Dig Dug (1982)

Dragon’s Lair (1983)

Marble Madness (1984)

King’s Quest IV (1988)

Space Quest III (1989)

Doom (1993)

Tyrian (1995)

Second Reality (1993)

Star Control II (1992)

Space Quest IV (1991)

Space Quest I VGA (1991)

Banjo-Kazooie (1998)

Loom (1990)

Legend of Zelda: Ocarina of Time (1998)

Monkey Island 2 (1991) / iMUSE

TIE Fighter (1994)

Legend of Zelda: The Wind Waker (2002)

Super Mario Galaxy (2007)

Red Dead Redemption (2010)

Auditorium (2008)

Quest for Glory 1 (1989)



Bonus 1: Women in Game Design (Sierra On-Line’s top designers)

Roberta Williams

Al Lowe

Mark Crowe & Scott Murphy

Jim Walls

Lori Ann Cole

Christy Marx

Jane Jensen

Josh Mandel

Lorelei Shannon


Bonus 2: Tom & Jerry trivia quiz

Bonus 3: Includes Dance of the Fairies tribute to Carlos Icaza


Transcript:


Hi, my name is Eric Wing. 


I requested extra time from Angelo because I have a lot of material. But I want to move fast and I will be skipping some slides for time. So let's get to it.



I call this 'an informal' history, because it is not exhaustive. However, there is also a method to my madness, so maybe I should have included the word, 'opinionated'.



I am an engineer, not a musician. But I am NOT an audio engineer. However I have been involved with implementing audio libraries. I've been involved with OpenAL for a long time. I co-authored a book which contains both Core Audio, and the most comprehensive resource to OpenAL ever written. I got just about every major OpenAL implementor to help in the proof-reading process. I also authored ALmixer, which is a convenience helper library on top of OpenAL. It is used by some commercial engines, such as the Corona SDK.


I am currently working on a new SDK called Blurrr, which is designed to help make cross-platform native development easier. And you can do it from C, Lua, JavaScript, or Swift.


But today, I decided I wanted to do a Game History talk instead of an audio programming primer. 



Quick rant: I am disturbed by what passes for video game history nowadays. Among other things, I'm seeing a worship of Super Mario Bros 1 and the original NES, as if game design starts there and people were all morons before that. And there is a general ignorance and lack of context of time. I kid you not, a friend had a college student that thought Sonic the Hedgehog was created in 1945.


So here's some trivia and context: Even though the copyright date of the NES and early games say 1986, it didn't launch until 1987 in America and didn't start hitting critical mass until 1988. People also forget that supply chains were a bigger problem back then and producing things like cartridges often severely limited supply.


In this talk, we're going to cover areas less traveled and more interesting.


Here is a timeline so you don't make the 1945 mistake.

Pong is in 1972.

I've highlighted the Great Video Game Crash which roughly spans 1983 to 1988.

The crash doesn't directly factor into this talk, but its effects are felt indirectly. We're going to start with arcades going up to 1984. Then we will jump to 1988 and focus on that following decade because that is where the next great leap occurs.


But speaking of history, here's a history of Animation quiz. All of these pictures are from Tom & Jerry from different periods. Can you order these chronologically from oldest to newest?


Sorry, I know this would be better if I could show you the animation in motion, but I only had a week to put this entire presentation together.


Since I'm in a time crunch, can you pick out which one is the oldest and which one is the newest?


Bonus points if you know when each was made.


So which is the oldest?  Show of hands? Top-left? Top-right? Center? Bottom-Left? Bottom-right?Here is the answer. About all of you were wrong which is typical for this test.

The oldest is the top-left.

It has the highest quality of animation. You can see shadows. They tried to make Tom look furry. Things like the pan have shading. It has the most foreground assets that need to be animated.  It also has the highest quality of frame animation which I was unable to show you.


Now most people find this unexpected. Isn't it interesting that with all our technology advancements, the one made in the 1940's is the highest quality?

We'll come back to this but ponder on that for awhile.


Adding music to video games is obvious. So in the beginning, it was just a matter of technology limitations.

So in 1975, we get our first taste of music in the arcade Gun Fight.

Space Invaders gave us both continuous music, and tempo increases as things get more challenging.


Rally-X gave us continuous melodic music.


Vanguard is a game of many firsts for the industry. 

It uses two real songs, Star Trek: The Motion Picture and Flash Gordon.

It has digital voice.

It is also the first game with both horizontal and vertical scrolling.

And it is the first game with a boss.

In the following clip, it starts with Star Trek. It will get the Invincibility Energy twice and play Flash Gordon.

Then it will transition to the Rainbow Zone which has its own level music and diagonal scrolling.


This arcade tried to capitalize on a rock band from the early 80's. The game lets you play all the different members of the band and plays their music. The game isn't particularly notable except for the fact that it was probably the first to do this.


Spy Hunter licensed the theme song from the 60's TV series Peter Gunn. I don't know what it is like today, but for awhile, more people recognized this as the Spy Hunter theme than knowing it came from something else.


Gyruss is notable for pushing harder on music than its peers at the time, so much so they decided to include an extra processor just for audio.

The game contains a remix of Bach.


It's hard to single out specific arcades because many were all trying things. So here are some honorable mentions.

Frogger is considered the first to switch out the music instantly, like when you beat a level.

Galaga had multiple ditties.

Dig Dug had both music that only played when you moved and changing tempo.

Marble Madness might be first game with what we would consider a "soundtrack" by today's standards. It had different music for every level.



As I said earlier, adding music to games is obvious.

The most interesting question is how do we harness music in ways unique to our medium... 

 to connect, to touch, and inspire players in ways that are not possible with any other medium?


So now we are going to jump to PCs and the year 1988.

And to do so, I must talk about Sierra On-Line, the biggest and most powerful company at the time.


Founded by husband and wife team, Ken & Roberta Williams at their kitchen table, they created a billion dollar public company.

At their height, they had a 28% marketshare in all entertainment software. This meant a quarter to a third of all the dollars you spent were on Sierra related titles.

Sierra was directly responsible for bringing sound cards and real music to games, and with it a paradigm shift in how the industry thought about music in games.


Digression: Because there was a talk tonight about the lack of women in game composing, here's my additional 2 cents.


Sierra was a special place, but nobody recognized it at the time.

Here are Sierra's top designers roughly in chronological order.


Roberta Williams, the Sierra co-founder, and best known as the creator of the King's Quest series and inventor of the graphical adventure game genre.


Al Lowe is most famous for the Leisure Suit Larry series.

Trivia: He was a high school music teacher and composed the Larry theme.


Mark Crowe and Scott Murphy created the Space Quest series.

Jim Walls created the Police Quest series and the fictional CIA thriller, Codename: ICEMAN


Lori Cole created the Quest for Glory series. My favorite Sierra game is Quest for Glory 2.


Christy Marx did a King Arthur game and a Robin Hood game.

And yes, she also created Jem & the Holograms.


Jane Jensen is best known for creating the Gabriel Knight series.

Trivia: She is the only Sierra designer with a formal Engineering background.


Before promoted to designer, Josh Mandel wrote a ton of dialog across many Sierra games.


And Lorelei Shannon.

That makes 5 women and 5 men.

So if we look at today...


What happened?!?!


Since LucasArts appears later in my talk, here they are.


Jane Jensen once said it seemed like a Boy's Club.


In Sierra's days, nobody gave this any thought at Sierra. They had balanced numbers of men & women. Of course! Why wouldn't it be?

It wasn't until Sierra was destroyed that people started noticing Sierra was a total anomaly.

Another interesting thing was that Sierra's customer base had pretty balanced demographics too. They pulled in far more women and girls than all their competitors.

When Sierra died, these customers seemed to disappear because none of the other games targeted them.


Anyway, I think this imbalance has been incredibly damaging in many subtle ways.

But this is a music talk...


 

So back to music. 

For completeness, yes, the Apple IIgs and Amiga had good sound capabilities, but DOS dominated, so we're not going to talk about these.


For context, this is what most PCs sounded like.


So, the story goes, Ken asked Roberta, if you could have one wish for your next game, what would it be?

'Real music'


The King's Quest series was the 800lb gorilla at the time, big enough to convince the industry and consumers to adopt technology.

So Ken worked out deals with Roland and Adlib to support and sell their sound cards.

And they recruited Hollywood composer William Goldstein to compose the score for King's Quest 4, which also marks the first time a professional film composer is used for a game.



I don't have time to really talk about the devices, so I'll just say that the MT-32 used recorded samples to build instruments and sounded really good. But it was expensive.


In contrast AdLib was a lot cheaper.

Adlib used something called FM synthesis. I don't have time to go into it, but basically you construct and manipulate waveforms trying to recreate sounds.

In theory, you could make it sound like anything.

In practice, it sounds fake and terrible when trying to reproduce real world sounds.

Drums especially sounded static-y and weak.


You may not recognize AdLib because, Sound Blaster came out a year later and was an AdLib compatible card. It also added a digital audio playback capability. This was usually used for sound effects and not music because of disk constraints. But this extra feature and competitive price point eventually crushed AdLib.


King's Quest 4 innovated in a lot of other ways too, but I don't have time to go over them.

But on a related point to women in games, King's Quest 4 was the first game to feature a female protagonist in a meaningful way.

Just for context, you play Rosella, the daughter of King Graham who you played as in the first 2 King's Quest games. In the opening, your father suffers a heart attack and will die in the next day. A good fairy tells you of a legendary magical fruit in a distant land that can save your father, but in order to get back, you must recover a stolen talisman from the evil fairy Lolette.


I will play you an excerpt of the King's Quest 4 opening on the MT-32.

Keep in mind what video games sounded like before from both the arcades and PC speaker.

Also notice that the way and type of music is a complete paradigm shift. Where we formerly came from action oriented twitch games, Sierra is now trying to connect music to the player in a completely different way. 


For completeness, I need to show you Adlib, though I don't want to spend a lot of time on it.

Here is just that first part.

Also don't forget to listen to the drums.



To help kick-off their music card launch, Sierra also hired the rock band Supertramp drummer Bob Siebenberg to compose for Space Quest 3. Space Quest was Sierra's sci-fi themed comedy adventure game.

This intro is shorter than the King's Quest 4 intro so I will show you more of this one.

Like King's Quest 4, notice that the way music is used is very different than games before it.


For completeness, I should also mention General MIDI and the Roland Sound Canvas. Because of time, I can't get into it, but basically General MIDI is an attempt to create a standard for MIDI cards, and the Roland Sound Canvas is the first and arguably the best of these.


Eventually some sound card manufactures made sound cards with a daughterboard connector where you could attach a piggy-back wavetable card. So for example, you could buy a Sound Blaster 16 and attach a Roland Sound Canvas daughterboard and you would get the best of both worlds.


So let's do a comparison. Here is Doom in AdLib.


Now here's a Roland Sound Canvas.


And finally for completeness, there were a few games that actually composed with AdLib as the "authentic" version. These are extremely rare, but the PC shooter Tyrian is one such game.


Here's the Sound Canvas. This is blasphemous to say, but it's still good and you get real drums.



The composers quickly realized that just trying to reapply their skills from other mediums like movies or albums wasn't fully working and more was desired. In addition to all the interactivity complexities, the players are still sensory starved, seeking as much information from the game as possible. And they are in front of the same scenes for long periods of time, so the songs need to be interesting and fun, and we need lots of it.


So music just to accompany a scene is not enough. In fact, it needs to help drive to fill in details.


This also leads to a lot more experimentation in the style of music; things that generally would not work in film or concert start becoming interesting to games.


This is a fan favorite from Space Quest 3. Bob Siebenberg originally started off more conservative but realized going really weird with this song would help define the salesman.


In this era, there is one more competing music technology, tracker based music. It is similar to MIDI except in addition to just storing what notes to play, it also stores the instrument samples.


This requires a sound card with digital audio playback, but no longer needs MIDI. And now the music playback sounds the same on all sound cards instead of varying widely between different devices. There are tradeoffs I don't have time to get into.

This did not really catch on with the game industry, but was heavily used by the Demoscene.


It would be criminal to not mention the Demoscene when talking about Tracked Music. Basically groups write graphical/musical artistic "demos" in Assembly Language and push the hardware to do things that people didn't think were possible.

The most famous group is The Future Crew, with their 1993 award winning demo, Second Reality.

Here is an excerpt from it.

Remember, this is 1993. It ran on a 386 at 33 Mhz. There was no graphics acceleration like today. And it was all written in Assembly, no libraries. And it runs in less than 640K of RAM.


Star Control 2 was one of the earliest games to use tracked music.

Star Control 2 really is one of the greatest games ever made. It's also one of the hardest games to describe. It is a hybrid game unlike anything made before it, or after it. And everything about it seems to be in contradiction to itself, yet it all works.

It's both an action game and adventure game. It has possibly the best traditional science fiction writing ever done for a video game, yet also manages to infuse humor in the game without compromising its seriousness.

The music is composed by 8 different musicians found during a contest. Despite the enormous differences in styles and absence of coordination, everything works.

 



 This is a montage of some different scenes. Star Control also reflects the "early lessons discovered". Every different part of the game has unique and memorable music. There are 24 different alien races you can talk to, each with a different unique song that helps define their personality. Also, easter egg: in alien conversation, notice the little oscilloscope music visualizer on the right side.


Back to MIDI.


Since MIDI music is all played back in real-time, musicians soon realized they could start doing dynamic things.

Here in Space Quest 4, we are at a shopping mall. Every store has its own song. The composer, Ken Allen, realized he could combine all the songs together and fade-in and fade-out the volume of the tracks as you walk closer or father to each store to give a sense of dynamic position.


This is a visualization of a MIDI file. In the top-left, you can see multiple tracks, each representing a different instrument to be played. With MIDI, it is very simple to turn on or off a track, change its volume, or even change the instrument type.


Here's a related example. If you look at our two Blues Brothers on stage, one of them has a harmonica that he sometimes plays. The music stays in sync with what the performers are doing. In this case, the MIDI file simply contains all the tracks of the music including the harmonica. When you need the harmonica part, you turn it on. When you don't, you turn it off.


Banjo-Kazooie is fairly famous for its use of dynamic music. Here's a clip where the instruments change when you go underwater.

But I also want to point out that as much praise as this game gets for this, this came almost a decade later than these other games we're talking about, using the exact same techniques.



Loom from LucasArts is a really innovative game. You use music to solve puzzles.


You carry a magic wand that is like a music instrument. To cast a spell, you must perform a musical sequence.

If you are thinking of Zelda 64: ocarina of Time, it's the exact same idea.

But Loom came almost a decade earlier.


So here's an example of casting an open spell on an egg.

This casts a sleep spell on the dragon.


In 1991, LucasArts deployed their dynamic music system, iMUSE, for their adventure games.

While not completely unique, iMUSE really pushed the envelope much farther and really is an impressive best-of-breed system.


iMUSE builds on top of all the dynamic ideas we've talked about so far, and now also adds seamless transitions between music.


As you go to each new area, a new variation of the song is transitioned into. What I'm about to play is not the authentic iMUSE system,

but I really like this live recreation because I think it conveys the spirit of what's going on really well.


Although iMUSE was created for adventure games, its pinnacle use was in the space flight simulator game, TIE Fighter.

In this game, you fly for the Empire against the Rebel traitors.

Since you are now playing from the perspective of the Empire, we suddenly need a new original soundtrack because happy Rebel music won't cut it. TIE Fighter delivered above & beyond. In addition to a unique soundtrack, iMUSE sequences were composed to detail all sorts of flight events. Now suddenly, the music isn't just passively along for the ride, but you can now build a mental model of the whole battlefield based on the changing music. 

Ships are jumping in, are they large or small, friend or foe?

I just lost a wingman.

I completed a mission objective.

As I said, earlier, players are information starved. So suddenly this extra layer of information is like growing a sixth-sense, or more aptly, like Using the Force.

And when flying without iMUSE, it was suddenly like losing a sense, like going blind.


TIE Fighter is not a great spectator game because it depends on so much spatial awareness, so I want to start with this slower audio-only mission start clip to get you acquainted with the music and the transitions and show how you can tell what's going on in the battle just by music. I will point out the different music transitions on the slide at the right time.



Now here's an in-game dogfighting clip.


So we've now come to the end of the era.

While dynamic music isn't completely dead, it's pretty sad. Most games will do a transition from calm to battle and back to calm. But that's about as far as it goes.

But let me say a few quick things about these games which do a bit more.


Ironically, Nintendo both was kind of late to the game, and also the last holdout on keeping MIDI.

All the Zelda games now continue to use dynamic music in different ways.

In Wind Waker, you can hear the music both changes when enemies are near and also the sword strike hits are connected with musical chords.

Even back in the Game Cube era, some people questioned why Zelda didn't use fully orchestrated digital audio by then. This is why.



Super Mario Galaxy hedged and had some fully orchestrated non-dynamic digital songs, and others that were dynamic MIDI. This is a dynamic MIDI example.


Red Dead Redemption developed their STEM technique. While interesting and innovative, we've lost the melodic qualities of iMUSE. It feels like reverting from Rally-X back to Space Invaders.


Auditorium is a game that uses music as feedback for puzzle solving.


[start rant]


So back to Tom & Jerry, this time period was called the Golden Age of Animation. It produced the highest quality stuff and the greatest animators came out of this time period: Joseph Hanna & William Barbera, Chuck Jones, Tex Avery, and so forth. While there are occasional pockets of brilliance that emerge from time-to-time, like Studio Ghibli, the animation industry as a whole has never been able to return to producing stuff in the quality and quantity of the Golden Age.


I bring this up because in our technology centric industry, we assume that everything newer is always better because that's how technology works. But I challenge that notion and think game music has stagnated. Real innovation and envelope-pushing seems far and few between.


In 1994, I couldn't wait to see the future, how iMUSE and the sequel to TIE Fighter would blow away what we had. That future never arrived.


With the exception of dedicated rhythm games, I cannot think of any game after TIE Fighter where losing the music feels like losing a sense.

And games as a whole feel like their music has shifted to background music, with the exception of cutscenes where they just apply standard cinematic techniques. This creates an artificial dichotomy that players can sense subconsciously and is an obstacle to creating sincere connections to the player.


And looking back, those past designers did so much with so little. Nothing existed so they had to invent everything themselves. Today, we have comparatively infinite computing resources available to do things they could only dream of, but in 20 some to 30 years we have little to show. They were innovative and ambitious. Today, we seem to find excuses to avoid it, the latest being, "My game engine doesn't support that." When they dreamed of new ways to improve the player experience, they created it.


Most of us here would like games to be realized as an art form. But how can we expect others to take our medium seriously when we don't even take every aspect of our craft seriously. And this goes beyond just audio to all aspects of our medium, even to how we make them. 



[end rant]



For more of my stuff, check out 'Why we loved Sierra games' on YouTube. It will challenge your thoughts on game design.


Also check out Blurrr SDK.


Finally, I want to switch gears and say a few words about my friend, mentor, and former co-founder, Carlos Icaza. He passed away unexpectedly one year ago.

He was a friend to this Indie Meetup and hosted it a few times. He co-founded Corona and Lanica, and he was a lead on Illustrator and Flash. Many of you here may have used products he had a hand in.



Being at a loss for words, I decided to write a "demo" in his memory. I took three things from our past:

He loved splines and we always wanted to do a spline project together but never found the time.

We had a bad history with particle engines, so I finally wrote a good one using Data Oriented Design techniques and SIMD intrinsics.

We loved the technique of applying real-time lighting to sprites using shaders.


It also happens to relate to this talk. The music is a remix of one of Sierra's most popular songs, Erana's Peace from Quest For Glory 1 from 1989. You will hear part of the original song in the bumper at the end. The demo itself is inspired from two scenes in the Quest for Glory series, but you also may get a demoscene vibe which is not an accident.


Copyright © PlayControl Software, LLC / Eric Wing