"Sound is very close to stimulating emotions in your brain," said Yamaoka via an interpreter. "What do humans fear?" He asked. In the case of audio, he noted that gaps between visual cues and accompanying audio. Specifically, if a sound precedes a visual cue, it can cause anxiety, while a sound that arrives too late is comforting. As an example, he noted the sounds of footsteps being made slightly before the onscreen action in a game. He noted that this effect can be felt even if the sync between sound and visual is off by a few frames of animation. Yamaoka said later in the panel that he used this technique in Silent Hill 3.
Another technique mentioned was the use of silence."When there's no sound, the brain fills in the gap," said Yamaoka. He then played a special version of Beethoven's "Für Elise" which had been injected with tiny, periodic gaps of silence. He then played the same piece, only this time injecting harsh noise instead of silence. Even packed in a fully lit lecture hall with hundreds of GDC attendees, it was unsettling. Unfortunately, Yamaoka wasn't able to give us the sound samples used during the panel, so we've done our best to recreate them:
Thankfully, Yamaoka closed the session with a soothing guitar performance, which we were able to capture for your enjoyment. Listen to it below and stay tuned for an interview with Mr. Yamoaka on Joystiq in the near future.
