MIGS06: Ken Perlin on "the illusion of life"
Ken Perlin, director of NYU's Media Lab, spoke at the Montreal International Game Summit on Wednesday on the topic of "revisiting the illusion of life." Who's Ken Perlin, you ask? Well, for starters, he's awesome.Perlin won an Academy Award in 1997 for his work in procedural textures. He's had a hand in a large number of CG movies (including TRON), and is getting more and more involved in the video game industry.
"The illusion of life" is a term that Walt Disney first used 60 years ago, and Perlin evokes it to ask the same question. Just as Disney hoped to do in animation, can games achieve "the illusion of life?"
Perlin has been focused recently on emotion in real-time interaction. He believes that the videogame industry is in a unique position; unlike the film industry, it has the potential to direct its digital "actors" in real-time. The industry, however, can no longer rely on animators to generate emotion in characters, and it's becoming more and more important for the code to carry the brunt of the work. His first example of this is a real-time render of a face, based on his observations of a then-girlfriend. "My actual girlfriend had a higher polygon count," he admits.
Perlin uses different interface switchers to move around parts of her face, and then shows that by conjoining movements, a programmer can create emotional "chords." He illustrates this by tightening her lips, half-closing her eyes, and cocking her head back in one direction. The effect is convincing: she looks displeased. Perlin's work on emotional chords has played a large part in the emotive facial responses of characters in Valve's Half-Life 2. In fact, his real-time model is almost a dead-ringer for Alyx.
Perlin goes on to discuss the increasing importance of emotion in games. He cites the example of Facade, which he says "takes the level of psychological violence much further than shooter games... I kept saying to myself 'I'm so glad this isn't a real dinner party.'"
Perlin then shows the audience the simplest character that can be made to portray emotion. Her name is Polly, and she's only five polygons. Polly displays emotion with limited movement, like scampering, swaggering, prowling, lumbering, feeling dejected, hotfeet, sprinting, and hopping. According to Perlin, we should refer more to people's own "database of emotional knowledge," playing into expectations and letting the brain do the rest. The anithesis of this is Final Fantasy: The Spirits Within. "Don't get me started on Polar Express," he says, shuddering. He asks us to contrast this hyper-realism with a Bugs Bunny cartoon. Bugs Bunny, as a character, is believable, but "that's not the same as realism. If Bugs Bunny walked through that door, people would freak out."He shows some more demos, the next one of two emotive virtual actors, with full-body awareness of actions. When Perlin grabs the actor's hand, the actor's entire body reacts: the feet move, the hips swivel, etcetera. Perlin considers this "full-body awareness" important to in-game animation: "the reason that all game characters are unbelievable... is because of what happens from the waist down."
His response to this is several experiments in footsteps. One applet is designed to make "smart feet." Perlin says that "we are, in our minds, magical floating heads and hands." We don't often think about the rest of our bodies, he explains, so it's hard for us to animate convincing people. In each example he shows, he focuses on creating convincing foot movements.
After a few more examples, Perlins begins to wrap up his keynote. In closing, he explains that "people care about the emotions of people... We can do things they can't do in the film world; we can go beyond film, but our techniques are much younger."
"The Graduate is a mindfuck," he says, then stops and asks "am I allowed to say that?" The audience laughs, encouragingly. "Well, anyway," he goes on, "we want [games] to be that, interactively."










Reader Comments (Page 1 of 1)
Optimus Prime @ Nov 13th 2006 10:52AM
Nice write up; the dude sounds like he knows what hes talking about.
Martin @ Nov 13th 2006 10:55AM
Its the Carver from Nip/Tuck!!!
josh @ Nov 13th 2006 11:47AM
programmers cannot animate, so good luck in trying to not rely on us animators to make convincing animations.. this point is obviously proven with the demo apps.
the illusion of life IS talking about animation, and animation IS in games, so to ask can games achieve the illusion of life? is nothing new. its more about the technology of games allowing the level of detail that most animators want to put into game animations.
i wonder if he's looked into the emotion and psychological research of paul ekman any...
it really sounds like he doesnt have much of a clue, honestly.
Antonio @ Nov 13th 2006 12:01PM
Neat article. Let's see more like these Joystiq!
-=Stitched=- @ Nov 13th 2006 12:12PM
Amen, Josh.
One just needs to play Oblivion to see what the NPC face models - reduced to values on sliders, look like without artist input.
It took a modder to make most of them looks *somewhat* decent.
wichenroder @ Nov 13th 2006 12:21PM
''it really sounds like he doesnt have much of a clue, honestly.''
josh vs ken perlin. I'd like to see this..
calthaer @ Nov 13th 2006 12:22PM
Sounds like another guy trying to make games more like movies. Who cares if the animation is impressive if the story is still essentially linear?
jcmschwa @ Nov 13th 2006 12:42PM
definitely agree with Stitched. unless coders are willing to make 100 variations of a single emotional reaction, emotions set to sliders are going to get extremely repetitive and tiring. ever try playing the sims and seeing how your character throws their arms in the air every time you won't let them go to sleep? think only a couple notches better than that and you've got depictions of emotions on a slider.
Spencer @ Nov 13th 2006 12:55PM
I saw Ken give this keynote at FuturePlay a month ago or so. His experimental programs that he uses to sketch up scripted sequences are damn amazing. The power they have comes from their ability to piece together previously animated sections, change them dynamically, and transition with realistic movements. He's a really interesting person to talk to, you can tell he really wants to make these things happen.
A note to Josh up there, he sure as hell knows what he's talking about. He showed a demo at FuturePlay that I don't believe they mention in the Joystiq article where he is able to drop in characters, and props, create actions on a timeline, and then draw (like in paint) a path on the ground and the characters will animate accordingly in real time. Its stunning how well it works, how the characters navigate a perform any combination of actions fluently the first time.
To Stitched: I agree, oblivion had some pretty wack faces, but last time I checked HL2 was more Perlins area, and HL2 is easily one of the highest achievers in the facial animation area. The modders ability to record dialogue and the software can lipsync accordingly is exactly the type of thing that Perlin is talking about. Characters need to be able to take the raw data (like sliders) and translate them into realistic movements that can reflect emotion. In fact, I'm pretty sure you and Perlin are after the same thing, I don't see why you agree with Josh.
JGH @ Nov 13th 2006 1:01PM
@3:
Are you crazy? He's Ken Perlin. As in Perlin Noise.
If you're really a graphical artist of any kind, you should know who he is, rather than speculating "it really sounds like he doesnt have much of a clue".
SE @ Nov 13th 2006 1:24PM
not only is he the "perlin noise" guy (clouds filter in photoshop for those who arent aware) the facial demo in the shot is MUCH older than half life 2, (about as old as half life 1 for that matter) but is the basis for the characters in HL2, the key isnt animation in sliders, but the ability to create convincing life like motions, which appear varied, through the application of multi-layer noise and other techniques...
this guy is a great mathematician, and a great artist, and you should do your research on him before you pass any kind of judgement.
josh @ Nov 13th 2006 1:31PM
yes yes, the BIG perlin, i know i know. but seriously, procedural animation looks like ass. maybe i'm partial to this because its what i do, but any animation needs an artists touch to look good. you can't simply break down human emotions and reactions into canned, computer generated clips.
i dont get why he mentions polar express or final fantasy. 2 totally opposite forms of technology behind each of those, and both having nothing to do with his automated sense of emoting. yes, both were pretty bad animation, but one - FF - because of the dreadful uncanny valley (and really a first attempt at its time for that level of human detail), and the other - polar express - was a technological decision by execs to use mocap, the first of its kind. they were trying to explore new tech, cut time and budgets, etc, but it still needed artists to correct it. this is proven by monster house, which is the same technology, but they put more artists in control of the final look. we'll see this again in another year or so with their next film - beowulf.
if he would take his ideas and use it to drive artist-created movements, this would be a whole lot better, but as i understand it... even those movements are procedural, right?
Topher @ Nov 13th 2006 3:03PM
Manual animation looks great for a single viewing, but is not dynamic and therefore becomes frustratingly repetitive when viewed dozens of times. Procedural movement and facial expression are the only hope for creating lifelike characters and a truly immersive gaming experience. I agree that the fruits of this fledgling field have been thus far less than impressive, but great work is being done in the artificial intelligence subfields of emotion modeling and lifelike characters. In fact, I think that it is advancements in this area, and not improved graphics, that will define next-generation gaming.
Dathos @ Nov 13th 2006 4:05PM
I think this guy, Perrin, has got the right idea. Set a bunch of variables for a person- how quick tempered are they, how patient are they, what's their sense of space. Many more and these collesce together to create a person who reacts realistically in situations. For example if in a game you walk up to someone and start talking to them they would get a bemused look, or if you stood behind them turn around cautiously, slightly fearfully- if that's what the situation entails. An animator can only do so many animation- sure they look good- but are in the long run limitted in the number of different reaction for similar situation which vary considerably. The idea is to use code to build the models and texture them so instead of making the same defencive posture and smile it will can depend on the situation. I for one can't wait for him to make his own game with the technology he describes.
Ayamikojima5 @ Nov 13th 2006 5:14PM
"maybe i'm partial to this because its what i do, but any animation needs an artists touch to look good"
Yeah, I'd say you are more than a little biased towards the need for artists being as you are one. That's too bad.
This is no longer an issue of artistry but one of pure science.
Science - requires important knowledge and logic, useful
Art - requires a high-horse to sit upon, useless
I am neither, but it is aparent enough that you never NEED an artist. An artist will convince others that they are required but they are not. "Artist" is a self-proclaimed position that carries no valuable distinctions other than an overinflated of one's own aesthetic sense.
Get a real job, hippie.
Chosin1 @ Nov 13th 2006 8:29PM
"maybe i'm partial to this because its what i do, but any animation needs an artists touch to look good"
My interpretation is that you are scared that this tech will put animators out of work. I can't say I blame you. That's your bread and butter. I'm a graphic designer so look at it this way: With all of off the shelf, destop publishing programs out there, designers still get a healthy amount of work. Why? Because some people still have the time and money to hire a professional to get professional results. If they don't they get software with a template to get them "acceptable" designs.
So basically the day is coming when you can get decent facial animations primarily from "canned, computer generated clips". Developers with better budgets will opt to hire real animators for better expressions. So animators can either adapt or find a new profession.
josh @ Nov 13th 2006 10:50PM
"Get a real job, hippie."
i work in the game biz/entertainment biz as a character technical director... its a real job, making real money, and it's real fun :)
you miss the point i think though. it IS about artistry, because he is striving to bring more appealing animation to games. this is called aesthetics... visually appealing... ie, looks good.
if you never need an artist, we're just gonna be poking around with box people in our games :)
chosin1: not really scared, no. there's plenty of work for good artists, so its not a worry at all. there just has been a lot of hoopla in our biz lately about automating/simplifying facial animation, so to see near-10 year old research that so desperately needs a kick in the nuts is ...disappointing.
facial animation in games has definitely come leaps and bounds recently, and that is mostly because we now have the ability hardware-wise, and engine-wise, to reach that level of detail. if we attempt to go down the procedural route, we wont be hitting the 'film quality' that we strive for anytime soon. its 'film quality' because we have a talented pool of people making sure that every frame is perfected.
now, with all that said... i'm sure he has his own team of artists saying the very same stuff, its just a matter of melding the ideas into an aesthetically and technologically pleasing one.
Ayamikojima5 @ Nov 13th 2006 11:05PM
hmm, despite the fact that my comments were meant to be mostly satirical and mostly for my own amusement, I am surprised to see that you A JOYSTIQ POSTER responded with civility and intelligence.
Weird.
You arent supposed to do that. Joystiq posters dont do that. Joystiq posters either ignore you or get pissed and try to start a flame war. Weird dude.
Utmost respect @ you.
oh and "there's plenty of work for good artists, so its not a worry at all"
If I can take your word for it, I'll have to tell that to my art major friends, being as most of them are already planning to be homeless and jobless. Poor saps. There is hope...oh wait, you said "good" artists...nevermind then, that doesnt apply to my friends.
josh @ Nov 13th 2006 11:41PM
hahaha
sorry for not following the guidelines set before me... ;)
yeah, plenty of jobs out here in the games/movie/entertainment industry... if you have the skills and contacts. bottom line, gotta be good, otherwise there are 500 other fresh out of school students that can suck right alongside you ;)
Matt @ Nov 14th 2006 12:29PM
Josh: As an animator, do you want to have to animate every little facial feature during, say, a cutscene?
No, you want to animate the character's kung-fu moves and walk cycle.
While his research about feet and hands would be worth looking at for you, I don't think you have much to fear from his procedural emotion work. It just means that as an animator you get to spend time working on the parts of the character that need it, and you can leave things like changing emotion to code.
Not to mention, *somebody* has to set up all the default emotion poses and the rules for how they relate. What Perlin describes isn't so much absolute rules for a human face, but a way in which the elements interact to create believable human motion. Just like rigging a skeleton or setting up blend shapes, an artist will need to define those elements in any real-world application.
Just like higher-level languages make a programmer's life easier, the only thing this research is going to do for animators is make their lives easier. With the increasing scope of games, it's not only convenient, it's neccesary.