Demonstrated: Controlling facial expressions in-game with camera
by Christopher Grant 
{ Oct 15th 2009 at 10:45AM } 
In his latest video (found after the break), Sko demonstrates how to control the facial expressions of a virtual character using nothing but a plain ol' web cam (oh, and the same FaceAPI software he used for his first demo). Much like Hideo Kojima's visions of a Natal-enabled future, Sko's never involve replacing the controller (or keyboard and mouse) but rather supplementing it with, in this case, your face. No word on support for particularly ugly faces, but we suspect Microsoft's got its best and most beautiful people on it.
Get a WordPress.com Blog
Reader Comments (Page 1 of 2)
SpyderTaco (PSN: Ar4chNova89) @ Oct 15th 2009 10:49AM
...Half-Life 2 was / is such a sexy looking game.
When the hell will they give us a taste of episode 3? WHEEEEEEEEEEEE...
"Rather than offer you the illusion of free choice, I will take the liberty of choosing for you." - G-Man
Enzemo @ Oct 15th 2009 11:23AM
The graphics in Half Life, and that of any game running on the Steam engine is, and always will be, substandard in todays market.
The engine is simply not powerful enough to perform at todays standards and the graphics serve only for hardcore fanboys who can't give it up and nostalgia
SpyderTaco (PSN: Ar4chNova89) @ Oct 15th 2009 11:26AM
What the hell are you talking about? I've been constantly playing Uncharted 2 for the past few days and I still think Half-Life 2 is a great looking game.
You could make the same argument about something like WoW. Yea, the graphics are not 'high-end' and 'cutting-edge', among other 'back-of-the-box' terms you want to use, but the style of both games is enough to keep the eyes pleased.
Jose [Steam & PSN: Assmar] @ Oct 15th 2009 11:32AM
I'm with Spyder.
At least Source games have style and don't look like they were shat out by the vaseline monster, except for TF2, but that was intentional!
abelpc @ Oct 15th 2009 12:19PM
@Enzemo.. Steam isn't a game engine. Steam is a service to purchase and download PC games.
RageOverdose @ Oct 15th 2009 12:24PM
I think Half Life 2 looks dated, but Episode 2 more than keeps the world pretty. As for style, I don't really think Half Life has ever had much of a "style" beyond realism.
But, the Source updates keep the engine looking fresh. What it lacks in specular maps it makes up for in lighting and animation, as well as performance.
Besides, TF2 is a great looking game, especially with it's style.
LegendaryRedass @ Oct 15th 2009 2:40PM
It's the source 2 engine. It was plain source for halflife 1, and became source 2 for halflife 2. Even though they refer to it as only source. And halflife 2 still looks awesome, even on 360, even though it's just a port.
Mike @ Oct 15th 2009 6:42PM
@Legendary - You could not be more wrong.
Powerlord @ Oct 15th 2009 11:58PM
Legendary:
Actually, there's a story behind the name
First, the HL1 engine is referred to as GoldSrc.
The HL2 engine is referred to as Source.
The story behind these names is on the Wikipedia GoldSrc page: http://en.wikipedia.org/wiki/GoldSrc
copa @ Oct 15th 2009 10:49AM
You haven't Quincy'ed... until you've Quincy'ed with Natal.
Courtney @ Oct 15th 2009 10:55AM
I can pretend to be Sean Connery! Fucking Awesome!
Fine Leather Jackets @ Oct 15th 2009 11:00AM
More importantly, Sean Connery can pretend to be you.
Space @ Oct 15th 2009 11:46AM
"I'm waiting to be impressed"
MaliceMajorE15 @ Oct 15th 2009 10:56AM
reminds me of the japanese PSeye tech demo from a couple of months ago
ChomskyKnows @ Oct 15th 2009 11:04AM
can we get Jenna Haze to do this please?
uncle jesse @ Oct 15th 2009 11:31AM
YOU can actually BE Jenna Haze...
*shudder*
Chas Winterbottom @ Oct 15th 2009 11:43AM
I can't tell if that would suck more or less than the game Haze.
Space @ Oct 15th 2009 11:48AM
I've seen her work, she sucks pretty hard.
uncle jesse @ Oct 15th 2009 11:53AM
I think we can all agree that no matter what Haze we are talking about, they all blow.
Exemplar @ Oct 15th 2009 12:45PM
*slow clap*
james @ Oct 15th 2009 1:20PM
well said gentlemen
WiNG [XBL&Steam: WiNGSPANTT] from lifeinagame.com @ Oct 15th 2009 11:04AM
It would be pretty hilarious to be playing Gears 3 and literally see the look on your enemy's face when you jump around the corner and chainsaw him in half!
SpyderTaco (PSN: Ar4chNova89) @ Oct 15th 2009 11:07AM
You're the reason why innovation fails to hit mainstream.
...BUT WHAT IF IN "HALO 26: REVENGE OF THE CHIEF" YOU COULD LOOK INTO MY ENEMY'S EYES WHEN YOU PRESS "CROUCH" A WHOLE BUNCH?!
PedoJokerBear [Planeteer | Power of Gall Bladder] @ Oct 15th 2009 11:22AM
whoa whoa whoa whoa....theres a halo 26?
The Dark Wayne [Planeteer: Power of the Batusi] @ Oct 15th 2009 11:29AM
yeah that's exactly what keeps innovation away from most games. Geez
ZaxCG2 @ Oct 15th 2009 12:37PM
I CAN'T WAIT FOR QUADRUPLE WIELDING!
Hey.
Guess where he holds one of his guns! :D
sonicspike41 @ Oct 15th 2009 2:19PM
Chief: "Sarge, cover my rear... I'm goin' in!"
Johnson: "But you're covering your own rear! Aw hell... don't go dyin' on my now soldier! Show those alien scum we ain't going down without a fight!"
sonofman @ Oct 15th 2009 11:21AM
Now they need to build a Star Trek game around this, where you control Spock and must lift your eye brow up at the correct moment during various encounters.
You could also make statements about how illogical things are by using a mic.
Poisoned Al @ Oct 15th 2009 11:31AM
...
HAAAAAAAAAX!!!!!
Das Boot @ Oct 15th 2009 4:06PM
Shut up Dr. Hax.
(In case you missed the reference, here's a link: http://www.youtube.com/watch?v=t2eUUHkpvQA )
Timerider @ Oct 15th 2009 11:32AM
Faceposer made easy. Maybe this will actually work on systems other than XP.
mr nimblewick @ Oct 15th 2009 11:34AM
Kind of cool, but not that much, really. Maybe this would be most useful for motion capture, having the voice actors themselves guide the facial expressions on characters?
I would like to see a practical, real-time application in a game.
uncle jesse @ Oct 15th 2009 11:58AM
Yeah, I don't really see how this could be something that would enhance actual gameplay.
Maybe Milo can tell see how you're reacting to things he says or situations and then respond accordingly or something.
That Fuzzy Bastard @ Oct 15th 2009 11:42AM
This could be the best enhancement to multiplayer gaming since XBL. Would do a lot to make it feel more personal, more like actually hanging out with friends.
ZaxCG2 @ Oct 15th 2009 12:39PM
Or you could ... you know... actually hang out with friends?
JeezWhiz87 @ Oct 15th 2009 11:46AM
I'm hard pressed to find an application for this that I can care about, but it might slightly enhance online communication, in say "Home" or something. I don't know, it doesn't really get me too amped though.
Space @ Oct 15th 2009 11:51AM
I don't get it either. how often do you see your characters face? Unless it's a game based around making stupid faces...
uncle jesse @ Oct 15th 2009 11:54AM
Confirmed: Stupid Faces - Q4, 2010.
Exemplar @ Oct 15th 2009 12:36PM
Oh, the porn industry will figure something out. They always do. You'll see.
uncle jesse @ Oct 15th 2009 12:43PM
Confirmed: 'Oh' Faces - Q4, 2010.
fuzzynyanko @ Oct 15th 2009 12:35PM
I remember Sony doing something similar. Sony's had problems with the face spazzing, and this seems less evident with this implementation. However, these guys didn't turn their heads to the side, where there were the most problems.
Dirty @ Oct 15th 2009 12:38PM
Im happy that my lifeless zombie stare can be captured and conveyed on screen while I play games.
Vcize @ Oct 15th 2009 12:46PM
Looks laggy :(
horngreen @ Oct 15th 2009 1:24PM
Everyone has done more with existing tech on game console than the console manufacturers themselves. Why is it always some guy who pulls this stuff of yet nobody in authority ever implements it into a game? I mean I'm so glad I bought the Live vision camera for R6Vegas and NOTHING ELSE!
Christian @ Oct 15th 2009 1:31PM
I find the picture a bit disturbing with the guy having such a long face...
Case @ Oct 15th 2009 1:36PM
Wow...that looked absolutely terrible. I guess if you want to be able to track whether or not the lips are moving (nevermind accurately), and track your eyebrows poorly, this is useful. Otherwise, file away as, "OK. Next."
There's really no application for this.
Asmerom (PSN: neon6plz) (Steam: neon6669) @ Oct 15th 2009 1:43PM
Is it possible I can mouth "Fuck you" without actually saying it?
gaddes @ Oct 15th 2009 1:50PM
I'm not really sure why people say there's no application for this. If the tech was refined a bit to get some better / more accurate expressions, then this could be a great tool to help improve facial animation in games... which is something that is currently severely lacking (with a handful of games as an exception). This is NOT to say that it would replace animators in the task, but that it would help easily create more base animations quickly and easily for animators to work with.
Ashitaka @ Oct 15th 2009 2:53PM
"If the tech was refined a bit to get some better / more accurate expressions"
You see, that's the problem.
Case @ Oct 15th 2009 5:28PM
Ashtika makes my point exactly. Perhaps I shouldn't say there's no application for this, but I should say there is no application for this product specifically. It looks terrible. There are already facial mocap applications and software out there that do a far more accurate job. All facial movements in games are not hand animated. Heck, a quick google search will get you to OptiTrack which is doing facial mocap for about $5-6,000. That's well within a game developer's budget, and they'll actually get exportable data and far more accurate motions.
This quality of tracking was a starting point...about 10 years ago.