Get your head in the game with Wii remote VR display
by Griffin McElroy 
{ Dec 23rd 2007 at 10:30AM }
While other people use the Wii for its intended purposes (playing party games and bludgeoning loved ones), Johnny Chung Lee sees its true potential. You might remember him as the guy who used a Wiimote to set up a Minority Report-esque finger tracking system, or the one who used the same peripheral to turn any surface into an an incredible multi-touch interactive whiteboard.
Further proving that Lee is a reverse engineer from the future, his newest video shows what might just be the next big thing in gaming -- immersive virtual reality displays using, you guessed it, a Wii remote. While this may conjure up bad memories of massive, plasticky helmets and unresponsive controls, Lee's method seems extremely functional, and only requires you to wear a pair of LED-infused safety glasses (which for all we know is the hip style in the futureworld Lee comes from). Seriously, how 'bout it, Nintendo?
(Thanks, Rich.)
Reader Comments (Page 1 of 2)
KayRazy Ken Kutaragi @ Feb 22nd 2008 2:52AM
I'm pretty sure you can do this with the Xbox Vision Camera or any web cam.
Feel pretty stupid now dont ya. The Wiimote is nothing special...Gods, get off the Plumber's cock b4 u guys get split in half.
FalseStream @ Dec 23rd 2007 10:41AM
That looks sick. This would work good for games like Time Crisis where your hiding behind things and have to jump out. Would probably make a game like that suck a little less.
Platy @ Dec 23rd 2007 1:25PM
somehow i thinked that the metroid prime 3 was going to use something like these because of those overreacted ads xD
Harbl @ Dec 23rd 2007 2:50PM
They have that already. It's that cooky Japanese cop game where it scans your head and you have to duck and weave bullets and cover. It's really awkward and you look like a total tard doing it.
Frowelishnu @ Dec 23rd 2007 9:45PM
@ Harbl:
Even though you look like a tard and the graphics on his software weren't great (stadium shot excluded) - this is mind blowing stuff.
I'm not a religious man, but when I saw the floating targets I saw god. Ok, hyperbole but it was awesome.
Autopsy15 @ Mar 11th 2008 2:04AM
He was talking about that Tokyo Cop game, not the guys tech demo. DOYYYY!
Fernando Rocker @ Dec 23rd 2007 10:44AM
Wow!
Jeff Smith @ Dec 23rd 2007 10:46AM
Didn't know what to expect at first. . . and I was starting to get bored. Wait! What's that!?
Damn that was cool! That would be a great addition to games.
Tyrus @ Dec 23rd 2007 10:49AM
I was going to rag on the guy for just finding different things to stick infrared LEDs to for another Wii experiment, but man, that was awesome. It's too bad that there's no real way to make this effect happen for multiple people on one display. I imagine you would be able to make it work for split screen, which would be disorienting for anyone that looked at a display not meant for them.
I wonder how 3D this truly appears to the average user. I imagine it's a step down from holograms, but I'd also think that the image would be that much easier on my eyes.
Chimp @ Dec 23rd 2007 1:30PM
That's what I want to see in the next Wii! But for the love of god don't tape to Wiis together nintendo! Just make something that will compete with the PS4 and Xbox 3 on terms of power and storage and connectivity.
SoulBlade @ Dec 23rd 2007 11:23AM
Nintendo won't do that - you'll probably see something with 360 capabilities from them the next time - they're all about making a cheap system that will sell and make them profit.
Platy @ Dec 23rd 2007 10:55AM
Nintendo On ! xD
timnc @ Dec 23rd 2007 10:57AM
Holy......
Obie @ Dec 23rd 2007 10:59AM
Nintendo needs to hire this guy and pay him whatever he wants, he is a genius! ( and obviously a fan)
Get on it Reggie,after selling all those Wii's you can afford it!
vidGuy @ Dec 23rd 2007 11:00AM
User Jayenkai brought up a good point about this demo on the WiiFanboy boards (two days ago, where's Joystiq been?)... the camera he uses to show the effect only has one 'eye'. The 3D effect may not be as dramatic to a person using both eyes to view the image.
Still impressive none the less. I'm sure this type of thing will be utilized by games within ten years, and we'll always know that the Wii did it first.
Jevanzz @ Dec 23rd 2007 11:41AM
Ehh, no, the Wii didn't do it first, that fat chinese guy did.
Spencer @ Dec 23rd 2007 11:54AM
Actually, this was done on a pc. The only Wii hardware used was the infrared camera of the Wiimote.
vidGuy @ Dec 23rd 2007 12:08PM
"the infrared camera of the Wiimote"
Which was first used in console gaming, where?
Blahblahblah @ Mar 17th 2009 5:06PM
Are you talking about consoles only? This has been available on PC for awhile through:
http://www.naturalpoint.com/trackir/02-products/product-videos.html
http://www.youtube.com/profile?user=NaturalPointKevin
http://www.youtube.com/watch?v=_AO0F5sLdVM
Blahblahblah @ Dec 23rd 2007 12:18PM
Ah, crap, replied to late. My first comment was in response to the one you posted at 11:00AM.
Fernando Rocker @ Dec 23rd 2007 12:19PM
blah
The Wiimote is the first infrared camera or consoles, to the mainstrem, used as the primary controller.
It's what Nintendo do first always... D Pad to consoles, to all people, analog stick, rumble, wireless controllers, and now motion and the infrared camera for he pointer. And touchscreen to a gaming device (wich Im sure the PSP 2 will have touchscreen too)
And believe me, Sony and Microsoft will use motion and pointer capabilites in their next systems. And all because Nintendo.
ill trooper @ Dec 23rd 2007 12:37PM
Ok, fernando...
And I'm sure that a few years from now when you compose your next list of Nintendo innovations, you'll likely omit this little gem: a report about a Sony patent filing from 2003 for a pointing system nearly identical to what's happening here that uses infrared indicators on the controller to interact with a camera facing the user to interact with the application running on the console:
http://www.joystiq.com/2005/10/03/sony-has-its-own-magic-wand-in-the-works/
vidGuy @ Dec 23rd 2007 1:00PM
And, ill trooper, when was this wonderful Sony device introduced to the public, reaching tens of millions of people?
bm @ Dec 23rd 2007 1:03PM
@"ill trooper":
Dude, there are pre-existing patents for just about EVERYTHING. The point is that Nintendo brought it to market, and not just as any damn optional peripheral, but the PRIMARY method of control standard for all games on the console.
Deny it all you want, but remember that Sony called motion sensitivity and the pointer device a gimmick right after it was announced... then what did they do when it became so popular? That's right.
And that's the point of the argument. It's not that Nintendo are the only ones who have any ideas, ever. It's just that Nintendo stick their neck out where nobody else will, test the water, and then the rest of the pussies (Sony et all) follow like sheep when they notice it's safe to do so.
Spartacus @ Dec 23rd 2007 5:06PM
That and "head tracking" isn't the same thing as 3D. A 3D image is viewable to anyone with the proper eyewear as it is a composite of 2 different (preset) cameras/viewing angles.
I would actually rather have 3D games than ones with head tracking capabilities for the same reason I would rather play on a standard controller than with a Wiimote- I don't have to move around and look like an idot while enjoying my gaming experience.
ill trooper @ Dec 23rd 2007 8:03PM
hmmm. You guys don't like factual timelines I see!When did nintendo do all this exactly? If you bother watching the video, you'll see that Johnny Lee devised this, and that the Wii-mote isn't even an essential part of the equation. So you can attempt to debate all you want, but what you're seeing conceptualized in this video is actually closer to what Sony has patented than what Nintendo themselves are doing, which is in itself very cool, but NOT what has been done here.
As for recognizing what you see in Johnny Lee's inspiring work (this is only one of a few that have regularily been posted days than Joystiq posts them), it seems to come down to how much you hate Sony or how often you like Mario's nuts on your chin. So just say that instead while you ignore that Nintendo had nothing to do with what you see Johnny Lee doing.
phihexy27 @ Dec 23rd 2007 9:57PM
"And believe me, Sony and Microsoft will use motion and pointer capabilites in their next systems. And all because Nintendo."
@Fernado
Well it would be awfully nice if Nintendo would take a page out of Microsoft's book and do something with there shitty online service. Or better yet take something from both Sony and Microsoft and have a formal licensing process for development on their console. It's all good and well that Nintendo is innovative with there control schemes, but how much does it matter when 90% of the games on their console are all shit. I enjoy playing good games on my Wii, just like everyone else, but they are definitely few and far between.
As far as the video goes, that was some really cool stuff.
Kassu @ Dec 24th 2007 7:01AM
BTW Fernando, Atari had the first wireless controller, if i'm not mistaken.
bm @ Dec 26th 2007 10:32AM
illtrooper: Let's not pretend that you're -not- one of those people who would like to believe motion sensitivity is totally Sony's thing or some bullshit. That's the core of the matter.
Lanturn of Ash @ Dec 23rd 2007 11:01AM
Holy shit, DUMB. Why you waste my time with crap videos Joystiq?
JK, that's awesome as hell. I'm completely envious of his TV alone, none less the fact that it's almost a ****ing portal to a 3D world.
Kangstarr @ Dec 23rd 2007 3:53PM
Nintendo might have innovative tech but their industrial design team might have borrowed some of their designs from Apple.
http://gonintendo.com/?p=920
*some of it is a stretch but pretty fascinating regardless*
Kn1ves @ Dec 23rd 2007 11:06AM
u guys ever play that one shooting game at the arcade that tracks players movement to duck for cover?
yea ur legs is gonna be sore.. along with flailing arms
great idea though. i'm going to try it out.
bm @ Dec 23rd 2007 1:05PM
You guys ever play that one game with the physical movement? You know, the one that's on TV all the time and people play in two teams on a field of grass? Man, your legs are gonna be sore from that.
CharleyTony @ Dec 23rd 2007 11:21AM
I think this is awesome,, Nintendo should really talk to this guy, buy his ideas, give him a job and use his help to mae the next nintendo system... the Wii 2 might only be 4 years away... and it will surely be BC with the first Wii, so the same tech this guy is playing with will surely be available on Nintendo's next system
Myke @ Dec 23rd 2007 11:27AM
Wow, that guy is a genius, seriously. I can see him getting a job offer from a number of different developers, it's just a matter of time.
ivan @ Dec 23rd 2007 11:29AM
is this a glimpse of the future?
Siegvar @ Dec 23rd 2007 12:14PM
Hmm... if the wiimote he uses as the 'gun' still tracks position based on calculation between the infra-red led on the glasses and the sensor in the wiimote, When he moves his head, his crosshair will move as well, even though his hands hasn't changed its aim direction.
I think you can catch a glimpse of this happening in the video if you're sharp. But then, maybe the position of the crosshair is correct from the view of the guy wearing the glasses? Gah, the maths is hurting my sleepy brain.
Raikage (ROFLBERRY PWNCAKES Defence Force) @ Dec 23rd 2007 3:18PM
Thats Interesting, because if it doesn't, then the whole movement in a 3D world is pointless unless your gun could move in it too.
But I'm not that great with this stuff so I'll let him figure it out :D
thyaosands! @ Dec 23rd 2007 5:31PM
in theory couldn't he set up an additional IR sensor in front of the TV that the second remote was tracking on? This way the second remote is tracked steady on the screen, while the movement tracked separately. Am I wrong in the assumption? I just assumed this is what happened.
DaiMac79 @ Dec 23rd 2007 12:23PM
Wow, this is truly awesome, I can see a game doing this on the Wii at some point. maybe a Wii Helmet periph with an even better IR cam that you place by the TV?
Mr Khan @ Dec 23rd 2007 12:43PM
Wow
Now he needs to send this about 13 years back, so that Nintendo can use it to make the Virtual Boy... work
Brucie B @ Dec 23rd 2007 1:23PM
Can't Agree More, That Was AWESOME!
BlackTye @ Dec 23rd 2007 1:40PM
Nintendo.... hire this man!
Bluebrake @ Dec 23rd 2007 2:07PM
Johnny Lee is some kind of techno superhero. And he has a badass name. Why hasn't somebody hired him?
And does anyone else feel like this setup does everything the Wii Balance Board does, but way better, and in a much simpler and cheaper package?
Kazi @ Dec 23rd 2007 2:22PM
No. The Wii Balance Board uses pressure sensors. While there's some things both his VR Glasses and the Wii Balance Board can detect they hardly overlap in that area. In fact the things they do overlap in (Leaning, Jumping, ect) is a pretty vague overlap in itself. The Wii Balance Board would actually detect the vague areas better but with the Glasses the detection would be improved. So while the Board would decect your shift in weight the Glasses would detect where your head is positioned making a crazy mutant technology where you can shift weight to one side and lean over. OMG CRAZY!
Bluebrake @ Dec 23rd 2007 2:23PM
And did anybody see his foldable displays demo? http://www.youtube.com/watch?v=nhSR_6-Y5Kg
Freaking crazy. It's less clear in that one how it works, though.
Seriously, in 1 month this guy has come up with a good 4 innovations that are each roughly a million times better than anything Nintendo or its 3rd party developers have even dreamed of in 2 years. They should be ashamed of themselves.
Lanturn of Ash @ Dec 23rd 2007 2:24PM
Pshhh. Everyone in China is named Jon Lee! (Just watched The Replacement Killers last night)
Bluebrake @ Dec 23rd 2007 2:27PM
@Kazi:
Leaning and jumping are pretty much all I've seen the balance board do so far, and the goggles method can do them far more precisely. I guess it can't calculate your body mass index, but... who cares?
Kazi @ Dec 23rd 2007 2:44PM
There's many things you can do with weight detection. Certainly not as many things as you can do with head position detection but the possibilities are there. On top of amazing detection on jumping and leaning it can detect when you lift your feet and at what weight/speed you're lifting them and moving them at (for instance more weight in your run would translate to longer, faster strides in the game), of course it's namesake it can detect balance (walking across a tight rope?). The Board is certainly more appropriate than head detection in Wiifit (as if we're going to see the Wii Balance Board applied to many games besides Wiifit 2 anyways).
Alcevious @ Dec 23rd 2007 2:25PM
This is the future of gaming. I'm stoked.