It seems so obvious: Using Kinect to help people learn American Sign Language. That's exactly what researchers at Georgia Tech College of Computing are working on, pairing Microsoft's oft-hacked motion sensing camera with custom software that previously required colored gloves kitted with wrist-mounted, 3-axis accelerometers. On a series of increasingly difficult tests, the software returned results with 100% accuracy, 99.98% accuracy, and 98.8% accuracy.
These promising results mean the team will be working on updates including a larger vocabulary which necessitates the need of "hand shape features." The initial proof-of-concept demo launched with a small vocabulary that excluded them in favor of broader gestural movements with the arms and body. We imagine that reported fourfold increase in Kinect image resolution would have a major benefit here, should Microsoft ever release it.
Reader Comments (42)
Posted: Dec 20th 2010 4:31PM leemahi said
this is going to be the most important hack yet. Way to go microsoft in creating such a amazing device. We all thought kinect was going to suck, but they pulled it off.
Posted: Dec 20th 2010 4:59PM Hunter141072 said
@leemahi
yeah, too bad the ones who didn´t make it suck were the guys from the mod community, as I said before... They are the ones that are doing awesome things with kinect, let´s see which "dance" game can be better than this......
Reply
yeah, too bad the ones who didn´t make it suck were the guys from the mod community, as I said before... They are the ones that are doing awesome things with kinect, let´s see which "dance" game can be better than this......
Posted: Dec 20th 2010 7:07PM Jack Kevorkian said
@leemahi I agree with it becoming the most important. I really hope they can use sign language recognition for something meaningful.
Reply
Posted: Dec 20th 2010 8:43PM This Little Man Says His Name Is said
@fohf
Microsofts software is the firmware and such that runs with Kinect.
Any games and such that suck are made by other companies.
Everything these hacks are doing are because of what MS did with making the Kinect.
Reply
Microsofts software is the firmware and such that runs with Kinect.
Any games and such that suck are made by other companies.
Everything these hacks are doing are because of what MS did with making the Kinect.
Posted: Dec 20th 2010 4:31PM dudes113 said
In terms of gaming I've been more interested in the Move, but the Kinect has so many more practical uses. All these hacks are much more interesting to me than gaming application of it.
Posted: Dec 20th 2010 5:01PM dudes113 said
@eNriqeu
Maybe not this hack specifically for you, but the web browsing hack, Minority Report-like UI hack, and holographic video hack could eventually be used in everyday life as the new standard. Its still early technology and application of it, but who knows what this device could lead to.
Reply
Maybe not this hack specifically for you, but the web browsing hack, Minority Report-like UI hack, and holographic video hack could eventually be used in everyday life as the new standard. Its still early technology and application of it, but who knows what this device could lead to.
Posted: Dec 20th 2010 4:33PM DragonRyder said
The more I read about these hacks the more I want a Kinect for my computer. Still not so sure about the Xbox gaming application of Kinect. Maybe if a game like Infinity Blade was actually made for Kinect, then we would have our killer app.
Sign language is such a smart application. I do hope Microsoft sees all the potential in Kinect.
Sign language is such a smart application. I do hope Microsoft sees all the potential in Kinect.
Posted: Dec 20th 2010 10:17PM Jack Kevorkian said
@DragonRyder Post script: Infinity Blade is one of the most overrated games of 2010.
Reply
Posted: Dec 20th 2010 10:59PM DragonRyder said
@Jack Kevorkian Have you played Infinity Blade? I can only imagine that it would work even better with Kinect, because my finger would not get in the way of the pretty graphics.
Reply
Posted: Dec 20th 2010 4:35PM HeriocZebra said
Hacks
Hacks everywhere
Hacks everywhere
Posted: Dec 20th 2010 4:42PM Robborboy said
"Awesome!" is all I can say for this.
Posted: Dec 20th 2010 4:44PM UnnDunn said
Someone needs to release an American Sign Language tutoring game, in the style of Dance Central, complete with Achievements.
This needs to happen now.
This needs to happen now.
Posted: Dec 20th 2010 4:56PM Tachyonic Cargo said
Wait a minute. I thought there was a news story several months ago (before Kinect launched), of Microsoft stating that Kinect (out of the box, with no hacks) could recognize ASL. And while none of the launch games would use the ASL recognition, that future games for the Kinect would.
In what manner you would use ASL in a game, is anyone's guess, but I use it every day to train my dogs, so I could guess at some uses for it in-game, but I am guessing the game would have to be built with ASL usage in mind - maybe something like ASL of the Dead (a Typing of the Dead sequel).
But if nothing else take things like the Kinect Dashboard . . . you know the bit where, "if you can see it, say it," rule applies. Well someone who is mute cannot exactly "say it". ASL support in the dash would defnitely be nice there.
In what manner you would use ASL in a game, is anyone's guess, but I use it every day to train my dogs, so I could guess at some uses for it in-game, but I am guessing the game would have to be built with ASL usage in mind - maybe something like ASL of the Dead (a Typing of the Dead sequel).
But if nothing else take things like the Kinect Dashboard . . . you know the bit where, "if you can see it, say it," rule applies. Well someone who is mute cannot exactly "say it". ASL support in the dash would defnitely be nice there.
Posted: Dec 20th 2010 8:45PM This Little Man Says His Name Is said
@Tachyonic Cargo
Or they could you know, just use their hands and press the button instead of using ASL to say "press this button"
Reply
Or they could you know, just use their hands and press the button instead of using ASL to say "press this button"
Posted: Dec 21st 2010 1:38AM Adinnieken said
@Tachyonic Cargo The news was (and is) that Microsoft patented the ability to recognize sign language using a device like Kinect. Thus, I'm sure Microsoft if it isn't sanctioning what Georgia Tech is doing, will be sending them a letter letting them know they own the patent.
Reply
Posted: Dec 20th 2010 5:03PM Fermie Prime said
Go Tech!
Posted: Dec 20th 2010 5:04PM Mannyman said
I admit that it's crazy to say, but I think there actually might be some potential for Microsoft or another publisher to release educational software for the Xbox 360. Or as they used to say back in the day, "edutainment".
Posted: Dec 20th 2010 5:35PM Mannyman said
@PR0F3TA
Hell yeah they were fun! Reader Rabbit, Oregon Trail, Mario Teaches Typing were all games I grew up with and were a lot of fun to play. Don't get me started talking about Carmen Sandiego, the damn game tricked me into reading encyclopedias for hours! There's probably still a market for it, if someone steps up.
Reply
Hell yeah they were fun! Reader Rabbit, Oregon Trail, Mario Teaches Typing were all games I grew up with and were a lot of fun to play. Don't get me started talking about Carmen Sandiego, the damn game tricked me into reading encyclopedias for hours! There's probably still a market for it, if someone steps up.
Posted: Dec 20th 2010 5:58PM Stevetrop Man of Mystery said
@Mannyman
I'm 28 and I'd love to see a new Where in The World is Carmen Sandiego. I spent hours playing that game on my nes growing up.
Reply
I'm 28 and I'd love to see a new Where in The World is Carmen Sandiego. I spent hours playing that game on my nes growing up.
Posted: Dec 20th 2010 6:10PM MrXBob said
I love all the publicity these hacks are getting. Finally all the unknowledgable fanboys who called Kinect "an EyeToy rip-off" are seeing that it's far from it.
People who took the time to listen to Microsoft at the start knew this from the beginning - but of course Anti-Xbox fanboys ignored it whilst playing with their Wii rip-offs (sorry, I mean Move)
Hate on Kinect games all you like, but there is now no denying that the device is a huge technical achievement. Pun intended. I still laugh when people say EyeToy can do everything Kinect can do. Lol.
People who took the time to listen to Microsoft at the start knew this from the beginning - but of course Anti-Xbox fanboys ignored it whilst playing with their Wii rip-offs (sorry, I mean Move)
Hate on Kinect games all you like, but there is now no denying that the device is a huge technical achievement. Pun intended. I still laugh when people say EyeToy can do everything Kinect can do. Lol.
Posted: Dec 20th 2010 6:32PM Hank Hill said
I literally shouted out "GENIUS" when I came across this. Ladybird gave me this funny look, I should stop doing that...
Posted: Dec 20th 2010 6:33PM beanninja said
Let's give them a hand!
Posted: Dec 20th 2010 7:25PM Snap Count said
Awesome so in the next GTA I can give someone the finger before I off him...the possibilities are endless.
Posted: Dec 20th 2010 7:29PM RoboZombie585 said
I am quite surprised. While Kinect might be the most over-hyped gaming attachment ever, I think it could be the most important thing ever invented in the history of everything else if this pace of innovation keeps going.
Posted: Dec 21st 2010 1:56AM theultimate13x said
This is my Killer app. Thank you Kinect!
Come on Microsoft. Release an updated Kinect(hardware) and the XNA Kinect software so these things can come to my console.
Come on Microsoft. Release an updated Kinect(hardware) and the XNA Kinect software so these things can come to my console.
Posted: Dec 21st 2010 3:01AM Machiavellian79 said
The problem with gamers is that they want everything now. Most of you just need some patience. Hacks are easy because they are just proof of concepts.
Making a game that will entertain for hours is something totally different.
There are some interestin stuff coming out of Japan and we really haven't seen anything yet from Kinect. It will take time for developers to go from Proof of concept to actual games that haven't been done before. Those take time and Kinect isn't something easy to get right the first time.
I wasn't all that convience about Kinect as a gameplay add on but now I am interested to see given enough time and MS coming through providing better support for Kinect where this device can go.
Making a game that will entertain for hours is something totally different.
There are some interestin stuff coming out of Japan and we really haven't seen anything yet from Kinect. It will take time for developers to go from Proof of concept to actual games that haven't been done before. Those take time and Kinect isn't something easy to get right the first time.
I wasn't all that convience about Kinect as a gameplay add on but now I am interested to see given enough time and MS coming through providing better support for Kinect where this device can go.
Posted: Dec 21st 2010 9:00AM MasqueradeRed said
This is brilliant!
Posted: Dec 22nd 2010 10:33AM insciens said
KINECT - It only does EVERYTHING!







