Augmented reality shooter shows the future of camera-enabled handhelds

by Randy Nelson { Jun 18th 2009 at 1:00AM }

Augmented reality (AR) has mostly been limited to research labs and the occasional cool webcam trick, but it's slowly becoming more plausible as a means for new forms of portable games. We saw one such title -- Sony's Invizimals for PSP -- at E3 and were wowed by its insertion of CG objects into "reality" using the system's camera add-on. Now a tech demo from Georgia Tech Augmented Environments Lab and the Savannah College of Art and Design (SCAD-Atlanta) gives us a glimpse at even more possibilities for these games on devices such as iPhone, PSP and Nintendo DSi.

Titled ARhrrr (geddit?), the game is a zombie shooter in which a flat tabletop map produces a cityscape seen on the handheld (in this case a prototype device from Nvidia). The city's populated with zombies, the handheld can be moved around at any angle and the overlayed city rotates accordingly. The idea behind the demo is that players are controlling a helicopter flying above the outbreak and can shoot zombies by tapping the screen. It's limited, but nonetheless really cool.

We've popped the demo footage after the break to rock your non-augmented reality. It's pretty impressive. So, what implimentations of AR on handhelds would you like to see?

[Thanks, Michael!]

Tags: arhrrr, augmented-reality, video

Reader Comments (Page 1 of 1)

deathraver @ Jun 18th 2009 1:06AM

that's actually extremely cool

Dr. Stabbingworth @ Jun 18th 2009 1:55PM

I think we're seeing the limitations imposed by our current input technology being washed away.

One of the greatest limitations of this incredible hardware we have is an inability to properly interact with it.

We can keep making stuff incrementally faster, but that won't do us much good unless we can quickly and intuitively access that power with new controls.

Joel @ Jun 18th 2009 1:06AM

left out a bit of info their. The nvidia processor is called the Tegra processor. very very powerful processor for mobiles. It is rumored to be powering the Zune HD. Article is on engadget :)

ifalldownstairs @ Jun 18th 2009 1:33AM

This is coming from a Zune 80 owner who plans on getting a Zune HD -

unfortunately, the Zune HD has a very un-HD resolution (480x272), which would make advanced games a bit difficult. Its actually a disappointing resolution when compared to phones like the HTC Touch HD or Samsung OmniaHD. The reasoning behind using Tegra would be to power the 720p video-out stream the Zune HD will be capable of, not to power a large screen. Also, the lack of a camera means games like this are obviously impossible.

Joel @ Jun 18th 2009 1:37AM

I am under the impression that the "HD" represents the HD Radio that is built in to the Zune HD, but I may be mistaken?

kojo87 @ Jun 18th 2009 2:39AM

Joel is right. and HD is still used as a buzz word for no reason. the HD in HD Radio stands for Hybrid Digital not High Definition. kinda disappointing that Microsoft still doing this.

i have a Zune 120GB. would really like it if Microsoft made a hard drive based Zune HD. even if its not 120GB i can still swap out HDDs.

Chuck @ Jun 18th 2009 1:50AM

I thought it was neat but weird.
For some reason I thought it was really cool once they tossed in the light and skittles, but overall, seem like too much work in general to me.

Sly [PSN SniperChameleon] @ Jun 18th 2009 1:18AM

skittle bombs FTW. normally i would be pissed if i have to run around a room to play the game, but since it's a handheld, it's alright.

Naota @ Jun 18th 2009 1:27AM

I'm facinated by AR gaming. I'm looking forward to what the future holds with it.

Manly Mcbeefington (Mr. ESC) @ Jun 18th 2009 1:31AM

Nvidia handheld,do want!

That's one of the most innovating games I have seen in a handheld.Hopefully users wil be able to create their own maps.

They should try to create a FPS next.Something like Virtua Cop would be awesome.

Lucas @ Jun 18th 2009 1:33AM

That's a great tech demo, looks kinda fun.... but.... after the novelty has worn off I don't think it'll be that fun to play. I don't see AR or motion controls replacing standard game-pad/controllers ever.

mike @ Jun 18th 2009 2:06AM

When did anyone ever suggest that AR would replace standard game controllers? Your cynicism is completely uncalled for.

MrKlorox @ Jun 18th 2009 9:11AM

Thank you, mike, for stating what we all feel when reading a comment like that.

Dr. Stabbingworth @ Jun 18th 2009 2:00PM

Standard controllers are horribly inefficient and not particularly intuitive. While I love me some dual analogs, I'll be happy to toss it in the bin as new, better input devices come along.

This is a huge part of the appeal of the Wii and the iPhone. Although the Wii is fairly crude (don't freak out, I mean we haven't yet figured out the best way to implement extremely accuracte motion controls!) and the iPhone is pretty limited in scale.

Still, it's pretty neat to watch huge revolutions unfold before our eyes.

SecretAgentHam @ Jun 18th 2009 1:34AM

made me want skittles...

NeoHumpty @ Jun 18th 2009 2:08AM

Wow. This could be great for pr0n. No, really. Playboy should investigate.

MrKlorox @ Jun 18th 2009 9:10AM

Playboy is not porn.

MystileArmor @ Jun 18th 2009 9:34AM

How would this be great for porn? To shoot Zombies that are crawling over boobs? I don't get it.

Pureshooter @ Jun 18th 2009 11:23AM

"How would this be great for porn? To shoot Zombies that are crawling over boobs? I don't get it."

I didn't get it either...until you said zombies crawling over boobs. You're so money and you don't even know it.

Joe H @ Jun 18th 2009 2:13AM

This has been done pretty well a couple years back on a common (at the time) handheld:

http://www.toyspring.com/arcade/

Zacqary Adam Green @ Jun 18th 2009 1:48PM

That one just overlays images onto a generic video feed. This is a step up because it actually renders based on the position around the map.

Shadsy @ Jun 18th 2009 2:45AM

What happens if you aren't pointing the camera at the playing field the right way (ie., too low to the ground)? The last time I checked, this kind of AR relied on being able to pick up tags to render stuff on top of, so unless that handheld is packing something else, there could still be technical problems.

Speaking of which, I wonder what happens if you ram it through a building. :D

mike @ Jun 18th 2009 2:56AM

It almost definitely uses what's known as temporal coherence, wherein the system uses knowledge of the previous frames to generate the current frame. In this case, you would take advantage of the fact that it's physically impossible to move the device any significant distance in a single frame (1/30th or 1/60th of a second). If the device is close enough such that the camera can only capture a small portion of the paper, you can safely combine the incomplete information from the current frame with information from the previous frames to calculate the new orientation for the AR overlay.

TLDR version: magic.

Spybreak @ Jun 18th 2009 3:11AM

You know this is pretty sweet. What are they sponsored by Skitties though lol? I think it would make expandable board pieces interesting since you could have almost unlimited possibilities there.

caserb @ Jun 18th 2009 4:08AM

This would be really cool with a Fatal Frame sort of game in a real world environment. I'd buy the psp so I could wander anywhere somewhere and get freaked out by what appears on the camera screen.

Neuromancer @ Jun 18th 2009 7:30AM

What ever happened to using a PSP to control a remote control car? That could have made a really cool AR game.

j.howlett @ Jun 18th 2009 8:51AM

this looks cool. i want to try it along with the Invizimals thingfor psp

VinceXombie @ Jun 18th 2009 9:06AM

Miniture War gaming! This tech would be amazing if applied to the Battletech or Warhammer universe!

MrKlorox @ Jun 18th 2009 9:08AM

Ooh yeah! That would be awesome... however the graphics don't look like it will allow much detail yet.

MrKlorox @ Jun 18th 2009 9:07AM

The graphics look a lot like Blast Corps for the N64. That would be a cool game to play like this. Destroy the buildings with your stylus before the nuclear equipped truck on the crash-course collides and blow up.

Dr. Stabbingworth @ Jun 18th 2009 2:02PM

Blast Corps FTW! That is a game that has held up remarkably well.

Hey Rare! Remember when you made awesome fucking games?

Chilly P @ Jun 18th 2009 9:52AM

Hey, I've been in that lab before

Kevin Dunn @ Jun 18th 2009 11:06AM

I admit, it's a pretty cool tech, but for that game I don't see why you need the board and the actual skittles. What about the game was enabled by addition of the camera? Couldn't my psp enable handle placement of bombs? Couldn't my DS handle the movement of the device in realtime? What did the actual map and actual skittles add? Now, for Fatal Frame in my own home - that would be awesome!

Shagittarius @ Jun 18th 2009 11:31AM

What if you took it a step further, and rather than requiring a camera to translate a flat surface into the game by rendering it, you simply just created it from some 3d models and points on a fictional X,Y,Z, plane.

Then you wouldn't need the camera and you could possibly display the 'game' on a regular TV screen, which is already available in most homes, and would require less of an initial investment and widen your target audience.

Then if you really want to get crazy your could design a way to control this from, say, your couch and you wouldn't even need to run all over the room to play it!!!!

oh wait...

Really though, I'm just kidding, certainly this tech could be adapted into an alternate and amusing use, this is just demo work, and certainly not how you would go about really using this kind of design.

Defac1a @ Jun 18th 2009 3:42PM

it's great to see this kind of technology ascend to a level of use. Imagine what this could be used for in the tabletop gaming world, it would open up a whole new dimension to the scale possible. I'll definately be following this in the future.

simple @ Jun 18th 2009 6:21PM

hey, i dropped this hint to them too. really neat, nonetheless.

NeoHumpty @ Jun 18th 2009 11:10PM

You can make your own 3-D camera angles on the fly. I think it'd be fun.

Abalistar @ Jun 18th 2009 11:27PM

Definitely neat, but there's no way I could play something like this at home. All the space you need to layout the board, and needing to actually move around the board would be really space consuming.

And while this could be fixed by playing on something higher up, hunching over and moving back and forth like that guy would become incredibly uncomfortable after a short period.

But, it's just a tech demo, so it's all good and still seems really neat.