Nintendo patents in displacement mapping discovered

by Jennie Lees { Dec 11th 2005 at 8:00PM }

Patent-surfing is the latest trend when trying to find out what console manufacturers are up to—some patents (and trademarks) give away a lot of information, but others are just red herrings or reveal little about the future. While Nintendo have been cagey about releasing the specifications for the Revolution, an eagle-eyed blogger from Germany has uncovered a patent that may tell us more about its graphical capabilities.

Nintendo have filed two patents relating to displacement mapping, a technique for rendering detailed surfaces while keeping the polygon count low. One involves specialist hardware for vector operations, optimising the graphics hardware so that displacement mapping is computed efficiently; the other relates to reducing the strain on the CPU caused by rendering a 3D world in 2D.

It's fairly likely that the Revolution will adopt this technology to some extent—an industry insider commented, when questioned, that advances in graphics technology, especially optimisations, are to be expected for the next generation and are "interesting, but not revolutionary".

[Thanks, Soothsayer]

Related From Joystiq

Nintendo trademarks Bii, Cii, Oii, Yii and Zii
12 days ago
Nintendo Power reveals Mega Man 10 for WiiWare
18 days ago
Nintendo's next step may be into the playpen
282 days ago

Reader Comments (Page 1 of 1)

Jirr @ Dec 18th 2005 9:38PM

hey , that is nice

SEE people , the revo is gonna have power in the graph department , hehehe

Ejflex @ Dec 18th 2005 9:38PM

the hammer is gonna drop real soon on all the Ninty haters. Booyah

Vandell @ Dec 18th 2005 9:38PM

All we can do is wait and see.

Clu @ Dec 18th 2005 9:38PM

why didn't you use the second picture, its way better than the ABC?

Ejflex @ Dec 18th 2005 9:38PM

The link isn't working for me. Whats uo with this shyte?

mike @ Dec 18th 2005 9:38PM

i dont want to pee on nintendo bonfires but http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=2&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=70&f=G&l=50&co1=AND&d=PG01&s1=20051117.PGPD.&s2=microsoft.AS.&OS=PD/20051117+AND+AN/microsoft&RS=

the article tries to make out this is some secret way of doing things lol

mike @ Dec 18th 2005 9:38PM

another http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PG01&s1=20050099417&OS=20050099417&RS=20050099417

Ise @ Dec 18th 2005 9:38PM

Mullinator your link is dead.

'...displacement mapping, a technique for rendering detailed surfaces while keeping the polygon count low.'

Sounds to me like a way to make it easy on the Revolution to produce graphics. Which isnt bad, its smart to have an efficient way to render stuff . But if the information about the tech specs on the Revo are true and the GPU is only going to add like 3mb of RAM...yea...this will definitely be needed.

jopojelly @ Dec 18th 2005 9:38PM

coool...seems this displacement mapping is better than normal mapping and Parallax mapping. If it runs twice as fast as before without the optimizations that'd be cool...

Mullinator @ Dec 18th 2005 9:38PM

By the sounds of it they havn't patented displacement mapping mike.

They patented a method of using and computing them. There is a big difference between the two.

Sean @ Dec 18th 2005 9:38PM

"Nintendo have filed two patents relating to displacement mapping, a technique for rendering detailed surfaces while keeping the polygon count low."

That's incorrect. Bump mapping is for rendering detailed surfaces while keeping the polygon count low. Displacement mapping actually requires a fairly high poly count.

Nintendo would use displacement mapping to, say, put ripples in water, etc. Anything where you would take a flat grid of polys and displace their Y values.

Mullinator @ Dec 18th 2005 9:38PM

Link isn't dead for me. Besides even if it is then it's just the second picture provided in the blog this news post links too.

mike @ Dec 18th 2005 9:38PM

nintendo have not patented displacement mapping it has been around for years and years, for example the wikioedia displays examples of applications that already do it, prior art and all that (relating to patent granting)

but it is possible to patent a certain technique of doing something, but that is different from saying any displacement mapping is patented

there are at least 20 that come back in google alone

Soothsayer @ Dec 18th 2005 9:38PM

"""
"Nintendo have filed two patents relating to displacement mapping, a technique for rendering detailed surfaces while keeping the polygon count low."

That's incorrect. Bump mapping is for rendering detailed surfaces while keeping the polygon count low. Displacement mapping actually requires a fairly high poly count.
"""

No, you're confused. Bump mapping creates the illusion of a rough surface because of the way it deals with lighting vectors. Displacement mapping actually crates a real rough surface, and drastically reduces the poly count. Click through, it explains.

Jason @ Dec 18th 2005 9:38PM

I don't like the idea of being able to patent things like this. Hardware architecture; fine, but this is like an artist patenting shading techniques. The only difference is that this is in the digital realm; otherwise, this would not be patentable.

jadnice @ Dec 18th 2005 9:38PM

For the record...for those of us who for some strange reason thinks the Revo GPU have 3mbs.... NO WAY IN HELL. I am sure it going to be at minimum 128mb.

The displacement mapping is a great idea.. By the way European developer of Raid Over The River NIBRIS have announced they've finally signed with a publisher. This is now officially first Revo exclusive gaming company.

http://www.revolutionreport.com/articles/read/223

Einhanderkiller @ Dec 18th 2005 9:38PM

http://en.wikipedia.org/wiki/Xbox_360#Graphics_processing_unit

Xbox 360's GPU only has 10mb of eDRAM running at 500 MHz.

Jake @ Dec 18th 2005 9:38PM

Ise, don't talk about specs, you clearly don't understand them.

3MB of GPU RAM on the REV? Damnit it's going to have GBA graphics! There won't be any room for textures! Everythings going to look muddy like the N64!

The 3MB is for framebuffer. They need to store 640x480 pixels (the revolutions resolution, i.e. SD output, 480progressive is the standard for the revolution) A full framebuffer at 24bpp (which is what TVs use) takes 0.879MBytes, and if you include a back buffer (which you would of course) thats 1.76MBytes. That leaves about 1MB for texture cache, which is not an indication of graphics capabilities. i.e. these are the EXACT same specs as the gamecube.

Anyone could have guessed 3MB. There is absoulutly no need for more.

But but, where do the textures go??? Consoles are not like PC's. Everything is VERY tightly integrated. This is most true with Nintendo, who are the undisputed kings at making well optimized video game hardware (look at the GPU's low specs and low manufacturing costs, and look at RE4's graphics compared to anything on the much more "powerful" and costly to manufacture xbox, and you'll see it. What does this mean? Most of the RAM is in one big pool. The graphics chip isnt on some AGP/PCIX slot, its integrated into the machine, so basically you just map areas of the main chunk of RAM to the gpu. This makes things faster and cheaper, but makes it impossible to upgrade (which isnt a goal with consoles) AFAIK, all current gen consoles do this, the only GPU ram is for the framebuffer, i.e. the last step before seeing it on the screen. It's such a better way of doing things, because if a game doesn't use much memory for textures than whatever, it doesn't need to, there is no clear line between video ram and normal ram, its however the developer wants to split it.

I really hate that people see 3MB of GPU ram and piss their pants. The GCN had 2MB framebuffer and 1MB texture cache.

Oh, and even if it was 3MB for GPU ram, youre still wrong. Decreasing polygon count wont decrease VRAM usage. VRAM is used for textures and junk mostly, so actually displacement mapping (or normal mapping, or any mapping) actually takes more VRAM in exchange for less CPU/GPU cycles.

I love technical stuff.

Mark Korsak @ Dec 18th 2005 9:38PM

that 10mb in the 360 is for anti aliasing along. if the game wasnt using it, it wouldnt make a difference. as for the ram on rev, its prob gonna be 108mb i think it was.
this whole thing is like the cube mapping rumor. its making amazing graphics with low power. rev graphics will be better than 360 and on par with ps3 im tellin ya. and this rendering 3d by making it 2d is sheer amazement, i didnt think it would be possible. that would be the graphics amazing with such low requirement, that would practically be making everything pre rendered in a way. think of it, if ur comp dosnt have the power to play something like doom 3 with all these graphic settings, it can play a video recording of it, which the video recording is 2d but looks 3d from the source. its kinda like that

jopojelly @ Dec 18th 2005 9:38PM

dude!!!!!

It says its a patent to "optimimize" the technique!!, READ PEOPLE "optimize", its not a patent for the technique itself. There are many ways of doing "displacement mapping", and Nintendo is patenting THEIR WAY on performing it. That means that they are patenting their way which they believe to be the most efficient.

This is NOT a NEW TECHNIQUE, but a NEW WAY of doing it....an efficient way.

(Man, did I really have to explain this to you peeps??)

My comment boils down to this: Read, Re-read, comprehend, then post. (not post, read, admit mistake later, act stupid)

jopojelly @ Dec 18th 2005 9:38PM

I hereby add "Jake" to my list of rational and sane people who are smart and cool . (RSSC)

props dude!

swarm @ Dec 18th 2005 9:38PM

Here's another example of displacement mapping:

http://www.cs.utah.edu/~bes/papers/height/paper-img17.gif

In actual polygons, that's an icosahedron. Yep: 20 triangles. Ray tracing and global lighting works on it, too. If Nintendo's research can put this to good use, wow.

Nolan Benbrook @ Dec 18th 2005 9:38PM

OMG! I sent in this data too :-(. I said that there was a patent filed about 3D but you guys didn't give me credit.... :-(

fawazr @ Dec 18th 2005 9:38PM

over 20 comments and Jake's (#18) is the only one worth reading. This is a bump, BTW b/c I so admire comments that don't come from the tail end of someone's colon.

STEEL @ Dec 18th 2005 9:38PM

Jake is nasty and i think this Jopojelly is on to something...heres hopin:)

Metusalem @ Dec 18th 2005 9:38PM

Kudos to Jake, simply.

Admomatic @ Dec 18th 2005 9:38PM

wow! look at this and read the whole page! http://en.wikipedia.org/wiki/Displacement_mapping

Jake @ Dec 18th 2005 9:38PM

Thanks for the praise :)

Anyone going to the wikipedia link Admomatic posted beware, it's being pretty heavily vandelized. I just reverted an old copy. The vandalized copy mentions the Xbox360 being the only system capable of this technique, and MS patenting it and planning to sue anyone that uses it. Silly. Basically, don't believe anything in it about the consoles, the unvandalized article doesn't mention video game consoles at all.

@Mark Korsack
Although I like pimping Nintendo as much as the next fanboy, I wouldn't compare the difference to playing prerendered footage. A movie in, its raw form, is a series of still frames with audio. Of course we always used compressed footage but same idea. The difference in performance won't be the same, because the way graphics works is all the pretty stuff is done to the 3D data, its fed to the gpu and it does the right distortions and transposes the 3D data onto a 2D framebuffer, rinse and repeat. Even if these optimizations are great it still won't be clsoe to playing a mpg or something, you still have to do the realtime calculations.

To quote the article "interesting, not revolutionary." jopojelly had it when he was saying it was just optimizations of a well known technique. Still awesome, but nothing to hump Miyamoto/Iwata/Reggie's leg over. (Not that I wouldn't hump any of their legs already...)

Migeru @ Dec 18th 2005 9:38PM

I'm sorry but instead of arguing on how it works, cant we just see that one example ...

http://photos1.blogger.com/blogger/1480/1168/1600/displacementmap7tx.jpg

...is most likely how the revolution will work and will actually support Nintendo's earlier comment that there will not be any difference between Revolution and Xbox360 and PS3?

I mean even if revolution isnt as powerful as PS3, this just goes to show that it doesnt HAVE to be as powerful to create such awesome graphics!

Anyway, if the rumors are true (possibly), and its a souped up Xbox (or 2xGamecube), that means Half-Life 2 Graphics are possible! and Remember Resident Evil/Biohazard 4?

If that is the standard and since this Displacement Mapping will make it EVEN EASIER for developers (meaning developers wont be scared of spending a lot of money to create the game that would be on par with PS3/Xbox360 Graphics-wise) to create for the Revolution ...

.. Well, everyone Meet the Winner of the console War!

(Even though there shouldnt be a war! stop Warring people! Peace!)

( *disclaimer* In my opinion, after hearing about many news of Revolution and its possibilities)

LSSLAVE @ Dec 18th 2005 9:38PM

I used to love Wikipedia but every few days ANOTHER article is having to be inspected because someone fucked with it.

I have played the Xbox 360 and it does NOT use this technology, so its obvious how stupid that is... in fact I even searched for the patent for this and Xbox 360 and I never found one that exists

Wikipedia needs someone to scan through stuff before its updated because im sick of articles being redone because someone put stupid shit in there. Its just sad.

epobirs @ Dec 18th 2005 9:38PM

It never fails. Nintendo files for a patent on a minor variation on a well known technique and the fanboys take it as evidence of Nintendo having some fabulous hidden technology. This same kind of misguided hopefulness happened a few months ago until the more technically inclined explained that this was a technique widely used, particularly in racing games, for many years. Before then it was patents filed for a GameBoy Color GPS peripheral that was never marketed. (It may be more viable as DS peripheral using the GBA slot but remains without a product announced.)

Spend some time wading through patent listings and you'll find many, many minor variants of widely used technologies. Real major breakthrough technologies are quite rare and are unlikely to first see commercial application in an inexpensive toy. If you want to see the future of console design look at places like Siggraph and the chips used in video card that cost more than an entire console system without factoring in the PC in which to use it.

Do not look to game consoles for new technologies. They aren't in that business. A console company's job is to gather available technologies and make them available in a standardized package to create a consistent development platform. For instance, there is nothing new about motion sensing systems, nor even their use in gaming as seen in the Xavix product. Nintendo's goal isn't to make something new but to make it into an item with wide appeal. Anybody can make a console with a motion sensing controller. It is up to Nintendo to not just make it but to make it a success.

This isn't trivial. Back in the late 80's CD-ROM seemed like a perfect medium for gaming. I worked at one of the first companies to ship a CD-ROM PC game. (It was the same as the existing floppy version but with Red Book audio for music and voice narration.) Yet it wasn't until the original Playstation that a CD-ROM based platform was a success. Not because Sony had better technology but because they found a better way to structure a business around optical media's strengths, especially in relation to Sony's own strengths.

If Nintendo can have the same kind of achievement with the Revolution control scheme, this will be a great accomplishment regardless of who was there first with similar hardware.

sam @ Dec 18th 2005 9:38PM

doesnt the unreal 3 engine do something like this? im not sure if i understand how this works but it seems to be along the same lines as the new unreal engine.

epobirs @ Dec 18th 2005 9:38PM

#29

Take a moment to consider how silly that statement is. IF you played a handful of SNES titles that didn't use Mode 7 would you then feel justified in claiming the function didn't exist in the hardware?

The actual text, in an article about normal mapping, is as follows: Currently, normal mapping has been utilized successfully on both the PC and gaming consoles, although only Xbox supports this effect. Next generation consoles rely heavily on normal mapping, and even use parallax mapping.

This is completely correct. The Xbox had a number of GPU functions that do not exist in the PS2 or GameCube (We aren't talking about the Xbox 360 here but the original Xbox.) With a great deal of effort these functions can sometimes be achieved in software if there is enough CPU time available and the video hardware provides a boost. This can be seen in some of the better GameCube games that use effects most developers would only attempt with dedicated shader hardware but groups like Factor 5 know how to get the most from the GC.

The features that were beyond the competition in the Xbox are now commonplace and all of the next generation consoles will have hardware support for normal mapping and other techniques. That is just the passage of time in GPU design. Likely among the difference between the GameCube's GPU and that in the Revolution will be the addition of dedicated shader hardware. It won't match what is in the Xbox 360 or PS3 but it isn't intended to. It is meant to deliver the most GPU functionality possible for a much lower price point than the other two consoles. This unavoidably means less functionality than the competition but Nintendo is intent on other aspect being the primary selling points.

Mullinator @ Dec 18th 2005 9:38PM

@epobirs

People aren't excited by the fact Nintendo has patented a method of using displacement mapping. They are excited because it is a very good indication of what they plan on doing with the console.

jopojelly @ Dec 18th 2005 9:38PM

People need to realize that this is just one aspect of games/graphics, and that there needs to be a lot more optimization thingys if the the "difference will not exist", as someone said.

If Nintendo has many more patents/hardware aspects showing streamlined processing and execution then the outlook looks good...

realvictory @ Dec 18th 2005 9:38PM

I don't think displacement mapping is any faster than normal, because it requires extra calculations to deform the original shape. Then it still has to draw as many polygons (as you get in the final shape).

In reality, it's a modelling technique rather than a rendering technique anyway, so I don't see the point in adding it to the GPU. The other kinds of mapping are what actually make thinks look more detailed than they really are.

shahar eldar @ Dec 18th 2005 9:38PM

too late to really read the entire thing, but this "patent" sounds a lot like the work of a certain fabio policarpo (look him up, CS degree holder) that has to do with real-time displacement mapping using the GPU, this is possible on current level hardware (x-box,GC) just useless beyond a tech demo level...

honestly guys graphics never depend on hardware, they depend on artists, and people who know how to juggle color/polygons/texture and so on... 90% of "features" are basically gimmicks you will be extremely tired of after a while, it's the developers that come up with creative beutiful designs and strong imagery will be the ones who'se game your remember

Either way, I'm just a student/nintendo fanboy mostly cus I enjoy their games, I couldnt care less about the X-360 (though the new controller is a huge improvement over the old one) and probably cant affoard it for at least a year (till I get my high paying job in the games industry)

Tom @ Dec 18th 2005 9:38PM

Ummm, I read the patent (I develop 3D modeling software which uses displacement mapping) and this looks like it's actually a patent for a method to increase the speed with which bump/normal mapping can be done in real time, not displacement mapping at all. They're displacing *texture* coordinates, not geometry. The people reporting this are misinformed. Though speed increases for normal/bump mapping will make for great looking graphics (displacement mapping by its nature increase the poly count and slows things down)

vectorelement @ Dec 18th 2005 9:38PM

I love how the dumbass [360 fanboy] misspelled 'patent' and 'rendering' on the last sentence of the vandalized Wiki. You would think that someone with factual comments and common sense would actually take the time to proofread; apparently that wasn't the case.

...and displacement mapping rocks!

kingmanor @ Dec 18th 2005 9:38PM

wouldn't it be great if nintendo could live off the royaltys of this from Sony and MS and coast thru the next console generation?

Petey @ Dec 18th 2005 9:38PM

Sean(post#11) & Tom(post#36) are actually right, this tech is quite good as it's faster for the 3D artist to work but it creates a high number of poly, no miracle...

Kamalot @ Dec 18th 2005 9:38PM

The Matrox Parhelia series of video cards implemented this feature years ago. It is a MUCH smarter way of working with 3D and allows for a whole new level of detail at amazing performance levels. The problem with Matrox is that no game development studios would mke games using displacement mapping since neither ATI or Nvidia supported it.

Here is an article about displacement mapping from Toms Hardware Guide.

http://www.tomshardware.com/2002/05/14/matrox_parhelia/pag...

Displacement mapping is a lot like bump-mapping but where bump mapping gives the illusion that the surface of a polygon has texture, displacement mapping actually deforms the shape of the polygon based on the texture applied. It is a brilliant way of working and allows extremely complex models to be created at a fraction of the cost it would take to build the same model in a traditional 3D program.

Displacement mapping, when done in hardware,, s much easier to render as well. Instead of the CPY having to move a large polygon-count model every frame, the CPU only handles a low-polygon count model while the 3D card applies the displacement map the same way it would a texture.

This is a very plausible strategy for Nintendo and would give them a graphical edge over the competition at a tiny fraction of the horsepower. Remember kids, if this displacement mapping is done in hardware, it could be blazingly fast.

shahar eldar @ Dec 18th 2005 9:38PM

Falafelkid @ Dec 18th 2005 9:38PM

Hello everyone. Just a quick message to the good people at Joystiq. I am flattered to be cited here (as the eagle-eyed German blogger), but why don�t you link to my blog? No big deal, but if you got the story from me I would prefer to be linked (mainly to get more clicks, admittedly). I am humbled by you picking up the story, though. Thanks.

Falafelkid @ Dec 18th 2005 9:38PM

By the way, the patent number changed again, so the old link returns an error. Here�s the new link:

http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=1&f=G&l=50&d=PG01&S1=%22emboss-style+bump+mapping%22&OS=%22emboss-style+bump+mapping%22&RS=%22emboss-style+bump+mapping%22

@Mike: If you meant my original article, I never said this was a secret technique. In fact, in the comments I say the very opposite. Just for the record ;)

Falafelkid @ Dec 18th 2005 9:38PM

Oops. I only just discovered the �Read� button with the link. I am so sorry for being such a noob ;) Would some admin please delete the first comment I made - and throw a bedsheet over me, while you�re at it. *glows with shame*

Pretty Obvious @ Dec 18th 2005 9:38PM

This is def good for Revolution games but you people need to realize an improvement in bump mapping does not put this system on par with the 360 or PS3. Hardware has the biggest impact on what developers can do with graphics, not the other way around like somebody stated.