A company called Allegorithmic recently held a presentation at London's Game Developers Conference and briefly talked about their advanced texture compression technology. The uninterestingly named ProFX and MaP Zone 2 programs aim to reduce the size of in-game texture files by up to 70% -- with no image quality lost. One of the games utilizing the technology turns out to be Roboblitz which, like every other Xbox Live Arcade game, is expected to come in at under 50MB. Allegorithmic's software supposedly reduced the entire game's collection of textures to a size less than 280KB. Efficient compression techniques, if implemented correctly, could easily be seen as a way to obviate the initial need for new storage mediums such as HD-DVD and Blu-Ray. However, the Bit-Tech article points out that designers are just as likely to fill the extra space to create even more rich and varied textures. Instead, the biggest winnner would be the world of digital distribution. Though broadband speeds are constantly increasing, nobody likes waiting for a progress bar to fill. Casual or arcade games could certainly benefit greatly from smaller overall file sizes as they vie for your limited time and attention.
[Via PS3 Fanboy, thanks to everyone that sent this in.]
_thumbnail.jpg "i am 8-bit: 2008")
(Page 1) Reader Comments
Reply
When I was at CMU we witnessed a demonstration of procedural texture generation that superceeded the need for large samples. The algorithim would take a tiny sample (3 flowers, a sentence, etc) and produce a field of flowers, a page in a book, etc. That tech was absolutly convincing, unless you tried to read the generated page of text.
I wonder if these are related?
Reply
Reply
for Brandon:
http://en.wikipedia.org/wiki/Kkrieger
kkrieger is 98 kilobytes compressed, 300 megs uncompressed. Thank procedural generation of everything from models to textures. However, the game requires a ton of RAM and processing power.
Reply
storage mediums such as HD-DVD and
Blu-Ray.HAHAHAHA.PS3 - Rip off waste of money.
Sony: "All that space is needed for all the new textures, cough up
the cash". Rest of World: "Piss off and play with ya UMD's and ya
minidiscs, ya blood sucking prostitutes of Art. Fuck off"God,
what a shit company. Hahaha.
Reply
Reply
Similar to your info on HD-DVD and Blu-Ray not being needed, there have already been HD movies down on standard DVDs. Microsoft aided with the codec to get these movies out. The only movie that came from hollywood was Terminator 2 though.
http://www.microsoft.com/windows/windowsmedia/musicandvideo/hdvideo/t2dvd.aspx
Specs to run the movie though:
Windows XP
Windows Media Player 9 Series
3.0 GHz processor or equivalent
512 MB of RAM
128 MB 3D video card
Those were harsh specs when the movie came out years ago, but these are a piece of cake now. I always wondered why MS didn't try to push this format with the 360.
Reply
no this means hd-dvd and blu ray can now store even more stuff making the game way better looking and with a lot more content
Reply
so my question is, if the PS3 achieves a market lead for Sony in this generation, will the major facets of the console's design (enormous storage media, Cell+SPE configuration) be enough to blunt, at least temporarily, the potential advantages of designing games to use these procedural algorithms?
Reply
Sure, they are gonna pack in enough textures to fill up the entire 50gig-mega-capacity-Bloo-Ray when this tech shrinks them down to nothing. Face it, with a 70% compression rate, an awful lot of that HD SPACE is completely unesseary. Like the blu-ray always was, its just obvious now.
I mean, ive seen the Graphically-everything-gameplay-nothing GT and Fight Night ps3 pics. Theres nowhere left to put more textures, the things already look like uncanny photos. Now it seems you can do that withouth all that CUTTING EDGE SUPER TECH.
makes me laugh so hard.
HAHAHAHAHAHAHAHAHaaaaaaaaaaa.
Reply
1. There should be a far heavier emphasis on physics. The Alan Wake demo at the Intel Developer Conference completely blew me away. Texture processing can only bring so much to the table. A lot more eye candy results from computing that makes those textures interact with one another. That's far more important for realistic gaming experiences than simple cosmetics.
2. More dynamic and engaging gameplay plots should also be another concern. I feel that games are getting too short and too straightforward all thanks to the new emphasis on eye candy. If this technology is used to simply pack more textures, it would ignore more important factors such as storyline and AI complexity.
Of course you can argue that texture compression can lead to more focuses on the points I bring up since it makes more space but that is totally up to the developer. This is a very powerful tool but I hope it doesn't lead to an even heavier emphasis on graphical fillers as the post suggests. This tool can be used in a way that would cause the industry to regress since lazy developers will merely make more polygons and textures and call it a day while other aspects of gaming falls to the wayside; unfortunately, this is far easier to do. Personally, I'm tired of seeing pretty but formulaic games like PS3's Ridge Racer.
The skilled developer will use the free space to emphasize the more important aspects of gaming I hope. I sincerely hope that the compression software doesn't lead to complacency.
Reply
I didn't see your post until now but wow. You really are an idiot. I think you are just trolling though. I've seen the tactic. You just pretend to be a rabid "Nintenork" to make the company and its loyalists look bad. Nintendo is my favorite company this generation by the way.
I'm mentally ashamed that my post has to come after yours. They share the same line of pixels on my computer screen and for that I dislike you immensely.
Reply
the 360 has been using procedural synthesis for awhile now and its getting better at it. this is the future of games and we should all be happy.
Reply
Reply
From the above comments, it seems like people think this is cab or zip (lossless) compression, it is not. It is also not DXT or jpeg (lossy) compression. It is more like simplifying textures into a list of procedures that in turn consistently reproduce the texture you've designed.
It is lossless in the sense that the data produced is identical to the original. However, rather than taking a texture as input and outputting a compressed file, this technology requires that you create your texture in their editor and the steps you performed are "recorded". The final result is a file that holds information on how to recreate the image at runtime.
The process of creating the textures at runtime is not as bad as you might think. They had Roboblitz on display at GameFest and game took about 3-4 seconds to generate the textures needed for the level. These levels are no joke, either; the texturing is very nice and very diverse - better than you would see in almost every AAA last-gen title and the game is for XBLA. The levels are also pretty big, so the delay isn't something you cannot live with.
They were working to implement streaming texture creation utilizing one of the Xbox 360's cores and hoped it would be implemented by the time Roboblitz shipped. When implemented (though it may not be for Roboblitz), this will effectively eliminate the load time related to texture generation, relative to pulling pre-generated textures off the disc.
This technology works on almost any kind of texture, be it a diffuse, bump (or normal) map, displacement map, etc. It cannot, as far as I was told, work effectively for lightmaps. Lightmaps are generated based on the world geometry and the work it would take to make them procedural wouldn't be worth the benefit since they are specific to a single level and not shared.
Creating textures didn't seem too difficult, just another tool an artist would have to master. A lot of pre-defined materials come packaged with their editor and if you go with their middleware, they will send someone out to train your team. Alternately, if you don't want to take the time to have your staff trained, you can contract out the creation of procedural textures, based on existing textures, to them.
In reading the article, I understand where the 70% number came from, but that just does not seem right. Based on what I saw, with the files they had there, to achieve the same lossless compression for one specific texture, it would take a 200+KB file compared to the 1KB file their software created. I think it was something like 100MB of DXT (lossy compression, probably 500+MB uncompressed) textures were converted to 300KB of procedural files - that's much higher than a 70% "compression" rate.
Anyway, this technology is great for XBLA titles as Microsoft seems to be sticking to their guns about the 50MB size limit (this was emphasized at GameFest). I'm still uncertain how widely adopted this will be for the games that carry the $60 price tag. When you're making a multi-million dollar game for retail, the scope is much larger than an XBLA title and you can never find enough processing power for your AI, physics, graphics, sound effects, etc.
If the Xbox 360 shipped with five 7GHz processors, developers would still find a way to max it out - be that with lazy programming or more complex logic, that all depends on the developer. I think this technology can still be vastly improved performance wise and the streaming would reduce the initial load time and memory usage, but regardless both Xbox 360 and PS3 developers would have to sacrifice something for a few seconds in order to make it work.
Jeff S, your fears are pretty unfounded. This simply means that developers can pack more content into less space, not that they can magically use more content at runtime. The textures generated from these "compressed" files still take up their fair share of RAM, just as much as any pre-generated texture would, so developers are still confined in that manner.
Regardless of the RAM limitations, I don't really understand what texturing has to do with physics, dynamic gameplay, or story. Texture artists do not program physics engines, direct gameplay experiences, or pen storytelling. Unsurprisingly, texture artists make textures and are often pretty non-technical and do not write stories.
I often see people post concerns about too much focus going into area X of development due to Y getting some press, but often times the man power involved in X and Y are in no way connected. Could more texture artists get hired in place of a physics programmer? Sure, but you don't want five physics programmers working on your game, that's a waste of money - same for people involved in storytelling.
Nick
Reply
Thanks for simplifying the technology into more understandable terms. It's really helpful and allays my fears quite a bit since it doesn't really give the developer an easy way out.
"Regardless of the RAM limitations, I don't really understand what texturing has to do with physics, dynamic gameplay, or story. Texture artists do not program physics engines, direct gameplay experiences, or pen storytelling. Unsurprisingly, texture artists make textures and are often pretty non-technical and do not write stories."
There is no link in terms of technology. I grant that in my original post. What I was trying to say was that from a marketing perspective, a developer can simply tout a feature as "new" and ignore other aspects of gaming; this isn't unprecedented. The new HD-centrism seems to emphasize the cosmetics of the game over the science. For example, Gran Turismo HD touts HD as a feature worth naming a game over. That worries me. This new tool seems to make it exponentially easier for a developer to do so. Why develop physics and storylines when you can simply make things look nicer? I detect a sad shortage of games on the new consoles that actually concentrate on these other aspects.
I'm not sure what your refutation concerning RAM limitations means. I didn't mention RAM in my post; remember, it was merely a concern regarding laziness on the part of the developer. Could you possibly clarify?
"I often see people post concerns about too much focus going into area X of development due to Y getting some press, but often times the man power involved in X and Y are in no way connected. Could more texture artists get hired in place of a physics programmer? Sure, but you don't want five physics programmers working on your game, that's a waste of money - same for people involved in storytelling."
Good point. There should be a balance. But unfortunately, the current state of the industry is biased heavily towards textures.
"Creating textures didn't seem too difficult, just another tool an artist would have to master. A lot of pre-defined materials come packaged with their editor and if you go with their middleware, they will send someone out to train your team. Alternately, if you don't want to take the time to have your staff trained, you can contract out the creation of procedural textures, based on existing textures, to them."
This reinforces my point. The easy way out will simply be more readily available. And yes, you can do all those things. Unfortunately, the easy way out is very popular.
I'd like to reiterate what I had said; this tool is very powerful. But I still think complacency is likely given the nature of marketing and the industry.
"The process of creating the textures at runtime is not as bad as you might think. They had Roboblitz on display at GameFest and game took about 3-4 seconds to generate the textures needed for the level. These levels are no joke, either; the texturing is very nice and very diverse - better than you would see in almost every AAA last-gen title and the game is for XBLA. The levels are also pretty big, so the delay isn't something you cannot live with."
Sure it can decrease load times. That's a given once this technology is implemented widely. Nothing stops the industry from calling that a "feature" though and making it a selling point or at least, tout it as revolutionary. I detest that. Maybe you think reducing load times is an end in and of itself but I respectfully disagree. The purpose of games is to enhance the general feel of the game. Tools like this only tempt the developer to parametricize the focus of the game. That's ridiculous and a very valid concern. HD is being misused this way as well.
I suppose it's a necessary evil for us to contend with. I'm sure all technology will cause people to merely focus on one aspect and hail it as revolutionary. Of course, there are a few gems in this new generation that avoid this but the problem is that these gems are so few. Yes, Nick, a developer can do everything you say a developer can do. But you have to admit that it probably won't use its tools in a utilitarian fashion.
Reply
2)Anyone ever think of how this could effect Nintendo's tiny little machine. It would be intresting to see what they can do with this.
I am glad to hear about this.
BTW I am no fanboy. I just refuse to buy an over-rated piece of equipment at a high mark up when there is competing formats and possibly the extinction of spinning drives may be on the way. Sure they always say "Well it won't be commercially abailable for years" but we are talking about 10 years. Do you really think the PS3 will be the end all say all for the next 10 years? I highly doubt it. How much would you be upset when they come out with a 10X blue ray drive 3 years from now anyhow. Good luck with that one.
The truest thing out of sony's camp is that they say that downloadable content is the wave of the future. If mini harddrives keep improving then I am willing to bet that optical discs will be on the way out. And who the hell wouldn't welcome that change because I for one was pissed when I had to replace the PS2 2X.
Reply
I don't care how much a game developer uses it...
I want it for my own personal use...
I've got a +500gb (growing as fast as my internet connection) collection of pr0n that needs backing up...
...
..
.
Reply
This technology is impressive, useful, and relatively new to commercial use; however, this technology is not all that you're giving it credit to be. It does not make it easier to make better looking textures, it just makes it easier to store them. The same amount of time and dedication (even more initially) is needed to create textures with these tools compared to traditional means.
Since the same amount of time is required to create a texture using this technology, it will cost the same amount of money. If the developers want more textures now that they have more space, they're going to need to hire more artists. The only aspects of development impacted by this technology are disc space and processor usage. Visual fidelity is unaffected by this technology.
What I was trying to point out about the RAM usage was that, while this technology saves on disc space it does not save on RAM usage. That means a developer can get an equal number of these textures and traditional textures into the system's memory at any given time. Therefore, this technology will not be a catalyst for more textures in a game any more than traditional textures.
The point I was trying to make with the artist vs physics programmer count was that you do not want more physics programmers, same goes for storytellers. They should not be balanced. Art generation takes substantially more man hours than physics implementation (especially with middleware like Havok around) and too many dedicated storytellers can bring lack of direction to a story. Arguably, the same goes for game designers.
I think you're misguided about the amount, or at least the focus, of work required to implement physics like those seen in Alan Wake. The bulk of the work for that tornado you saw was done in the middleware they used, Havok 4. They had to interface the Havok engine with their game engine and define some variables for each piece of content on the screen. You should not need more than 1-2 people dedicated to physics implementation and content variables are defined as the content is created.
Now I know that's just one of your examples, but the same largely applies for storytelling and gameplay mechanics. Game design is not a problem you can throw a dozen people at and have the result come out better than a single designer - it'll probably be worse. The same goes for story, if you don't have a very small group of people deciding your overall direction and storytelling mechanisms, your story will become muddied and confusing.
Disregarding for a moment the fact that the procedural texture middleware does not make the creation of good textures easier, if a development studio decided they wanted to focus on more textures they would have to hire more artists to do the work. If they didn't decide to focus on more, better textures when they had a traditional texture workflow, the move to this technology is not going to change their mind.
You seem confused about my load times comment, too. I am saying that the generation of these textures, right now, requires more time than loading traditional textures from the game disc. Using streaming, they can reduce the load time to the point that it is on par with the load times associated with loading traditional textures. It does not mean developers can advertise faster loading times than before.
I think your concerns are with the overall next generation of games, not with anything this middleware directly affects. It's fine to not be happy about the selling point of graphics over gameplay, I agree in that area, you just picked the wrong technology to illustrate your concern with. However amazing, there's nothing this technology can do to convince developers to put more emphasis on graphics.
J (#17), I'm not sure how useful this technology will be for Nintendo. Compared to the Xbox 360 and the PS3, the Wii doesn't have nearly as much processing power. Developers making games for the Wii are much harder pressed to make use of every microsecond the Wii's processor can offer them.
Nick
Reply
Alright. That definitely clarifies things a bit more but I'm still a bit confused. Regrettably, I'm not really prepared to offer a counterstatement seeing how I'm not familiar with the intricacies of this technology and how we're talking on two different levels. I'll go read up more on this technology.
I still think my link is tenable. Unless the blog post is misleading, the idea that graphical artists can simply add more graphical details proves the link between this technology and "better graphics".
You say that RAM imposes a ceiling on the graphical capabilities of the console. That may be true in the future but I have a hard time believing that early generation systems are utilizing system resources to the maximum. Even if that ceiling is being hit already, future copy/paste jobs of textures can simply max out the technology with redundant filler crap that adds nothing significant to the game. That's also a bad thing.
If I am correct though, your first statement should justify what I was saying. You said this technology makes it easier to store the textures. If that's the case, I fail to see where I've gone wrong in understanding this technology. According the blog post, this would allow graphics to be packed even more densely because of freed up space. I don't understand why you seem to think that the textures would still cost the same to develop. I think it would be cheaper. For example, if a model is made up of x number of textures, why wouldn't this technology simply allow even more textures packed into the same physical space? Perhaps my understanding of textures is wrong but isn't the heart of every graphic component simply triangles and polygons? I don't see why a developer can't amplify the amount of these triangles and polygons and call it a day. If that isn't technologically valid, then yes, my argument wouldn't make sense.
But even if I were misguided, I still think the extra space can be exploited without hiring more artists. For example, if a game map had 10 trees, there can now be 100 trees. Or if there are 10 buildings, there can now be 100 buildings. Or if there are 10 soldiers, there can now be 100 soldiers. It seems like this tool opens the door to easy and rudimentary copy and paste jobs.
I think you overestimate the part that the graphical designer plays in games. I don't see much variation in artistic styles or even unique and distinctive characters in many games. I think a small core of artists can conceive of and draw all the figures in a large RPG or FPS. I don't see why the heavy emphasis needs to be art and textures.
You also say that physics is easy to execute. I don't know if that is true but I think at the very least, constant research should be done in the physics department. Yes, the physics technology is disjointed from the compression math that is the subject of this post but you are missing my point about complacency. I think that physics development may fall aside because "features" were accessed with a new compression algorithm. I don't know why Havok was brought into the discussion. It only reinforces my point. Perhaps developers should spend more time and money on Havok compatibility. I realize what I am saying isn't mutually exclusive but the market will simply be flooded with crappy games because rarely will the developer go the extra length to do both.
I disagree with the point regarding storytelling and gameplay mechanics. While you may have a point when it comes to a general barebones storyline, I think the finer aspects and offshoots of a good story need many individuals acting as a collective. Even if you disagree, I think your argument is only true if the director of the game development is incompetent. Even if you still disagree, I think that development can at least improve even further by doing more research or additional testing.
I think my comment on load times was totally fine. Your counter about how it would allow more complex games to have the same load time as simplex games only applies if the extra space is utilized. If the developer merely uses it as a tool of compression without any modification, my concern about advertisement of load times becomes activated.
In an ideal world, all of this would happen at the same time. However, due to time and budget constraints, they don't. I still think this technology, by allowing rudimentary copy/paste jobs, can lead to complacency to a certain degree. I'm a big RTS player and I know that companies will prop up 10000 people on the field or more realistic settings due to more trees and rocks as a major achievement. Unfortunately a lot of developers (or publishers) think that makes a game. This is a very direct and clear link to what this technology can enable later on in the future.
Nothing stops the developer from advancing gameplay mechanics even further except themselves. And from a pure marketing perspective, they may get to a point where they can justify to themselves cutting down development and research costs simply by doing a copy/paste job because it is cheap. This would ultimately, be a disservice to the gaming community.
Reply
Compression implies that you take a texture and make it 70% smaller, when in reality, the tech is a procedural texture SDK, so you create a texture using their toolkit, and the engine generates it on runtime. so smaller memory footprint, smaller file size. It does not compress anything.
Reply
Reply
Maybe a bit long link, but it explains quite good how it works. Someone of Allegorithmic even explains their method quite well.
However, I refuse to give my thought about this technology and how it will affect the future, because I feel that I have to do somewhat more research on this instead of assuming things. The truth is out there.
greetz, kite
Reply
Reply
Most just commented on the basis that blu-ray was mentioned.
@Jeff S, this does NOTHING to RAM, AT ALL!
Forget RAM, it doesn't automagically improve how you can utilise RAM.
This ONLY means you can store MORE objects in the final game, it doesn't mean you can have more things on screen at once.
Lets say this game is on disc.
You copy the files over from the disc into RAM, this then requires the textures to be created, which uses CPU cycles.
But in the end, it will still take the SAME amount of RAM it took if it was done the in the traditional sense.
This still requires the same artists to do their jobs at the same rate, it only means they can store less data on the final game (whether it be a disc or file)
@everyone mentioning compression
IT ISN'T COMPRESSION!
@everyone mentioning 50GB BD-ROMS
God damn you idiots need laid, why does EVERY idiot assume PS3 will use 50GB BDs?
PS3 can still use 25GB discs you know...
@dude wondering about Cell
Cell is excellent for doing this job!
Cell was actualy built to do repetitive crap like this, despite what the "popular" opinion is.
The streaming thing you mention is in fact one of the things this could be used to help this.
The SPEs could be coded so that they could execute these algorithms in a cycle (not CPU cycle) around Cell.
It would probably only take 1-3 cycle(s) around to actually finish one, depending on how complex the algorithm is. (also, no, this isn't slow, this is incredibly fast! heh)
@generally everyone
What we really need to make texture creation easier (in fact, any area of design) is better tools, smarter tools that "know" what we want to do.
This is only solving one problem and that is final-size. (but its definitely a good problem to solve if we intend moving to this digital distribution model)
Reply
Therefore, the PS3 just because a bigger ripoff.
Sure, developers COULD use the space to put in more game content, levels etc. But do you really think they will? Theres already a huge rise in development costs due to the necessity of ultra detaled textures etc. What makes you think that devs will but their guts to fill up a 50gig disc with content when they had prevously been told that the space would be needed for textures?
More PS3 woe, I see.
Reply
Reply
As Nick said before there is equivalent trade. It comes at a cost. It takes processing power to do it and it takes the same amount of RAM once these textures are generated.
Reply
Already many upcoming DS games are using far more FMV and even full speach then prior titles. This is due to better compression tech.
FF3 and Rune Factory have lots of FMV and Rune Factory has full speach going on.
I could also see this tech being used for say popcap or other casual games. Simple stuff you know that wouldn't be too much of a draw on a system's specs.
Reply
Now, on to this technology. If everything Nick says is true (and I think it is highly likely that it is), then yes, this is not compression of any sort. That's good because I was afraid I was going to have to come in here and go on an Information Theory rant. Procedural texture creation though puts most of that out the window. What is does open up, however, is some limitations of the technology. From the description of having to use their tool to generate the procedural texture procedure files, it sounds to me like this will be useful for a great deal of the textures in a game, but not all. Any textures which include things like logos, defined symbols, etc, would probably be easier to leave as normal bitmapped textures, unless their tool includes some sort of tracing tool that would let you try to bring in bitmapped images into what sounds very much like a vector-based format.
As for storage needs... this technology won't help the size of map files, or HD video cutscenes, or things like that, so it's not really a relevant conversation. More storage is always better. Software is a gas, it expands to fill the space it is given. That is a Law of Software Engineering, and you'd do well to remember it.
What this technology needs is on-die support, like DXT. Hardware support of the format by the video subsystem could really make significant changes. As it is, once the textures are rendered into texture memory ont he video card, any scaling or shader work on the data will work just like a normal texture. That's great for current day technology, but some neat effects and possibly greater quality might be had if they were working on the procedural definition itself rather than its rendered output.
Reply
As stated above, compression technology is the wave of the future, not adding in a lot of extra space. Yes, eventually that space will get used, but compression technology is moving faster than developers. As stated above by Nick, this program can pull a 200-1 compression ratio. Does that mean developers are going to put in 200 times more textures? No. All that means is they can now utilize more space on more important functionalities of a game, as opposed to the fancy wrapper. Those important things, like physics and AI, take up considerably less space than graphics and sound, and when graphics and sound are being cut down in size without giving up equivelant quality of a larger file, what use does a product like BluRay have in the next 10 years? Next to none, especially from a consumer standpoint.
If a 360 developer can produce the same game as a PS3 developer and use 1/4 of the space with little loss in visual quality, what reason is there to spend $100 or $200 more? Is a minor increase in polygon count (the PS3 and 360 have no difference in ability to process on-screen textures) really worth that much money, especially to the core consumer that actually makes these things popular?
I'm very surprised some enterprising company doesn't use the DVD texture compression mentioned above to seriously undercut and destroy HD-DVD and BluRay entirely. I guess for $100 more than a current DVD player ($250 I'd wager) a company could sell a DVD able to throw out the 1080p sized movies without ever having to once consider a $500 or $1000 option or ever have the words "upconvert" uttered.
Yes, we will eventually have use for all that extra space, but the reason the storage requirements have exploded over the past 30 years is because compression technology has been pretty much ignored along the way. Had compression technology been moving at the same pace it is now back when hard drives were being developed, then we'd all be happily going along with our 2 gig hard drives and having lots of extra space to spare.
Reply
then why don't you just get an external HDD? You could back up that entire collection on one source, rather than 20+ blu-ray discs.
Reply