Bill Gates on the future of computer interaction
by Ross Miller 
{ Jun 3rd 2007 at 12:30PM }
In a very rare joint interview with Apple co-founder Steve Jobs on stage at last week's D 2007, Microsoft founder Bill Gates told hosts Walter Mossberg and Kara Swisher, both of the Wallstreet Journal, of his vision for the future of interaction with technology, and therefore gaming, and inadvertently used the Nintendo Wii as a stepping stone."Imagine a game machine where you can just pick up a bat and swing it or the tennis racket," said Gates, to which Mossberg noted that one exists, referencing Wii Sports.
"No, no, that's not it," said Gates, "you can't pick up your tennis racket and swing it, ['Oh, I see what you mean,' said Mossberg] you can't sit there with your friends and do those natural things. That's a 3D positional device, this is video recognition, this is a camera seeing what's going on ... the camera will be ubiquitous."
Gates laid out his vision in four words: touch, ink, speech, vision. Was this a subtle jab at Nintendo? We don't think so, though Gates' quick response to Mossberg's statement gives the impression that he was prepared for the comment. Simply, Gates feels that long down the line, it will be better (though he admits there are privacy concerns) to use video recognition for the same functions, and his words apply to all technology and not just games. (It should be noted, however, that Gates' opinion of the Wii and its controller have undulated over time.)
Speech as an input is something we're starting to see more often (e.g. Tom Clancy's EndWar), and touch has been prevalent with Microsoft Surface, university research and the "money-printing" Nintendo DS (though that might fall under his "ink" category, given the use of a stylus). Video recognition is also something that has worked with mixed results, though games like the upcoming Eye of Judgment show a clever use of the technology.
But, returning to Gates example, is swinging an actual tennis racket for a virtual environment the best choice? Later in the interview, from Steve Jobs, talking about the post-PC market: "They don't want to drive with a joystick, they like the steering wheel. And so, you have to ... as Bill was saying, in some cases you have to augment what exists there and in some cases you can replace things." Perhaps in this case, the use of a small, plastic wand as a replacement for a racket is more ideal in an indoor environment. It certainly lowers the risk of breaking anything -- or maybe not.
[Via Engadget]
Speech as an input is something we're starting to see more often (e.g. Tom Clancy's EndWar), and touch has been prevalent with Microsoft Surface, university research and the "money-printing" Nintendo DS (though that might fall under his "ink" category, given the use of a stylus). Video recognition is also something that has worked with mixed results, though games like the upcoming Eye of Judgment show a clever use of the technology.
But, returning to Gates example, is swinging an actual tennis racket for a virtual environment the best choice? Later in the interview, from Steve Jobs, talking about the post-PC market: "They don't want to drive with a joystick, they like the steering wheel. And so, you have to ... as Bill was saying, in some cases you have to augment what exists there and in some cases you can replace things." Perhaps in this case, the use of a small, plastic wand as a replacement for a racket is more ideal in an indoor environment. It certainly lowers the risk of breaking anything -- or maybe not.
[Via Engadget]
Reader Comments (Page 1 of 2)
2ez @ Jun 3rd 2007 12:47PM
Hmm... so maybe Nintendo really is on to something here.
Aunalis @ Jun 3rd 2007 12:52PM
I would absolutely hate the gaming market to go that direction. I want full immersion in a video game, sure, that's what we all want. But if I wanted to play a tennis game with a tennis racket i'd actually go outside and play tennis. How would you do a game like Gears of War for instance? You're going to have to have 10 different kinds of guns laying on your coffee table to switch to a new gun? throw immitation grenades at the correct position on your screen (and end up breaking it). Going even further, how would you do an RPG? Final Fantasy would just straight up not work.
How do I see games going then? Anyone ever play .hack? That's full immersion. Don't cut out the keyboard and controllers.
AirIntake @ Jun 3rd 2007 1:02PM
Well, all controls will be obsolete once they have a system that communicates directly with the brain. No monitors, speakers, or controllers will be required after that.
Psaakyrn @ Jun 3rd 2007 1:04PM
to #2 Aunalis
Just because it can detect a tennis racket, doesn't mean that it MUST be a tennis racket. What I think is meant here is that the whole idea of controls be removed entirely; It should not matter what you are using, as long as the actions can still be intepreted. So, instead of using a gun-like device, I can pretend my pointed finger is the gun, and still use it to fire and register in the game world.
Lewis @ Jun 3rd 2007 1:05PM
M$ is gonna buy Steve Jobs and Shigeru Miyamoto. I'm telling man...god.
Aunalis @ Jun 3rd 2007 1:13PM
to #3 AirIntake
That's a very good point. I saw something recently about them getting even closer with that technology on Engadget.
to #4 Psaakyrn
That may all be well and good but the fact still remains that you're swinging your arms wildly to use a "sword" or swing a "racket". I can't honestly see ALL gaming taking that direction. But i do admit that it works perfect for games like Guitar Hero and a few choice Wii games.
crusher @ Jun 3rd 2007 1:22PM
Anyone else notice how Bill and Steve are dressed in that photo? Looks exactly like those Apple TV commercials with the "young, hip" Apple guy talking to the Microsoft guy!
XGamerX @ Jun 3rd 2007 1:39PM
i dunno an actual racket may not be the best. Not everyone has alot of space to swing such things in their rooms. Having one smaller device preform many functions seems easier to me.
http://gamergeddon.com/
Mr Khan @ Jun 3rd 2007 1:46PM
I think he realized he was referencing Wii Sports after he said it, then said no when he realized he was glorifying competition
Slvrgun @ Sep 3rd 2007 8:11PM
You might as well go outside and do the real thing. Oh I forgot, with all that air outside, and that thing called the sun, and the breeze, and the grass, and the prospect of falling over a rock or something, its just too dangerous to go outside.
Slvrgun @ Jun 3rd 2007 1:48PM
You might as well go outside and do the real thing. Oh I forgot, with all that air outside, and that thing called the sun, and the breeze, and the grass, and the prospect of falling over a rock or something, its just too dangerous to go outside.
Kye @ Jun 3rd 2007 1:55PM
I bet he's gona go and say it would be good if you had one device that could mimic the actions of all sorts of things, bats, rackets, swords, guns etc.
Wrench @ Jun 3rd 2007 2:00PM
Isn't the keyboard like the ultimate controller already? So why change things around?
Ted @ Jun 3rd 2007 2:07PM
lol, they mirror each others' body language. Look at their poses @ 4:38
ExMcloud @ Jun 3rd 2007 9:00PM
Apple and Microsoft should definitely do something together because it would definitely change the software world. And also the gaming world
DjHyjak @ Jun 3rd 2007 2:26PM
This isnt the first time Microsoft has brought this up. I remember when they first introduced the vision camera for the 360 they said it can already do those cool things and even recognize when multiple users are in frame. They have yet to impliment its use tho and i wonder why? With the Wii being the new pet rock and all, they could get in on some of that casual money.
gone with the wind @ Jun 3rd 2007 2:24PM
to #3 well if all games in da future are going to require mind control im gueesing that the learning curve will be difficult and will probally alienate the gaming community from the non gaming comunity. plus wat would u have to think in order to perform moves, i dont know how u would unlock new moves it cant tell u to think apples for example in order to perform your finishing move ,unless they make u think of the button configuration in your mind in order to play in wich case while ergonmically comfterble it is far to complicated idk mabey im to brain dumb to think of trillions of colors n the games physics while being shot at while picturing a controller in my mind, ill stick with the ways things are now, just better graphics n super duper ultra hi-def tv
Ted @ Jun 3rd 2007 2:30PM
I want to drive a car with a joystick, as long as that means the car would be able to fly.
incrediblebulk92 @ Jun 3rd 2007 2:32PM
I'm sorry but I really hate the wii control system, I mean theres nothing wrong with it, you like it fine play it but me, I want a controller. If i have to memorize 300 slightly different hand jerks for games like mortal kombat or look like a complete toss pot every time I throw a grenade or reload in Call of Duty I'm really not going to enjoy playing them. I really think that this whole motion control thing is really overrated it doesn't bring anything new to the table to me. Also I don't want to dance around my living room for some sort of webcam, I'm happy pressing A to pick up my weapon, its a lot simpler and less annoying then bending over and grabbing at the ground for a couple of minutes.
Disclaimer: This is all my opinion, don't take offense at this wii/ps3 fans
Ted @ Jun 3rd 2007 2:32PM
@ number 14
It can already be done, dude. Do a bit of research on the net, you'll soon find clips of preliminary work to be able to control software with the brain. For example, you think of pushing a chair forward in the game, and it gets pushed forward. You would probably still have the mouse/keyboard combo, so you move around and point with mouse, but you can then interact using your mind.
incrediblebulk92 @ Jun 3rd 2007 2:34PM
I'm sorry but I really hate the wii control system, I mean theres nothing wrong with it, you like it fine play it but me, I want a controller. If i have to memorize 300 slightly different hand jerks for games like mortal kombat or look like a complete toss pot every time I throw a grenade or reload in Call of Duty I'm really not going to enjoy playing them. I really think that this whole motion control thing is really overrated it doesn't bring anything new to the table to me. Also I don't want to dance around my living room for some sort of webcam, I'm happy pressing A to pick up my weapon, its a lot simpler and less annoying then bending over and grabbing at the ground for a couple of minutes.
Disclaimer: This is all my opinion, don't take offense at this wii/ps3 fans
Nessuno @ Jun 3rd 2007 2:37PM
When "Next-Gen" starts referring to advances in gameplay and interaction rather than polygon counts, gaming will have made a revolutionary change for the better.
So, basically, I'd love it if the XBox720 and PS4 marketing campaigns brag about how accurate and immersive their control setups are.
And, btw, if that happens, don't expect them to credit Nintendo for the paradigm shift.
aaron @ Jun 3rd 2007 2:38PM
I believe Bill Gates was talking about the standalone -like interactive Xavix or some cleverly stupid name. It had a central hub, but you had to buy the baseball bat separately as well as the golf and tennis games.
The keyboard is the ultimate controller for PC games, not the entire game industry.
jipi @ Jun 3rd 2007 2:44PM
M$ act like they own nintendo.
sporge @ Jun 3rd 2007 2:47PM
@ 17
is needing to memorize 30 different button combos for mortal combat really all that much easier than memorizing a motion? Of course I don't believe they are aiming to destroy the classic controller as you still use normal style controllers for the virtual counsel. The new controller just gives you a new way to play games, which in some cases can work wonders, but in others they can feel awkward and tacked on. No different than games on any other system though as controls can be just as bad in some games with a well used controller.
___________________________________________
I have to say I don't think we can see full immersion into a video game world until we can hook our brains up to the computer or something like that. Honestly anything virtual now that creates full immersion is hard to think of fitting in a living room. Think of laser tag, would you call that a video game? I wouldn't but that is a game using technology to full immerse you in a game, your controller the light gun and suit that senses if you are hit. Where is the line between what is a video game and a game using computer technology?
incrediblebulk92 @ Jun 3rd 2007 2:57PM
@23 I hadn't really thought the mortal kombat thing through, I just saw the article on it and thought WHOA, I'm not really a fighting game sorta guy anyways. I don't think that I'd be opposed to a choice system where both controllers would work for all games, or maybe even only one of the controllers work for certain games (warioware would NOT work with a normal controller). The only problem with the motion control really is that their rarely well done, most do feel tacked on or rushed. Red steel and far cry on the Wii were both completely ruined by the controls imo
SKI @ Jun 3rd 2007 3:10PM
"The only problem with the motion control really is that their rarely well done, most do feel tacked on or rushed."
And this is what, 6 months into the Wii/PS3 launch? Just think what will happen in the next couple of years when developers get more comftorable, plus how much better it can get with better sensors in the next generation.
smartguy1600 @ Jun 3rd 2007 3:56PM
I agree that immersion won't truly occur until there is some form of brain manipulation. I don't want to say The Matrix exactly, but something related to that concept certainly. A player could sit in some kind of chair, perhaps a specialized one to ensure that your body maintains comfort and so on, and then you connect somehow, obviously not physically (aka like through the back of your head like the Matrix) but from that point on things becomes like real motions to you. It's hard to explain, but still using the Matrix analogy, Neo and Morpheus in the karate simulation were fighting, normally neo couldn't do anything like that, but then the program sort of told his brain how. So when you unlock moves, it would sort of be like telling your brain how your able to do it, and in the game world, you would be fully capable of doing so. That I think, would be awesome. Of course, we have to keep a way to prevent people from staying in the system too long, or becoming so absorbed that they cannot distinguish when they are hooked up and not. I think graphically, until we get to complete realism it wouldn't be much of a problem. This is of course, speaking years and years down the line. Just some thoughts.
J rod @ Jun 3rd 2007 5:09PM
I remember when Bill Gates dissed the Wii controller and said motion sensing isn't mainstream and that he don't like it.
He only cares about money.
FrankTheCrank @ Jun 3rd 2007 5:11PM
This guy has been wrong so many times that it's laughable you would even bother to listen.
He was the one that didn't think the internet would take off.
Innovation and Microsoft just don't mix. I'm no Apple fan boy, but they have it right about Redmond, their just one big copy machine.
Aarron soans @ Jun 3rd 2007 5:22PM
yh i agree the wii control system is rubbish, alot of games that get releaesed on the Wii are rubbish because you cant implement those features into a game like GOW for example, succesfully enough and that is why there have been no extraordinary games for the wii ( you might argue wii sports but its still not a game youll be playing 2 years l8r like halo and gears). thats because the game developers cant create gd enough games for this system because it doesnt work you cant play games like final fantasy by shaking a plastic 3d rectangle its impossible and thats why there will never be any hugely popular games that have a huge lifespan for the wii with the same credibillity as counterstrike which is still being played years l8r enough with poor graphics compared to the current market, its a simple game which would fail miserably on the wii along with every toher succesfull game microsoft and sony has released
Dave @ Jun 3rd 2007 5:20PM
I think I'd pass on any device that interfaces with my brain directly. My brain allows me to interact with other objects--not the other way around.
jay @ Jun 3rd 2007 5:40PM
See Aarron might have a point (if you needed to get what he said translated) if the Wii was a failing system. But it isn't. It's been 6 months and I'm still playing Wii Sports alongside SSX Blur and Mario Party.
After just 3 months I had grown bored of Gears of War, Resistance and co. 2 minutes after beating Halo 2 I realised I will never play it again (and haven't).
After 2 years I'm still playing HL2 Deathmatch.
gone with the wind @ Jun 3rd 2007 6:03PM
@ #18 im not saying it cant b done i know it can b done ,i saw this special on a physically disabled person who could control a mouse on the computer with a cable on his head ala matrix, i know thats not what video games are gona do (at least i hope not cause i dont want cables in my head ) but its a step in the same direction but ether way im sure thats a long ways from now it b to expensive in a video game, at least 30 years down the road when ps10 and xbox infinity come out. all i know is that if that happens theres going to be a whole lot more lawsuits in the gaming indrusty, if a wii mote can break your tv screen imagine what something connected to your mind can do, BE VERY SCARED OF THE FUTURE !!!!!!!!!! when games systems become sentinet and you have a wii strapped to your spine telling you wat to do !!!
Ken @ Jun 3rd 2007 6:04PM
jay
That's not fair. You know that HL2DM is about 1000x better than GOW. As much as people don't want to believe it. :)
smartguy1600 @ Jun 3rd 2007 6:07PM
@ 30:
No offense to Halo players, but Halo is just another FPS. What it did well was it's multiplayer aspect and bringing FPS onto a console, which before that there were some successes, but Halo was the "next gen" FPS on consoles. I think if you look at a different example, say, I dunno, DS, I think that fits Wii's future a bit better. When the two handhelds, PSP and DS, came out. I went with the PSP. Why? Because i looked at the games for DS and saw tacked on, simple uses of the touch screen that did nothing to make me go "oooo." But now, you have all these creative and unique uses of the dual screen and touch pad that games on the DS are addictive and fun, and they are selling like crazy. I think down the line we are going to see alot more really cool games on the Wii, Manhunt 2 for one, I think, looks amazing with its motion controlled kill scenes.
Anyway, point is I'm more inclined to side with jay here, because the games you spout out as amazing are games that have done the same things as before, just in a better way. In defense I think Gears is a great game, but its short, and online the modes are nothing but rehashes of "kill or be killed." Also, Counterstrike, while being a well balanced FPS, is just another FPS. Its still being played because it has a rabid fanbase. That does NOT make a good game. I think games like "Shadow of the Colossus" are masterpieces, yet it sold like shit. Not to mention Counterstrike is simply a mod of another well known FPS, a game with a certain Mr. Freeman...
Evan @ Jun 3rd 2007 6:11PM
@Aunalis "But if I wanted to play a tennis game with a tennis racket i'd actually go outside and play tennis. How would you do a game like Gears of War for instance?"
If you wanted to play something like Gears of War, you could go to a paint ball range. If you wanted to play something like Grand Turismo, you could go to a track. Even a go-kart track (with no power or speed limiters) is 10 times more exhilerating than the video game, but it also requires a kind of skill and physical endurance that isn't needed in the game. Real tennis and golf require real skills that take years of practice to master. And paint ball is very physically exhausting, I doubt many video gamers have the stamina for a whole morning of paint ball.
Video games require no physical stamina and very little physical skill compared to their real-life counterparts. Some games, like MMORPGs, require no skill whatsoever - you just click on stuff and are almost garaunteed to level up if you spend enough hours logged in. This actually makes games attractive to many people. A gamer can be an obese slob in real life, but feel like a powerful mage in a game. A gamer can be a scrawny 12-year-old kid in real life, but feel like an elite marine in a game.
But as soon as you move beyond a mouse or standard controller, to a more realistic interface, you start to require real abilities of the player. A gamer's lack of real physical skills will limit their in-game abilities and break the ego-fantasy.
This is a problem that the Wii faces. When game control is dumbed so that the game is accessible to everyone, like in Wii Sports, some people complain that it doesn't require any skill. But when game control is not dumbed down as much, like in Super Swing Golf, the people who lack the real-life skill to get into the game complain that the game control must be broken (in fact, Super Swing Golf's control is more solid than either Wii Sports or Tiger Woods if, but only if, you actually know how to swing a club). The Wii is an interesting experiment, but I think we'll always have games that use a conventional controller or mouse for those players who don't want to have their skills tested.
Tron Knotts @ Jun 3rd 2007 6:29PM
Aaron, don't you think that FPS game like Counter Strike and Halo have some potential on the Wii, given the Wii-Remotes capacity for precision on screen aiming?
The first company to make an Online competitive FPS for the Wii that plays relatively well is going to make millions.
Matt B @ Jun 3rd 2007 6:54PM
Bill wasn't inadvertently referencing the wii, he was really hinting about the possibilities with the Playstation EYE. ;) Which will be in my home soon.:D
Cinder @ Jun 3rd 2007 7:12PM
Kudos, Evan, for one of the most interesting and subjective posts I've read on this site in a while.
Andrew Fong @ Jun 3rd 2007 8:10PM
Bill is simply applying a platform-software approach to where the Wii is heading. As he sees it, almost everyone has a tennis racket and almost everyone will have an HD camera in the future. Whereas Nintendo is trying to sell hardware (Wiimote), Gates is talking about using software (e.g. image recognition) to give the same capabilities to existing hardware (computer + racket + camera).
laesperanzapaz @ Jun 3rd 2007 8:19PM
regarding gates talking of the wii sports part:
notice how angry he gets - in one split second - when he rejects the wii tennis controls....
ahh, gates. momentarily caught off guard.
Andrew Fong @ Jun 3rd 2007 8:20PM
Tron, that depends on whether or not the developers can figure out how to use the Wiimote as a precision aiming device. As it stands, it actually isn't that precise, especially for an activity such as, say, sniping. Unlike the light gun, there isn't a one-to-one relationship between where you're pointing the Wiimote and where the "cursor" is.
There are also issues like lag and sensitivity to deal with. To see what I mean, go to the Wii main menu and try quickly snapping from one menu button to the other. Then try holding the cursor steady in one spot. Both are much harder than with a mouse or even a joystick.
I think there are things developers can do to fix this (Metroid may be promising), but I think we're going to be seeing more Red Steels and Far Crys before we start getting some consistently good Wii FPSes.
David @ Jun 3rd 2007 8:44PM
Umm... Mr. Bill, buy a Wii.
Like someone said what Bill Gates wants is a holodeck. Why would I want to run back and forth in my living room when I can go outside and do the exact same thing?
mike @ Jun 3rd 2007 9:20PM
Apple and Microsoft should definitely do something together because it would definitely change the software world. And also the gaming world
--
Fuck you, MS Office for Mac is bad enough.. We don't need any more 'ideas' from Redmond~
kingofwale @ Jun 3rd 2007 10:45PM
tennis game where you don't control the footwork, baseball game where you don't field. Combine that with graphics that looks like 4 yr olds drawn it. Yeah, Nintendo got it right. ;)
And who's Bill Gates to talk about electronics future? Their two main products, xbox 360 and Zune, none of them are making profits at all.
I bet Steve Job's sitting there thinking, what is this guy smoking? :)
samfish @ Jun 3rd 2007 11:07PM
What the Hell is with all the people on this site lately who can't fucking type?! This isn't text messaging, kids.
Occasional typos are fine, but capital letters, punctuation, proper grammar and correctly spelled words make you seem intelligent.
but if u want 2 b stoopid and type like i m rite now then u shld no dat its hard 2 read. plz dnt doo it nemore, kthxbi
dbeech @ Jun 3rd 2007 11:38PM
I don't know what Microsoft is planning, but I hope to God they bring back Matthew Perry and Jennifer Aniston to test it out and give us the sequel to the 'first ever cyber sitcom'.
dontplaygameswithme @ Jun 6th 2007 5:14PM
J rod He didnt say he didnt like it. He said they released it on the pc for one of there flight games and people complained. Also he said, we found out people didnt want to crash into the ground. They just wanted to put the controller down! He was explaining the compliants they got about it. Not saying he didnt like it he loved it thought it was a good idea and released it.
Sean @ Jun 3rd 2007 11:41PM
Bill Gates was talking about the PS3!The PS eye which will have 3d mapping and object recognition. Much like Billy Bob said. (I can't believe this) I actually agree with Bill Gates. Steve Jobs looks pissed off.