Joystiq

Accuracy
The accurate flight model is probably X-Plane's strongest aspect. Because the sim can read the geometric shape of every plane, it can accurately calculate how the plane should fly through the virtual air. This results in an extremely realistic feeling when flying the different types of aircraft in the game. It's not hard to let your imagination take you away as you attempt an altitude record in a F-104 Starfighter, or attempt to infiltrate a oil rig with a military helicopter.

While the game lacks the range of preset situations that its main competitor (MSFS 2004) features, if you can do it in real life, you can do it in X-Plane. Anything from a full shuttle approach from orbit to a mid-air launch of the X-15 Mach 6 experimental plane is possible with X-Plane. Unfortunately new users will most likely not take the initiative here and give up early instead of exploring the game's massive range.

As a tool for training budding pilots, X-Plane is very effective. My father used X-Plane combined with a photographic scenery package to help him pass the landmark based navigation section of his pilot's license. However, you will need a joystick to get the most out of X-Plane. Mouse control is possible but in no way recommended.
Scope
The range of aircraft that X-Plane is capable of simulating is massive: light planes (such as clunky old Cessna's), radio controlled planes, C-5 Galaxy super haulers and even GDI Orcas from Command and Conquer all fly in a realistic fashion. If you opt to buy the full earth scenery package then you'll have an entire world of accurate topography (mountains, hills and rivers) to buzz around. In a practical sense what this means is that you can fly from any location on any route to any airport in the world. An excellent way of helping pilots plan their journey but also pretty fun for aviation lovers too. Newbies could find this daunting due to the lack of a highlights feature. How are you supposed to know what the best airports are if you're given an entire world of them!

It's worth noting that X-Plane is not a combat simulator. Whilst it can simulate weaponry, including missiles, bombs and cannon fire there aren't actually any targets and they have little effect on physical objects in the game. That's annoying but not a deal breaker -- people looking for a combat flight simulator probably weren't in X-Plane's target audience in the first place. X-Plane is aimed at people that would prefer to map out a route from Heathrow to LAX and then execute it via knobs and dials in the auto-pilot rather than take 25 minutes to intercept a ME109 in a Spitfire.

Add-ons and constant development
One of the chief complaints that many gamers have with X-Plane is that Austin has never released a "final" version of X-Plane. In a constant stage of development, X-Plane 8 has been moving forwards since its inception. In practice what it means is that any changes you would like fixed, you can raise the issue in the community and if it's a reasonable request there's a good likelihood that it'll get done. X-Plane is very much the user's simulator in contrast to the commercial Microsoft Flight Simulator.

Graphics
To me, the graphics feel like just another functional aspect of the simulation of flight. Sure, there are occasional moments where you'll say "damn, that looks real" but if you're looking for pure eye candy then pick up the game's less realistic but possibly more user friendly competitor, Microsoft Flight Simulator. Hardcore users will occasionally want to use X-Plane to simulate IFR (instrumental flying rules) anyway -- which means all there is to see is a thick cloud of fog and your instruments.

Loading times and bugs
When you first boot X-Plane there's a monstrous loading screen (that can take much longer than normal if you've installed custom scenery). Once you get past this there's very little loading but if you change the graphics settings you'll have to restart the game and sit through the monstrous load sequence again. Unfortunately there's also a few bugs: you'll see land that acts like water, runways that hover magically above the ground and the occasional crash (not the aircraft kind, the application kind) if you seriously mess up your aircraft. [Amendment: these issues are primarily the result of the user's failure to install scenery (where airports will appear to hover in mid air) or the user's use of crappy aircraft models (none of the aircraft included with X-Plane caused any crashes). Also, it appears as if the land acting as water bug was fixed in an earlier version of X-Plane - it was previously a major issue. Not any more. As long as you have all the scenery installed and stick with well coded aircraft, X-Plane should remain a practically bug free experience.]

One of the most annoying things that I've seen on every machine I've ever tried X-Plane with is the occasional skip or pause whilst playing the game whilst the game loads terrain textures. This severely reduces immersion and in a worst case scenario can force the game to freeze for up to 30 seconds. As long as you stay within a certain region, the game won't need to re-load the terrain, but that's not a viable solution considering the nature of flight.
Interface
There's no way around it: X-Plane's interface is confusing. Practically every menu screen features a bewildering array of options that will confuse newbies no end. You'll learn to love the vast range of options that can be tweaked, but only after you've torn your hair out wondering what the hell this or that button does. One of the worst examples of this is the in-game cockpits. In some of the poorer cockpit designs, sometimes the switches will be completely unlabeled requiring guesswork as to what button does what. [Clarification: again, the poorer cockpit designs are only seen in fan designed cockpits: the planes that come with the aircraft are of a high quality. Regarding the interface - the complex nature of the sim means that amount of information displayed in X-Plane is on the same level as some graphics design packages like Photoshop. The problem is with GUI's themselves: they're physically not capable of displaying clearly to the user the large amount of options that X-Plane requires. Fortunately, the location of options will eventually become second nature so if you stick with it you'll eventually overcome your initial confusion.]

Massive learning curve
X-Plane really isn't a pick up and play type of game. Unless you have some flying experience it'll take you a long time to work out what every feature does and how it affects your flight. That's not to say X-Plane has a steep learning curve, just a very long one. It should only take a few hours to grasp the basics of taking off, flying and landing aircraft, but due to the immense depth of the game it takes a while to progressively learn how to use the more advanced features - like the auto-pilot for instance.

Conclusion
If you have even a passing interest in the workings of aviation then you need to own a copy of X-Plane. Setting aside expensive simulators used to train pilots to fly airliners, X-Plane is the closest you can come to flying a plane without actually setting foot in one. Unfortunately, that means X-Plane shuts out casual gamers. With a little bit of work, Austin could implement some more situations (e.g. favorite airports, tricky approaches) and offer even a basic tutorial but until he does, X-Plane will remain a very difficult game to master, just like a real life flying course.

How much you enjoy X-Plane depends entirely upon your interests: if you're a simulator buff, you can't get much better than this game for comprehensive flight simulation. Unfortunately, the lack of even a basic tutorial and "quick start" option, the practical requirement of a joystick and limited number of pre-set situations means that X-Plane is inaccessible to players new to the genre.

If you've regularly played more than two or three flight simulators in the last five years or are a pilot then the following score will apply to you. However, newbies to flight simulators and aviation in general would do well to check out more basic simulators before they try out the "daddy" of flight simulation.

Score: 8.5

[Update: made amendments to the bugs section]
[Update 2: clarified user interface section]