One of the major problems with Xorg is also one of the major problems with Linux: they've broken it up into individually coded, (almost) separate projects, that all develop at their own pace. There's not such thing as "Xorg 7.4" just as there is no such thing as "Debian Linux 5.0" -- it's just a (virtually) random collection of various modules with different version numbers thrown together as "a release".
It's great that each of the pieces can now be worked on separately, but it really sucks that now each of the pieces is released separately. Work on them separately, but release them as whole. It's like there's no Q/A done to "the collection formerly known as X11", it's all just "compiles? loads? ship it". Just like a Linux distro. It's very irritating.
And the whole "everything that is not Windows is Linux" mentality is getting to be very annoying. So long as it works on Linux, it must work everywhere (since everything is Linux), so it's shipped. And (from what I've heard around the virtual water cooler) the number of Linux-isms in the tree is increasing. I wouldn't really be too surprised if Xorg 8.0 became a Linux-only release, with tight ties into the kernel.
I've never had any real issues running Xorg on my systems, but the reliability and speed has been going down over time, compared to XFree86 4.0.