You're definitely correct about Windows (it was something I was excited about in Vista), but I'm not sure about Linux. Looking at the Nvidia GLX drivers for my Linux machine, it still seems like quite a hefty kernel module. I can't really examine it, though, because it's not open source.

Actually the part of it that runs in kernel mode does have it's source visible. It's part of the standard packaging provided by NVIDIA so that it can work on many different kernel versions. For the most part their driver is also userspace and the kernel driver exposes a communications pathway to the userspace. That's not to say I know how robust that pathway is, but they have separated things somewhat.

The open source drivers also follow this kind of model where the kernel modules handle hardware access and memory management but not a whole lot more. Mesa (and Gallium 3D) all work in userspace to handle compiling things to the native formats of the cards.