It may be interesting to offload encoding of VNC data to some kind of more dedicated hardware. It would offload the CPU, and might also provide more performance than the CPU is capable off. We would have to be careful about not becoming overly reliant on such help though, and not ending up requiring one specific vendor.
AMD has released code to access a H.264 encoder on their GPUs: http://lists.freedesktop.org/archives/mesa-dev/2014-February/053203.html