See issue 12620. This has yet to be tested/confirmed, but apparently it affects normal Linux terminal server sessions as well as VirtualBox, so could be a bug in the client. Perhaps we should investigate.
Works just fine with the current code.
Seems to work great, tested loading the Spanish layout with "setxkbmap es".