I have Ctrl+Shift+Alt configured as the key combination to release the keyboard grab in VMware. However this does not work reliably when I'm using the Windows client. Sometimes it breaks the grab, sometimes it doesn't. It turns out after some experimenting that the problem is Shift+Alt. If I press the Alt before shift, then it works. But Shift before Alt breaks.
I had a look at this and the issue is that VMware looks at physical keys rather than symbols. On Linux Shift+Alt generally generates Meta, but not on Windows. So Xvnc will find a different key when it sees the client sending Alt with Shift already pressed. I've implemented a heuristic upstream to handle this: https://github.com/TigerVNC/tigervnc/commit/3b532f87b26d791b0b64b87aa39141d1a81098e8 The basic principle is that if Shift+Alt in the server keyboard map generates Meta, then we'll use that key and send a Meta when we see the symbols Alt_* from a client and Shift is currently pressed.
Also note bug 5258 which is indirectly affected by this heuristic.
Tested on a Windows 10 client against a RHEL 8 server with build 6364. Tested by pressing Shift+Alt and verified that it sends Meta with keycode 64 instead of 204 as previously. Also tested that Linux clients still have the correct behavior.
