x0vncserver 1.13.0 is doing some incorrect xkb calls that causes our X server to crash. We can see the following in the log: > corrupted size vs. prev_size > (EE) > (EE) Backtrace: > (EE) 0: /opt/thinlinc/libexec/Xvnc (xorg_backtrace+0x41) [0x76a2f1] > (EE) 1: /opt/thinlinc/libexec/Xvnc (0x400000+0x36d909) [0x76d909] > (EE) 2: /lib64/libc.so.6 (0x7f4406200000+0x3ea00) [0x7f440623ea00] > (EE) 3: /lib64/libc.so.6 (0x7f4406200000+0x8ebec) [0x7f440628ebec] > (EE) 4: /lib64/libc.so.6 (raise+0x16) [0x7f440623e956] > (EE) 5: /lib64/libc.so.6 (abort+0xcf) [0x7f44062287f4] > (EE) 6: /lib64/libc.so.6 (0x7f4406200000+0x82d3e) [0x7f4406282d3e] > (EE) 7: /lib64/libc.so.6 (0x7f4406200000+0x9893c) [0x7f440629893c] > (EE) 8: /lib64/libc.so.6 (0x7f4406200000+0x9940e) [0x7f440629940e] > (EE) 9: /lib64/libc.so.6 (0x7f4406200000+0x9a91b) [0x7f440629a91b] > (EE) 10: /lib64/libc.so.6 (free+0x73) [0x7f440629d133] > (EE) 11: /opt/thinlinc/libexec/Xvnc (SrvXkbFreeClientMap+0x136) [0x576fc6] > (EE) 12: /opt/thinlinc/libexec/Xvnc (SrvXkbFreeKeyboard+0xfb) [0x57365b] > (EE) 13: /opt/thinlinc/libexec/Xvnc (ProcXkbGetKbdByName+0xaf8) [0x55f198] > (EE) 14: /opt/thinlinc/libexec/Xvnc (Dispatch+0x325) [0x71c925] > (EE) 15: /opt/thinlinc/libexec/Xvnc (dix_main+0x388) [0x720778] > (EE) 16: /lib64/libc.so.6 (0x7f4406200000+0x29510) [0x7f4406229510] > (EE) 17: /lib64/libc.so.6 (__libc_start_main+0x89) [0x7f44062295c9] > (EE) 18: /opt/thinlinc/libexec/Xvnc (0x400000+0xc2fa0) [0x4c2fa0] > (EE) > (EE) > Fatal server error: > (EE) Caught signal 6 (Aborted). Server aborting > (EE) On a different test run with a manually built Xvnc I got: > (EE) > (EE) Backtrace: > (EE) 0: ./builddir/x86_64/unix/xserver/hw/vnc/Xvnc (xorg_backtrace+0x41) [0x609071] > (EE) 1: ./builddir/x86_64/unix/xserver/hw/vnc/Xvnc (0x400000+0x20c669) [0x60c669] > (EE) 2: /lib64/libc.so.6 (0x7f09fa000000+0x3ea00) [0x7f09fa03ea00] > (EE) 3: ./builddir/x86_64/unix/xserver/hw/vnc/Xvnc (0x400000+0x17c749) [0x57c749] > (EE) 4: ./builddir/x86_64/unix/xserver/hw/vnc/Xvnc (ProcXkbSetMap+0x152) [0x582b22] > (EE) 5: ./builddir/x86_64/unix/xserver/hw/vnc/Xvnc (Dispatch+0x325) [0x5bb6b5] > (EE) 6: ./builddir/x86_64/unix/xserver/hw/vnc/Xvnc (dix_main+0x388) [0x5bf508] > (EE) 7: /lib64/libc.so.6 (0x7f09fa000000+0x29510) [0x7f09fa029510] > (EE) 8: /lib64/libc.so.6 (__libc_start_main+0x89) [0x7f09fa0295c9] > (EE) 9: ./builddir/x86_64/unix/xserver/hw/vnc/Xvnc (0x400000+0xe7fb0) [0x4e7fb0] > (EE) > (EE) Segmentation fault at address 0x68 > (EE) > Fatal server error: > (EE) Caught signal 11 (Segmentation fault). Server aborting > (EE)
Could be the same issues as bug 7971.
TigerVNC reports about this: https://github.com/TigerVNC/tigervnc/issues/1595 https://github.com/TigerVNC/tigervnc/issues/1598 https://github.com/TigerVNC/tigervnc/issues/1589
This issue does not seem to be fixed by the patches added in bug 7971.
The issue that was causing our X server to crash had been fixed upstream, the fixes have been patched.
The main cause of the issue was that a piece of code had been moved from a function that returned 0 on failure to a function that returns 0 on success, without taking the flipped meaning of the return value into account. Hence, causing invalid inputs to pass as valid.
The following was done to verify that the issue have been solved, using tigervnc v. 1.13.0 (vncviewer and x0vncserver): 1. In a running Thinlinc session, run "x0vncserver -display=:10 -SecurityTypes=None" - Port 5900 needs to be open for clients to have access from another machine. - Change -display to whatever display the session is running on. 2. Connect with vncviewer. 3. Close vncviewer. Before patching: Closing the vncviewer caused the X server and hence the session to crash. After patching: Closing the vncviewer causes the x0vncserver to crash with "X Error of failed request: BadValue (integer parameter out of range for operation)" The session is still running, without any error messages in xinit.log.
The patches applied changes to the internal function _XkbSetMap, which should be called when setxkbmap is used. Verified that we don't have any regressions when changing keyboard layout with setxkbmap (changed from se to us layout): 1. Check current layout with "xmodmap -pk". 2. Change layout with "setxkbmap us". 3. Verify that layout has been changed with "xmodmap -pk". 4. Verify that no errors related to the change are reported in xinit.log.
This bug is fixed.
Tested with server build 3286 on Ubuntu 22.04. > Must: > * Our X server should be protected from crashes caused by third party programs. Yes, I tested the scenario with opening vncviewer inside a ThinLinc session, and saw that it no longer crashes when closing. These are the steps I went through to test it: 1) Downloaded tigervnc-1.13.0.x86_64.tar.gz from https://sourceforge.net/projects/tigervnc/. Unpack the tar, should find binary files 'vincviewer' and 'x0vncserver' there. 2) Inside a ThinLinc session, run "x0vncserver -display=:10 -SecurityTypes=None", where -display is the displaynumber of the session. 3) Still inside the session, connect with vncviewer. 4) Close vncviewer. Before the fix, the Xserver crashes on vncviewer close and the ThinLinc session shuts down. After the fix, x0vncserver crashes with error 'X Error of failed request: BadValue (integer parameter out of range for operation)', but the ThinLinc session is still up and running. > * Potential fixes must not cause regressions in functionality. I couldn't trigger any regressions. Tested both the scenario with 'xmodmap' and 'setxkbmap' described in comment 11, and through 'xev' that registers keyboard and click events. Did not see any errors in xinit.log. > * Bugs found should be reported upstream if they are still present in newer versions of X server. No additional bugs found, so no bug reports made. Also checked that the release notes look good, and that the commits have the same commit message and code changes as the upstream ones. Closing.
