Bug 5572 - Xvnc segfault after client-initiated resize
Summary: Xvnc segfault after client-initiated resize
Status: CLOSED WORKSFORME
Alias: None
Product: ThinLinc
Classification: Unclassified
Component: VNC (show other bugs)
Version: trunk
Hardware: PC Unknown
: P2 Normal
Target Milestone: 4.5.0
Assignee: Pierre Ossman
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-06-17 10:14 CEST by Samuel Mannehed
Modified: 2015-09-21 10:33 CEST (History)
1 user (show)

See Also:
Acceptance Criteria:


Attachments

Description Samuel Mannehed cendio 2015-06-17 10:14:09 CEST
I got the follow segmentation fault when resizing a session from the HTML5 client. I haven't found a way to reproduce the error however. Was using ThinLinc 4.4.0




Tue Jun 16 21:54:24 2015
 XserverDesktop: Got request for framebuffer resize to 1920x629
 XserverDesktop: 1 screen(s)
             0 (0x00000000): 1920x629+0+0 (flags
              0x00000000)

 VNCSConnST:  FramebufferUpdateRequest 1920x626 at 0,272 exceeds
 framebuffer
              1920x629
 VNCSConnST:  FramebufferUpdateRequest 1920x3 at 0,898 exceeds
                  framebuffer
              1920x629
 VNCSConnST:  FramebufferUpdateRequest 1920x629 at 0,272 exceeds
                  framebuffer
              1920x629
 PixelBuffer: reallocating managed buffer (1920x629)
(EE) 
(EE) Backtrace:
(EE) 0: /opt/thinlinc/libexec/Xvnc (xorg_backtrace+0x34) [0x5d4b64]
(EE) 1: /opt/thinlinc/libexec/Xvnc (0x400000+0x1d88f9) [0x5d88f9]
(EE) 2: /lib64/libpthread.so.0 (0x3cdfe00000+0xf6d0) [0x3cdfe0f6d0]
(EE) 3: /lib64/libc.so.6 (0x3cdfa00000+0x14e9b0) [0x3cdfb4e9b0]
(EE) 4: /opt/thinlinc/libexec/Xvnc
                  (_ZNK3rfb11PixelBuffer8getImageEPvRKNS_4RectEi+0x8c)
                  [0x5fa27c]
(EE) 5: /opt/thinlinc/libexec/Xvnc
                  (_ZN3rfb13EncodeManager13findSolidRectERKNS_4RectEPNS_6RegionEPKNS_11PixelBufferE+0xb1)
                  [0x6149e1]
(EE) 6: /opt/thinlinc/libexec/Xvnc
                  (_ZN3rfb13EncodeManager15writeSolidRectsEPNS_6RegionEPKNS_11PixelBufferE+0x61)
                  [0x614d51]
(EE) 7: /opt/thinlinc/libexec/Xvnc
                  (_ZN3rfb13EncodeManager11writeUpdateERKNS_10UpdateInfoEPKNS_11PixelBufferEPKNS_14RenderedCursorE+0xb1)
                  [0x616451]
(EE) 8: /opt/thinlinc/libexec/Xvnc
                  (_ZN3rfb16VNCSConnectionST22writeFramebufferUpdateEv+0x4e2)
                  [0x60f802]
(EE) 9: /opt/thinlinc/libexec/Xvnc
                  (_ZN3rfb16VNCSConnectionST15processMessagesEv+0xdc)
                  [0x61033c]
(EE) 10: /opt/thinlinc/libexec/Xvnc
                  (_ZN14XserverDesktop17readWakeupHandlerEP6fd_seti+0x1f9)
                  [0x5f08d9]
(EE) 11: /opt/thinlinc/libexec/Xvnc (vncCallReadWakeupHandlers+0x2a)
                  [0x5e69fa]
(EE) 12: /opt/thinlinc/libexec/Xvnc (0x400000+0x1ee25c) [0x5ee25c]
(EE) 13: /opt/thinlinc/libexec/Xvnc (WakeupHandler+0x6b) [0x57b09b]
(EE) 14: /opt/thinlinc/libexec/Xvnc (WaitForSomething+0x492)
                  [0x5d1942]
(EE) 15: /opt/thinlinc/libexec/Xvnc (Dispatch+0xa2) [0x576622]
(EE) 16: /opt/thinlinc/libexec/Xvnc (main+0x3ca) [0x45adea]
(EE) 17: /lib64/libc.so.6 (__libc_start_main+0xf5) [0x3cdfa21d65]
(EE) 18: /opt/thinlinc/libexec/Xvnc (0x400000+0x5c45d) [0x45c45d]
(EE) 
(EE) Segmentation fault at address 0x3e88f50

Fatal server error:
Caught signal 11 (Segmentation fault). Server aborting
Comment 1 Samuel Mannehed cendio 2015-06-23 11:32:17 CEST
Got more segfaults, slightly different but triggered again by a client-initiated resizes. I have found a way to reproduce the crash almost always (with Xvnc from both ThinLinc 4.3.0 and 4.4.0):

1. Open a session in the HTML5 client on a touch device
2. Do a server-side resize to make the session large (xrandr -s 1920x1200)
3. Use panning to move the viewport away from the top left corner of the session
4. Rotate the device to trigger a client-initiated resize
5. Xvnc crashes here 9 out of 10 times, if not -> just repeat steps 2 to 4

It would seem like our HTML5 client doesn't change the coordinates of the viewport when the session is resized and thus try to use coordinates which are outside of the framebuffer. Xvnc doesn't seem to be able to handle this case properly.




Tue Jun 23 09:24:50 2015
 XserverDesktop: Got request for framebuffer resize to 768x891
 XserverDesktop: 1 screen(s)
             0 (0x00000000): 768x891+0+0 (flags
              0x00000000)

 PixelBuffer: reallocating managed buffer (24x24)
 Cursor:      cropping 24x24 to 7x16
 PixelBuffer: reallocating managed buffer (24x24)
 Cursor:      cropping 24x24 to 12x20
 PixelBuffer: reallocating managed buffer (24x24)
 Cursor:      cropping 24x24 to 18x17
 PixelBuffer: reallocating managed buffer (18x17)
 VNCSConnST:  FramebufferUpdateRequest 768x635 at 120,91 exceeds framebuffer
              768x891
 VNCSConnST:  FramebufferUpdateRequest 768x256 at 120,726 exceeds framebuffer
              768x891
 VNCSConnST:  FramebufferUpdateRequest 768x891 at 120,91 exceeds framebuffer
              768x891
 PixelBuffer: reallocating managed buffer (768x891)
(EE) 
(EE) Backtrace:
(EE) 0: /opt/thinlinc/libexec/Xvnc (xorg_backtrace+0x34) [0x5d4b64]
(EE) 1: /opt/thinlinc/libexec/Xvnc (0x400000+0x1d88f9) [0x5d88f9]
(EE) 2: /lib64/libpthread.so.0 (0x3cdfe00000+0xf6d0) [0x3cdfe0f6d0]
(EE) 3: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager14checkSolidTileERKNS_4RectEPKhPKNS_11PixelBufferE+0xee) [0x61454e]
(EE) 4: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager22extendSolidAreaByBlockERKNS_4RectEPKhPKNS_11PixelBufferEPS1_+0xa0) [0x6146a0]
(EE) 5: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager13findSolidRectERKNS_4RectEPNS_6RegionEPKNS_11PixelBufferE+0x159) [0x614a89]
(EE) 6: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager13findSolidRectERKNS_4RectEPNS_6RegionEPKNS_11PixelBufferE+0x30c) [0x614c3c]
(EE) 7: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager15writeSolidRectsEPNS_6RegionEPKNS_11PixelBufferE+0x61) [0x614d51]
(EE) 8: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager11writeUpdateERKNS_10UpdateInfoEPKNS_11PixelBufferEPKNS_14RenderedCursorE+0xb1) [0x616451]
(EE) 9: /opt/thinlinc/libexec/Xvnc (_ZN3rfb16VNCSConnectionST22writeFramebufferUpdateEv+0x4e2) [0x60f802]
(EE) 10: /opt/thinlinc/libexec/Xvnc (_ZN3rfb16VNCSConnectionST15processMessagesEv+0xdc) [0x61033c]
(EE) 11: /opt/thinlinc/libexec/Xvnc (_ZN14XserverDesktop17readWakeupHandlerEP6fd_seti+0x1f9) [0x5f08d9]
(EE) 12: /opt/thinlinc/libexec/Xvnc (vncCallReadWakeupHandlers+0x2a) [0x5e69fa]
(EE) 13: /opt/thinlinc/libexec/Xvnc (0x400000+0x1ee25c) [0x5ee25c]
(EE) 14: /opt/thinlinc/libexec/Xvnc (WakeupHandler+0x6b) [0x57b09b]
(EE) 15: /opt/thinlinc/libexec/Xvnc (WaitForSomething+0x492) [0x5d1942]
(EE) 16: /opt/thinlinc/libexec/Xvnc (Dispatch+0xa2) [0x576622]
(EE) 17: /opt/thinlinc/libexec/Xvnc (main+0x3ca) [0x45adea]
(EE) 18: /lib64/libc.so.6 (__libc_start_main+0xf5) [0x3cdfa21d65]
(EE) 19: /opt/thinlinc/libexec/Xvnc (0x400000+0x5c45d) [0x45c45d]
(EE) 
(EE) Segmentation fault at address 0x4460010

Fatal server error:
Caught signal 11 (Segmentation fault). Server aborting






Tue Jun 23 11:29:33 2015
 Timer:       handleTimeout(0x2222368)
 VNCSConnST:  FramebufferUpdateRequest 768x635 at 87,121 exceeds framebuffer
              768x891
 VNCSConnST:  FramebufferUpdateRequest 768x256 at 87,756 exceeds framebuffer
              768x891
 VNCSConnST:  FramebufferUpdateRequest 768x891 at 87,121 exceeds framebuffer
              768x891
 PixelBuffer: reallocating managed buffer (768x891)
(EE) 
(EE) Backtrace:
(EE) 0: /opt/thinlinc/libexec/Xvnc (xorg_backtrace+0x36) [0x5dac96]
(EE) 1: /opt/thinlinc/libexec/Xvnc (0x400000+0x1deb69) [0x5deb69]
(EE) 2: /lib64/libpthread.so.0 (0x3cdfe00000+0xf6d0) [0x3cdfe0f6d0]
(EE) 3: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager11analyseRectEPKNS_11PixelBufferEPNS_8RectInfoEi+0x3dd) [0x61af2d]
(EE) 4: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager12writeSubRectERKNS_4RectEPKNS_11PixelBufferE+0xef) [0x61bdcf]
(EE) 5: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager10writeRectsERKNS_6RegionEPKNS_11PixelBufferE+0x117) [0x61c167]
(EE) 6: /opt/thinlinc/libexec/Xvnc (_ZN3rfb13EncodeManager11writeUpdateERKNS_10UpdateInfoEPKNS_11PixelBufferEPKNS_14RenderedCursorE+0xbe) [0x61c92e]
(EE) 7: /opt/thinlinc/libexec/Xvnc (_ZN3rfb16VNCSConnectionST22writeFramebufferUpdateEv+0x4eb) [0x61600b]
(EE) 8: /opt/thinlinc/libexec/Xvnc (_ZN3rfb16VNCSConnectionST15processMessagesEv+0xdc) [0x61676c]
(EE) 9: /opt/thinlinc/libexec/Xvnc (_ZN14XserverDesktop13wakeupHandlerEP6fd_seti+0xe8) [0x5f7858]
(EE) 10: /opt/thinlinc/libexec/Xvnc (0x400000+0x1ee1b4) [0x5ee1b4]
(EE) 11: /opt/thinlinc/libexec/Xvnc (WakeupHandler+0x5b) [0x57f9fb]
(EE) 12: /opt/thinlinc/libexec/Xvnc (WaitForSomething+0x4b6) [0x5d7976]
(EE) 13: /opt/thinlinc/libexec/Xvnc (Dispatch+0xb2) [0x57ae92]
(EE) 14: /opt/thinlinc/libexec/Xvnc (main+0x3da) [0x56888a]
(EE) 15: /lib64/libc.so.6 (__libc_start_main+0xf5) [0x3cdfa21d65]
(EE) 16: /opt/thinlinc/libexec/Xvnc (0x400000+0x57529) [0x457529]
(EE) 
(EE) Segmentation fault at address 0x4b86000

Fatal server error:
Caught signal 11 (Segmentation fault). Server aborting
Comment 2 Pierre Ossman cendio 2015-09-21 10:33:13 CEST
We are unable to reproduce this crash anymore, even when using old versions of ThinLinc. Closing for now.

Note You need to log in before you can comment on or make changes to this bug.