Something has regressed, and it is no longer possible to use the client in full-screen mode on macOS. This upstream bug report claims it was macOS 13 that broke things, but we don't have any machines to test this theory: https://github.com/TigerVNC/tigervnc/issues/1561 You get a completely black screen whenever you enable full-screen. It doesn't matter if you use a single monitor, or multiple, or if "Displays use separate spaces" is on or off. For some odd reason, it works if you start the client in full-screen mode. But if you disable it and re-enable it, then you get a black screen again. Tested on both ARM and x86. The problem only happens with "Send system keys" enabled, which suggests the problem has something to do with layering and the shielding window.
Apple seems to have done something with regard to levels in macOS 13, as this page states they've added a new type for the level: http://codeworkshop.net/objc-diff/sdkdiffs/macos/10.13/CoreGraphics.html No notes about any changes in semantics, though.
The following thread can be updated with any workarounds until a solution is found: https://community.thinlinc.com/t/thinlinc-client-in-full-screen-results-in-black-screen-on-macos-13/519
Our server-mac wasn't upgraded to 13 yet, so I can confirm that things work fine on macOS 12. Unfortunately, I don't see any changes in level switching between macOS 12 and 13. I added debug output to the level changes, and both machines do the same changes. The shielding level is also at the same level on both machines. If I increase the level to shielding+1, then I no longer get a black screen. Unfortunately, it screws with FLTK somehow and now the F8 menu is no longer visible. I also discovered a bit of a workaround. Opening one of the dialogues (options, about, ...) and closing it again gets the display working. I am not sure why, as the levels set are the same. One difference is that in the bad case we seem to capture the display multiple times, whilst in the good case only once.
It has something to do with the window ordering, not levels. If I put a call to orderFront: after adjusting the level, then everything works fine. This shouldn't be needed, though, as setLevel: is still explicitly documented as also ordering a window to the front. I'm also seeing another difference between macOS 12 and 13. When leaving full screen, the window loses focus on macOS 13. At first, I thought it had something to do with the menu window, but it happens even if I add the ability to toggle full screen using a key press. I can also see that it loses focus before doing anything with the displays or levels. I am uncertain if this is related, but it might be, since stacking and focus tend to have some correlations.
The loss of focus happens without keyboard grab as well, so perhaps not related. I got this backtrace for it though: > (lldb) bt > * thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGUSR1 > * frame #0: 0x00007ff807a82e5a libsystem_kernel.dylib`__kill + 10 > frame #1: 0x000000010001cfc2 vncviewer`DesktopWindow::fltkDispatch(int, Fl_Window*) + 162 > frame #2: 0x00000001000d2616 vncviewer`-[FLWindowDelegate windowDidResignKey:] + 70 > frame #3: 0x00007ff807b8b6e6 CoreFoundation`__CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__ + 137 > frame #4: 0x00007ff807c24ccc CoreFoundation`___CFXRegistrationPost_block_invoke + 86 > frame #5: 0x00007ff807c24c23 CoreFoundation`_CFXRegistrationPost + 534 > frame #6: 0x00007ff807b5ee61 CoreFoundation`_CFXNotificationPost + 809 > frame #7: 0x00007ff8088d1f2c Foundation`-[NSNotificationCenter postNotificationName:object:userInfo:] + 82 > frame #8: 0x00007ff80ad8d8dd AppKit`-[NSWindow resignKeyWindow] + 758 > frame #9: 0x00007ff80b53af66 AppKit`-[NSWindow _orderOut:calculatingKeyWithOptions:documentWindow:] + 319 > frame #10: 0x00007ff80ac3d1c2 AppKit`NSPerformVisuallyAtomicChange + 132 > frame #11: 0x00007ff80b53d008 AppKit`-[NSWindow _reallyDoOrderWindowOutRelativeTo:] + 634 > frame #12: 0x00007ff80b53d413 AppKit`-[NSWindow _reallyDoOrderWindow:] + 99 > frame #13: 0x00007ff80b53d689 AppKit`-[NSWindow _doOrderWindow:] + 295 > frame #14: 0x00000001000d5eb1 vncviewer`Fl_Window::fullscreen_off_x(int, int, int, int) + 113 > frame #15: 0x00000001000b7f75 vncviewer`Fl_Window::fullscreen_off(int, int, int, int) + 85 > frame #16: 0x0000000100028c24 vncviewer`Viewport::handleSystemEvent(void*, void*) + 436 > frame #17: 0x000000010008daf2 vncviewer`fl_send_system_handlers(void*) + 50 > frame #18: 0x00000001000d4555 vncviewer`+[FLApplication sendEvent:] + 21 > frame #19: 0x00000001000d4d39 vncviewer`fl_mac_flush_and_wait(double) + 473 > frame #20: 0x00000001003e2fc6 vncviewer`main + 2070 > frame #21: 0x0000000100019f74 vncviewer`start + 52
The focus loss seems to be its own thing, so let's move it to bug 8049 for clarity.
I've constructed a minimal Cocoa application, to rule out any strange behaviour from FLTK or TigerVNC. And I can see the issue there as well, confirming that this is some OS bug. I tried to understand the issue a bit better by using CGWindowListCreate() and +[NSWindow windowNumbersWithOptions:]. Unfortunately, I don't think it made things much clearer. Some observations though: * +[NSWindow windowNumbersWithOptions:] doesn't see all windows. Fortunately it sees our window and the shielding window, which are the important ones. * CGWindowListCreate() doesn't seem to properly update its information right away, so it cannot be fully trusted. On macOS 12 it fails to see restacking right away, and on macOS 13 it also fails to notice the creation of the shielding window. * +[NSWindow windowNumbersWithOptions:] shows the expected behaviour on macOS 12: 1. Shielding window gets created in front of our window once we grab the display 2. Our window is ordered in front of the shielding window once we change levels (no restacking required) * +[NSWindow windowNumbersWithOptions:] shows some wonky data on macOS 13 though: 1. Shielding window gets created when we grab the display, but is claimed to be at the bottom of the stack. Something that is clearly incorrect as we see it in front of everything else. 2. Changing levels claims to put our window in the front, ahead of another window that was previously in front of ours. The shielding window remains at the bottom, though. 3. Restacking our window in the front has no visible change in +[NSWindow windowNumbersWithOptions:], but in reality it fixes the issue and puts our window in front. If you call CGWindowListCreate() or +[NSWindow windowNumbersWithOptions:] at a later time, then any oddities will have been resolved and they will return data that corresponds to what you see on screen.
I've reported the issue to Apple, and it has been given feedback ID FB11893978.
Perhaps it might be possible to work around this by delaying the steps somehow, allowing the OS to resync itself. That will likely be very complex though, and risky. We normally expect the switch to full screen to be fairly atomic. Since the -[NSWindow orderFront:] hack works, and is very simple, I think we're safer off staying with that. It should really be a no-op, so the risk of side effects should be minimal.
Works well now with the workaround in place. Tried toggling back and forth between full screen and windowed. It works fine in the current build, but breaks in 4.14.0.
Tested using server build #2976 on Fedora37 and Ubuntu22.04, and client builds #2866 (pre-fix) and #2881 (post-fix) on MacOS 13.1. The client fullscreen scenarios are (1) starting in window mode, then enter fullscreen via f8 menu (2) starting in fullscreen on current monitor (3) starting in fullscreen on all monitors (4) starting in fullscreen on one of multiple monitors When using the pre-fix build, as mentioned in comment 0, it seems to be working as intended when starting in full screen, but goes completely black when going from windowed to full screen mode. In the post-fix build, however, all of the scenarios above checks out, so this should be resolved now. Closing.
