Bug 5845 - Automatically hide the control bar of the HTML5 client
Summary: Automatically hide the control bar of the HTML5 client
Status: CLOSED FIXED
Alias: None
Product: ThinLinc
Classification: Unclassified
Component: Web Access (show other bugs)
Version: trunk
Hardware: PC Unknown
: P2 Normal
Target Milestone: 4.7.0
Assignee: Samuel Mannehed
URL:
Keywords: derfian_tester, hean01_tester, relnotes
Depends on:
Blocks: 4889
  Show dependency treegraph
 
Reported: 2016-04-18 16:20 CEST by Samuel Mannehed
Modified: 2016-12-05 15:10 CET (History)
4 users (show)

See Also:
Acceptance Criteria:


Attachments

Description Samuel Mannehed cendio 2016-04-18 16:20:23 CEST
Some users might need all the screen real estate possible, especially on mobile devices. The control bar takes up quite a lot of space at the top of the screen.
Comment 1 Samuel Mannehed cendio 2016-04-18 16:22:45 CEST
One solution for showing the control bar could be a floating, movable button along the edge. We could also use the F8 key on desktop to show the control bar, but we still need a solution for mobile devices. See bug 4922.
Comment 2 Pierre Ossman cendio 2016-09-05 10:26:08 CEST
We now have two pull requests upstream as a result of this work:

https://github.com/kanaka/noVNC/pull/651
https://github.com/kanaka/noVNC/pull/653

We don't have time to wait for these to be merged, so we'll do a local commit and get back in sync at a later point.
Comment 11 Pierre Ossman cendio 2016-09-06 12:59:30 CEST
All done. Tester should verify the toolbar with our usual matrix of tested browsers.
Comment 12 Samuel Mannehed cendio 2016-09-07 11:29:56 CEST
(In reply to comment #11)
> All done. Tester should verify the toolbar with our usual matrix of tested
> browsers.

More specifically, the tester should verify the buttons in the toolbar and the general functionality for controlling the toolbar appearance. Lastly, the tester should verify that automatic resize of the session still works as well.

The buttons in the toolbar are:

* Panning (touch only)
* Mouse buttons (touch only)
* Clipboard
* Extra keyboard keys (also displays on-screen keyboard on touch)
* Disconnect

The matrix of tested browsers can be found here:

http://intranet/ThinLinc/Testing#HTML5_client_Tests_.2BIxg-
Comment 13 Karl Mikaelsson cendio 2016-09-08 10:47:00 CEST
When the browser window is made small enough, the menu handle can disappear off-screen.
Comment 14 Thomas Nilefalk cendio 2016-09-08 10:53:39 CEST
When disconnecting, the "disconnected"-message is shown in a white rounded box which often will show up on a white background page. A thin border around the box should improve visual appearance in this case.
Comment 15 Thomas Nilefalk cendio 2016-09-08 11:03:51 CEST
The Shift and AltGr buttons on the keyboard fold out does not work in
situations where you would expect them too.

E.g. it is not possible to activate Shift and then press a single letter key on
 the keyboard to get an uppercase letter. Neither can you create an alternate
character such as 'ø' (created by AltGr+ö on Swedish keyboards) by activating
AltGr and then pressing 'ö' (this simply generates 'ö').

It seems this is related to the way the browser captures and propagates
keypresses, characters, symbols and events.

There might be situations where the AltGr could be used, e.g. superuser
creating isoteric keyboard shortcuts.

With the Shift and AltGr toggles present a common user might think that they
can be used in the way described above. This would be confusing and generate
support issues.
Comment 20 Pierre Ossman cendio 2016-09-09 10:44:41 CEST
Should be fixed now.
Comment 22 Samuel Mannehed cendio 2016-09-13 12:18:09 CEST
As discussed on the developer meeting today, we have decided to reopen this due to the layout of the new toolbar. Currently there is unused space at the bottom in the toolbar which we feel should be removed in order to make the toolbar smaller.

This becomes an issue when using the HTML5 client in small browsers (small height) since the toolbar always attempts to center itself. As the browser height becomes increasingly lower, and surpasses the height of the toolbar, more and more of the buttons will be placed outside the viewport.

We can mitigate this problem by removing the unused space at the bottom.
Comment 24 Samuel Mannehed cendio 2016-09-13 15:49:28 CEST
(In reply to comment #13)
> When the browser window is made small enough, the menu handle can disappear
> off-screen.

This is fixed now
Comment 26 Pierre Ossman cendio 2016-09-15 15:33:24 CEST
Should be all fixed now.
Comment 27 Karl Mikaelsson cendio 2016-09-19 13:53:11 CEST
Test report from Linux browsers, both tested with ThinLinc 4.6.0post, build 5239.

 * Firefox 48.0.1 on Fedora 24 x86_64

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete 
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide menu
   ✓ Move menu handle

 * Google Chrome 53.0.2785.116 on Fedora 24 x86_64

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete 
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide menu
   ✓ Move menu handle
Comment 28 Karl Mikaelsson cendio 2016-09-19 16:18:25 CEST
Test report from Windows browsers, all tested with ThinLinc 4.6.0post, build
5239. Test server is an Ubuntu 16.04.

 * Firefox 48.0.2 on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete 
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle

 * Google Chrome 53.0.2785.116m on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete 
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle

 * Internet Explorer 11 (11.545.10586.0/11.0.34) on Windows 10

   Clicking on the show/hide handle makes the mouse pointer disappear
   and all clicks will toggle the toolbar visibility.

 * Edge 25.10586.0.0 on Windows 10

   Fails to redirect to the agent - the client never starts.
Comment 32 Samuel Mannehed cendio 2016-09-20 15:29:18 CEST
(In reply to comment #28)
>  * Internet Explorer 11 (11.545.10586.0/11.0.34) on Windows 10
> 
>    Clicking on the show/hide handle makes the mouse pointer disappear
>    and all clicks will toggle the toolbar visibility.

Fixed in 31726.

>  * Edge 25.10586.0.0 on Windows 10
> 
>    Fails to redirect to the agent - the client never starts.

Can reproduce on Edge 25 but not on Edge version 39. It seems like Windows updated Edge with the "Anniversary Update" and this seems to have fixed something.

However, this seems to be a regression since Edge 25 works in the HTML5 client on a ThinLinc 4.6.0 server. Will investigate further.
Comment 33 Samuel Mannehed cendio 2016-09-20 16:16:01 CEST
(In reply to comment #32)
> Can reproduce on Edge 25 but not on Edge version 39. It seems like Windows
> updated Edge with the "Anniversary Update" and this seems to have fixed
> something.
> 
> However, this seems to be a regression since Edge 25 works in the HTML5 client
> on a ThinLinc 4.6.0 server. Will investigate further.

Further investigation shows that this isn't a regression after all. The issue is that Edge doesn't show the dialogue for accepting an unsigned certificate for the agent if the agent hostname is different from the address of the master you connected to. Added bug 5997 for this. Marking as resolved - ready for testing once again.
Comment 34 Samuel Mannehed cendio 2016-09-21 12:40:34 CEST
Moving the controlbar handle doesn't work in Safari on iOS, we get an exception:

TypeError: TouchEventConstructor is not a constructor (evaluating 'new e.constructor(e.type, e)')
Comment 35 Karl Mikaelsson cendio 2016-09-21 12:54:30 CEST
More test report from previously failing Windows browsers. All tested with ThinLinc 4.6.0post, build 5241. Test server is an Ubuntu 16.04.

 * Internet Explorer 11 (11.545.14393.0/11.0.35) on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle

 * Edge 38.14393.0.0 on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete 
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle

   Edge is still REALLY picky about hostnames. See bug 5997 for more details.
Comment 37 Samuel Mannehed cendio 2016-09-22 14:28:39 CEST
(In reply to comment #34)
> Moving the controlbar handle doesn't work in Safari on iOS, we get an
> exception:
> 
> TypeError: TouchEventConstructor is not a constructor (evaluating 'new
> e.constructor(e.type, e)')

Fixed in r31740.

Due to the amount of changes made, we think it's best to retest everything:

The tester should verify the buttons in the toolbar and the general functionality for controlling the toolbar appearance. Lastly, the tester should verify that automatic resize of the session still works as well.

The buttons in the toolbar are:

* Panning (touch only)
* Mouse buttons (touch only)
* Clipboard
* Extra keyboard keys (also displays on-screen keyboard on touch)
* Disconnect

The matrix of tested browsers can be found here:

http://intranet/ThinLinc/Testing#HTML5_client_Tests_.2BIxg-
Comment 38 Thomas Nilefalk cendio 2016-09-22 16:58:21 CEST
MacOS: Chrome, Safari and Firefox tested : Passed.
Comment 39 Karl Mikaelsson cendio 2016-09-22 17:48:54 CEST
Test report from Windows browsers, all tested with ThinLinc 4.6.0post-5241.r31740. Test server is an Ubuntu 16.04.

 * Firefox 48.0.2 on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

 * Google Chrome 53.0.2785.116m on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

 * Internet Explorer 11 (11.545.14393.0/11.0.35) on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

 * Edge 38.14393.0.0 on Windows 10

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize
Comment 40 Karl Mikaelsson cendio 2016-09-23 13:09:28 CEST
Partial test report from Android browsers, all tested with ThinLinc
4.6.0post-5241.r31740. Test server is an Ubuntu 16.04.

 * Google Chrome 53.0.2785.124 on Android 5.1.1 (Nexus 7)

   The on-screen keyboard disappears if you:

    - hide the toolbar
    - press the mouse button button
    - press the clipboard button

   I haven't got panning to work at all yet, but I believe the keyboard will
   disappear if you press the panning button as well.
Comment 41 Henrik Andersson cendio 2016-09-23 13:47:29 CEST
I have tested chrome and firefox on linux platform and have a few inputs to the
implementation, otherwise it works as expected.

Firefox 48.0.1

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

Chrome 53.0.2785.116

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize


[1] I expect a toolbar to have a few buttons / indicators showing states, with
the current implementation the keyboard states are buried into sub menu. A user
can not hide and then show the toolbar and see states directly without clicking
down the keyboard submenu. One possible workaround could be that menu / submenu
states are saveed and when showing toolbar the keyboard submenu is also
expanded.

[2] the items in the toolbar are seleectable which looks not so great when
accidentally done.

[3] The alt state indicator is easily taken out of sync. Press alt through menu
and then alt on keyboard.
Comment 42 Henrik Andersson cendio 2016-09-23 13:53:06 CEST
(In reply to comment #41)
> 
> [1] I expect a toolbar to have a few buttons / indicators showing states, with
> the current implementation the keyboard states are buried into sub menu. A user
> can not hide and then show the toolbar and see states directly without clicking
> down the keyboard submenu. One possible workaround could be that menu / submenu
> states are saveed and when showing toolbar the keyboard submenu is also
> expanded.
> 

Also, this is inconsitent behaviour between keyboard key states and mouse states which actually is visible on the main toolbar.
Comment 43 Karl Mikaelsson cendio 2016-09-23 15:13:31 CEST
(In reply to comment #40)
> Partial test report from Android browsers, all tested with ThinLinc
> 4.6.0post-5241.r31740. Test server is an Ubuntu 16.04.
> 
>  * Google Chrome 53.0.2785.124 on Android 5.1.1 (Nexus 7)
> 
>    The on-screen keyboard disappears if you:

Also:

     - Press the Ctrl-Alt-Delete button - pressing other "key" buttons does not hide keyboard.
Comment 45 Samuel Mannehed cendio 2016-09-23 16:23:02 CEST
(In reply to comment #40)
>    The on-screen keyboard disappears if you:
> 
>     - hide the toolbar
>     - press the mouse button button
>     - press the clipboard button

(In reply to comment #43)
>      - Press the Ctrl-Alt-Delete button - pressing other "key" buttons does not
> hide keyboard.

These items were fixed in r31744.
Comment 46 Samuel Mannehed cendio 2016-09-23 16:36:58 CEST
(In reply to comment #45)
> (In reply to comment #40)
> >    The on-screen keyboard disappears if you:
> > 
> >     - hide the toolbar
> >     - press the mouse button button
> >     - press the clipboard button
> 
> (In reply to comment #43)
> >      - Press the Ctrl-Alt-Delete button - pressing other "key" buttons does not
> > hide keyboard.
> 
> These items were fixed in r31744.

Correction:

> >     - press the mouse button button
> >     - press the clipboard button

Disappearing keyboard in these above cases were not fixed, not a regression.
Comment 47 Samuel Mannehed cendio 2016-09-26 11:52:30 CEST
(In reply to comment #41)
> [1] I expect a toolbar to have a few buttons / indicators showing states, with
> the current implementation the keyboard states are buried into sub menu. A user
> can not hide and then show the toolbar and see states directly without clicking
> down the keyboard submenu. One possible workaround could be that menu / submenu
> states are saveed and when showing toolbar the keyboard submenu is also
> expanded.

I believe that users use these buttons temporarily and don't regard them as permanent states. I think the current behavior is good enough.

> [2] the items in the toolbar are seleectable which looks not so great when
> accidentally done.

Not a regression, opened bug 6010 for this.

> [3] The alt state indicator is easily taken out of sync. Press alt through menu
> and then alt on keyboard.

Not a regression, opened bug 6009 for this.
Comment 48 Bojan Memetovic cendio 2016-09-26 12:36:36 CEST
Test report from IOS browsers, all tested with ThinLinc
4.6.0post-5241.r31746. Test server is an Fedora 24.

 * Firefox Version 5.1(1) on Ipad mini 

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

* Google chrome Version 52.0.2743.84 on Ipad mini 

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

* Safari Version 537.36 on Ipad mini 

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize
Comment 49 Karl Mikaelsson cendio 2016-09-26 15:40:13 CEST
Test report from Android browsers, both tested with ThinLinc
4.6.0post, build 5243-r31742 (from jenkins).

 * Firefox 49 on Android 5.1.1

   ✓ Panning button
   ✓ Mouse button
   ✓ Toggle keyboard
   ✓ Hotkey submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move toolbar handle

 * Google Chrome 53.0.2785.124 on Android 5.1.1

   ✓ Panning button
   ✓ Mouse button
   ✓ Toggle keyboard
   ✓ Hotkey submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move toolbar handle
Comment 50 Bojan Memetovic cendio 2016-09-27 12:14:47 CEST
Test report from IOS browsers on ISO10, all tested with ThinLinc
4.6.0post-5241.r31746. Test server is an Fedora 24.

 * Firefox Version 5.3(2) on Ipad mini 

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

* Google chrome Version 53.0.2784.109 on Ipad mini 

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize

* Safari Version 10 on Ipad mini 

   ✓ Keyboard submenu: Ctrl, Alt, Tab, Escape and Ctrl-Alt-Delete
   ✓ Clipboard
   ✓ Disconnect
   ✓ Show/hide toolbar
   ✓ Move menu handle
   ✓ Session resize
Comment 51 Pierre Ossman cendio 2016-10-17 09:43:37 CEST
Found two issues with the handle:

 - Sometimes it will start at the very top of the control bar

 - The coordinates for the dragging get screwed up if the page is scrolled down a bit.
Comment 54 Samuel Mannehed cendio 2016-10-17 16:59:58 CEST
(In reply to comment #51)
> Found two issues with the handle:
> 
>  - Sometimes it will start at the very top of the control bar

Fixed in r31807.

>  - The coordinates for the dragging get screwed up if the page is scrolled down
> a bit.

Fixed in r31806.

The tester should verify that moving the handle still works as expected on all platforms and browsers.
Comment 55 Henrik Andersson cendio 2016-10-18 16:47:11 CEST
(In reply to comment #54)
> (In reply to comment #51)
> > 
> >  - Sometimes it will start at the very top of the control bar
> >

How to force reproduce the bug:

 - Open debug mode in browser and find element named noVNC_control_bar
 - Change css and add 'display: none' for the above element
 - Resize browser window to trig a resize
 - Remove 'display: none' as added above to show toolbar

 
> >  - The coordinates for the dragging get screwed up if the page is scrolled down
> > a bit.

How to reproduce:

 - use xrandr within a session and set a big screen size to view the scrollbars
 - scroll down a bit using scrollbar
 - now while dragging the control bar handle it has an offset of pixels matching scroll
Comment 56 Karl Mikaelsson cendio 2016-10-19 08:58:11 CEST
We got a report from a customer that got this traceback on the beta release.

> About Safari:
> - Error is "TypeError: UI.popupStatus is not a function. (In
>   'UI.popupStatus(msg)', 'UI.popupStatus' is undefined)" within
>   ui.js.
Comment 58 Samuel Mannehed cendio 2016-10-19 10:40:28 CEST
(In reply to comment #56)
> We got a report from a customer that got this traceback on the beta release.
> 
> > About Safari:
> > - Error is "TypeError: UI.popupStatus is not a function. (In
> >   'UI.popupStatus(msg)', 'UI.popupStatus' is undefined)" within
> >   ui.js.

Fixed in r31812. This fix was very small and limited to error handling in a corner case, so this does not mean that we have to retest anything.

Easy way to reproduce:

1. login using the HTML5 client 4.7.0beta1
2. disconnect (don't log out)
3. on the server, remove /var/opt/thinlinc/sessions/<username>/last/vncpasswd
4. reconnect

As soon as the new build (5267) is completed this can be tested.
Comment 59 Pierre Ossman cendio 2016-10-19 16:38:28 CEST
(In reply to comment #56)
> We got a report from a customer that got this traceback on the beta release.
> 
> > About Safari:
> > - Error is "TypeError: UI.popupStatus is not a function. (In
> >   'UI.popupStatus(msg)', 'UI.popupStatus' is undefined)" within
> >   ui.js.

Verified that this works now.
Comment 60 Pierre Ossman cendio 2016-10-21 09:29:46 CEST
(In reply to comment #51)
> Found two issues with the handle:
> 
>  - Sometimes it will start at the very top of the control bar
> 
>  - The coordinates for the dragging get screwed up if the page is scrolled down
> a bit.

No regressions seen on Android or iOS for these fixes. Tested Safari, Chrome and Firefox.
Comment 61 Henrik Andersson cendio 2016-10-21 09:41:47 CEST
Verified that Thinlinc 4.7.0rc3 fixes the problem on linux platform.

- Chrome 54.0.2785.116
- Firefox v49.0
Comment 62 Pierre Ossman cendio 2016-10-21 10:01:18 CEST
Tested Safari, Firefox and Chrome on macOS as well.

That should be all. Closing.

Note You need to log in before you can comment on or make changes to this bug.