Comment 1 Peter Åstrand 2015-11-10 10:34:36 CET

Can probably be fixed if we implement the proper cache handling support in tlwebaccess.

Comment 4 Samuel Mannehed 2017-01-26 12:03:27 CET

Moving from 'MediumPrio' to '---' for discussion.

Comment 5 Samuel Mannehed 2018-07-01 16:40:38 CEST

https://github.com/novnc/noVNC/issues/1097

Comment 11 Pierre Ossman 2023-08-29 10:44:41 CEST

https://community.thinlinc.com/t/webaccess-in-4-15/686

Comment 12 Alexander Zeijlon 2025-01-24 13:10:55 CET

Right now, a browser will use cached files (if available) whenever it connects to our web services.

This is problematic if JavaScript calls in the HTML code, which is often rendered from a template on each request, is incompatible with older, cached JavaScript files.

The most straight forward way of handling this is to make sure that the browser always updates its cache if newer versions of the cached files are available.

Comment 13 Alexander Zeijlon 2025-01-24 13:17:56 CET

It should be sufficient to look at file modification timestamps. And this is something that is already a part of the HTTP protocol, and already implemented in our HTTP server.

Comment 14 Alexander Zeijlon 2025-01-24 13:55:29 CET

As I understand it, cache handling based on modification dates are handled by a combination of a few headers..

"Last-Modified" contains the modification date of the file on the server.

"Cache-Control", when set to "no-cache" in responses from the server, in practice tells the browser to only use cached files if they are up-to-date.

"If-Modified-Since" contains the server-side modification date of the file in the browser cache.

Comment 15 Alexander Zeijlon 2025-01-24 14:00:03 CET

In our desired scenario, we want the server to use the headers "Last-Modified: <timestamp>", and "Cache-Control: no-cache" in responses to file-requests.

This will trigger browsers to do validation for its cached files by performing a get-request with the header "If-Modified-Since: <timestamp>".

On such requests, the server should then respond with either:
- 200 OK, and the requested file, if the server side file has been modified since the provided timestamp.
- 304 Not Modified, and no file, if the serverside file has not been modified since the timestamp.

Comment 19 Adam Halim 2025-01-27 12:24:00 CET

Tested build 3891 with Web Access on Firefox and Chromium.

For it to work properly on Chromium, I had to use a signed certificate for Web Access. It seems like Chromium won't cache anything if there are certificate errors [1].

Could confirm that I get status "304 Not Modified" when refreshing.

[1] https://issues.chromium.org/issues/40140471

Comment 20 Alexander Zeijlon 2025-01-27 13:14:03 CET

We are now asking the browsers to verify their cached files, and this means that each visit to the web client url will result in more GET requests, even if these likely will result in a response "304 Not Modified" more often than not.

We should therefore look at if there are scenarios where the increased request rate has any effect on the overall usability.

Comment 21 Alexander Zeijlon 2025-01-27 14:52:38 CET

I tested loading the web client's agent page in different scenarios with heavy throttling enabled.

No throttling:
==============

  Without cache present: 0.35 s
  With cache present: 0.35 s

Low bw (500/500 kbit/s):
========================

  Without cache present: 28 s
  With cache present: 2.6 s

High RTT (200 ms):
===================

  Without cache present: 4 s
  With cache present: 3.9 s

Low bw (500/500 kbit/s), High RTT (200 ms):
============================================

  Without cache present: 29 s
  With cache present: 4.3 s


The measurements where we have a cache present are of interest for us here. In the worst case scenario with both low bandwidth and high round-trip-time, a page load takes around four seconds, which definitely should be OK from a usability perspective.

Comment 22 Alexander Zeijlon 2025-01-27 15:04:59 CET

We also made the decision to be a bit flexible when interpreting how to use the HTTP header "If-Modified-Since". See the HTTP specification [1].

[1] https://httpwg.org/specs/rfc9110.html#field.if-modified-since

The specification states that the header is to be used to check if a cached file has a newer state (as in modification time) on the server than what is in the browser cache.

But since they do not define a *hard* MUST for this usage, we can choose to interpret the purpose of the header as a way to check if the file has been modified at all. 

A scenario where this is convenient is when a server is downgraded, such that files are installed with older modification times than what a browser may have in cache.

Therefore, we now check for if the "If-Modified-Since" timestamp is not equal to the modification time of the file on the server, hence allowing cached files to be updated also when a server downgrade is done.

Comment 23 Alexander Zeijlon 2025-01-27 15:34:23 CET

Another option to what is stated in comment 22, would be to use the ETag and If-None-Match headers instead [1]

[1] https://httpwg.org/specs/rfc9110.html#field.if-none-match

But these are really meant to be used for managing cache for dynamically generated content.

Comment 24 Alexander Zeijlon 2025-01-27 15:36:23 CET

Testing:
========

Tested that we get "304 Not Modified" as expected with Chrome/Chromium, Firefox and Safari.

Comment 25 Alexander Zeijlon 2025-01-27 15:58:43 CET

> MUST:
> * Users shouldn't get a dysfunctional web
>   client after a server upgrade due to a
>   stale browser cache.
The server is now responding to file requests with files whenever these differ from what's in the browser cache.
> * The web client should be usable in terms
>   of fetching files in non-optimal scenarios,
>   w.r.t latency and bandwidth.
It is, see comment 21.
> SHOULD:
> * Users shouldn't get a dysfunctional web
>   client after a server downgrade due to a
>   stale browser cache.
Downgrades (older state on server than in cache) will now be handled just as with upgrades. See comment 22.

Comment 27 Adam Halim 2025-01-28 13:55:03 CET

Tested server build 3892 on RHEL 9 server.

I verified that the cache is used properly after an initial visit to Web Access and Web Admin on:
* Firefox
* Chromium
* Edge
* Safari

I also ran some experiments, similar to the ones in comment #21 on Web Admin, and compared them to loading times in 4.18.0:

No throttling (4.18.0 vs. build #3892):
==============

  Without cache present:  0.32 s -> 0.28 s
  With cache present:     0.12 s -> 0.20 s

Low bw (500/500 kbit/s):
========================

  Without cache present: 23  s -> 23 s
  With cache present:    5.1 s -> 0.5 s (*)
     (*) Firefox refused to cache thinlinc.iso, which took 5s to load.
         Ignoring this, the page loaded almost instantly.

High RTT (200 ms):
===================

  Without cache present: 1.1  s -> 1.1 s
  With cache present:    0.46 s -> 1.1 s

Low bw (500/500 kbit/s), High RTT (200 ms):
============================================

  Without cache present: 24  s -> 24  s
  With cache present:    5.5 s -> 1.3 s (*)
     (*) Firefox refused to cache thinlinc.iso, which took 5s to load.
         Ignoring this, the page loaded quickly.


In general, I think the values when the cache is present are acceptable.

Comment 28 Adam Halim 2025-01-28 14:57:45 CET

> MUST:
> ✅ Users shouldn't get a dysfunctional web
>   client after a server upgrade due to a
>   stale browser cache.
Indeed, upgrading from build 3981 to 3982 made the browser to download the newer files.
> SHOULD:
> ✅ Users shouldn't get a dysfunctional web
>   client after a server downgrade due to a
>   stale browser cache.
Indeed, downgrading from build 3982 to 3981 made the client to download the older files.
> ✅ The web client should be usable in terms
>    of fetching files in non-optimal scenarios,
>    w.r.t latency and bandwidth.
With the experiments performed in comment #21 and comment #27, I think the performance is acceptable.

Looked through the commits and release notes and think they look good!

Closing.

Comment 29 Adam Halim 2025-01-28 14:59:36 CET

(In reply to Adam Halim from comment #28)
> Indeed, upgrading from build 3981 to 3982 made the browser to download the
> newer files.
To clarify, both builds had the fix for this bug, the latter only included the release notes.