Bug 8277 - SyntaxWarning errors in log when starting tlwebadm/tlwebaccess
Summary: SyntaxWarning errors in log when starting tlwebadm/tlwebaccess
Status: CLOSED FIXED
Alias: None
Product: ThinLinc
Classification: Unclassified
Component: Web Administration (show other bugs)
Version: trunk
Hardware: PC Unknown
: P2 Normal
Target Milestone: 4.18.0
Assignee: Emelie
URL:
Keywords: linma_tester, prosaic
: 8364 (view as bug list)
Depends on:
Blocks:
 
Reported: 2023-12-29 14:21 CET by Pierre Ossman
Modified: 2024-10-22 14:50 CEST (History)
4 users (show)

See Also:
Acceptance Criteria:
MUST: - Web services should not generate SyntaxWarnings. Should not happen during installation or usage. - Web Access and Web Administrator should continue working SHOULD: - Make sure no new warnings appear


Attachments

Description Pierre Ossman cendio 2023-12-29 14:21:15 CET
Cheetah has some improper strings that historically have given DeprecationWarning:s when in debug mode:

https://github.com/CheetahTemplate3/cheetah3/issues/31

However, they've now been upgraded to SyntaxWarning:s that are visible even in production builds:

> Dec 29 12:56:48 lab-154.lkpg.cendio.se systemd[1]: Starting tlwebadm.service - ThinLinc Web Administration...
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Template.py:142: SyntaxWarning: invalid escape sequence '\*'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   '''
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Template.py:1617: SyntaxWarning: invalid escape sequence '\*'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   """Compile the template. This method is automatically called by
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Template.py:1728: SyntaxWarning: invalid escape sequence '\*'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   """Method for importing web transaction variables in bulk.
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Template.py:1984: SyntaxWarning: invalid escape sequence '\d'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   re.search('[ \t]*File.*line (\d+)', formatedExc).group(1))
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Template.py:2025: SyntaxWarning: invalid escape sequence '\d'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   cheetahPosMatch = re.search('line (\d+), col (\d+)', formatedExc)
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Template.py:2035: SyntaxWarning: invalid escape sequence '\d'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   cheetahPosMatch = re.search('line (\d+), col (\d+)',
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/SettingsManager.py:19: SyntaxWarning: invalid escape sequence '\('
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   '[\(]*' + Number + r'[ \t]*\+[ \t]*' + Number + '[\)]*')
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/SettingsManager.py:19: SyntaxWarning: invalid escape sequence '\)'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   '[\(]*' + Number + r'[ \t]*\+[ \t]*' + Number + '[\)]*')
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Parser.py:464: SyntaxWarning: invalid escape sequence '\*'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   + r'(?P<REFRESH_CACHE>\*' + interval + '\*)'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/Parser.py:557: SyntaxWarning: invalid escape sequence '\$'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   """Unescape any escaped Cheetah \$vars in the string.
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]: /opt/thinlinc/modules/thinlinc/Cheetah/SourceReader.py:8: SyntaxWarning: invalid escape sequence '\s'
> Dec 29 12:56:48 lab-154.lkpg.cendio.se bash[95267]:   ENCODINGsearch = re.compile("coding[=:]\s*([-\w.]+)").search
> Dec 29 12:56:48 lab-154.lkpg.cendio.se systemd[1]: Started tlwebadm.service - ThinLinc Web Administration.

Fortunately, it looks like upstream Cheetah has fixed the issue. So all we hopefully need is an upgrade.
Comment 1 Tobias cendio 2024-06-14 10:26:26 CEST
*** Bug 8364 has been marked as a duplicate of this bug. ***
Comment 4 Emelie cendio 2024-10-07 12:55:10 CEST
Before upgrading, we first compared our Cheetah in the vendor snapshot tree with the current Cheetah used in ThinLinc to see what see we changed last vendor drop.
>
> svn diff svn://svn.lkpg.cendio.se/vendor/cheetah/cheetah-2020-09-10-14-42/ svn://svn.lkpg.cendio.se/trunk/ctc/tlmisc/cheetah | less
>

We found two types of differences. The first one was the Python imports. ThinLinc uses the relative paths to Cheetah modules. This was done in bug 4245. The second difference was the removal of the CGI part of Cheetah, done in bug 8161.

On GitHub, these changes have been merged with some modifications.
- The imports changed in commit https://github.com/CheetahTemplate3/cheetah3/commit/b1c8cdce2ce1ae32faa787462cde64d34049207f
- Instead of removing CGI, the Cheetah project solved it using try. See commit https://github.com/CheetahTemplate3/cheetah3/commit/faf45251aafc7dadc6945260a6b371d821f5389a
Comment 6 Emelie cendio 2024-10-07 13:11:53 CEST
While trying to solve merge conflicts when trying to upgrade Cheetah, we think the best solution is to just use the code in GitHub and modify if needed. Svn made some strange suggestions during the merge conflict, where it removed necessary code, so this is hopefully the best solution. We hope unmodified Cheeath code from GitHub works, since all modifications mentioned in comment 4 are merged.

With new Cheetah in ThinLinc, we need to investigate if we will get DeprecationWarnings about CGI now that we want to use the code from GitHub where it still attempts to import CGI. We don't know which Python versions this applies to.
Comment 9 Emelie cendio 2024-10-08 11:31:18 CEST
With new Cheetah in ThinLinc we still got DepricationWarning, and the warning will show from Python 3.11+. As mentioned earlier, Cheetah has built in handling for Python 3.13+ where CGI is completely removed.

The warning is hidden inside Cheetah, since we otherwise need to hide it everywhere it is imported in ThinLinc.

A pull request was made with our solution to Cheetah on GitHub:
https://github.com/CheetahTemplate3/cheetah3/pull/62
Comment 10 Emelie cendio 2024-10-08 11:34:33 CEST
The new Cheetah works and shows no SyntaxWarning nor DepricationWarning.

It was tested on Fedora39 with Python 3.12.0, by tailing the journal- and thinlinc log-files.
Comment 11 Emelie cendio 2024-10-08 11:35:47 CEST
Note that both SyntaxWarning and DepricationWarning showed in the journal-log when starting Wab Administration and Web Access services.
Comment 12 Samuel Mannehed cendio 2024-10-08 12:50:27 CEST
> MUST:
> - Web services should not generate SyntaxWarnings. Should not happen during installation or usage.
> - Web Access and Web Administrator should continue working

Using jenkins build 3721 on RHEL 9:

✓ We tailed the journal and ThinLinc-logs on while:
  ✓ installing and running tl-setup
  ✓ visiting Web Access login, connecting, and lastly the "disconnected"-page
  ✓ visiting and using each page in Web Administration

> SHOULD:
> - Make sure no new warnings appear

As mentioned in comment 9, we noticed that the DeprecationWarning showed. After fixing it in commit r41229, we re-tested on Fedora 39 as mentioned in comment 10.
Comment 13 Linn cendio 2024-10-10 14:09:32 CEST
Tested with server build 3729 on Ubuntu 24.04 (Python v. 3.12.3).

Checked journalctl, tlwebadm.log and tlwebaccess.log. Did not see any Python warnings when installing or running tl-setup. Also logged in to Web Access and clicked around briefly on Web Admin, all logs looked good in that case too.

Checked the code changes, looks good and the addition for handling CGI deprecation warning is minimal.
Comment 14 Emelie cendio 2024-10-17 09:15:45 CEST
While trying to merge our pull request [1] with our solution to hide the warning to Cheetah on GitHub, we found the real reason why we get the DepricationWarning. During development, we set the environment variables PYTHONDEVMODE=1 PYTHONWARNINGS=default. These variables are the reason we get the warning, and the Cheetah team does not when they're testing.

We will therefore remove our solution, since our ThinLinc users will not see this warning.

[1] https://github.com/CheetahTemplate3/cheetah3/pull/62

Note You need to log in before you can comment on or make changes to this bug.