Bug 7629 - VSM Agent still uses Python 2
Summary: VSM Agent still uses Python 2
Alias: None
Product: ThinLinc
Classification: Unclassified
Component: VSM Agent (show other bugs)
Version: trunk
Hardware: PC Unknown
: P2 Normal
Target Milestone: 4.13.0
Assignee: Niko Lehto
Keywords: prosaic
Depends on:
Blocks: 7572
  Show dependency treegraph
Reported: 2021-01-22 12:46 CET by Linn
Modified: 2021-07-22 12:35 CEST (History)
2 users (show)

See Also:
Acceptance Criteria:


Description Linn cendio 2021-01-22 12:46:25 CET
It needs to be converted to work with Python 3.
Comment 1 Linn cendio 2021-01-22 13:00:31 CET
Here is a list of shared dependencies between VSM Agent and VSM Server (bug 7630):

* vsm.xmlrpc
* ctcommon
* vsm.vsmcommon
* vsm.extproc
* vsm.async
* which
* vsm.authoritychecker
* vsm.check_environment
Comment 57 Frida Flodin cendio 2021-02-17 08:59:27 CET
As a part of this bug we no longer run all external programs with LC_ALL set to 'C'. We have investigated and not found a direct need for this anymore. I have tested on Ubuntu20.04 with locale set to zh_CN.GB18030 and everything seems to work fine:

* Normal login and reconnection works
* Sending license warning mail with non-ASCII content works
* tl-limit-printers works
Comment 58 William Sjöblom cendio 2021-02-18 13:17:03 CET
Clarification related to revision r36189:

After the conversion of `thinlinc.selinux' a couple of functional tests were successfully performed:
* Running tl-setup on a VM running CentOS-8, verifying that tl-setup can detect SELinux-enabled systems.
* Creating a session with a user created using `useradd --no-create-home', verifying that the agent is able to create new home directories on SELinux-enabled systems.
Comment 119 Frida Flodin cendio 2021-03-24 13:36:30 CET

As a consequence of converting handler_reqsession to Python 3 we have change the allowed format for the configuration of '/vsmagent/make_homedir_mode'. This value is easiest written as an octal integer, e.g. '0700'. The configuration files should not depend on what programming language is used to parse them. Therefore we consider it a bug that we allowed the mode to be written as '0o700', which is Python specific syntax. Now we only support octal written as '0700' and decimal '448'.

We want hiveconf files to support as narrow format as possible. Since it's really cumbersome to change in the future as we always need to support old configuration files. Also, the more we allow the more we need to keep track of when testing, so it's better to promise as little as possible.
Comment 146 Frida Flodin cendio 2021-03-29 16:20:50 CEST
As a part of this bug (and bug 7630, bug 7572) some tlmisc modules no longer need Python 2 support:
 * daemon
 * pydes
 * xmlrpcutil

None of them needed any changed to drop Python 2 support. The only change I did
was not running the tests with Python 2 and using a context manger in 'daemon'
when opening the pid file.

I have tested that the following still works after my change: 
- [X] tlwebaccess (pid file creation, logging in, reconnecting)
- [X] tlwebadm (pid file creation, logging in and basic usage)

Tested on Ubuntu 20.04, Python 3.9 and Chrome 89.0
Comment 148 Frida Flodin cendio 2021-03-30 10:27:58 CEST
Tested build 1982 on SUSE 12 and got the following traceback when trying to reconnect to a session: 

> 2021-03-30 10:24:55 ERROR vsmagent: Unhandled exception with async process: <class 'TypeError'> startswith first arg must be bytes or a tuple of bytes, not str Traceback (most recent call last):
>   File "/opt/thinlinc/modules/thinlinc/vsm/asyncbase.py", line 103, in ooOOOoOO0
>     obj . handle_read_event ( fd )
>   File "/opt/thinlinc/modules/thinlinc/vsm/extproc.py", line 318, in handle_read_event
>     self . handle_close ( fd )
>   File "/opt/thinlinc/modules/thinlinc/vsm/extproc.py", line 288, in handle_close
>     self . handle_exit ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/extproc.py", line 464, in handle_exit
>     self . callback ( OO , self . stdout_data , self . stderr_data )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_verifysessions.py", line 28, in <lambda>
>     i11IiIIiii1II = lambda O0o000ooO , ooo0O , IIIi : self . _parse_ss_output ( O0o000ooO , ooo0O , IIIi , i1ii1 )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_verifysessions.py", line 120, in _parse_ss_output
>     if IIi1I1I1i . startswith ( "Recv-Q" ) :
> TypeError: startswith first arg must be bytes or a tuple of bytes, not str
> 2021-03-30 10:24:55 ERROR vsmagent: Killing process with pid 5695
> 2021-03-30 10:24:55 ERROR vsmagent: [Errno 3] No such process killing process with pid 5695
Comment 156 Pierre Ossman cendio 2021-04-07 14:56:54 CEST
Getting resource warnings for the main socket:

> sys:1: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('', 904)>

For some subprocess (like a session):

> /usr/lib/python3.8/subprocess.py:942: ResourceWarning: subprocess 421656 is still running

And when figuring out the hostname:

> /opt/thinlinc/modules/thinlinc/vsm/vsmagent.py:273: ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('', 33264)>
>   oo0000oOOoOOO = get_agent_hostname ( self . hive , self . log , warnings = 1 )
Comment 161 Pierre Ossman cendio 2021-04-08 10:50:40 CEST
Resource warnings have now been fixed.

Note that the one for hostname detection only happened when the network was down.

Note You need to log in before you can comment on or make changes to this bug.