Comment 1 Aaron Sowry 2021-12-08 22:14:08 CET

Worth noting that as of Python 3.9, hashlib supports a "usedForSecurity" flag. 

MD5 invocations with this flag set to "False" will not cause FIPS verification to fail, since the standard allows MD5 if it is not being used for security purposes.

Comment 3 Pierre Ossman 2021-12-15 15:37:48 CET

Two areas discovered where we use MD5:

 * Computing signatures for the SSH host keys. Done by the agent and sent to the client. Not used by current clients, but unsure if older clients need this.

 * Used in generating the mcookie for Xvnc

Comment 4 Pierre Ossman 2021-12-17 15:00:07 CET

(In reply to Pierre Ossman from comment #3)
> Two areas discovered where we use MD5:
> 

For reference, this are the errors you see:

>  * Computing signatures for the SSH host keys. Done by the agent and sent to
> the client. Not used by current clients, but unsure if older clients need
> this.
> 

In vsmagent.log:

> 2021-12-16 16:57:40 WARNING vsmagent: Error parsing SSH key: 127.0.0.1 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDwt3T8mwEitzpxeW9zaWcZ09T4Ug9VdtZdYM1F02lDJ5QIKmUYGtuRl90Epo1b8HuOzCY6XxgJuUDJgbEl3oXFDcxBIcWYu/pVA/IuG/lykaoA/u3s3QBbpgZ3L7z5It5bt+4y5VzTBjzPFTVJmGhJ+Xo6EZS68TZFws0PqSSYhBMeIc3617LlrDHMzVYFIspaXUeUHXj93LRxUjrDqVRanjBL0Ey217+625CkWEQlfzbe6/InaZ5Fc5AqYBPIQx8fqhsRBZf8nFhF7ZQHyUTkSKB9F4yq+aHfYw1hj5QYMpmwvna4jMsybFV86tNn7Ogg3uafXGrTrgTxXjBsf+jr
> 2021-12-16 16:57:40 WARNING vsmagent: Error parsing SSH key: 127.0.0.1 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMt3xd2SaE3oryCkJzFbxTwjiT7cqQo3Lhpd3++9hDCYh9Jab/qPYLlwSXjJH1Pg5BP3rjw0u5o0g5rPDCOYnCM=

>  * Used in generating the mcookie for Xvnc

Also in vsmagent.log:

> 2021-12-17 13:06:26 ERROR vsmagent: Unhandled XMLRPC exception: <class 'ValueError'> [digital envelope r
> outines: EVP_DigestInit_ex] disabled for FIPS Traceback (most recent call last):
>   File "/opt/thinlinc/modules/thinlinc/vsm/asyncbase.py", line 104, in ooOOOoOO0
>     obj . handle_read_event ( )
>   File "/usr/lib64/python3.6/asyncore.py", line 423, in handle_read_event
>     self.handle_read()
>   File "/usr/lib64/python3.6/asynchat.py", line 151, in handle_read
>     self.found_terminator()
>   File "/opt/thinlinc/modules/thinlinc/vsm/xmlrpc.py", line 555, in found_terminator
>     self . handle_request ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/xmlrpc.py", line 569, in handle_request
>     self . handle_method ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/vsmagentchannel.py", line 120, in handle_method
>     self . params )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 50, in handle
>     self . check_old_client ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 62, in check_old_client
>     self . check_valid_user ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 69, in check_valid_user
>     self . setup_session_info ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 95, in setup_session_info
>     self . find_free_display ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 177, in find_free_display
>     self . find_free_display ( socket_status = SOCKET_NOT_INUSE )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 138, in find_free_display
>     self . set_sessionkey ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 201, in set_sessionkey
>     self . set_vncpassword ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 243, in set_vncpassword
>     self . correct_system_sockets ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 306, in correct_system_sockets
>     self . check_homedir ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 347, in check_homedir
>     self . start_session ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 352, in start_session
>     II1 = oO00 . start ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/sessionstart.py", line 40, in start
>     self . create_session_env ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/sessionstart.py", line 72, in create_session_env
>     self . session_env [ "TLSESSIONMCOOKIE" ] = self . mcookie ( )
>   File "/opt/thinlinc/modules/thinlinc/vsm/sessionstart.py", line 122, in mcookie
>     Oo0O0o0oO000 = hashlib . md5 ( )
> ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

Comment 7 Pierre Ossman 2022-05-23 15:59:13 CEST

(In reply to Pierre Ossman from comment #3)
> Two areas discovered where we use MD5:
> 
>  * Computing signatures for the SSH host keys. Done by the agent and sent to
> the client. Not used by current clients, but unsure if older clients need
> this.
> 

They do. Any client older than 4.1.0 will need an exact match of the fingerprint, compared to what the client's ssh produces. This was changed in bug 4557 where we check the entire key instead.

So removing MD5 here would effectively drop support for clients older than 4.1.0.

(If that is done, when we should probably have some dummy information instead of the fingerprint. It looks like the client will crash and burn if this field is missing, rather than present an error message.)

Is there a workaround for this bug?

I was forced to upgrade from Centos 7 to RHEL8 w/ fips enabled. I am trying to use the latest server 4.14 and latest client 4.14. I have no need for support clients older than 4.1.0.

Trying to connect on a fresh install yields same backtrack reported above in Comment #4.

Comment 9 Pierre Ossman 2022-06-27 08:33:58 CEST

No, I'm afraid we are not aware of any workaround for this right now.

I hand modified the thinlinc/modules/thinlinc/vsm/sessionstart.py file and changed hashlib.md5() to hashlib.sha256(). That worked. I did this based on this bug description indicating this hash calculation doesn't even matter except for supporting of very old 4.1 clients. So maybe you just need a flag or configuration option that does something similar to what I did by hand or a way to disable this altogether? Looking on your site, version 4.1 is 9 or 10 years old, right?

Comment 11 Pierre Ossman 2022-08-15 12:52:55 CEST

https://community.thinlinc.com/t/thinlinc-fails-due-to-fips-mode/392/6

Comment 13 Adam Halim 2024-02-27 09:34:25 CET

The mcookie generated and used by Xvnc is generated pseudo-randomly and
cryptographically secure. Looks like any 128-bit data is sufficient, as long as
it is difficult for someone to guess it [1]. There are many ways to generate
random data. For cryptographically secure randomness, os.urandom() can be used,
together with binascii.hexlify() to make it a hexadecimal string (similar to
how it worked previously)

When testing an old client (4.0.0) on Fedora 38 against a server (build 3507)
on RHEL 9 with FIPS enabled, I get the following on the client log when I try
to connect:
> 2024-02-26T16:12:56: Processing SSH output: no matching mac found: client hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 server hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512
And the following message on the client:
> "Couldn't setup a secure tunnel to ThinLinc Server.
> (Couldn't establish SSH tunnel, SSH termianted.)
On the server side, sshd logs the following:
> 47 lab-190.lkpg.cendio.se sshd[16448]: Unable to negotiate with 10.47.1.80 port 42914: no matching host key type found. Their offer: ssh-rsa,ssh-dss [preauth]
Looks like clients older than 4.0.0 won't be able to connect to FIPS-enabled
systems, and it seems like we won't be able to display a proper error message.

Moving forward, I would like to keep compatibility with older clients if FIPS
is disabled on the system.

[1] https://www.x.org/releases/X11R7.5/doc/man/man7/Xsecurity.7.html

Comment 14 Linn 2024-03-01 10:46:46 CET

(In reply to Adam Halim from comment #13)
> When testing an old client (4.0.0) on Fedora 38 against a server (build 3507)
> on RHEL 9 with FIPS enabled, I get the following on the client log when I try
> to connect:
> > 2024-02-26T16:12:56: Processing SSH output: no matching mac found: client hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 server hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512
> 
> And the following message on the client:
> > "Couldn't setup a secure tunnel to ThinLinc Server.
> > (Couldn't establish SSH tunnel, SSH termianted.)
> 
> On the server side, sshd logs the following:
> > 47 lab-190.lkpg.cendio.se sshd[16448]: Unable to negotiate with 10.47.1.80 port 42914: no matching host key type found. Their offer: ssh-rsa,ssh-dss [preauth]
> Looks like clients older than 4.0.0 won't be able to connect to FIPS-enabled
> systems, and it seems like we won't be able to display a proper error
> message.

I tested running a 4.0.0 client on Fedora 39 against server build 3508 on RHEL 9 with FIPS disabled, and the results were almost identical to the server with FIPS enabled.

The slight difference is in tlclient.log:
> 2024-03-01T09:23:17: Processing SSH output: no hostkey alg
The popup message in the client is still the same, as well as the sshd log on server side:
> Mar 01 09:23:17 lab-92.lkpg.cendio.se sshd[51839]: Unable to negotiate with 10.47.1.10 port 46086: no matching host key type found. Their offer: ssh-rsa,ssh-dss [preauth]

I also tested connecting the 4.0.0 client to the same server build on RHEL 8 (FIPS disabled), and could connect without any issues.

I have not yet looked deeper into why a non-FIPS RHEL 9 have this issue.

Comment 15 Linn 2024-03-04 09:32:15 CET

The issue seen with client 4.0.0 and server on RHEL 9 in comment 14 seems to be bug 7933. Below is a quote from bug 7933 comment 3:
> So Red Hat gave a clue as to what the issue is. It seems sshd explodes if the 
> client supports the older "ssh-rsa" algorithm, instead of the newer 
> "rsa-sha2-256" or "rsa-sha2-512". The problem seems to be the corner case that
> all three of these use the same key, just different hash algorithms. The issue
> cannot be seen for other deprecated algorithms, e.g. "ssh-dss".
When testing the backwards compatibility, we should test on RHEL 8 or another dist that does not have the sshd issue seen on RHEL 9.

Comment 20 Linn 2024-03-07 16:18:29 CET

Through the changes made for the mcookie in this bug, some default behaviour when logging in will change.

Firstly, we still rely on /dev/urandom for generating the base of the mcookie (though using it indirectly through os.urandom()). Previously, if /dev/urandom did not exist, due to how hashlib.md5() and hexdigest() works, a valid mcookie would be generated anyway. As a result, users were able to log in and use ThinLinc. However, the mcookie would have the same value at every login, making it easy to guess.

After the changes in this bug, it will no longer be possible to log in if /dev/urandom does not exist. Despite the change, not being to log in feels like a more secure and proper way of handling this scenario.


Also, note that the scenario that /dev/urandom is missing is very unlikely. Since it is so rare, clean error handling was considered an EXTRA/COULD feature, and other things were prioritised instead.

Comment 21 Linn 2024-03-07 16:53:13 CET

Tested running with server on RHEL 8 against a client build 3407 on Fedora 39.

> MUST:
>   * ThinLinc server should function if MD5 is disabled on the system
It does. MD5 is disabled when FIPS is enabled, and I can log in to the server while it is in FIPS mode. 

> SHOULD:
>   * Client compatibility should not be affected by any changes made (if FIPS
> is disabled)
Tested logging in with a 4.0.0 client, and can log in when if FIPS is disabled, so backwards compatibility has been kept.


Acceptance criteria that were not fulfilled:
> SHOULD:
>   * If client compatibility can't be upheld, it should be communicated in a
> clear way.
Not applicable, client compatibility was upheld.

> EXTRA:
>   * The client should function if MD5 is disabled on the system
>   * Client compatibility should not be affected by any changes made (even
> with FIPS enabled)
These criteria were down-prioritised and not done.


Tips for tester
===============
* To enable FIPS, run: sudo fips-mode-setup --enable
* To check if FIPS is enabled or not, look for "fips=1" in: cat /proc/cmdline

* If you are interested to test client 4.0.0 on RHEL 9 (FIPS disabled), the theory is that login will work if sshd is configured to get around bug 7933. Martin may know how to configure this.

Comment 23 Tobias 2024-03-11 16:22:09 CET

General
=======
Tested connecting to servers 4.16.0 and 4.16.0post-3514 on FIPS enabled/disabled RHEL8 and FIPS enabled RHEL9 using clients 4.0.0, 4.16.0, and 4.16.0post-3412 on Ubuntu22.04 with the following results

FIPS disabled RHEL8
   Server 4.16.0
      ✅ Client 4.0.0
      ✅ Client 4.16.0
      ✅ Client 4.16.0post-3412
   Server 4.16.0post-3514
      ✅ Client 4.0.0
      ✅ Client 4.16.0
      ✅ Client 4.16.0post-3412

FIPS enabled RHEL8
   Server 4.16.0
      ❌ Client 4.16.0
      ❌ Client 4.16.0post-3412
   Server 4.16.0post-3514
      ✅ Client 4.16.0
      ✅ Client 4.16.0post-3412

FIPS enabled RHEL9
   Server 4.16.0
      ❌ Client 4.16.0
      ❌ Client 4.16.0post-3412
   Server 4.16.0post-3514
      ✅ Client 4.16.0
      ✅ Client 4.16.0post-3412

Note that the FIPS enabled RHEL9 was installed FIPS compliantly as per RHEL recommendations [1], while on RHEL8 FIPS mode was merely enabled/disabled on a regular install,
> sudo fips-mode-setup --enable
> sudo reboot

Evidently ThinLinc clients cannot connect to ThinLinc servers installed on FIPS compliant systems without the latest changes in place.

[1] https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/security_hardening/assembly_installing-the-system-in-fips-mode_security-hardening

Acceptance criteria
===================
MUST:
✅ ThinLinc server should function if MD5 is disabled on the system

  It does.

SHOULD:
✅ Client compatibility should not be affected by any changes made (if FIPS is disabled)

  Working as intended.

❌ If client compatibility can't be upheld, it should be communicated in a clear way.

  This criteria wasn't handled since client compatibility is upheld.

EXTRA:
❌ The client should function if MD5 is disabled on the system
❌ Client compatibility should not be affected by any changes made (even with FIPS enabled)

  These EXTRA criteria were not considered.

Comment 27 Linn 2024-03-12 08:59:14 CET

The leftover function has now been removed since it is no longer in use. I also updated the release notes to have the correct grammar.

Comment 28 Tobias 2024-03-12 10:05:50 CET

Things seem to be in order and the primary acceptance criteria are fulfilled. Closing.