Certain security standards (notably FIPS 140-2) forbid the use of MD5, even for non-sensitive operations like checksum. ThinLinc makes use of MD5 in at least one location (presenting the server host key fingerprint), which means it cannot be used by customers who require the FIPS certification. This bug is about finding where MD5 is used in ThinLinc, and determining whether we can use something else like SHA-256 instead.
Worth noting that as of Python 3.9, hashlib supports a "usedForSecurity" flag. MD5 invocations with this flag set to "False" will not cause FIPS verification to fail, since the standard allows MD5 if it is not being used for security purposes.
Two areas discovered where we use MD5: * Computing signatures for the SSH host keys. Done by the agent and sent to the client. Not used by current clients, but unsure if older clients need this. * Used in generating the mcookie for Xvnc
(In reply to Pierre Ossman from comment #3) > Two areas discovered where we use MD5: > For reference, this are the errors you see: > * Computing signatures for the SSH host keys. Done by the agent and sent to > the client. Not used by current clients, but unsure if older clients need > this. > In vsmagent.log: > 2021-12-16 16:57:40 WARNING vsmagent: Error parsing SSH key: 127.0.0.1 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDwt3T8mwEitzpxeW9zaWcZ09T4Ug9VdtZdYM1F02lDJ5QIKmUYGtuRl90Epo1b8HuOzCY6XxgJuUDJgbEl3oXFDcxBIcWYu/pVA/IuG/lykaoA/u3s3QBbpgZ3L7z5It5bt+4y5VzTBjzPFTVJmGhJ+Xo6EZS68TZFws0PqSSYhBMeIc3617LlrDHMzVYFIspaXUeUHXj93LRxUjrDqVRanjBL0Ey217+625CkWEQlfzbe6/InaZ5Fc5AqYBPIQx8fqhsRBZf8nFhF7ZQHyUTkSKB9F4yq+aHfYw1hj5QYMpmwvna4jMsybFV86tNn7Ogg3uafXGrTrgTxXjBsf+jr > 2021-12-16 16:57:40 WARNING vsmagent: Error parsing SSH key: 127.0.0.1 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMt3xd2SaE3oryCkJzFbxTwjiT7cqQo3Lhpd3++9hDCYh9Jab/qPYLlwSXjJH1Pg5BP3rjw0u5o0g5rPDCOYnCM= > * Used in generating the mcookie for Xvnc Also in vsmagent.log: > 2021-12-17 13:06:26 ERROR vsmagent: Unhandled XMLRPC exception: <class 'ValueError'> [digital envelope r > outines: EVP_DigestInit_ex] disabled for FIPS Traceback (most recent call last): > File "/opt/thinlinc/modules/thinlinc/vsm/asyncbase.py", line 104, in ooOOOoOO0 > obj . handle_read_event ( ) > File "/usr/lib64/python3.6/asyncore.py", line 423, in handle_read_event > self.handle_read() > File "/usr/lib64/python3.6/asynchat.py", line 151, in handle_read > self.found_terminator() > File "/opt/thinlinc/modules/thinlinc/vsm/xmlrpc.py", line 555, in found_terminator > self . handle_request ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/xmlrpc.py", line 569, in handle_request > self . handle_method ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/vsmagentchannel.py", line 120, in handle_method > self . params ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 50, in handle > self . check_old_client ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 62, in check_old_client > self . check_valid_user ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 69, in check_valid_user > self . setup_session_info ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 95, in setup_session_info > self . find_free_display ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 177, in find_free_display > self . find_free_display ( socket_status = SOCKET_NOT_INUSE ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 138, in find_free_display > self . set_sessionkey ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 201, in set_sessionkey > self . set_vncpassword ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 243, in set_vncpassword > self . correct_system_sockets ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 306, in correct_system_sockets > self . check_homedir ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 347, in check_homedir > self . start_session ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/handler_reqsession.py", line 352, in start_session > II1 = oO00 . start ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/sessionstart.py", line 40, in start > self . create_session_env ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/sessionstart.py", line 72, in create_session_env > self . session_env [ "TLSESSIONMCOOKIE" ] = self . mcookie ( ) > File "/opt/thinlinc/modules/thinlinc/vsm/sessionstart.py", line 122, in mcookie > Oo0O0o0oO000 = hashlib . md5 ( ) > ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
(In reply to Pierre Ossman from comment #3) > Two areas discovered where we use MD5: > > * Computing signatures for the SSH host keys. Done by the agent and sent to > the client. Not used by current clients, but unsure if older clients need > this. > They do. Any client older than 4.1.0 will need an exact match of the fingerprint, compared to what the client's ssh produces. This was changed in bug 4557 where we check the entire key instead. So removing MD5 here would effectively drop support for clients older than 4.1.0. (If that is done, when we should probably have some dummy information instead of the fingerprint. It looks like the client will crash and burn if this field is missing, rather than present an error message.)
Is there a workaround for this bug? I was forced to upgrade from Centos 7 to RHEL8 w/ fips enabled. I am trying to use the latest server 4.14 and latest client 4.14. I have no need for support clients older than 4.1.0. Trying to connect on a fresh install yields same backtrack reported above in Comment #4.
No, I'm afraid we are not aware of any workaround for this right now.
I hand modified the thinlinc/modules/thinlinc/vsm/sessionstart.py file and changed hashlib.md5() to hashlib.sha256(). That worked. I did this based on this bug description indicating this hash calculation doesn't even matter except for supporting of very old 4.1 clients. So maybe you just need a flag or configuration option that does something similar to what I did by hand or a way to disable this altogether? Looking on your site, version 4.1 is 9 or 10 years old, right?
https://community.thinlinc.com/t/thinlinc-fails-due-to-fips-mode/392/6
