Some of our test cases break when run locally with Python >= 3.11 [1], specifically tests for our scripts. [1] To our knowlege, did not test further back. The root cause for this is that we are importing scripts with "-", and imports in python may not have "-" in their names. We believe that the unittest-module in older Python versions (e.g. 3.6) let this pass.
We need to replace "-" with e.g. "_", but not everywhere. Lines like these need to remain consistent with the file hyphenated filename we are importing and testing ("tl-script-name" as an example): > fn = os.path.join(testdir, "..", "scripts", "tl-script-name") > loader = SourceFileLoader("tl-script-name", fn) > spec = importlib.util.spec_from_loader("tl-script-name", loader) > script = importlib.util.module_from_spec(spec) > spec.loader.exec_module(script) But all other instances of "tl-script-name" need to be replaced with "tl_script_name": > sys.modules["tl_mount_localdrives"] = script > [...] > with patch("tl_script_name.SomeClass", MagicMock()) as some_mock: > script.main()
There are also some instances of > @patch("tl-script-name.open", ... ) where we could simply change the call to > @patch("builtins.open", ... ) This would make it clearer that we are in fact mocking a built-in function.
It looks like modules defined in sys.modules may not have "-" in their names, so I grepped after the files that match the pattern "sys.modules\[.*-.*\]", and it seems like these are the files that do the invalid import operation: test_notify.py test_thinlinc_login.py test_tl_collect_licensestats.py test_tl_desktop_builder.py test_tl_env.py test_tl_ldap_certalias.py test_tl_limit_printers.py test_tl-mount-localdrives.py test_tl_rsync_all.py test_tl_run_profile.py test_tl_select_profile.py test_tl_serial_redir.py test_tl_session_param.py test_tl_set_sso_helper.py test_tlsetup_main.py test_tl_show_licenses.py test_tl_ssh_all.py test_tl_ssh_askpass.py test_tl_sso_password.py test_tl_sso_token_passphrase.py test_tl_sso_update_password.py test_tl_startup_bg.py test_tl_support.py test_tl-umount-localdrives.py test_tl_wait_smartcard.py test_tl_xstartup_desc.py
The test_tl_startup_bg.py is still broken when run outside cenbuild for reasons not related to hyphenated module names. > ERROR: test_valid_pixel_data_size (__main__.TlStartupBgTest.test_valid_pixel_data_size) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/local/home/aleze/git/ctc-git/tlmisc/startupbg/tests/./test_tl_startup_bg.py", line 74, in setUp > self.Context = context_patcher.start() > ^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1603, in start > result = self.__enter__() > ^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1540, in __enter__ > raise InvalidSpecError( > unittest.mock.InvalidSpecError: Cannot autospec attr 'Context' as the patch target has already been mocked out. [target=<MagicMock name='cairo' id='139752818861184'>, attr=<MagicMock name='cairo.Context' id='139752803762544'>] I tried removing autospec=True from all such patch()-calls in the file, but this then produced another error: > ERROR: test_gdk_missing (__main__.FallbackTests.test_gdk_missing) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/usr/lib64/python3.12/unittest/mock.py", line 1840, in _inner > return f(*args, **kw) > ^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1384, in patched > with self.decoration_helper(patched, > File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ > return next(self.gen) > ^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1366, in decoration_helper > arg = exit_stack.enter_context(patching) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/contextlib.py", line 526, in enter_context > result = _enter(cm) > ^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1439, in __enter__ > self.target = self.getter() > ^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/pkgutil.py", line 513, in resolve_name > mod = importlib.import_module(modname) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module > return _bootstrap._gcd_import(name[level:], package, level) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "<frozen importlib._bootstrap>", line 1387, in _gcd_import > File "<frozen importlib._bootstrap>", line 1360, in _find_and_load > File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked > File "<frozen importlib._bootstrap>", line 935, in _load_unlocked > File "<frozen importlib._bootstrap_external>", line 995, in exec_module > File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed > File "/usr/lib64/python3.12/site-packages/gi/__init__.py", line 40, in <module> > from . import _gi > File "/usr/lib64/python3.12/unittest/mock.py", line 1134, in __call__ > return self._mock_call(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1138, in _mock_call > return self._execute_mock_call(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1208, in _execute_mock_call > return self._mock_wraps(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/local/home/aleze/git/ctc-git/tlmisc/startupbg/tests/test_tl_startup_bg.py", line 800, in _import > return __orig_import(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1134, in __call__ > return self._mock_call(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1138, in _mock_call > return self._execute_mock_call(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/usr/lib64/python3.12/unittest/mock.py", line 1208, in _execute_mock_call > return self._mock_wraps(*args, **kwargs) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > File "/local/home/aleze/git/ctc-git/tlmisc/startupbg/tests/test_tl_startup_bg.py", line 795, in _import > ret = getattr(ret, sub) > ^^^^^^^^^^^^^^^^^ > AttributeError: partially initialized module 'gi' has no attribute '_error' (most likely due to a circular import) The test case still passes when run in cenbuild. This may be out of scope for this bug depending on how much it takes to repair the test.
The test in question, test_gdk_missing, was mocking gi-internals in a way that broke if gi was actually installed on the system (gi is not present in cenbuild). This is easily fixed, and as a side effect of this, the test now looks more consistent with other tests in the same unit test class.
> MUST: > * Unittests should run, without errors, on both newer (3.12) and > older (3.6) version of Python. All tests listed in comment 3 now work with old and modern versions of Python 3.
Looks good. Ran all test files in comment 3 both in `cbrun x86_64 python` and in Python 3.12.2 from Fedora 39. Note that the detailed discussed in comment 2 was not committed as part of this bug.
