Bug 8402 - Very old base packages in cenbuild
Summary: Very old base packages in cenbuild
Status: CLOSED FIXED
Alias: None
Product: ThinLinc
Classification: Unclassified
Component: Build system (show other bugs)
Version: trunk
Hardware: PC Unknown
: P2 Normal
Target Milestone: 4.18.0
Assignee: Bugzilla mail exporter
URL:
Keywords: prosaic
Depends on: 8432
Blocks: 8368 8412
  Show dependency treegraph
 
Reported: 2024-07-29 13:25 CEST by Joel Leesment
Modified: 2024-12-06 15:40 CET (History)
4 users (show)

See Also:
Acceptance Criteria:
MUST: * We should have package versions in cenbuild that allows us to use more modern tools. SHOULD: * Packages should be upgraded to the newest version possible. * %file definitions use a consistent format between spec files. * New packages should use the smallest packaging method (e.g. xz instead of gz). EXTRA: * All packages should be upgraded to their latest versions.


Attachments

Description Joel Leesment 2024-07-29 13:25:45 CEST
Cenbuild uses very old base packages, we should upgrade to a recent version.
Comment 6 Alexander Zeijlon cendio 2024-10-08 07:45:43 CEST
When attempting to upgrade to coreutils version 9.5 in cenbuild, the configure-step of the build exits with an error that says:

> configure: error: could not enable timestamps after mid-January 2038.
> This package recommends support for these later
> timestamps. However, to proceed with signed 32-bit
> time_t even though it will fail then, configure with
> '--disable-year2038'.

I tested running the configure script manually in all cenbuild archs and this error is seen in all but the x86_64-arch. I was expecting to be able to successfully run configure in all archs that are used for compiling 64-bit binaries. But apparently this is not the case.
Comment 7 Alexander Zeijlon cendio 2024-10-08 07:57:58 CEST
The error itself is triggered by a check in the configure script that tests whether the time_t type can be made to overflow for a large enough value, and it has been added to make developers aware of if they are compiling on an architecture where this problem can occur in the future.

See also: https://www.gnu.org/software/gnulib/manual/html_node/Avoiding-the-year-2038-problem.html
Comment 8 Alexander Zeijlon cendio 2024-10-08 09:02:36 CEST
The problem that this error warns about seems to be handled, or at least mitigated, in glibc version 2.38 [1][2].

[1] https://lists.gnu.org/archive/html/info-gnu/2023-07/msg00010.html
[2] https://sourceware.org/glibc/wiki/Release/2.38

But since we aren't upgrading glibc to 2.38 as a part of this bug, for now, we will have to use the option '--disable-year2038' when compiling coreutils for cenbuild.
Comment 9 Adam Halim cendio 2024-10-10 16:20:41 CEST
When upgrading pkg-config to 0.29, I get this error when trying to upgrade from our old 0.28-2 to 0.29.1-1:
> Error: Transaction test error:
>  file /opt/cendio-build/arch/win64/usr/share/doc/pkg-config/pkg-config-guide.html from install of cendio-build-pkg-config-native-win64-0.29.1-1.noarch conflicts with file from package cendio-build-pkg-config-win64-0.28-2.noarch
Comment 30 Adam Halim cendio 2024-10-11 17:10:20 CEST
Lots of packages are moving over to meson which we currently have in cenbuild,
albeit an old version. We're stuck with this old version since never version
require Python 3.7, while we have 3.6 (bug 8383, bug 8156). 

Some of the packages we've looked at require a newer version of meson than we
have, so we figure we might as well upgrade Python3 while we're at it. We are
aiming to upgrade to Python 3.9.
Comment 31 Alexander Zeijlon cendio 2024-10-14 10:12:35 CEST
A follow-up on comment 30. Right now, we can't upgrade past Python 3.9, since Pyobfuscate currently does not work with versions above that.
Comment 33 Alexander Zeijlon cendio 2024-10-16 12:51:57 CEST
Work on upgrading python has been moved to bug 8432.
Comment 35 Adam Halim cendio 2024-10-17 14:21:14 CEST
Upgrading pkg-config to 0.29.2 causes build failures when building python3-pillow:
> The headers or library files could not be found for zlib,
> a required dependency when compiling Pillow from source.
There is not a missing dependency in our .spec, and I've double-checked that zlib is installed correctly.

This might be a bug in Pillow/setuptools or something, but I haven't dug deeper into this. The issue still persists with pkg-config 0.29.1 and 0.29. Also tried upgrading to Pillow 8.4 (latest version we can build with Python 3.6).

We're leaving pkg-config for now. It's a very slow-moving project and hasn't had a release in 7 years.
Comment 52 Adam Halim cendio 2024-10-18 16:38:55 CEST
We're looking to prioritize packages that **must** be upgraded to build glibc 2.28. I've looked through the release notes for glibc 2.13-2.28 and tried to take note of some changes that might be interesting to us:

* 2.13 https://sourceware.org/legacy-ml/libc-alpha/2011-01/msg00017.html
    - Nothing
* 2.15 https://lwn.net/Articles/488032/
    - Nothing
* 2.16 https://sourceware.org/legacy-ml/libc-alpha/2012-06/msg00807.html
    - Support for old ARM API removed from ports.
    - ARMHF EABI uses /lib/ld-linux-armhf.so.3 instead of 
                      /lib/ld-linux.so.3
* 2.17 https://sourceware.org/legacy-ml/libc-announce/2012/msg00001.html
    - glibc-ports is shipped in the ports/ directory instead of a separate
      release.
* 2.18 https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
    - Nothing
* 2.19 https://sourceware.org/legacy-ml/libc-alpha/2014-02/msg00224.html
    - Nothing
* 2.20 https://sourceware.org/legacy-ml/libc-alpha/2014-09/msg00088.html
    - Minimum Linux kernel 2.6.32
* 2.21 https://sourceware.org/legacy-ml/libc-alpha/2015-02/msg00119.html
    - GCC 5 with GNU binutils 2.25 (current version) onwards.
* 2.22 https://sourceware.org/legacy-ml/libc-alpha/2015-08/msg00609.html
    - Nothing.
* 2.23 https://sourceware.org/legacy-ml/libc-alpha/2016-02/msg00502.html
    - header <regexp.h> has been removed. Programs should use <regex.h> instead.
* 2.24 https://sourceware.org/legacy-ml/libc-alpha/2016-08/msg00212.html
    - Minimum kernel version 3.2
    - The deprecated __malloc_initialize_hook variable has been removed from
      the API.
* 2.25 https://sourceware.org/legacy-ml/libc-alpha/2017-02/msg00079.html
  - Some C libraries once required _REENTRANT and/or _THREAD_SAFE to be defined
    by all multithreaded code, but glibc has not required this for many years.
      * We have some places where we define these explicitly. These can
        removed.
  - The inclusion of <sys/sysmacros.h> by <sys/types.h> is deprecated. This
    means that in a future release, the macros “major”, “minor”, and “makedev”
    will only be available from <sys/sysmacros.h>.
      * This broke the old util-linux. Version 2.32.1 is patched.
  - Most of glibc can now be built with the stack smashing protector enabled.
    It is recommended to build glibc with --enable-stack-protector=strong.
      - We disable stack smash protection for RPM.
* 2.26 https://sourceware.org/legacy-ml/libc-announce/2017/msg00001.html
  - The obsolete function cfree has been removed.  Applications should use free
    instead.
  - binutils >= 2.25 required to build glibc.
  - GCC 4.9 or later
* 2.27 https://sourceware.org/legacy-ml/libc-announce/2018/msg00000.html
  - Support for statically linked applications which call dlopen is deprecated
    and will be removed in a future version of glibc. Applications which call
    dlopen need to be linked dynamically instead.
  - The nonstandard header files <libio.h> and <_G_config.h> are deprecated and
    will be removed in a future release. Software that is still using either
    header should be updated to use standard <stdio.h> interfaces instead.
  - bison >= 2.7 required to generade code in the 'intl' subdirectory.
* 2.28 https://sourceware.org/legacy-ml/libc-alpha/2018-08/msg00003.html
  - The nonstandard header files <libio.h> and <_G_config.h> are no longer
    installed.  Software that was using either header should be updated to use
    standard <stdio.h> interfaces instead. 
  - The macros 'major', 'minor', and 'makedev' are now only available from the
    header <sys/sysmacros.h>; not from <sys/types.h> or various other headers
    that happen to include <sys/types.h>.  These macros are rarely used, not
    part of POSIX nor XSI, and their names frequently collide with user code;
    see https://sourceware.org/bugzilla/show_bug.cgi?id=19239 for further
    explanation.
  - The obsolete function ustat is no longer available to newly linked
    binaries; the headers <ustat.h> and <sys/ustat.h> have been removed.  This
    function has been deprecated in favor of fstatfs and statfs
  - In this release, if the configure option --disable-crypt is used, glibc
    will not install <crypt.h> or libcrypt, making room for the separate
    project's versions of these files.  The plan is to make this the default
    behavior in a future release.
  - GNU make 4.0 or later is now required to build glibc.

From what I can tell, we currently **need** to upgrade util-linux, kernel-headers, m4 and findutils. m4 and findutils need to be upgraded/patched to handle the <libio.h> and <_G_config.h> removal.

Regarding util-linux and kernel-headers, we're aiming to upgrade these to the same version our build server (RHEL 8) uses, which means:
* util-linux 2.32.1
* kernel-headers 4.18
Comment 54 Adam Halim cendio 2024-10-21 16:18:43 CEST
Had a look through our open bugs and looked at all open issues with the keyword "upstream" to try and find if there are any bugfixes we can get "for free" by upgrading some of our packages.

It looks like:
* bug 8184 can be closed by upgrading svglib >= 1.3.0
* bug 7167 can be closed by upgrading gettext >= 0.20.2
Comment 56 Adam Halim cendio 2024-10-24 10:47:10 CEST
Here are the packages that have been updated so far:

git ls-tree -r --name-only HEAD | grep "\.tar\." | while read filename; do  echo "$(git log -1 --date=short --format="%ad | %h |" -- $filename) $filename" ; done | sort | grep 2024-

> 2024-09-23 | dcf84cb | compression/src/zlib-1.3.1.tar.xz
> 2024-09-23 | dcf84cb | crypto/src/gnutls-3.8.7.1.tar.xz
> 2024-09-23 | dcf84cb | crypto/src/nettle-3.10.tar.gz
> 2024-10-04 | 0a44aba | rpm/src/popt-1.19.tar.gz
> 2024-10-04 | 294952e | compression/src/gzip-1.13.tar.xz
> 2024-10-04 | 3c76bd8 | compression/src/cpio-2.15.tar.bz2
> 2024-10-04 | 569452a | compression/src/bzip2-1.0.8.tar.gz
> 2024-10-04 | 9c8251d | compression/src/libarchive-3.7.6.tar.xz
> 2024-10-04 | c5f8cc2 | base/src/bash-5.2.tar.gz
> 2024-10-07 | 2683100 | base/src/grep-3.11.tar.xz
> 2024-10-07 | 425567b | perl/src/perl-5.40.0.tar.gz
> 2024-10-07 | 425567b | perl/src/perl-cross-1.6.tar.gz
> 2024-10-07 | 94e8c9f | base/src/mpc-1.3.1.tar.gz
> 2024-10-07 | a1e9c77 | base/src/sed-4.9.tar.xz
> 2024-10-07 | dd953cb | base/src/gawk-5.3.1.tar.xz
> 2024-10-07 | fc7b503 | base/src/mpfr-4.2.1.tar.xz
> 2024-10-08 | 2552afe | network/src/rsync-3.3.0.tar.gz
> 2024-10-08 | 46825e9 | base/src/coreutils-9.5.tar.xz
> 2024-10-08 | 547cc66 | build/src/diffutils-3.10.tar.xz
> 2024-10-08 | 65a9665 | astyle/src/astyle-3.4.13.tar.bz2
> 2024-10-08 | 9453a2e | graphics/src/graphviz-12.0.0.tar.xz
> 2024-10-08 | a6d266c | network/src/ldns-1.8.4.tar.gz
> 2024-10-08 | b0f9b48 | build/src/nasm-2.16.03.tar.xz
> 2024-10-08 | deb017a | build/src/patch-2.7.6.tar.xz
> 2024-10-08 | eb4911e | build/src/bison-3.8.2.tar.xz
> 2024-10-08 | fa22883 | network/src/curl-8.10.1.tar.xz
> 2024-10-08 | fed2514 | xml/src/expat-2.6.3.tar.xz
> 2024-10-10 | 38293eb | base/src/readline-8.2.tar.gz
> 2024-10-10 | 60e369b | python/src/Python-3.9.20.tar.xz
> 2024-10-10 | 66a2bba | build/src/meson-1.5.2.tar.gz
> 2024-10-10 | b4ba7b7 | graphics/src/librsvg-2.34.2.tar.xz
> 2024-10-21 | 9ae740a | perl/src/XML-Parser-2.47.tar.gz
Comment 57 Adam Halim cendio 2024-10-24 13:30:35 CEST
Some of the modern tools we want to upgrade to in Cenbuild are GCC 14.2,
glibc 2.28 and QT 6. Following are the requirements for each package:

GCC 14.2
========
* glibc [1]
* python3
* sh/bash
* flex
* gawk
* binutils (>= 2.35 for LTO [2])
* gzip     1.2.4
* bzip2    1.0.2
* make     3.80
* tar      1.14
* GMP      4.3.2
* MPFR     3.1.0
* MPC      1.0.1
* gettext  0.22
* texinfo  4.7

[1] Unknown which version of glibc works with GCC 14.2.
[2] LTO is optional.

glibc 2.28
==========
* binutils 2.25
* gcc      4.9
* make     4.0
* texinfo  4.7
* gawk     3.1.2
* bison    2.7
* sed      3.02

QT 6
====
* CMake     3.21
* ninja
* c++17    (GCC 8 for full support?)
* Python >= 2.7
* libclang
* libgl
* libegl
* libfontconfig1
* libinput

On Linux, QT needs
* libxcb              1.11
* libxcb-render-util  0.3.9
* libxcb-icccm	      0.3.9
* libxcb-keysyms      0.3.9
* libxcb-image        0.3.9
* libxcb-util	      0.3.9
* libxcb-cursor0      0.1.1
* libxkbcommon	      0.50
* libxkbcommon-x11    0.50
* libfontconfig	      2.6
* libfreetype	      2.3.0
* libXext	      6.4.3
* libX11	      6.2.1
* libX11-xcb	      1.3.2
* libSM  [3]	      6.0.4
* libICE [3]	      6.3.5
* libglib-2.0	      2.8.3
* libpthread	      2.3.5

Out of the packages we already have in cenbuild, the following packages MUST be
upgraded before we can start work on upgrading the other tools:
* gettext 0.19.3 -> >= 0.22 (0.22.5 is latest release). Needed for GCC 14.2.
* make    3.82   -> >= 4.0  (4.4.1 is latest release). Needed for QT 6.

[3] Unsure about the version numbers here. We have the latest version of libSM and libICE (1.2.4 & 1.1.1 respectively)
Comment 58 Adam Halim cendio 2024-10-24 13:33:07 CEST
(In reply to Adam Halim from comment #57)

Forgot to mention that glibc 2.28 also requires kernel-headers >= 3.2. This upgrade will be done together with glibc, as they are closely tied together.
Comment 59 Adam Halim cendio 2024-10-24 13:38:35 CEST
Upgrading gettext from 0.19.3 to 0.22.5 has caused some issues.

First, building TRE with an upgraded gettext fails with the following error:
> error: File not found: /home/cendio/cenbuild/repo/rpmbuild/BUILDROOT/cendio-build-tre-win64-0.7.6-5.i386/opt/cendio-build/arch/win64/usr/x86_64-w64-mingw32/sys-root/mingw/bin/libtre-*.dll
Not sure why this happens, but running autoreconf -fiv before building TRE solves this issue.

Another issue that occurs is that the windows client fails to build with a new gettext:
> /usr/i686-pc-mingw32/sys-root/mingw/lib/libintl.a(relocatable.o): In function `DllMain@12':
> /local/home/adaha/devel/cenbuild/repo/rpmbuild/BUILD/gettext-0.22.5/gettext-runtime/intl/gnulib-lib/relocatable.c:333: multiple definition of `DllMain@12'
> pulsecore/.libs/libpulsecommon_6.0_la-dllmain.o:pulsecore/dllmain.c:34: first defined here
> collect2: error: ld returned 1 exit status
Comment 63 Adam Halim cendio 2024-10-29 17:45:38 CET
Follow-up on comment #59:

The windows-client build failures were fixed with two patches from upstream [1, 2]. 

With the patches in place, the osx64 build failed as well with the following error:
> libtool:   error: '/usr/x86_64-apple-darwin10/sys-root/usr/lib/libintl.la' is not a valid libtool archive
It looks like libintl.la stopped being installed some time ago [3], and I found a patch upstream with a fix [4].

With patch [4] in place, the macOS client built successfully, and I quickly tested the behaviour described in bug 7167, and the macOS client now seems to work as expected.

Since patch [4] doesn't build relocatable.c any more, patch [1] and [2] might not be needed as it was due to relocatable.o the windows builds failed. I will test this sometime soon.

[1] https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commit;h=8752367c1cbd1a5c231e423dbd1c186ea1c67841
[2] https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commit;h=8e914861708ed459a2ae8157eb60d89375ab46cd
[3] https://savannah.gnu.org/bugs/?64536
[4] https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commit;h=b182c17096847cb581c3e2c3f42c145c47c7c7a6
Comment 66 Adam Halim cendio 2024-10-30 15:45:44 CET
(In reply to Adam Halim from comment #63)
> Since patch [4] doesn't build relocatable.c any more, patch [1] and [2]
> might not be needed as it was due to relocatable.o the windows builds
> failed. I will test this sometime soon.
Indeed, patch [1] and [2] could be removed. I could still build gettext for all architectures, and the client for all architectures.
Comment 68 Adam Halim cendio 2024-10-31 15:44:22 CET
gettext has now been updated to 0.22.5 as of [1].

As mentioned in comment #66, only one patch [2] was required to fix the issues
seen on macOS. An issue I stumbled upon was that the patch modified cofigure.ac
and Makefile.in, which required running autoreconf before building.

Unfortunately, adding autotools/autoreconf to gettext as a dependency resulted in a
circular depdency between
cendio-build-build -> cendio-build-gettext -> ... -> cendio-build-build.
To get around this, I applied patch [2] manually and ran autoreconf by hand,
and could from there create a new patch with all diffs (more details in [1]).

As a bonus, it looks like bug 7167 got fixed, and bug 7871 might be affected as
well.

[1] https://git.cendio.se/thinlinc/cenbuild/-/merge_requests/24
[2] https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commit;h=b182c17096847cb581c3e2c3f42c145c47c7c7a6
Comment 70 Adam Halim cendio 2024-11-04 16:10:07 CET
The upgrade of util-linux has caused some issues.

It seems like (at least) libmount is linked to the system's glibc instead of the one in cenbuild:
> nm -gD /opt/cendio-build/arch/x86_64/usr/lib64/libmount.so  | grep clock
>                  U clock_gettime@GLIBC_2.17
I haven't been able to determine the cause of this, but I found this when trying to build glib2, which gave this error:
> FAILED: gio/glib-compile-resources 
> cc  -o gio/glib-compile-resources gio/glib-compile-resources.p/gvdb_gvdb-builder.c.o gio/glib-comp...>
> //usr/lib64/libmount.so.1: undefined reference to `memcpy@GLIBC_2.14'
> //usr/lib64/libmount.so.1: undefined reference to `clock_gettime@GLIBC_2.17
Comment 71 Adam Halim cendio 2024-11-04 16:57:29 CET
(In reply to Adam Halim from comment #70)
Looks like this issue persists in newer versions as well tested (2.37.4, 2.38.1, 2.39.4), but I haven't been able to test the latest 2.40.2.

I tried installing the version of util-linux before the upgrade (2.19.1), and could see that this bug is probably not a new one:
> nm -gD /opt/cendio-build/arch/x86_64/usr/lib64/libmount.so | grep memcpy
>                  U memcpy@GLIBC_2.14
Comment 72 Adam Halim cendio 2024-11-05 08:59:32 CET
libattr was removed along with the libcap upgrade, as libcap was the only package that (seemingly) required libattr. It seems like there are a few more packages that still link to libattr without having it as a hard dependency. The ones I've run in to so far are rsync and patch. Simply re-building the packages solves this.

It's probably easiest to include libattr again in the base package, as several packages have it as an optional dependency.
Comment 73 Adam Halim cendio 2024-11-05 16:51:01 CET
(In reply to Adam Halim from comment #71)
This looks like it might be a bug in libtool [1] which is shipped with util-linux.

The binaries that come from just building (running make) are linked correctly, and the issue seems to happen somewhere in the install step when libtool is relinking. There is a "-L/usr/lib64" in the relinking step, which probably shouldn't be there.

[1] http://savannah.gnu.org/support/?107416
Comment 74 Adam Halim cendio 2024-11-05 17:25:06 CET
(In reply to Adam Halim from comment #73)
I've created bug 8441 for this issue.
Comment 77 Adam Halim cendio 2024-11-07 16:01:29 CET
make has now been upgraded to 4.3, which means cenbuild should now satisfy all requirements that are needed to begin upgrading glibc/qt/gcc.
Comment 78 Adam Halim cendio 2024-11-07 16:03:14 CET
Unfortunately, make was not upgraded to the latest version (4.4.1).

Upgrading to 4.4.1 showed a bug where building the package cendio-build-glibc-stage1-armhf would cause make to get stuck in a recursive loop. I have made a report upstream [1].

[1] https://savannah.gnu.org/bugs/index.php?66424
Comment 79 Adam Halim cendio 2024-11-08 16:56:51 CET
(In reply to Adam Halim from comment #35)
> We're leaving pkg-config for now. It's a very slow-moving project and hasn't
> had a release in 7 years.
Just kidding, we're upgraing pkg-config to the latest version.

It was indeed an issue with python3-pillow which we could work around. I've created a bug upstream [1].

[1] https://github.com/python-pillow/Pillow/issues/8542
Comment 80 Adam Halim cendio 2024-11-11 17:43:11 CET
As part of this bug, the following packages have been updated so far:

* coreutils   9.0    -> 9.5
* findutils   4.4.0  -> 4.10.0
* m4          1.4.16 -> 1.4.19
* util-linux  2.19.1 -> 2.32.1

* gawk 3.1.6  -> 5.3.1
* grep 2.5.6  -> 3.11
* mpc  1.0.3  -> 1.3.1
* mpfr 3.1.2  -> 4.2.1
* sed  4.5.0  -> 4.9

* bzip2      1.0.6  -> 1.0.8
* cpio       2.10   -> 2.15
* gzip       1.3.13 -> 1.13
* libarchive 3.5.2  -> 3.7.6

* diffutils 2.8.1 -> 3.10
* patch     2.5.4 -> 2.7.6

* gettext 0.19.3  -> 0.22.5
* make    3.82    -> 4.3

* graphviz 2.24.0 -> 12.0.0
* librsvg  2.26.0 -> 2.34.2

* libcap 2.24 -> 2.70

* python 3.6.12 -> 3.9.20
* meson  0.61.5 -> 1.5.2

* astyle 2.02.1  -> 3.4.13
* bison  2.4     -> 3.8.2
* nasm   2.14.02 -> 2.16.03
* expat  2.2.26  -> 2.6.3

* curl  7.21.2  -> 8.10.1
* rsync 3.0.6   -> 3.3.0
* ldns  1.7.1   -> 1.8.4

* bash     4.3.30 -> 5.2
* readline 8.1    -> 8.2
* perl     5.34.0 -> 5.40.0
* popt     1.16   -> 1.19

 
The following packages have not yet been merged, but are planned to be synced
as part of this bug:
* automake   1.16.5 -> 1.17
* libtool    2.4.6  -> 2.5.3
* pkg-config 0.28   -> 0.29.2
* tre        0.7.6  -> 0.9.0
Comment 86 Adam Halim cendio 2024-11-19 17:33:30 CET
The final packages have finally been merged and synced with our repo, which concludes the work for this bug. In total, all packages listed in comment #80 were upgraded.
> MUST:
> * We should have package versions in cenbuild that allows us to use more modern
>  tools.
With the upgrades done, we should be able to build modern versions of glibc, GCC/minGW, and QT6.
> SHOULD:
> * Packages should be upgraded to the newest version possible.
Almost all packages have been upgraded to their latest versions. The ones that have been excluded are ones that require newer glibc/GCC (util-linux, librsvg, astyle, graphviz). Python was also not upgraded to the latest version due to bug 8227. The package "make" was also not upgraded to the latest version (more info in comment #78 and https://git.cendio.se/thinlinc/cenbuild/-/merge_requests/28)
> * %file definitions use a consistent format between spec files.
This has been done continuously on a best effort basis. We have tried to be more explicit in library/binary file ownership, and less explicit with man/infopages.
> * New packages should use the smallest packaging method (e.g. xz instead of gz).
I looked through all upgraded packages and confirmed that all used the smallest package method from official upstream sources. Note that some releases use the .tar.lz format, which we don't have support for.
> EXTRA:
> * All packages should be upgraded to their latest versions.
All packages that were part of the original plan (Joel's work) were not upgraded, due to time constraints. We focused on the packages we deemed most important, especially build tools and requirements for glibc/GCC/QT6.
Comment 87 Adam Halim cendio 2024-11-20 07:53:00 CET
In total, there were 60 packages that were planned to be upgraded. Out of these 60 packages, 34 were upgraded. Three additional packages were upgraded as well (ldns, meson, and python).

Here is a list of the packages that were not upgraded:
* elfutils
* libjson
* pango
* fontconfig
* nss
* nspr
* speex
* libsndfile
* gtk2
* glib2
* atk
* cairo
* libffi
* libpng
* libiconv
* texinfo
* pam
* krb5
* libogg
* alsa-lib
* file
* libjpeg
* tiff
* imagemagick
* autoconf
* ncurses
Comment 88 Madeleine cendio 2024-11-22 17:22:04 CET
The testing for this bug was primarily aiming for analyzing the output from Jenkins builds, as the commits had already gone through continuous code review in Git. The aim was to examine metrics like the number of build errors and the time it takes to complete a build, using tools like "Build Time Trend" for insights.

However, older builds in Jenkins are automatically deleted, which resulted in that many logs that could have been helpful for investigating this bug were no longer available. It might be relevant to consider changing this setting, to be able to analyzing builds after working with larger bugs like this.

Another issue that was encountered during the testing was that the console output from Jenkins builds isn’t deterministic. The order in which packages are installed or removed changes between builds, which makes it difficult to compare logs and track what might have changed. Additionally, the Jenkins logs only show which packages are installed or removed and don’t include all the errors or warnings that might appear if the build was done on a virtual machine.

We noted a significant increase in build time for Cenbuild after upgrading `make` to version 4.3. Initially, it seemed like the upgrade was causing the issue, but it turned out to be unrelated. The real problem was a Jenkins misconfiguration that happened when Cenbuild was moved to Git. Before, Jenkins only rebuilt the packages affected by changes, so builds were much faster. Now, it rebuilds the entire Cenbuild from scratch every time something is pushed, even for the smallest change, which takes around 12 hours.
Comment 89 Frida Flodin cendio 2024-11-25 13:28:47 CET
I looked over some of the base packages that have been upgraded. This bug is hard to test thoroughly since so much has changed and a lot of base base packages have been upgraded. Regressions should have surfaced with the continuous testing by the team that has been using this updated cenbuild for over a month.

I looked through the packages and looked for upgrades that could potentially have effects that have not been tested yet.

> * bzip2      1.0.6  -> 1.0.8
I think the could effect the fonts in the client. They look good!
> * graphviz 2.24.0 -> 12.0.0
> * librsvg  2.26.0 -> 2.34.2
The TAG could be effected. I looked through most of the TAG and everything looks good.


I feel confident that no large regressions or bugs have been introduced by these changes. Closing
Comment 90 Pierre Ossman cendio 2024-12-04 16:40:58 CET
ImageMagick is broken:

> $ cbrun i386  magick
> magick: error while loading shared libraries: libgvc.so.5: cannot open shared object file: No such file or directory

Might be prudent to do a general check for things that need to be rebuilt because of upgrades.
Comment 91 Alexander Zeijlon cendio 2024-12-05 15:40:11 CET
I checked all binaries in all arches with ldd, looking for libraries that aren't found. There are two packages we have to look at

cendio-build-ImageMagick-i386:
==============================
i386/usr/bin/animate
i386/usr/bin/compare
i386/usr/bin/composite
i386/usr/bin/conjure
i386/usr/bin/convert
i386/usr/bin/display
i386/usr/bin/identify
i386/usr/bin/import
i386/usr/bin/magick
i386/usr/bin/magick-script
i386/usr/bin/mogrify
i386/usr/bin/montage
i386/usr/bin/stream

cendio-build-util-linux-x86_64:
===============================
x86_64/usr/bin/eject
x86_64/usr/bin/lslocks
x86_64/usr/bin/lsns
Comment 92 Alexander Zeijlon cendio 2024-12-05 15:45:26 CET
For ImageMagick, all the installed binaries listed in comment 91 all list some combination of the libs below as not found:

libcdt.so.4
libgraph.so.4
libgvc.so.5

Turns out that all these are part of the previous version of cendio-build-graphviz-i386, which was upgraded as a part of this bug.
Comment 93 Alexander Zeijlon cendio 2024-12-05 15:51:06 CET
Although it is apparent that ImageMagick can depend on Graphviz if the latter is installed during build time, Graphviz is not declared as a requirement for cendio-build-ImageMagick-i386.

I tested to build ImageMagick both with and without the Graphviz package installed, and both builds go through successfully.

It therefore looks like we have two options moving forward. Either add a requirement on Graphviz for ImageMagick, or make sure to build ImageMagick without Graphviz installed.
Comment 94 Alexander Zeijlon cendio 2024-12-05 15:58:50 CET
I tested running our make targets that run some ImageMagick-related commands, with a package that does not depend on Graphviz. And everything seems to build just fine.

Let's therefore, for now, not depend on packages we don't need to depend on.
Comment 95 Alexander Zeijlon cendio 2024-12-05 15:59:00 CET
Note that there is bug 7977, but the issue there is present regardless of if the older (compatible) packages or the new ImageMagick without dependency is used.
Comment 96 Adam Halim cendio 2024-12-06 10:00:18 CET
(In reply to Alexander Zeijlon from comment #93)
> It therefore looks like we have two options moving forward. Either add a
> requirement on Graphviz for ImageMagick, or make sure to build ImageMagick
> without Graphviz installed.
An alternative would be to see if we can build ImageMagick with graphviz disabled, regardless if it is installed or not.

But since we already have graphviz as a dependency in Cenbuild, I think it's easier to add it as a dependency to ImageMagick as well. This is also how Fedora builds ImageMagick.
Comment 98 Adam Halim cendio 2024-12-06 10:49:39 CET
I decided to add graphviz as an explicit dependency to ImageMagick and re-built ImageMagick.

Tested running all binaries in ImageMagick without issue.

Also had a brief look through the TAG and client and did not see any issues.
Comment 99 Pierre Ossman cendio 2024-12-06 15:40:40 CET
ImageMagick now works fine. Tested by generating the metainfo images and inspecting the result.

Could not find any other binaries in /usr/bin or /usr/lib* that had a missing library in the same way.

Note You need to log in before you can comment on or make changes to this bug.