The cenbuild service is supposed to unmount up everything that is mounted in each chroot environment. It does this by going through /proc/mounts and simply doing umount on each entry found that starts with the chroot's path. The problem is if there are mounts within mounts in the chroot. Since we examine /proc/mounts from start to end, we'll get the mounts in the order they were created. Parent mounts will then be before child mounts, and we can't unmount the parent before the child. The end result is that we get things left mounted and not cleaned up. We also see the problem in the logs: Mar 31 09:06:07 be34ca00db9c cenbuild[900675]: umount: /opt/cendio-build/arch/x86_64/opt/docker-cenbuild: target is busy. The issue is seen constantly in our container helper because of the way we map files in to it.
For full robustness, we should probably do multiple passes over /proc/mounts. But simply reversing the order should hopefully be good enough in practice.
Fixed and works well now.