Currently we cheat a bit for our build system in that we include the sysroot path in the prefix. E.g. the bindir for an arm package is /usr/arm-linux-gnueabi/sys-root/usr/bin. The proper way is to keep bindir at the normal /usr/bin and have all the tools compensate for the sysroot offset. The main problem with our current approach is that sometimes those incorrect paths leak into the binaries. The problem with doing it properly is that some build environments aren't compatible with a sysroot cross compile and may need fixes.
This also breaks packages that do special things when prefix is /usr, such as glibc.
In theory, this could lead to things like fontconfig not finding it's configuration files in /etc, making it impossible for the system administrator to configure things like font overrides, cache directories, and custom font locations. I have not verified that this is the case.
We should also adjust our CMake toolchain file when fixing this and set CMAKE_SYSROOT.