[Guide] HowTo: Multilib LFS

emmett1 · 03-13-2019, 03:57 AM

I got some minor failed test on glibc 64bit and 32bit. So i guess its good enough for multilib. So far i got no problem running 32bit binaries.

here the test result for glibc 64 & 32

64:

Quote:

XPASS: conform/UNIX98/ndbm.h/linknamespace
XPASS: conform/XOPEN2K/ndbm.h/linknamespace
XPASS: conform/XOPEN2K8/ndbm.h/linknamespace
XPASS: conform/XPG42/ndbm.h/linknamespace
UNSUPPORTED: elf/tst-audit10
UNSUPPORTED: elf/tst-avx512
XPASS: elf/tst-protected1a
XPASS: elf/tst-protected1b
FAIL: elf/tst-ptrguard1
FAIL: elf/tst-stackguard1
UNSUPPORTED: math/test-double-libmvec-alias-avx2
UNSUPPORTED: math/test-double-libmvec-alias-avx2-main
UNSUPPORTED: math/test-double-libmvec-alias-avx512
UNSUPPORTED: math/test-double-libmvec-alias-avx512-main
UNSUPPORTED: math/test-double-libmvec-sincos-avx2
UNSUPPORTED: math/test-double-libmvec-sincos-avx512
UNSUPPORTED: math/test-float-libmvec-alias-avx2
UNSUPPORTED: math/test-float-libmvec-alias-avx2-main
UNSUPPORTED: math/test-float-libmvec-alias-avx512
UNSUPPORTED: math/test-float-libmvec-alias-avx512-main
UNSUPPORTED: math/test-float-libmvec-sincosf-avx2
UNSUPPORTED: math/test-float-libmvec-sincosf-avx512
UNSUPPORTED: misc/tst-pkey
UNSUPPORTED: nptl/test-cond-printers
UNSUPPORTED: nptl/test-condattr-printers
UNSUPPORTED: nptl/test-mutex-printers
UNSUPPORTED: nptl/test-mutexattr-printers
UNSUPPORTED: nptl/test-rwlock-printers
UNSUPPORTED: nptl/test-rwlockattr-printers
FAIL: nptl/tst-cleanup0-cmp
FAIL: nptl/tst-stackguard1
FAIL: resolv/tst-resolv-ai_idn
FAIL: resolv/tst-resolv-ai_idn-latin1
FAIL: stdio-common/tst-setvbuf1-cmp
FAIL: stdio-common/tst-unbputc
FAIL: stdlib/tst-fmtmsg
Summary of test results:
9 FAIL
5953 PASS
21 UNSUPPORTED
13 XFAIL
6 XPASS

32:

Quote:

XPASS: conform/UNIX98/ndbm.h/linknamespace
XPASS: conform/XOPEN2K/ndbm.h/linknamespace
XPASS: conform/XOPEN2K8/ndbm.h/linknamespace
XPASS: conform/XPG42/ndbm.h/linknamespace
XPASS: elf/tst-protected1a
XPASS: elf/tst-protected1b
UNSUPPORTED: misc/tst-pkey
UNSUPPORTED: nptl/test-cond-printers
UNSUPPORTED: nptl/test-condattr-printers
UNSUPPORTED: nptl/test-mutex-printers
UNSUPPORTED: nptl/test-mutexattr-printers
UNSUPPORTED: nptl/test-rwlock-printers
UNSUPPORTED: nptl/test-rwlockattr-printers
UNSUPPORTED: resolv/tst-resolv-ai_idn
UNSUPPORTED: resolv/tst-resolv-ai_idn-latin1
UNSUPPORTED: time/tst-y2039
Summary of test results:
5882 PASS
10 UNSUPPORTED
13 XFAIL
6 XPASS

And here sanity check for 64 and 32 bit (6.21 part in the book).

64:

Quote:

+ echo 'int main(){}'
+ cc dummy.c -v -Wl,--verbose
+ readelf -l a.out
+ grep ': /lib'
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
+ grep -o '/usr/lib.*/crt[1in].*succeeded' dummy.log
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib/crt1.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib/crti.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib/crtn.o succeeded
+ grep 'SEARCH.*/usr/lib' dummy.log
+ sed 's|; |\n|g'
SEARCH_DIR("/usr/x86_64-pc-linux-gnu/lib64")
SEARCH_DIR("/usr/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib32")
SEARCH_DIR("/usr/x86_64-pc-linux-gnu/lib");
+ grep '/lib.*/libc.so.6 ' dummy.log
attempt to open /lib/libc.so.6 succeeded
+ grep found dummy.log
found ld-linux-x86-64.so.2 at /lib/ld-linux-x86-64.so.2

32:

Quote:

+ echo 'int main(){}'
+ cc -m32 dummy.c -v -Wl,--verbose
+ readelf -l a.out
+ grep ': /lib'
[Requesting program interpreter: /lib/ld-linux.so.2]
+ grep -o '/usr/lib.*/crt[1in].*succeeded' dummy.log
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib32/crt1.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib32/crti.o succeeded
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../../lib32/crtn.o succeeded
+ grep 'SEARCH.*/usr/lib' dummy.log
+ sed 's|; |\n|g'
SEARCH_DIR("/usr/i386-pc-linux-gnu/lib32")
SEARCH_DIR("/usr/x86_64-pc-linux-gnu/lib32")
SEARCH_DIR("/usr/lib")
SEARCH_DIR("/lib")
SEARCH_DIR("/usr/lib32")
SEARCH_DIR("/usr/i386-pc-linux-gnu/lib");
+ grep '/lib.*/libc.so.6 ' dummy.log
attempt to open /usr/lib32/libc.so.6 succeeded
+ grep found dummy.log
found ld-linux.so.2 at /usr/lib32/ld-linux.so.2

emmett1 · 03-13-2019, 04:07 AM

In LFS they did not use /usr/lib64 and /lib64 only for linkers thats why its not in sanity test result. And actually i just noticed the difference from the book today. I dont know if the book got it wrong, because the book use /lib and /usr/lib for 64bit libraries. I haven't build pure64 LFS for so long, maybe i will try build pure64 follow the book and fix the instructions here.

hazel · 03-13-2019, 06:50 AM

They changed the filesystem, I think at LFS 8.0. Before that, /lib64 was a link to /lib and /usr/lib64 linked to /usr/lib. I suspect that you were using an earlier book.

I'm surprised that you didn't get that swathe of glibc-32 errors that I got. Maybe that's because I was modifying an existing setup rather than doing it from scratch. I won't know until I build my next LFS

.

sr_ls_boy · 03-13-2019, 08:25 AM

This is different than my experience.
I place 64-bit libs in /lib64 & /usr/lib64, 32-bit went in /lib & /usr/lib.

emmett1 · 03-14-2019, 11:04 PM

Quote:

Originally Posted by hazel

They changed the filesystem, I think at LFS 8.0. Before that, /lib64 was a link to /lib and /usr/lib64 linked to /usr/lib. I suspect that you were using an earlier book.

Yeah i know theres was symlink /lib64 to /lib before, i started LFS around 7.9

Quote:

Originally Posted by hazel

I'm surprised that you didn't get that swathe of glibc-32 errors that I got. Maybe that's because I was modifying an existing setup rather than doing it from scratch. I won't know until I build my next LFS

.

Yeah maybe. I've experimenting with pure64 LFS and i found the book was right about sanity test result. And i got very hard time enabling multilib from pure64, its just work before, maybe changes on recent toolchain version failing it. Last thing i can think of to enable multilib on pure64 is by build from scratch using my multilib method until gcc on chapter 6 then copy over all files into current running system replacing everything (be careful, i'm not test this method yet). Or build over everything from scratch

Note: i've updated the instruction, add sanity test result for 64 and 32 bit, and stripping 32bit libraries (/tools/lib32/*)

emmett1 · 03-14-2019, 11:14 PM

Quote:

Originally Posted by sr_ls_boy

This is different than my experience.
I place 64-bit libs in /lib64 & /usr/lib64, 32-bit went in /lib & /usr/lib.

Its CLFS method, i've successfully build multilib system using CLFS method with working full DE and 32bit binary before but i dont like how it put the libraries. CLFS use default libraries hierarchy, 64bit in /lib64 and /usr/lib64, and 32bit in /lib and /usr/lib. Then I need add "PKG_CONFIG_PATH=/usr/lib64/pkgconfig" and "--libdir=/usr/lib64" to every 64bit build. I found this is not relevan since its 64bit system and just need some 32bit libraries. gcc and glibc dev should change this already in its code.

hazel · 03-22-2019, 12:46 PM

Triumph! I installed cups-filters, copied over the Brother driver files from Slackware and I can now print in LFS for the first time. The sheaf of check errors in glibc-32 don't seem to stop it working in practice.

Thanks, Emmett1! I shall build my next LFS as multilib from the start, using your method.

sr_ls_boy · 05-22-2019, 10:07 AM

Meson is new hassle. I can't get it to cross compile 'glib'.
I need to compile the package as 32-bit. The online tutorials
treat me like developer.

voncloft · 05-24-2019, 09:06 PM

Thanks for this guide!

I got steam to work.

I am one step closer to stepping away from main stream distros.

Luridis · 05-26-2019, 12:18 AM

IIRC the reason the devs moved from using lib64 to lib was because of the Linux Standard Base. Standards committees trash things like existing structures that are working well all the time for what they assure us are very good reasons. However, such reasons usually make little to no actual sense to anyone outside of said comity.

As for multilib, it needed for very little on a user compiled system. That said, I know there is some commercial software (Steam) that needs 32 bit natives. I'd like to see a setup with only enough lib32 to support the few things that actually need them, rather than building 2 versions of every single library on the system. LFS is busy enough without having to build things like libgpm32.so and libgpm64.so when they former will never be used by anything.

hazel · 05-26-2019, 06:41 AM

+1 on that. I made my LFS multilib so that I could use my printer in it. It's a Brother and uses 32-bit drivers. But the only 32-bit library it needs is glibc. I suspect most proprietary drivers are like that.

Incidentally I used to use Crux, and the way they do it is to include 32-bit glibc and libstdc++ in the core repository and put all the other 32-bit libraries in a separate repo so that people can just install those they need.

emmett1 · 05-29-2019, 09:48 PM

Quote:

Originally Posted by voncloft

Thanks for this guide!

I got steam to work.

I am one step closer to stepping away from main stream distros.

Yep, steam, wine and other 32bit program running fine for me. And I already ditch Arch completely from my machine. I can play game on steam on my LFS based distro now.
Check out here for my distro: https://venomlinux.org

hazel · 08-21-2019, 11:12 AM

I've been working through this for LFS 9.0-rc1. So far I've found one necessary change in Chapter 6 32-bit glibc. This version requires a symlink to be made to the just-compiled linker-loader before the tests are run. The book places this link in /lib.

Code:

case $(uname -m) in
  i?86)   ln -sfnv $PWD/elf/ld-linux.so.2        /lib ;;
  x86_64) ln -sfnv $PWD/elf/ld-linux-x86-64.so.2 /lib ;;
esac

For a 32-bit build, it needs to go in /usr/lib32 which is the directory specified by the libc_cv_slibdir config variable. If it is not there, the tests won't run.

This requires a consequent change in the install directions:
ln -sv ../usr/lib32/ld-linux.so.2 /lib/ld-linux.so.2
becomes
ln -sfv ../usr/lib32/ld-linux.so.2 /lib/ld-linux.so.2
CORRECTION: That change is not necessary after all. ln -sv is good enough for both links.

I got a huge number of failures in these tests, though if you look closely, you will see that many of them fall into a single family. When you take out all the nptl failures, the number drops from 144 to 25.
CORRECTION: It drops to 3 if both the build and the test are carried out with -j1.

hazel · 08-22-2019, 09:03 AM

Another important point. The book has a detailed sanity check after installing glibc and adjusting the paths used by binutils and gcc. This check does not work for glibc-32 because the dummy file doesn't compile successfully. The problem, as far as I can see, is that the compiler (or more likely the loader) does not find a library called libgcc_s.so. There's a version in /tools but the gcc adjustment means that this tree is no longer searched. A version needs to be provided in /usr/lib32 by using symbolic links, as is done for 64-bit.

Unfortunately, in trying to fix this, I screwed up my /tools directory, so I shall have to start again from chapter 5.

Postscript: OK, recovered. The tests worked this time around, except with one oddity: readelf on the 32-bit dummy file reported an attempt to use a loader called /lib32/ld-linux.so.2. Now /lib32 does not exist on this system. Nevertheless, all the other tests worked and the final one showed that /usr/lib32/ld-linux.so.2 was the linker that was actually used.

krash314 · 09-24-2019, 05:12 PM

I am trying to follow this procedure with the following variations:
1) I am following the 5.9 book here. It seems to have included the modifications of Emmett1.
2) I am trying to use the packages from the LFS book 9.0 because I need something newer.

Everything is working fine until I get to the config of binutils in 5.9. Binutils-2.25 - Pass 2. I get the following result.

lfs:/lfs/sources/binutils-build$ CC=$LFS_TGT-gcc \
> AR=$LFS_TGT-ar \
> RANLIB=$LFS_TGT-ranlib \
> ../binutils-2.32/configure \
> --prefix=/tools \
> --disable-nls \
> --disable-werror \
> --with-lib-path=/tools/lib \
> --with-sysroot \
> --enable-64-bit-bfd
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /bin/install -c
checking whether ln works... yes
checking whether ln -s works... yes
checking for a sed that does not truncate output... /bin/sed
checking for gawk... gawk
checking for gcc... x86_64-lfs-linux-gnu-gcc
checking whether the C compiler works... no
configure: error: in `/lfs/sources/binutils-build':
configure: error: C compiler cannot create executables
See `config.log' for more details
lfs:/lfs/sources/binutils-build$

Has anyone else run into this problem?
What additional information can I provide to help troubleshoot this?
Has anyone else tried this procedure using newer LFS 9.0 packages?