investigate ELFHack

[nickdesaulniers]

via https://glandium.org/blog/?p=1177 (@glandium).

I was thinking we might be able to use RELR
https://reviews.llvm.org/D48247
https://groups.google.com/g/generic-abi/c/bX460iggiKg
but @pcc had to implement relocation processing support in the Linux kernel for aarch64 to support this relocation type.
https://patchwork.kernel.org/project/linux-arm-kernel/patch/[email protected]/
So I doubt any existing dynamic linker could process these for x86-64 quite yet.

I'm not sure how much of this might be irrelevant though with #150 or with @MaskRay 's work on enabling -fno-semantic-interposition in Clang's Cmake itself.
https://reviews.llvm.org/D102453

I don't know if ELFhack is easy to fetch/use outside of mozilla-central.

[MaskRay]

llvm-project has been using -fvisibility-inlines-hidden for a long time. It recently started to use -Bsymbolic-functions for libLLVM.so and libclang-cpp.so. There are much fewer JUMP_SLOT relocations. Many absolute relocations shift to R_*_RELATIVE. (The dynamic relocation benefit of -fno-semantic-interposition is entirely covered by -Bsymbolic-functions.)

# regular, -Bno-symbolic
% readelf -r libLLVM-13git.so | awk '$3~/R.*/{s[$3]++} END {for (k in s) print k, s[k]}'
R_X86_64_DTPOFF64 2
R_X86_64_DTPMOD64 3
R_X86_64_RELATIVE 87992
R_X86_64_64 17637
R_X86_64_GLOB_DAT 3117
R_X86_64_JUMP_SLOT 20147

# -Bsymbolic-functions
% readelf -r symbolic.so | awk '$3~/R.*/{s[$3]++} END {for (k in s) print k, s[k]}'
R_X86_64_DTPOFF64 2
R_X86_64_DTPMOD64 3
R_X86_64_JUMP_SLO 348
R_X86_64_RELATIVE 104870
R_X86_64_64 1060
R_X86_64_GLOB_DAT 2816

RELR can optimize R_*_RELATIVE. Android Bionic is the only dynamic loader which supports RELR.

# -Bsymbolic-functions
% llvm-readelf -WS symbolic.so | egrep 'Nr|\.rel[ar]'
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 8] .rela.dyn         RELA            00000000003c8d60 3c8d60 27d368 18   A  1   0  8
  [ 9] .rela.plt         RELA            00000000006460c8 6460c8 0020a0 18  AI  1  25  8

# -Bsymbolic-functions --pack-dyn-relocs=relr
% llvm-readelf -WS relr.so | egrep 'Nr|\.rel[ar]'    
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 8] .rela.dyn         RELA            00000000003c8d60 3c8d60 016bd8 18   A  1   0  8
  [ 9] .relr.dyn         RELR            00000000003df938 3df938 007ba8 08   A  0   0  8
  [10] .rela.plt         RELA            00000000003e74e0 3e74e0 0020a0 18  AI  1  26  8

On a Linux desktop, this needs interest in glibc and musl. They haven't expressed interest. (I have a musl patch in 2019 but it cannot be accepted because it is not accepted ABI. However, ELF as an ABI has stopped evolving.)

Realized that I sent https://sourceware.org/pipermail/binutils/2020-May/111086.html which got no response in binutils. Just sent a GNU ld feature request https://sourceware.org/bugzilla/show_bug.cgi?id=27923 and a glibc ld.so feature request https://sourceware.org/bugzilla/show_bug.cgi?id=27924

[nickdesaulniers]

@MaskRay what about -z rel; does GNU ld.so support REL for x86? Could we link clang itself with -z rel?

[MaskRay]

glibc ld.so's x86-64 port doesn't support REL. ld.so will not apply dynamic relocations in a clang built with ld.lld -z rel and thus the clang will not run.

With appropriate abstractions, supporting both REL and RELA should take relatively small efforts, e.g. ld.lld has proper abstraction so supporting -z rel is not too hard. musl has proper abstraction so supporting REL and RELA don't need extra code. However, glibc ld.so doesn't have that appropriate abstraction;-)

[nickdesaulniers]

ah, so then ELFHack is not converting RELA relocation types to REL, since glibc's dynamic linker would not process those relocation types?

I guess clang could try to relocate itself; then it wouldn't be constrained to relocation types of the host's dynamic linker? I guess that would entail a non-standard PT_INTERP since you'd need to do that before the dynamic linker is run? (or maybe it doesn't matter?)

[pcc]

I don't think you'd need a custom PT_INTERP. I think an executable could relocate itself by adding an entry at the start of .preinit_array. But the relocation for this entry would need to be stored in RELA format (assuming that you use packed relocations for all other relocations) to solve a bootstrapping problem: we need the call from the dynamic loader to the self-relocator to succeed and that's obviously going to happen before relocation.

You would also need to make sure that any GOT entries required by the self-relocator (e.g. dlsym) are stored in RELA format.

So if you wanted to do this I think it would require some kind of custom linker extension.