Adding support for new OS - I'm lost with configuring std/crt0 linking error "crtbeginT.o relocation against hidden symbol __TMC_END__"

1

Hi all! I'm trying to add a x86_64-unknown-helenos target to Rust (HelenOS website if you're curious - it's a Unix-like-but-not-exactly microkernel OS).

I already have a local libc fork with added bindings, and I have added this target to the compiler.

// rustc_target/spec/base/helenos.rs
pub(crate) fn opts() -> TargetOptions {
    TargetOptions {
        os: "helenos".into(),
        dynamic_linking: true,
        crt_static_default: true,
        crt_static_allows_dylibs: true,
        position_independent_executables: true,
        static_position_independent_executables: true,
        linker: Some("/home/volfmatej/.local/share/HelenOS/cross/bin/amd64-helenos-gcc".into()),
        has_rpath: true,
        relro_level: RelroLevel::Full,
        panic_strategy: PanicStrategy::Abort,

        pre_link_objects: crt_objects::new(&[
            (LinkOutputKind::DynamicNoPicExe, &["libstartfiles.a"]),
            (LinkOutputKind::DynamicPicExe, &["libstartfiles.a"]),
            (LinkOutputKind::StaticNoPicExe, &["libstartfiles.a"]),
            (LinkOutputKind::StaticPicExe, &["libstartfiles.a"]),
        ]),
        pre_link_args: TargetOptions::link_args(
            LinkerFlavor::Gnu(Cc::No, Lld::No),
            &["--hash-style=sysv", "-m64", "-nostartfiles"],
        ),
        ..Default::default()
    }
}

// rustc_target/src/spec/targets/x86_64_unknown_helenos.rs
pub(crate) fn target() -> Target {
    let mut base = base::helenos::opts();
    base.cpu = "x86-64".into();
    base.plt_by_default = false;
    base.max_atomic_width = Some(64);
    base.stack_probes = StackProbeType::Inline;

    Target {
        llvm_target: "x86_64-unknown-helenos".into(),
        metadata: crate::spec::TargetMetadata {
            description: Some("64-bit HelenOS".into()),
            tier: Some(3),
            host_tools: Some(false),
            std: Some(true),
        },
        pointer_width: 64,
        data_layout:
            "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128".into(),
        arch: "x86_64".into(),
        options: base,
    }
}

Some of the values are guessed, I'll need to do some more reading to figure everything out, but I believe it's mostly correct, and some random changes that I tried didn't have any effect on my current problem.

Right now, my next step is to try and get the whole std crate to compile.

I filled in some of the sys gaps (I could readily use the unix implementation for thread-local storage and allocation, but most of it still uses just sys/pal/unsupported), and the compilation itself seems to run fine. The problem right now seems to be linking, and I'm stuck without any idea what to do next:

./x build library --target "x86_64-unknown-helenos" finishes with this error:

Building stage1 library artifacts {alloc, core, panic_abort, panic_unwind, proc_macro, std, sysroot, test, unwind} (x86_64-unknown-linux-gnu -> x86_64-unknown-helenos)
   Compiling core v0.0.0 (/home/volfmatej/dev/helenos-rust/rust-lang/library/core)
   Compiling rustc-std-workspace-core v1.99.0 (/home/volfmatej/dev/helenos-rust/rust-lang/library/rustc-std-workspace-core)
   Compiling compiler_builtins v0.1.140
   Compiling libc v0.2.169 (/home/volfmatej/dev/helenos-rust/rust-libc)
<... snip ...>
   Compiling std v0.0.0 (/home/volfmatej/dev/helenos-rust/rust-lang/library/std)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/volfmatej/dev/helenos-rust/rust-lang/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/volfmatej/.asdf/shims:/home/volfmatej/.asdf/bin:/home/volfmatej/.local/bin:/home/volfmatej/.cargo/bin:/home/volfmatej/.local/share/pnpm-bins:/home/volfmatej/.local/share/coursier/bin:/home/volfmatej/.local/share/go/bin:/home/volfmatej/.local/share/zig/bin:/home/volfmatej/.bun/bin:/home/volfmatej/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/volfmatej/.dotnet/tools" VSLANG="1033" "cc" "-Wl,--version-script=/tmp/rustcaB5XOQ/list" "-Wl,--no-undefined-version" "/tmp/rustcaB5XOQ/symbols.o" "<16 object files omitted>" "/home/volfmatej/dev/helenos-rust/rust-lang/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-helenos/release/deps/std-33d2962de8102689.5kkb4de76ftxkx9w73cvsyg0x.rcgu.rmeta" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/home/volfmatej/dev/helenos-rust/rust-lang/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-helenos/release/deps/{libpanic_abort-690107261f35da7d.rlib,librustc_demangle-2cc0eec046619471.rlib,libstd_detect-935a530832259f31.rlib,libhashbrown-834a29268c17de19.rlib,librustc_std_workspace_alloc-85fcd0ed332c58a0.rlib,libminiz_oxide-676f715bd6d7278e.rlib,libadler2-b8c84c93ab563772.rlib,libunwind-936fcac46c0958c0.rlib,libcfg_if-79fdd395fdd3067f.rlib,liblibc-d5a7dc039d86499d.rlib,liballoc-fc5ac0d66141cadb.rlib,librustc_std_workspace_core-5a3470c0de75d393.rlib,libcore-8baa648399148487.rlib,libcompiler_builtins-243fdf62afc35a2d.rlib}" "-Wl,-Bdynamic" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-o" "/home/volfmatej/dev/helenos-rust/rust-lang/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-helenos/release/deps/libstd-33d2962de8102689.so" "-static" "-shared" "-Wl,-soname=libstd-33d2962de8102689.so" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin" "-Wl,-rpath,$ORIGIN/../lib"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
          /usr/bin/ld: failed to set dynamic section sizes: bad value
          collect2: error: ld returned 1 exit status
          

error: could not compile `std` (lib) due to 1 previous error

So it seems to be trying to use a C runtime object file from the host linux-gnu, which is probably wrong. Moreover, it still seems to be linking with cc and not the linker that I specified? I still feel like I should be able to add some more linker configuration somewhere, but when looking at the targets of other small OSs, I didn't really find anything.

I think I'm not too far from the goal here. I'm able to compile a simple project that uses libstd and some pure-rust dependencies using rustup toolchain link to use my patched compiler + libc, when I add the following into build.rs:

fn main() {
    let helenos_export = "../../helenos/amd64/export-dev";
    println!("cargo:rustc-link-search=native={helenos_export}/lib");

    // ideally this would be
    // println!("cargo:rustc-link-lib=dylib=c");
    // but for some reason rustc decides that it likes the static version better
    println!("cargo:rustc-link-arg=-l:libc.so.0");
    println!("cargo:rustc-link-arg=-l:libposix.so.0");

    println!("cargo:rustc-link-lib=static=startfiles");
    println!("cargo:rustc-link-arg=-nostartfiles");
}

then cargo b --target x86_64-unknown-helenos -Zbuild-std works.

So how do I move these steps into stdlib, to get it to compile on its own? I suppose the link-libs should be added to libc, as #[link] attributes, but what about the link-args and link-search path?

Thanks in advance. I can try to provide a full replication of my setup, but it's quite annoying - you have to clone multiple git repos, checkout my patches, and then link everything together with [patch.crates-io] etc.

2

It looks like rustc is incorrectly using the linker for the host (cc) instead of the linker you configured. Maybe you need to add the following to your config.toml?

[target."x86_64-unknown-helenos"]
linker = "/home/volfmatej/.local/share/HelenOS/cross/bin/amd64-helenos-gcc"
3

thanks a lot, this fixes my immediate issue and unblocks me to work further

update: see next comment, I figured most of it out
however, why is it necessary? If my understanding of the whole bootstrap process is correct, the stage1 std build (which was failing for me) is done using stage1 compiler, which is built from my patched source, and so it contains the specification of the linker that I added in the target description? Why is that ignored?

And secondly, the config.toml is a local development file. If I'm aiming to eventually upstream this as a tier3 target, what is some permanent way to solve this? I suppose that linker is supposed to be just amd64-helenos-gcc, and the binary should be somewhere in $PATH. What about the link search path? How can I get rid of this configuration line from build.rs, what standard location do I need to move the .so and .a libraries to?

Some background:
I'm working on this as a part of my bachelor thesis, so as my ultimate goal, I hope to eventually upstream as much of my changes and possible (definitely at least the rustc target, libc and stdlib changes), and become able to clone a Rust project, run cargo build --target x86_64-unknown-helenos, and get a binary that I can copy to HelenOS and run. Or at least achieve this with as little special local setup as possible.

I know this is a large undertaking, I'm already for example running into various crates with OS-specific implementations (sysinfo, dirs) which I'll need to patch to build any reasonably useful software. I obviously have a supervisor from my university, but he isn't familiar with Rust build system at all, so if anyone knowledgeable would be willing to periodically answer my questions here, I'd be very grateful (and perhaps get you credited as and advisor in the thesis? idk how any of this works :D)

4

Hm, so I think I figured it out. I added amd64-helenos-gcc and binutils to $PATH, and changed the linker in the target to just the name instead of full path, and now it seems to work. Though I'm still not sure why full path from my original question didn't work - shouldn't binary in path and full path be equivalent?

(I also added the amd64-helenos to cc here. I'm honestly not sure if this can be why it started working, it's quite hard to keep track of the whole process, with all the movement across four different patched repos and long recompile times... I really really should do some writeup of my process for any poor soul attempting this in the future.)