Skip to content

Add --nmagic linker arg, for unaligned flash origin support. #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 13, 2020

Conversation

Dirbaio
Copy link
Member

@Dirbaio Dirbaio commented Sep 3, 2020

Without this, the linker places some extra program header entries that can confuse flashing tools.

Many thanks to @jonas-schievink for pointing me to this flag in Matrix.

How to reproduce the bug

  • cargo generate --git https://github.com/rust-embedded/cortex-m-quickstart
  • change flash ORIGIN = 0x00027000 in memory.x
  • cargo build
  • llvm-objdump -x target/thumbv7em-none-eabihf/debug/repro

Program header entries will look something like this:

Program Header:
    PHDR off    0x00000034 vaddr 0x00020034 paddr 0x00020034 align 2**2
         filesz 0x00000100 memsz 0x00000100 flags r--
    LOAD off    0x00000000 vaddr 0x00020000 paddr 0x00020000 align 2**16
         filesz 0x00000134 memsz 0x00000134 flags r--
    LOAD off    0x00007000 vaddr 0x00027000 paddr 0x00027000 align 2**16
         filesz 0x00000400 memsz 0x00000400 flags r--
    LOAD off    0x00007400 vaddr 0x00027400 paddr 0x00027400 align 2**16
         filesz 0x00000570 memsz 0x00000570 flags r-x
    LOAD off    0x00007970 vaddr 0x00027970 paddr 0x00027970 align 2**16
         filesz 0x00000180 memsz 0x00000180 flags r--
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

The entry at 0x00020000 causes flashing tools (probe-rs at least) to flash data at 0x00020000, corrupting whatever's there. This happens whenever flash ORIGIN is not a multiple of 0x10000.

With --nmagic, program headers are correct and flashing does what it's supposed to do:

Program Header:
    LOAD off    0x000000f4 vaddr 0x00027000 paddr 0x00027000 align 2**2
         filesz 0x00000400 memsz 0x00000400 flags r--
    LOAD off    0x000004f4 vaddr 0x00027400 paddr 0x00027400 align 2**2
         filesz 0x00000570 memsz 0x00000570 flags r-x
    LOAD off    0x00000a64 vaddr 0x00027970 paddr 0x00027970 align 2**2
         filesz 0x00000180 memsz 0x00000180 flags r--
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

Without this, the linker places some extra program header entries that can confuse flashing tools.

Many thanks to @jonas-schievink for pointing me to this flag in Matrix.
@rust-highfive
Copy link

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jonas-schievink (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

Copy link
Contributor

@jonas-schievink jonas-schievink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We've done the same in a closed-source project a while ago, and it hasn't caused any issues there.

I'll leave this open for a while just in case someone has any comments or objections.

@adamgreig
Copy link
Member

I'm trying to replicate this on any of my projects without success, any idea if there are more conditions than reported here to trigger the bug?

@jonas-schievink
Copy link
Contributor

We were seeing this only with GNU LD, but @Dirbaio also saw it with LLD. Maybe the linker or linker version matters?

@adamgreig
Copy link
Member

If I follow the instructions in the first post, I still don't reproduce this issue; the ELF looks fine in both debug and release modes, and I get the same thing in other projects too. I'm using rustc 1.46.0, but I'm sure I've generated non-aligned images for ages without issue.

@Dirbaio, could you double check your env doesn't have any stale settings like RUST_FLAGS or something, just in case? Alternatively could you possibly zip up a minimal crate which demonstrates this problem?

@Dirbaio
Copy link
Member Author

Dirbaio commented Sep 3, 2020

I'm using rust nightly from rustup, and lld from Arch Linux repos. (not sure if rust uses system's lld or bundles its own though)

[dirbaio@mars nrfconnect]$ rustc --version
rustc 1.48.0-nightly (d006f5734 2020-08-28)
[dirbaio@mars nrfconnect]$ ld.lld --version
LLD 10.0.1 (compatible with GNU linkers)
[dirbaio@mars nrfconnect]$ pacman -Qo $(which ld.lld)
/usr/bin/ld.lld is owned by lld 10.0.1-1

Nothing about rust in env.

Repro is literally cargo generate and changing memory.x. Attached tarball with everything in target as is. -> repro.tar.gz

Just cargo build and llvm-objdump -x target/thumbv7m-none-eabi/debug/repro

@adamgreig
Copy link
Member

Aha, I can duplicate this using nightly but not on stable. Looks like this is an LLVM regression in nightly, then? Rust uses its own copy of LLD by default, it doesn't require one on the system.

@Dirbaio
Copy link
Member Author

Dirbaio commented Sep 13, 2020

I did some more investigation.

  1. ld.lld aligns to the "page size" by design, and --nmagic is the intended way to disable that:
[dirbaio@mars repro]$ ld.lld --help | grep magic
  --nmagic                Do not page align sections, link against static libraries.
  --no-nmagic             Page align sections (default)
  --no-omagic             Do not set the text data sections to be writable, page align sections (default)
  -N                      Alias for --omagic
  -n                      Alias for --nmagic
  --omagic                Set the text and data sections to be readable and writable, do not page align sections, link against static libraries
  1. It works on stable because stable is using a page size of 0x1000 by default, and nightly is using 0x10000.
STABLE

rustc 1.46.0 (04488afe3 2020-08-24):
    LOAD off    0x00001000 vaddr 0x00027000 paddr 0x00027000 align 2**12
         filesz 0x00000acc memsz 0x00000acc flags r-x
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

rustc 1.46.0 (04488afe3 2020-08-24) with --nmagic:
    LOAD off    0x000000b4 vaddr 0x00027000 paddr 0x00027000 align 2**2
         filesz 0x00000a7c memsz 0x00000a7c flags r-x
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

rustc 1.46.0 (04488afe3 2020-08-24) with -zmax-page-size=0x1000
    LOAD off    0x00001000 vaddr 0x00027000 paddr 0x00027000 align 2**12
         filesz 0x00000acc memsz 0x00000acc flags r-x
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

rustc 1.46.0 (04488afe3 2020-08-24) with -zmax-page-size=0x10000 ===> BROKEN!!
    PHDR off    0x00000034 vaddr 0x00020034 paddr 0x00020034 align 2**2
         filesz 0x000000c0 memsz 0x000000c0 flags r--
    LOAD off    0x00000000 vaddr 0x00020000 paddr 0x00020000 align 2**16
         filesz 0x000000f4 memsz 0x000000f4 flags r-x
    LOAD off    0x00007000 vaddr 0x00027000 paddr 0x00027000 align 2**16
         filesz 0x00000acc memsz 0x00000acc flags r-x
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

NIGHTLY

rustc 1.48.0-nightly (d006f5734 2020-08-28) ===> BROKEN!!
    PHDR off    0x00000034 vaddr 0x00020034 paddr 0x00020034 align 2**2
         filesz 0x00000100 memsz 0x00000100 flags r--
    LOAD off    0x00000000 vaddr 0x00020000 paddr 0x00020000 align 2**16
         filesz 0x00000134 memsz 0x00000134 flags r--
    LOAD off    0x00007000 vaddr 0x00027000 paddr 0x00027000 align 2**16
         filesz 0x00000400 memsz 0x00000400 flags r--
    LOAD off    0x00007400 vaddr 0x00027400 paddr 0x00027400 align 2**16
         filesz 0x00000550 memsz 0x00000550 flags r-x
    LOAD off    0x00007950 vaddr 0x00027950 paddr 0x00027950 align 2**16
         filesz 0x00000180 memsz 0x00000180 flags r--
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

rustc 1.48.0-nightly (d006f5734 2020-08-28) with --nmagic
    LOAD off    0x000000f4 vaddr 0x00027000 paddr 0x00027000 align 2**2
         filesz 0x00000400 memsz 0x00000400 flags r--
    LOAD off    0x000004f4 vaddr 0x00027400 paddr 0x00027400 align 2**2
         filesz 0x00000550 memsz 0x00000550 flags r-x
    LOAD off    0x00000a44 vaddr 0x00027950 paddr 0x00027950 align 2**2
         filesz 0x00000180 memsz 0x00000180 flags r--
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

rustc 1.48.0-nightly (d006f5734 2020-08-28) with -zmax-page-size=0x1000
    LOAD off    0x00001000 vaddr 0x00027000 paddr 0x00027000 align 2**12
         filesz 0x00000400 memsz 0x00000400 flags r--
    LOAD off    0x00001400 vaddr 0x00027400 paddr 0x00027400 align 2**12
         filesz 0x00000550 memsz 0x00000550 flags r-x
    LOAD off    0x00001950 vaddr 0x00027950 paddr 0x00027950 align 2**12
         filesz 0x00000180 memsz 0x00000180 flags r--
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

rustc 1.48.0-nightly (d006f5734 2020-08-28) with -zmax-page-size=0x10000 ===> BROKEN!!
    PHDR off    0x00000034 vaddr 0x00020034 paddr 0x00020034 align 2**2
         filesz 0x00000100 memsz 0x00000100 flags r--
    LOAD off    0x00000000 vaddr 0x00020000 paddr 0x00020000 align 2**16
         filesz 0x00000134 memsz 0x00000134 flags r--
    LOAD off    0x00007000 vaddr 0x00027000 paddr 0x00027000 align 2**16
         filesz 0x00000400 memsz 0x00000400 flags r--
    LOAD off    0x00007400 vaddr 0x00027400 paddr 0x00027400 align 2**16
         filesz 0x00000550 memsz 0x00000550 flags r-x
    LOAD off    0x00007950 vaddr 0x00027950 paddr 0x00027950 align 2**16
         filesz 0x00000180 memsz 0x00000180 flags r--
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

The exact commit that changed this is llvm/llvm-project@87383e4 . Apparently it's for compat with GCC linker and with systems with bigger pages. It's not a nightly regression.

IMO the right fix is --nmagic.

The alternative is adding -zmax-page-size=0x1000 to restore the old smaller alignment, but someone might need even lower alignment. There are chips out there with flash pages smaller than 0x1000. eg nrf51 is 0x400.

Copy link
Contributor

@jonas-schievink jonas-schievink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches my expectations. LLD tries to match GNU LD's behavior.

bors r+

@bors
Copy link
Contributor

bors bot commented Sep 13, 2020

Configuration problem:
bors.toml: not found

@therealprof
Copy link
Contributor

Yeah, no bors in here. Let's just merge it then, shall we? ;)

@therealprof therealprof merged commit e1e9885 into rust-embedded:master Sep 13, 2020
japaric added a commit to knurling-rs/app-template that referenced this pull request Nov 11, 2020
rationale is in rust-embedded/cortex-m-quickstart#95
we want to avoid potentially running into that issue in the future
bradjc added a commit to tock/tock that referenced this pull request Jan 8, 2021
Previously we told the linker the page size was small so that it would
in effect remove any padding between program sections. This was a hack
when switching linkers, but the proper way to do this is using the
"-nmagic" flag which explicitly tells the linker to not align sections
to page sizes.

Some relevant discussions:
- rust-embedded/cortex-m-quickstart#95 (comment)
- #2180

Also copying in the linker docs which might be useful:

```
$ ld.lld --help | grep magic
  --nmagic                Do not page align sections, link against static libraries.
  --no-nmagic             Page align sections (default)
  --no-omagic             Do not set the text data sections to be writable, page align sections (default)
  -N                      Alias for --omagic
  -n                      Alias for --nmagic
  --omagic                Set the text and data sections to be readable and writable, do not page align sections, link against static libraries
```

Fixes #2219
@hydra
Copy link

hydra commented May 24, 2022

If one switches to using the gnu linker (ld) instead of llvm because of this bug: rust-lang/rust#88704 then you get this:

 = note: arm-none-eabi-gcc.exe: error: unrecognized command line option '--nmagic'

linker switched using this .cargo/config.toml snippet.

[target.thumbv7m-none-eabi]
rustflags = [
  "-C", "linker=arm-none-eabi-gcc",
 ...

Is there a solution that also works for gnu ld ?

@newAM
Copy link
Member

newAM commented May 24, 2022

Try replacing arm-none-eabi-gcc with arm-none-eabi-ld:

[target.thumbv7m-none-eabi]
rustflags = [
  "-C", "linker=arm-none-eabi-ld",

--nmagic is a flag in the GNU linker[1], so it should work.

@hydra
Copy link

hydra commented Jun 1, 2022

Try replacing arm-none-eabi-gcc with arm-none-eabi-ld:

doh! yes, overlooked that... I'll give it a whirl and report back as time permits.

@hydra
Copy link

hydra commented Jun 2, 2022

I have to use 'gcc' because other arguments need to be passed to it, for map file and for keeping generated assembly files.

[target.thumbv7m-none-eabi]
rustflags = [
  "-C", "linker=arm-none-eabi-gcc",
  "-C", "link-arg=-nostartfiles",
  "-C", "link-arg=-Wl,--print-memory-usage",
  "-C", "link-arg=-Wl,-Map=out.map",
  "-C", "link-arg=-Wl,--verbose",
  "-C", "save-temps",
  "--emit", "asm",
]

a solution that works is to use a build.rs file:

    // for GNU LD
    println!("cargo:rustc-link-arg-bins=-Wl,--nmagic");
    // for LLVM
    //println!("cargo:rustc-link-arg-bins=--nmagic");

but you have to know which linker you're using to uncomment the right line. Some automatic way of doing it would be ideal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants