Compute Module 3/3+ does not boot from eMMC with 1.20250127 or later (U-Boot)

[jens-maus]

Describe the bug
After having received a bug report on one of my public RaspberryPi projects (https://github.com/jens-maus/RaspberryMatic) regarding Compute Module 3/3+ equipped systems not being able to boot anymore (see jens-maus/RaspberryMatic#3046) I investigated the issue myself and found that after simply replacing the start.elf and fixup.dat files in the boot partition with the versions from 1.20241126 the system starts booting correctly again. Further investigation shows that even with the start.elf/fixup.dat of version 1.20250127 Compute Module 3/3+ equipped systems cannot boot anymore and run into the following U-Boot errors and console only:

Please note, that the referenced project (RaspberryMatic) uses U-Boot as a second-level bootloader which supports boot into a recovery partition on GPIO input. This U-Boot boot sequence works fine with rpi-firmware versions older than 1.20250127, Thus, 1.20241126 is the last version allowing to correctly bootup. As soon as start.elf and fixup.dat is used from versions later than 1.20241126 it runs into the shown error messages and cannot obviously load the boot script located on the /boot FAT partition where also config.txt & co are located.

Please note, that the same image boots perfectly with a standard RaspberryPi3(+), so only a Compute Module 3/3+ setup is currently affected as it seems.

To reproduce

1. Install a latest RaspberryMatic image and try to boot it – note the errors mentioned above
2. replace start.elf and fixup.dat on the boot partition with the ones from rpi-firmware release 1.20241126
3. try to perform a boot again and note that U-Boot is now able to find the boot.scr and continues to bootup correctly.

Expected behaviour
Newer rpi-firmware versions > 1.20241126 should still work as expected

Actual behaviour
As mentioned, with rpi-firmware 1.20250127 or newer the included start.elf/fixup.dat combo results in non-bootable U-Boot images.

Logs
see screenshot

Additional context
This issue could be related to #1619 as it shows similar read timeout erros and hsts mentionings.

[pelwell]

If Linux boots on that firmware but u-boot doesn't then it sounds like u-boot is assuming too much.

[jens-maus]

I am still slightly investigating but I found it strange that simply downgrading start.elf/fixup.dat solves the issue.

[pelwell]

I would trace the interactions over the mailbox API.

[jens-maus]

After some more investigation it seems that this issue is really related to a combination of rpi-firmware versions newer than 1.20241126 together with newer U-Boot versions being slightly incompatible. After downgrading U-Boot from 2024.10 to 2023.01 the affected compute module 3/3+ systems are able to boot again, thought they also show some slightly different warnings:

So question remains: What might have changed in start.elf/fixup.dat combo in rpi-firmware 1.20241126 -> 1.20250127 that could trigger newer U-Boot versions of fail to boot?!?!

[pelwell]

The label on the 1.20241126 firmware in the rpi-firmware repo (https://github.com/raspberrypi/rpi-firmware/commits/stable) says:

This is effectively latest firmware before the initial_turbo=60 change

I suggest you try with initial_turbo=0 in config.txt.

[pelwell]

Alternatively, core_freq_fixed=1 will allow the ARM cores to run fast but keep the main core/peripheral clock at a fixed value.

[jens-maus]

Thanks for the hints. I just checked if initial_turbo=0 solves my issues and indeed it does solve the booting issues with newer u-boot version. In fact, I could now update to U-Boot 2025.04 and get it running with initial_turbo=0.

Interestingly enough after searching for similar issues where initial_turbo=0 solves issues I found that one here:

home-assistant/operating-system#3973

In fact, HomeAssistantOS also uses U-Boot and thus ran into the same issue at some point.