Skip to content

build_library/oem/ami: Drop bare earlycon on arm64 - #4314

Merged
chewi merged 1 commit into
flatcar:mainfrom
jordeu:aws-arm64-drop-earlycon
Oct 9, 2026
Merged

chewi merged 1 commit into
flatcar:mainfrom
jordeu:aws-arm64-drop-earlycon

Conversation

@jordeu

@jordeu jordeu commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Flatcar on arm64 EC2 boots with a bare earlycon, which takes the UART address from the ACPI SPCR table. On Graviton that address is wrong, and when an NVMe device lands there (t4g with an extra EBS volume) the kernel hangs during early boot. This drops earlycon and keeps console=ttyS0, so the serial console still works; only pre-console early output is lost.

Details

  • earlycon was added in d30359a together with the ttyAMA0 -> ttyS0 console fix. The console fix stays.
  • SPCR on Graviton reports 0x80000000 while the real UART is elsewhere (e.g. 0x8000c000). With an NVMe BAR mapped at 0x80000000, the early 8250 console polls it forever.
  • Scaleway and Azure set earlycon with an explicit driver/address and are not affected.
  • nvme_core.io_timeout is unchanged.
  • Stable 4757.2.1 ships the same line and is affected too; up to maintainers whether to backport.

How to use / testing

Boot an arm64 AMI built from this branch on a t4g instance with an additional EBS volume attached; it should reach userspace and the serial console should still show kernel output. tormath1 offered to build a test image for this.

The change was prepared with the help of an LLM.

Fixes flatcar/Flatcar#2446

CI 🟢 : https://jenkins.flatcar.org/job/container/job/packages_all_arches/825/cldsv/

Bare earlycon picks the UART address from the ACPI SPCR table, which on
Graviton reports 0x80000000 instead of the real UART. When an NVMe BAR
is mapped at that address (e.g. t4g with an additional EBS volume), the
early 8250 console spins forever and boot hangs. The serial console is
still set up by console=ttyS0, so only early boot output is lost.

Signed-off-by: Jordi Deu-Pons <jordi@jordeu.net>
Copilot AI balanced review requested due to automatic review settings October 9, 2026 10:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused configuration change matches the documented failure and preserves the regular serial console.

0 open findings

What changed in this PR

Removes unsafe automatic UART discovery from AWS arm64 boot configuration while retaining normal serial output.

Changes:

  • Drops bare earlycon from EC2 arm64 kernel arguments.
  • Documents the resulting Graviton boot-hang fix.
File Description
build_library/​oem/​ami/​grub.cfg Removes bare earlycon on arm64.
changelog/​bugfixes/​2026-10-09-ec2-arm64-earlycon.md Records the AWS arm64 boot fix.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@tormath1

tormath1 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

@jordeu thanks for the PR - here's an AMI image for ARM64 if you want to give it a try: https://bincache.flatcar-linux.net/images/arm64/9999.9.100+aws-earlycon/

@jordeu

jordeu commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for building it! I've tested it on t4g.large in eu-west-2, with Ignition config {"ignition":{"version":"3.3.0"}}:

Image Extra 8 GiB gp3 Result
stable 4757.2.1 (ami-0203c6338a91abaf0) yes hangs at EFI stub: Exiting boot services...
9999.9.100+aws-earlycon yes boots, reaches multi-user, status checks ok
9999.9.100+aws-earlycon no boots, reaches multi-user, status checks ok

With the test image, /proc/cmdline has console=tty1 console=ttyS0,115200n8 and no earlycon, and the second NVMe controller (00:1f.0) is detected normally. The serial console still shows the full kernel log.

@chewi
chewi merged commit 9f6cc91 into flatcar:main Oct 9, 2026
2 checks passed
@chewi

chewi commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Backported to 4820 (Alpha), 4790 (Beta), 4757 (Stable).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

AWS arm64: earlycon in the EC2 OEM grub.cfg hangs boot on t4g when an extra EBS volume is attached

4 participants