A blog series recounting our adventures in the quest to port the BEAM JIT to the ARM32-bit architecture.

This work is made possible thanks to funding from the Erlang Ecosystem Foundation and the ongoing support of its Embedded Working Group.

EEF Logo

First Shell Prompt with the ARM32-bit JIT ​

We are excited to announce that on 20 March 2026, at 18:00 CET, we were able to boot into the shell using the ARM32 JIT for the first time!

shell.png

What happened since last time ​

Since the last post, we went through all the code in hello.erl. Once we completed all test functions, we were left with 150 emitters. Then we uncommented and ran the Erlang STONE test suite, a simple benchmark in hello.erl that computes a performance score. This allowed us to expose and implement a few more emitters, leaving us with 120.

At this point, it was clear to us that hello.erl was not enough to test all Erlang operations. More precisely, it does not cover all possible operation combinations. We are still using the ARM64 translation scheme (.tab files), so the generic-to-specific operation translation is mostly the same. This proved to work, except for rare cases in which the different word size, from 64 to 32 bits, required small edits to the .tab files.

To move forward, we removed the boot bypass that was calling hello:start/2 instead of init:start/2 and reinstated all standard processes. This restored the classic boot sequence. Given that we needed to execute the real OTP startup, we had to make sure we called BEAM with the proper arguments.

To make our lives simpler, we installed binfmt-support and configured it to run ARM32-bit executables with qemu-arm and the arm-linux-gnueabihf interpreter. Configuration is handled via Vagrant here. The result is that we can call the cross-built erlexec binary on Ubuntu ARM64 as if we were on ARM32.

shell
export EMU=beam.debug
export ROOTDIR=/home/vagrant/arm32-jit/otp/RELEASE
export BINDIR=/home/vagrant/arm32-jit/otp/RELEASE/erts-15.0/bin
export PROGNAME=erl
$BINDIR/erlexec -boot start -- +A0 +S 1:1 +SDcpu 1:1 +SDio 1 +JMsingle true

From this point, the work was the same as what we did with hello: wait to trip on an NYI emitter, or stare at a crash and investigate the backtrace or Erlang error. Luckily, the majority of the bugs manifested as Erlang function_clause or badmatch errors. These errors are directly linked to the Erlang code. This makes it easy to add erlang:display/1 calls and debug terms. The annoying part is that it has often been very hard to find the root cause of a bug. More than once, I had to go back to some old emitter that worked perfectly for hello.erl but was plagued with a bug that surfaced only when loading the shell.

Things that help with debugging are:

  • Fill the failing code with Erlang erlang:display/1 calls to understand which specific step introduced an anomaly.
  • Check the module.S compilation result to get an idea of which generic OPs are involved.
  • Compare with the module.asm JIT dump file to inspect the assembly.
  • Look at the suspect emitter functions; if all looks OK, then either:
    • the error is upstream in another emitter
    • we need to brutally add prints in the BEAM OP itself.

Since I am already emitting assembly to implement an OP, nothing stops me from emitting assembly that calls a runtime function such as erts_printf and prints the content of a register.

And of course, all the above is faster with LLMs; I just had to make sure I would not shoot myself in the foot. The great part is that I was able to go from just running the hello.erl module to booting into the shell in about a week of work. This effort is summarized in this PR. I did not have to add many emitters, but the bugs were very hard to deal with.

See you soon ​

Now that the shell is working, we are going to need a test suite application to trigger all remaining emitters and finally finish the job.

node-crash.png

As you can see, it is not that hard to spot the missing pieces: just 75 emitters to go!