Extending LLVM's BOLT-based Binary Analyser to Validate Stack Variable Initialisation

2026-06-09T00:00:00+02:00

Introduction

BOLT, originally developed at Meta and now part of the LLVM project., is a post-link optimiser that rewrites compiled applications to improve their performance. In 2024, Kristof Beyls at Arm built on top of BOLT a prototype static binary analyser to validate compiler code generation for security-related features. This tool operated directly at the binary level to "verify that a given hardening feature has been applied correctly across the whole binary," by checking whether the compiled program satisfies the security properties the feature is intended to provide.

The first scanner implemented for the analyser targeted the "pac-ret" hardening scheme for AArch64, associated to the compiler flag -mbranch-protection=pac-ret, seeking to detect unsafe instruction patterns that could weaken the security guarantees expected from code that uses Pointer Authentication correctly. A second scanner that focuses on code generation mechanisms that prevent Stack Clash attacks (i.e. -fstack-clash-protection) is in the works.

With the goal of further developing the BOLT-based binary analyser, OSTIF commissioned Quarkslab to extend it to support additional compiler flags for security hardening. The initial mandate was intentionally broad, leaving it to us to identify which code generation schemes would be appropriate to tackle, balancing relevance, complexity, and the resources available for the project. Following a preliminary assessment, Quarkslab was then expected to propose one or more hardening features to work on and, upon agreement with OSTIF, implement corresponding scanners for the binary analyser.

Work Summary

The project was carried out between November 2025 and April 2026.

Candidate compiler hardening options were first reviewed, and -ftrivial-auto-var-init was selected as the most suitable target. This flag is handled by both GCC and Clang, is relevant to current hardening practice, and has a clear binary-level property to check: automatic variables¹ should be initialised before use.

Two complementary approaches were then developed for validating the flag statically at the binary level:

a load-oracle approach, which uses stack loads as evidence of automatic variables residing on the stack; and
a store-witness approach, which checks whether the bytes read by each stack load were written on every relevant control-flow path before the load.

These strategies were implemented in a new BOLT-based scanner for x86-64 binaries, published at quarkslab/bolt-stackinit-scanner. The scanner recovers direct and derived stack accesses, recognises common block initialisation patterns, verifies initialisation within and across functions, and finally reports stack reads that may read uninitialised memory. The same scanner can also incidentally be used as a general detector for uninitialised stack reads in binaries not compiled with -ftrivial-auto-var-init.

The scanner was tested on GNU coreutils 9.5 binaries across GCC and Clang x86-64 builds, at several optimisation levels, with and without -ftrivial-auto-var-init=zero. At this stage, this evaluation was only intended to carry the implementation forward and identify its remaining limits.

Interested readers are encouraged to consult the full project report, published on Quarkslab's public reports repository, presenting the theoretical and practical challenges, the limitations of the scanner, and possible ways forward.

Conclusion

This first iteration of the scanner shows that the approach works in principle, though the observed residual reports make apparent the limits of static analysis. Large binaries still generate too many false positives, to the point where manual, unassisted triage becomes too cumbersome. Possible next steps to refine the analysis further include using symbolic execution to prune infeasible control-flow paths, and exploring auxiliary techniques such as DWARF-assisted variable recovery or differential compilation.

This study proved both challenging and stimulating, and gave us the opportunity to explore another facet of binary analysis. Quarkslab would like to thank Kristof Beyls for putting the project forward and OSTIF for the longstanding trust and collaboration over the years.

Quarkslab's blog - automatic initialisation

Extending LLVM's BOLT-based Binary Analyser to Validate Stack Variable Initialisation

Introduction

Work Summary

Conclusion

Further Reading