Compiler performance and self-hosting in Zig

Andrew Kelley and Jacob Young lead the development of the self-hosted compiler. Jacob Young recently upstreamed a new backend that passes 1547 of 1960 behavior tests. This rate exceeds the 79% success rate of the LLVM backend. The x86_64 backend now passes 1987 behavior tests, surpassing the 1980 tests passed by the LLVM backend. Andrew Kelley builds the Zig compiler in 10 seconds or less using the x86_64 backend, while the LLVM version takes 75 seconds. Parallelizing the compiler pipeline improved wall-clock time for x86_64 by 5% to 50%. This new backend uses machine code instruction encoding for the internal MIR structure. This architectural decision allows instruction encoding to occur on N codegen threads instead of one single linker thread. The backend also uses a bespoke two-pass liveness analysis where the second pass generates all the MIR in reverse. This allows backend-specific instruction lowerings to affect liveness and cause earlier instructions to become unused. This removes the need for the complex deferred value tracking required by the x86_64 backend used previously. The self-hosted compiler also reduces memory usage by 3x compared to the old C++ implementation. Building the compiler used to require 9.6GB of RAM, but now it requires only 2.8GB. Incremental compilation is approaching the point where it can be turned on by default to allow small changes to rebuild in milliseconds.

Manual memory management and language comparisons

Teams choosing between Rust and C find that Zig requires explicit memory management. Rust relies on a borrow checker to provide safety through ownership. Zig uses explicit allocators, meaning you pass a struct to functions to describe the allocation action. This prevents the hidden memory allocations that happen in C. You already know how C handles malloc, so consider the implications of passing an allocator struct directly to every function that requires memory. Zig uses defer and errdefer to manage resource deallocation. If a function returns an error, errdefer handles the cleanup. This prevents the memory leaks that often occur in C. Emil Lerch notes that while Zig’s approach can be painful, it forces programmers to consider allocations carefully. This can reduce the working set of many programs. The language also provides error handling through the try and catch keywords. Using try allows a function to return an error if an expression fails. This provides a unified interface for errors without much syntactic overhead. The Zig toolchain weighs around 300MB uncompressed, which is much smaller than the 806MB Docker image for Rust 1.53-alpine. Compared to Go, Zig’s grammar is similar in its simplicity, but it lacks the garbage collector and goroutines. Zig’s binary size is significantly smaller than Go’s 2MB "Hello World" in debug mode.

Cross-compilation and C interop

The Zig build system simplifies cross-compilation for various targets. Alex Ronne Petersen states that users can build binaries for FreeBSD 14.0.0+ and NetBSD 10.1+ from any machine. This strategy builds libc and related libraries for every supported target and extracts public symbol information from the resulting ELF files. The system combines this information into a compact abilists file. For FreeBSD and NetBSD, the toolchain imports crt0 code and applies patches to accommodate different OS versions. The toolchain also ships FreeBSD/NetBSD system and libc headers. Zig facilitates C interop by parsing C headers and source files directly. The toolchain uses libclang to import these files. This allows developers to call C functions from Zig code using the @import keyword. Using zig cc provides better defaults like implicitly enabling -fsanitize=undefined. This change helps catch subtle bugs. The new UBSan runtime provides helpful error messages instead of obscure SIGILL errors. The new Io interface also improves asynchronous tasks without requiring libc. Standard operating systems often hide nameservers behind libc, but the new interface allows for a more direct approach. glibc provides getaddrinfo_a and gai_cancel, while Windows uses GetAddrInfoExW and GetAddrInfoExCancel. OpenBSD uses getaddrinfo_async and asr_abort, and macOS uses CFHostStartInfoResolution and CFHostCancelInfoResolution. The implementation sends DNS queries to configured nameservers and immediately attempts TCP connections as responses arrive, canceling all in-flight connection attempts and DNS queries once a single connection succeeds. This implementation performs no heap allocation. It also works with single-threaded, blocking Io. Does the increasing complexity of the compiler pipeline threaten the simplicity of the language?