0009: The binaries and the Python module allocate with mimalloc¶
2026-11-01. Accepted.
Context¶
Large parallel batches stopped scaling with threads on Windows. core.i, 200 000
lines of 100 cm (37.6 M segments), 16 threads: 2.4 M segments/s, against 10.4 M at
20 000 lines. One thread stayed linear. Timing the phases showed the system heap: just
freeing the per-chunk buffers took 2.3 s, and Model::volumes on the same model grew
faster than its line count (×7, then ×10 for 4× the lines).
The same code with mimalloc as the global allocator (release, laptop, 16 threads):
| Workload | System heap | mimalloc |
|---|---|---|
core.i trace, 200 000 × 100 cm |
16.2 s | 4.0 s |
core.i volume tracing, h = 2 cm |
42.5 s | 5.7 s |
| Reference model, 2 M points | 1.16 s | 0.73 s |
| Reference model trace, 200 000 × 100 cm | 2.29 s | 2.16 s |
llevant-bench view, CPU slices and first hits |
5–30 % faster, every view | |
| Viewer orbit (GPU) | 16.6 ms median | 16.7 ms median (vsync) |
Peak memory goes up 4–6 % on the reference model (mimalloc keeps freed pages).
Decision¶
llevant, llevant-bench, llevant-viewer and the Python module (_llevant) set
mimalloc::MiMalloc as their #[global_allocator]. The library crates do not: an
application that embeds them chooses its own allocator.
Consequences¶
- mimalloc reuses memory, so a fresh
vec![0; n]is cleared in full; the Windows heap instead maps large zeroed blocks lazily. Code that relied on that has to reuse its buffers: eachTracertakes the last one's surface slots on its thread (a fresh array per small task doubled the slice times before that). - mimalloc is C code built by
cc: the wheels and binaries need a C compiler, which the manylinux images and MSVC have. - The Python module builds mimalloc with
local_dynamic_tls(2026-11-12). By default mimalloc's thread-locals are initial-exec, which marks_llevant.abi3.soSTATIC_TLS: a module loaded withdlopenthen has to fit in glibc's static TLS surplus (512 bytes by default), andimport llevantfailed on Linux after numpy and others ("cannot allocate memory in static TLS block"). Raising the surplus withGLIBC_TUNABLES=glibc.rtld.optional_static_tls=…is not a way round it: glibc carves static TLS out of every thread's stack, so 64 MB of it crashed the 2 MB rayon threads at the first batch. Local-dynamic TLS costs nothing measurable (Python, reference model: 200 000 lines of 100 cm 0.12–0.16 s and 2 M points 0.07 s, before and after).test_no_static_tls(crates/llevant-py/tests) reads the module's ELF flags. The binaries keep initial-exec (acargo build --workspaceunifies the feature into them too, harmlessly).