0004: Compiled models are cached on disk¶
2026-10-17, format revised 2026-10-31 and 2026-11-01 (sections, memory map). Accepted.
Context¶
Compiling the reference model takes about 24 s (boxes 7.6 s, octrees 13.8 s), and every frontend run paid it. The model is read-only after compiling, so it can be written out as it is.
Decision¶
- The engine writes and reads models as bytes:
Model::saveandModel::read(saved.rs). The bytes start with a header: a magic number,FORMAT_VERSIONandENGINE_HASH, a digest of the engine's source, version andCargo.lockmade bybuild.rs(the packaged lock when built from crates.io; the parser, migjorn, is pinned exactly so that the version fixes it). Then sections, each au64length and bitcode: the model without its octrees, then each octree's arrays in pieces of 4 M items. Only the build that saved a model reads it; anything else (cut short, bytes after the end) is refused withError::Format. - Saving writes section by section, each encoded just before it is written, and
reading decodes from a slice (
Model::from_bytes): the cache maps the entry into memory (memmap2) instead of reading it into a buffer. Sound because entries are never changed in place (written to a temporary file and renamed). - The cache policy belongs to the frontends (
llevant_scene::store::Store), not to the engine: - an entry's key is a BLAKE3 hash of the input's bytes, the
OptionsandENGINE_HASH; - entries are written to a temporary file and renamed;
- the most recently used entries are kept within 8 GB (
LLEVANT_CACHE_GB); - entries live in
LLEVANT_CACHE_DIRor the user's cache directory; LLEVANT_CACHE=offturns the cache off, and an unreadable cache is ignored.
Evidence¶
- Reference model: a 1.2–1.6 GB entry, read in 1.5–2.2 s instead of a 24 s compile.
llevant sliceon it: 25.6 s → 2.9 s wall time.-
Encodings tried for reading the reference model: bincode 1.3 s (unmaintained), postcard 2.1 s, bitcode about 1.5 s (kept).
-
Memory, reference model (
llevant info, peak commit / working set from the OS):
| one bitcode payload (before) | sections + memory map | |
|---|---|---|
| compile, no cache | 5.11 / 4.39 GB | 5.30 / 4.52 GB |
| compile + store | 9.14 / 7.92 GB | 5.42 / 4.42 GB |
| cached load | 4.47 / 3.97 GB | 2.33 / 3.28 GB |
The whole payload was built in memory before writing, and read into a buffer that lived while it was decoded; the input (378 MB) was kept to the end. A cached load's working set includes the mapped file's pages, which the OS can drop. Entry 1.58 → 1.47 GB; decoding 1.2 → 1.4 s (pages fault in from the map).
Consequences¶
- No migration code: any engine change gives a new hash, and stale entries simply age out.
tests/saved.rschecks that a model read back answers exactly as the compiled one.- The instance numbering is not cached; it is rebuilt on first use, in milliseconds.