benchmark

baseline_timings.json and baseline_counters.json preserve INSERTION order, not sorted order — do NOT re-dump them with json.dump(sort_keys=True), which scrambles the whole file into a noisy diff (CADET_* jumps to top because ASCII 'C'<'a'). run_regression.py's --update path DOES use sort_keys=True (would reorder on next gate-update), but for a manual single-bench add, insert the new entry as a text block before the final closing brace to keep the diff minimal. When adding a bench: (1) FAST_SUITE tuple (name,timeout,strategy,rust_only) in run_regression.py; (2) baseline_timings.json entry {callback_count,peak_memory_mb,python_time,rust_time,state_creations,steps}; (3) baseline_counters.json full --counters-json stats dict (required or the gate's expected_counter_keys check ERRORs on missing keys). Note --counters-json output has the flag string on line 2 which contains '{}' — bench_diff.load_counters' naive text.find('{') breaks on it; parse from the first line that is exactly '{'.

add_constraint_raw dedup distribution sweep (angr-55e3, 2026-06-02). 21-bench sweep of ADD_CONSTRAINT_RAW_TOTAL / DEDUP_SCANNED / DEDUP_HIT counters via python tests/benchmarks/run_single.py --counters-json.

Headline numbers:

• 11/21 benches: zero add_constraint_raw calls (lineage dedup path never fires): aarch64/arm/mips32/mips64 synthetics, ais3_crackme, codegate-angrybird, defcamp_r100, flareon2015_2, mma_howtouse, strcpy_find, sym-write, unmapped_analysis.
• csaw_wyvern: 33/40 hit = 82.5% (best — 33 push+assert ops saved)
• defcon2016quals_baby-re: 12/13 = 92.3% (highest %, low count)
• flareon2015_5: 126/403 = 31.3% (largest absolute total; most calls still novel)
• whitehatvn2015_re400: 1/2 = 50%
• csgames2018: 0/13 = 0% (calls fire but no dups)
• ekoparty_rev250 / fauxware / flareon2015_10 / google_unbreakable_0: 0% hit, low counts

Key sanity check: scanned == total on EVERY bench. Every add_constraint_raw call enters the dedup path; nothing exempts itself.

Comparison vs zdho-structural-duplicate-rate memory (different metric):

• csaw_wyvern: 53.4% structural dups across run vs 82.5% per-call ptr-equality hits — per-call dedup CATCHES MORE than structural rate because the SAME identical AST gets re-asserted multiple times within a session.
• ekoparty_rev250: 25.5% structural vs 0% per-call — structural dups exist across run but each is asserted only once to lineage solver.

No code change: dedup is doing its job where it fires. Total absolute savings even on best case (csaw_wyvern: 33 push+assert ops over 220ms run loop) is small. The 11 zero-call benches don't benefit and don't pay for the lazy-seed overhead (amortized to nothing at 0 calls).

This is sweep #4 in the characterization series after angr-0wam (syscall fallback, null), angr-twyr (simprocedure fallback, null), angr-3x4u (concretize, found csaw_wyvern 85% loop-time concentration).

android_arm_license_validation bench landed (angr-p3da, iter 64): first real ARM ELF in FAST_SUITE. Rust 0.25s / Python 0.20s (SLA WARN ~0.8x, not FAIL). Marked rust_only=True due to output divergence (multiple valid solutions). 17/17 fast-tier benches pass. baseline_timings.json now has 28 entries.

baseline_timings.json gotchas (learned angr-zjro9, 2026-07-13): (1) DFS-strategy suite entries get their OWN baseline key, name + '__dfs' (see baseline_key assignment in run_regression.py's REGRESSION_SUITE loop) — e.g. defcamp_r100 AND defcamp_r100__dfs. Refreshing only the base key leaves the dfs twin stale and the gate keeps flapping. (2) run_regression prints the dfs run's header as '--- (DFS) ---', so a naive '^--- (\S+) ---' parse silently drops it and mis-attributes its timing to the previous bench. (3) The file on disk is INSERTION-ordered indent=2, but run_regression's json.dump uses sort_keys=True — hand-editing must round-trip with sort_keys=False to avoid a whole-file reorder diff (see bench-add-baseline-files-insertion-order).

run_single.py _run_in_child loads each bench solve.py via exec_module inside an 'except Exception' guard. argparse with required=True (and any sys.exit) raises SystemExit, a BaseException NOT caught by 'except Exception' — so it escapes exec_module, propagates, and HANGS the multiprocessing pool worker until the parent --timeout hard-kills it, mislabeling the bench as TIMEOUT. Fix pattern: (1) add an EXAMPLE_CATALOG 'argv' entry for solve.py scripts that read sys.argv (e.g. sharif7_rev50 -> ['-f','getit'], insomnihack_aeg -> ['./demo_bin']); (2) _run_in_child now has an explicit 'except SystemExit' clause that fast-fails ok:False with a 'missing argv?' hint. Lesson: a catalog 'timeout >60s' label can be a masked argv/SystemExit hang, not a real slow bench — sharif7_rev50 actually solves in Rust ~0.85s / Py ~3.3s.

argv-injection mechanism (merged from bench-harness-argv-injection, commit 5e87a3064, angr-86c4, 2026-06-03): the EXAMPLE_CATALOG 'argv' key is a list of post-script-name argv elements; run_single.py sets sys.argv=[solve_script, *argv] before exec_module and restores it in a finally. Backward compatible — entries without 'argv' leave sys.argv as the parent run_single.py invocation (usually argv[1]=example_name). Use for solve.py that reads sys.argv[1] as a binary path (insomnihack_aeg {'argv':['./demo_bin']}, secuinside2016mbrainfuzz pattern).

cmu_binary_bomb upstream solve.py issues found while landing cmu_binary_bomb_partial wrapper (2026-06-03, angr-w5op closed in commit 9cb790c27):

• flag_2: stack_push of 32-bit BVS raises SimMemoryError 'Not enough data for store' in both engines.
• flag_3: uses simulation_manager(state, veritesting=True). Rust silently no-ops veritesting and returns empty solution list. See memory rust-veritesting-silently-unsupported.
• flag_5: uses proj.kb.obj.get_symbol(...). The kb.obj plugin was renamed/removed years ago; raises AttributeError.
• flag_6: ~73s path-explosion enumerating linked-list combinations (Python timing). Too slow for FAST/MEDIUM tier.

Working flags: 1 (~0.3s), 4 (~0.3s, but Rust returns (0,0) vs Python (7,0) — different valid Z3 models), secret (~1.4s).

Bench now exists as cmu_binary_bomb_partial in tests/benchmarks/synthetic_examples/, see memory bench-cmu-binary-bomb-partial. This memory supersedes the prior wrap-list (flag 3/5/6 were assumed working — they aren't).

cmu_binary_bomb wrapper landed as cmu_binary_bomb_partial in tests/benchmarks/synthetic_examples/. Runs flags 1, 4, secret. Skip rationale per flag:

• flag_2: upstream stack_push of 32-bit BVS raises SimMemoryError 'Not enough data for store' in both engines.
• flag_3: upstream uses simulation_manager(state, veritesting=True). Rust manager silently no-ops veritesting and returns empty solution list.
• flag_5: upstream uses proj.kb.obj.get_symbol(...) — kb.obj plugin removed years ago, AttributeError.
• flag_6: ~73s path explosion enumerating linked-list paths; would push past MEDIUM tier.

Rust timing: 1.32-1.34s (very stable across 5 runs, NOT bimodal). Python: 1.38s. Speedup ~1.04x. peak_mem=383MB. Wrapper chdirs to ~/repos/angr-examples/examples/cmu_binary_bomb so upstream's relative './bomb' path resolves. Added to FAST_SUITE rust_only=True because solve_flag_4 returns a different valid Z3 model than Python (Python 7 0, Rust 0 0) — both reach find without bomb_explode. Closed bd angr-w5op in commit 9cb790c27.

Concretize counter distribution sweep across 20 fast-tier benches (angr-3x4u, 2026-06-02):

Headline finding: 17/20 benches show ZERO concretize activity. The 3 with activity:

Notes:

• Bench-wide: mem_load_symbolic_addr = mem_store_symbolic_addr = 0. Concretize calls fire from somewhere other than the symbolic-addr path (likely write-side concretize_for_write for SimMemory write resolution).
• csaw_wyvern: 28 calls × 7.2ms/call, all max_cands=1 (single-value queries). Bench is already 16.9x speedup so this is not blocking the headline number, but is the dominant Rust-loop cost on that bench.
• flareon2015_5 has the most calls (48) but high candidate count (avg 61, max 64) so per-call cost (~110μs) is much lower than csaw_wyvern (~7ms).
• concretize_disjunction_count = 0 across the whole corpus — disjunction-style concretization is never triggered on these benches; all concretize sites use the bounded-eval path.
• flareon2015_2 reports 71 concretize_read_count but rust_concretize_count=0. Likely a fast-path where the count increments but no z3 query fires (e.g. cache hit on already-concrete addr).

Comparison to prior characterization sweeps:

• syscall_python_fallback_by_num (angr-0wam): near-zero across corpus.
• simprocedure_fallback_by_name (angr-twyr): near-zero across corpus.
• concretize_* (this sweep): concentrated cost in ONE bench (csaw_wyvern), real but already in the speedup leader.

DECISION: Characterization only. csaw_wyvern is the fastest bench at 16.9x; even halving concretize cost would push it to ~30x but the absolute ms savings (~100ms) is dwarfed by other benches' loop times. Bigger ROI lives elsewhere. Memory preserved for future iters that may want to revisit concretize batching / caching as a structural improvement.

cow_fork_scaling synthetic bench (angr-vx8p.2) is the corpus's ONLY fork-scaling regression gate: x86_64 fork_tree_8 reads 16 symbolic stdin bytes through 8 independent branches -> 256 leaf states. Key metric note: the Rust engine reports state_creations=0 for this workload (Rust forks via deferred CoW, not Python SimState copies); the real fork signal lives in steps=513, deadended_count=256, rust_deferred_fork_count=255. So when wiring fork-heavy benches, gate on steps/deadended, NOT state_creations. rust_only=True (Python OOMs past N=8). Lives in tests/benchmarks/synthetic_examples/cow_fork_scaling/ (prebuilt binary + fork_tree_8.c source). Char tooling+writeup in tests/benchmarks/characterization/cow_fork_scaling/.

Bimodal TIMING (merged from benchmark-cow-fork-scaling-bimodal-timing, angr-5cx4r): its Rust timing is bimodal (fast ~2.2s / slow ~2.7s, from Z3 SAT-search nondeterminism over the 256 fork-leaf constraints) but COUNTS are stable (steps=513, peak ~999MB). baseline_timings rust_time is pinned to the SLOW mode (2.7, was 2.28) so the 0.15 gate tolerates both modes — the fast-mode 2.28 baseline caused a false gate failure on iter-77's doc-only commit. It is deliberately NOT in BIMODAL_BENCHMARKS: that frozenset drops a bench from the whole --skip-bimodal gate, and the steps=513 count signal (the O(1) CoW-fork claim) is this bench's entire purpose. Same slow-mode-baseline convention as CADET_00001_partial.

native/angr/examples/bench_deserialize.rs is a one-shot microbenchmark for IRSB deserialize timing. Capture inputs by adding a CAPTURE_IRSB env-gated dump in angr/exploration/rust_irsb_serializer.py serialize_irsb() and running 'CAPTURE_IRSB=/tmp/sample_irsbs.json python tests/benchmarks/run_single.py codegate_2017-angrybird --engine rust' (writes 500 samples). Build: cargo build --manifest-path native/angr/Cargo.toml --release --example bench_deserialize. Run: ./target/release/examples/bench_deserialize /tmp/sample_irsbs.json. Reports full deserialize µs/call and json-parse-only µs/call.

bench-gate-cold-start-noise

run_regression.py gate retry pass (--retry-failures N) re-checks ALL THREE failure dimensions — timing, SLA, and peak_memory — not just timing. A single anomalously slow/heavy bench run emits up to 3 correlated failures from the SAME run; before angr-4v8g the retry only cleared the timing failure_msg, so a transient outlier still failed the gate on SLA+memory even after the re-run proved it noise (sank iter-105 on defcon2016quals_baby-re: 636MB/0.39x in gate, 288MB/0.47s/3x in 3 isolated reruns). Mechanism: per-bench dict retry_info keyed by name (built via _retry_record helper) accumulates whichever of timing_msg/sla_msg/mem_msg fired plus bl/baseline_mem/sla_py_time; retry pass re-evaluates each failed dimension and discards from the 'pending' set + removes from canonical failures[] on first within-bounds measurement. Each bench runs in its OWN subprocess so ru_maxrss is per-bench, not contaminated by earlier benches — a memory spike is genuine transient Z3-path variance, not cross-bench leakage.

tests/benchmarks/run_single.py harness supports per-example argv injection via the EXAMPLE_CATALOG dict (key='argv', list of post-script-name argv elements). Use this when a solve.py reads sys.argv[1] as a binary path (insomnihack_aeg, secuinside2016mbrainfuzz pattern). Added 2026-06-03 in commit 5e87a3064 (angr-86c4). Example: {'argv': ['./demo_bin']} sets sys.argv = [solve_script, './demo_bin'] before exec_module; restored in finally. Backward compatible: entries without 'argv' get the previous behavior (sys.argv unchanged from the parent run_single.py invocation, which usually means argv[1] = example_name).

When changing benchmark-touched code, single-shot run_regression.py results can show 15-22% regressions that disappear under a 5-sample median (within 2%). Don't trust a one-sample timing failure — re-run with: 'for i in 1 2 3 4 5; do python tests/benchmarks/run_single.py --engine rust 2>&1 | grep ^OK; done'. cmu_binary_bomb_partial showed 1.62s in one shot but median 1.36s (vs baseline 1.33s, within noise) across 5 samples. Re-running run_regression.py once also typically suffices to confirm a noise spike.

bench-noise-subsecond

simple_heap_overflow (angr-nf32, 2026-06-03): bench cannot run locally due to two blockers. (1) Python: solve.py needs angr/binaries CI repo (libc 2.27 + ld-linux-x86_64.so.2) — absent locally, no network. With system glibc 2.39, solve.py raises 'Uh oh! Couldn't explore to the crashing state. It's possible your libc is too new.' because heap layout differs. (2) Rust: entry_state() default options include SYMBOL_FILL_UNCONSTRAINED_REGISTERS which is in _RAISE_OPTION_NAMES (apre-root-cause), raises NotImplementedError. Verified that remove_options={SYMBOL_FILL_UNCONSTRAINED_REGISTERS} satisfies state creation but Python is still blocked, so this workaround alone doesn't unblock the bench. Closed angr-nf32 as deferred. Catalog entry in run_single.py:76 marks rust_ok=False. Future revival path: vendor a libc 2.27 + ld-linux-x86_64.so.2 (e.g., extract from an angr/binaries snapshot) OR rewrite solve.py to target a different heap-overflow pattern that doesn't depend on libc internals.

Bench-wide simprocedure fallback distribution sweep (angr-twyr, 2026-06-02): 17 benches surveyed, 74 total simprocedure_python_fallback calls.

ZERO-fallback benches (7/17, fully native-covered): ais3_crackme, codegate_2017-angrybird, csgames2018, defcamp_r100, flareon2015_2, strcpy_find, sym-write.

Top fallback names (aggregate): 14 __stack_chk_fail (defcon2016quals_baby-re only) 13 my_scanf (defcon baby-re, app-specific user proc) 7 std::operator<<<...> (csaw_wyvern only) 6 operator new(unsigned long) (csaw_wyvern only) 6 UserHook (flareon2015_5,10 + whitehatvn re400) 5 memmove (csaw_wyvern, all SYMBOLIC fallbacks) 5 operator delete(void*) (csaw_wyvern only) 2 std::string ~... 2 CallReturn (mma_howtouse + flareon2015_10) 2 strncpy (mixed: 1 symbolic, 1 other) 1 each: open, strncmp, get_flag, std::allocator, std::string ctor, length()

Breakdown by native_proc_*_by_name subcategory: symbolic_fallbacks: memmove=5, open=1, strncpy=1 (already-native procs giving up on symbolic args) not_implemented_fallbacks: other_fallbacks: strncmp=1, strncpy=1

Implications:

• Native registry already covers the hot path of all 17 benches at 100% procedure count parity — zero fallbacks on 7/17, single-digit on others.
• The only would-be 'easy native port' candidates are __stack_chk_fail (14 calls, single bench, trivial stub) and the C++ stdlib procs (csaw_wyvern 7+6+5+5+2+...=~25 calls); csaw_wyvern is already 16.9x speedup so adding C++ stdlib natives is not a bench mover.
• UserHook (6) and CallReturn (2) are Python-only by construction.
• my_scanf and get_flag are app-specific CTF hooks (not libc), no reusable native procedure can address them.
• Symbolic-arg gap on memmove/open/strncpy is small (7 total calls); see native_proc_symbolic_fallbacks_by_name counters to drive any future refinement of these handlers' symbolic-input acceptance.

Like the syscall-fallback sweep (bench-syscall-fallback-zero-coverage), the take-away is that the simprocedure native coverage is not a current bottleneck; the counter remains as a diagnostic surface for new workloads. Saved during ralph iter 34, post angr-0wam.

run_regression.py SLA gate: the speedup-vs-Python SLA check (in the REGRESSION_SUITE loop, 'SLA check' block) falls back to baseline['name']['python_time'] when py_time is None. As of angr-ngver this fallback is GUARDED by 'not entry_rust_only': the global --rust-only flag still gets SLA coverage via cached python_time, but per-bench rust_only=True entries (4th FAST_SUITE tuple field) are SLA-EXEMPT. Rationale: rust_only benches are arch-edge/tiny workloads (e.g. android_arm_license_validation ~0.42x Python — PyO3/lift init tax dominates a 0.2s ARM run) where Rust is legitimately slower; the timing gate (vs cached rust_time, 15% threshold) still protects them. Don't 'fix' a per-bench rust_only SLA exemption by reinstating the unconditional fallback.

Bounded-bench soft-timeout harness (tests/benchmarks/run_single.py, added angr-9w6ad.7): opt-in via env ANGR_BENCH_SOFT_TIMEOUT=, rust engine only. _run_in_child arms TWO watchdogs before exec_module: (1) SIGALRM (signal.setitimer ITIMER_REAL) whose handler raises a local _SoftTimeout(Exception) so the existing _collect_rust_diagnostics except-path dumps partial mgr.stats() counters; (2) faulthandler.dump_traceback_later(repeat=True) on a separate thread. KEY LIMITATION the harness is designed around: the SIGALRM handler only runs at a Python bytecode boundary, so it CANNOT interrupt a bench stuck inside one long _rust_mgr.run() PyO3 call (a single dominating Z3 solve or VEX loop) — that is exactly when faulthandler (separate thread) is what pins the hang location. explore() loops in Python in 50-step batches (_run_predicate_batch), so the signal DOES fire for benches that return between batches. Parent run_example still hard-kills via multiprocessing pool.terminate on its own --timeout; set parent --timeout > soft cap so the child watchdogs fire first. Both watchdogs cancelled in the inner finally. Used for P2c attribution; see benchmark-p2c-veryslow-profile.

Survey of untracked angr-examples CTF crackmes for bench-add (angr-528r, 2026-06-06): exhaustive enumeration found ALL viable candidates have blockers. Categorized: (1) too-slow: 9447_nobranch (hours), mma_simplehash (~1hr), hitcon2017_sakura (~6min), sharif7_rev50 (timeout), 0ctf_momo_3 (timeout), b01lersctf2020_little_engine (~150s); (2) Rust-engine semantic gap (list-index/AST-complexity): asisctffinals2015_fake, asisctffinals2015_license, whitehat_crypto400, ekopartyctf2015_rev100; (3) structurally incompatible: secuinside2016mbrainfuzz (4-binary harness), 0ctf_trace (uses factory.successors() not simgr), defcon2017quals_crackme2000 (no binary checked in). NEW data point from iter 277: CSCI-4968-MBE/crackme0x00a — clean 32-bit x86 i386 ELF, Python engine solves in 0.62s, but Rust engine returns 0 found states in 0.10s when going through full entry_state init path (likely __libc_csu_init unconstrained-memory fills diverge — flareon2015_2 works because it uses blank_state, not entry_state). Future bench-add work should require (a) ground-truth checked-in binary, (b) blank_state OR pre-validated entry_state path, (c) Rust engine smoke pass before wrapping. xmllint (angr-75mc) remains the only path forward for angr-vx8p completion.

xmllint_getenv FAST-tier bench (synthetic_examples/xmllint_getenv/solve.py, angr-11djq.3): first real-world libc-heavy utility in the gate. KEY GOTCHA: with use_sim_procedures=True angr hooks getenv INSIDE libc and redirects the call straight to the hook, BYPASSING the main-object PLT stub (proj.loader.main_object.plt.get('getenv') == 0x4062e0 is NEVER executed -> find returns 0 states). The correct find target is proj.loader.find_symbol('getenv').rebased_addr (the resolved libc symbol where the SimProc lives). Bounded single-state slice: symbolic 16B stdin + args [--noout --nonet --recover --noent -], reaches getenv in ~27 manager steps, Rust 3.5s/Py 3.84s. Binary resolved via ANGR_EXAMPLES_DIR (xmllint/xmllint_bin in angr-examples), not vendored. Fallback surface: syscall fallbacks 0, simproc fallbacks 27 (intentional libc pthread/malloc). rust_only in FAST_SUITE (realism bench not speed win).

Full sweep delta after angr-nnoh fix (commit aaaa25779) on 2026-05-05 (HEAD=89132680b). 22/22 benchmarks pass. Mostly recoveries from the 9 regressors tracked in project_benchmark_status. Notable changes baseline_pre→post: hackcon2016_angry-reverser 35.0s→12.0s (-65.7%, biggest win; prior baseline was stale), securityfest_fairlight 9.65s→12.83s (+33% — FP-theory work added overhead, profiled in fairlight-bottleneck-2026-05), codegate_2017-angrybird 0.88s→2.82s (+220% — old baseline was stale per angrybird-baseline-stale memory; new captures truth), ekopartyctf2016_rev250 1.48s→2.03s (+37%), google2016_unbreakable_1 2.95s→3.52s (+20%), mma_howtouse 5.76s→6.74s peak_mem 1346MB→283MB (mem leak fix landed; runtime still slow, SLA WARN 0.63x — Callable-heavy, known), sym-write 0.37s→0.42s (+14%, modest variance after seg_selector fix), flareon2015_5 6.27s→6.56s, whitehatvn 1.20s→1.26s. csaw_wyvern, ais3_crackme, defcamp variants, fauxware all flat. ONLY SLA warning: mma_howtouse 0.63x (Callable-heavy, expected). Test status: cargo test --release --lib = 391 passing, pytest tests/engines/test_rust_exploration.py = 214 passing.

Pre-existing benchmark regression discovered 2026-05-17 (HEAD 036716c2a) while validating angr-518z cleanup work. mma_howtouse: 6.513s/286MB baseline (2026-05-09) -> 55.04s/1888MB current — 8.5x slower and 6.6x more memory. Peak memory exactly matches pre-leak-fix scale (~1606MB) from benchmark-mma-howtouse-leak-fix memory, but commits 342df4a7f's traverse/clear helpers on PythonCallbacks are still in place and traverse_fields/clear_fields enumerates all 21 fields correctly. So the original cycle GC path still works; a new leak path opened between cafc8e702 (2026-05-09 baseline refresh) and 036716c2a. Companion regression at smaller scale: ais3_crackme 0.84s/367MB -> 2.01s/929MB (same direction, ~2.4x slower / ~2.5x more mem). fauxware unaffected (0.24s, baseline 0.385s). Filed as angr-9maq P1. Both regressions reproduce on tree without angr-518z changes (verified via cargo-only rebuild after git stash). Bisect candidates: 4j5u orchestrator extraction series, uq4n inspect dispatch series, 0e4fe9223 (move per-state metadata into RustSimState — touches lifetime). Recommended starting point: check whether _state_cache or _ast_handle_cache grows unbounded in mma_howtouse under HEAD vs cafc8e702.

angr-5hpn (concrete-RHS shl/mul rewrites) landed 2026-05-20 with no measurable change on the fast-tier regression matrix (16/16 pass, all within ±0.01s of baseline: fauxware 0.22s, defcamp_r100 0.27s, csaw_wyvern 2.65s). The optimization is theoretically valuable (avoids Z3 O(N^2) bit-blast for shl/mul-by-const) but the fast-tier benches don't have enough symbolic-LHS const-RHS shifts on the hot path to register. Worth keeping for: (1) workloads with heavy eflags packing/unpacking, (2) workloads that stress symbolic arithmetic through Z3, (3) cleaner ASTs for downstream simplification. Companion bead angr-ya00.1 (mul2concat=true Z3 param) addresses the same pattern from the Z3 side.

Benchmark results after angr-9maq fix (commit fced54a07, 2026-05-17, on HEAD which is 77333aecc+1): mma_howtouse: 55.04s/1888MB -> 7.23s/277MB (baseline 6.5s/286MB - back to baseline) ais3_crackme: 2.0s/929MB -> 0.91s/369MB (baseline 0.84s/367MB) fauxware: 0.21s/176MB (unchanged) unmapped_analysis: 0.79s -> 0.96s (slight slowdown, possibly within noise; 18% > 15% threshold) Per-call timing inside mma_howtouse Callable loop: 1.93s -> 0.18s (10x faster). 437/437 test_rust_exploration.py tests pass.

aarch64_le_branch added to FAST_SUITE 2026-05-14 (commit e0d2746d1). ARM64 promoted Experimental -> Supported. Synthetic inline-ELF (no angr-examples binary needed). 10 AArch64 instructions; add+add imm+cmp on symbolic w0; solution w0=42 (because 2*42+16=100). Avoids NEON ops (still scaffolded). rust_time=0.51s on local box; python_time=null in baseline so SLA gate is skipped. Total benchmark count: 24 (22 angr-examples + 2 synthetics).

Post-angr-8j16 (read/write FD generalization) callback profile across 6 benches (codegate_2017-angrybird, fauxware, sym-write, ais3_crackme, defcamp_r100, csgames2018) shows ZERO syscall_python_fallback for file FDs and ZERO native_proc_fallbacks for read/write. Only fauxware still has 1 simprocedure_python_fallback (open), unchanged from pre-8j16 baseline_counters.json. Iter 79 hypothesis was that codegate_2017-angrybird would show callback drop — but codegate runs entry_state(addr=mid-binary, LAZY_SOLVES) entirely from symbolic memory with no file IO at all (844 native_proc_calls are libc string ops). The 8j16 benefit will only surface on a multi-fd benchmark like xmllint (pending angr-75mc) that does open + read/write on non-stdin fds. Don't add to baseline_counters.json followups expecting a measurable shift on existing benches.

Bench post-Arc-concretize-cache (2026-05-05): fauxware 0.38s (baseline 0.387s), sym-write 0.42s (baseline 0.423s), flareon2015_5 6.27-6.43s (baseline 6.563s). All within noise to slightly faster. The 5-10% speedup in angr-6uhh's description didn't show on these benchmarks because pending_stores flushes are not the dominant cost; concretize cache savings are most visible on address-heavy multi-soln workloads.

Bench post-Arc-wrap-fork (2026-05-05, angr-0dgj): fauxware 0.38s (baseline 0.387s), ais3_crackme 0.86s (baseline 0.845s), flareon2015_5 6.36s (baseline 6.56s), defcamp_r100 0.25s (baseline 0.250s). Full regression run: 12/12 pass, total 21.7s. Changes are structurally cheaper but micro-benchmark wins fall within noise — typical CTF binaries do not fork heavily enough for the O(n) → O(1) HashMap/HashSet/Vec clone savings to dominate. Big win would surface on heavily-branching workloads or with very large hook_addrs maps.

arm_le_branch synthetic benchmark added 2026-05-17 (commit f503436b8) for angr-duta.2. Inline ELF32 ARM LE, ~10 instructions, drives r0 through 2*r0+16==100, solver picks r0=42. ARM moved Experimental -> Supported in arch matrix. rust_time=0.31s, python_time=null (rust_only=True since PyO3 init dominates the workload). Total benchmark count: 25 (22 angr-examples + 3 synthetics: ARM, AArch64, MIPS32). Avoids NEON / VFP / Thumb (those still use scaffold paths).

AST-shrinking Z3 optimizations on bench-bottlenecked workloads may UNMASK Z3 SAT-search nondeterminism that was previously hidden by a deterministic-but-slow solve path. The 2026-06-01 angr-rbnk SignExt fix on hackcon2016 collapsed AST 13.7x; pre-fix bench was tightly clustered (5-sample median 14.84s, 10-sample 30.79s ±1.33s) but post-fix went bimodal (8.97-34.88s, median ~22.5s). Pattern: tight variance + slow median can hide simpler search-space-of-Z3 behavior that becomes visible once the AST is no longer the bottleneck. When landing AST-simplification fixes on slow benches, always re-measure variance distribution; consider whether the bench should move into BIMODAL_BENCHMARKS even if median improves.

Benchmark 'regressions' on this box are usually BOX DRIFT, not code. Proof method (used 2026-07-13, angr-zjro9): check out the commit that recorded the baseline, rebuild cargo-direct, and re-measure with run_single --both. If python_time moves too (flareon2015_2 python 4.13 -> 4.68, +13% on frozen Python-engine code), no Rust change can explain it. Corroborating tell: some benches run FASTER than baseline (sharif7_rev50 0.22x, arm_le_branch 0.64x) — a uniform code regression cannot do that. Baselines refreshed 2026-07-13 to max-of-N (not median) so the 15% threshold absorbs run-to-run variance; flareon2015_2 alone swings 3.92-4.59s across runs, so a median baseline is too tight to gate at 15%. ALWAYS re-measure vs a stashed/checked-out HEAD before blaming your diff.

Benchmark baseline expansion (2026-05-01, commit 15395dd6b): tests/benchmarks/baseline_timings.json went from 16 entries / 7 with full stats → 22 entries / 22 with full stats. New entries: defcamp_r100__dfs, defcon2016quals_baby-re, ekopartyctf2016_sokohashv2, hackcon2016_angry-reverser, mma_howtouse, unmapped_analysis. Pulled from orphan pool (in baseline but not in suite): codegate_2017-angrybird, csgames2018, whitehatvn2015_re400. Full suite runs in ~120s; expect ~30+ seconds added per future medium-tier addition.

angr-wnjy (commit be674991d, 2026-05-25): refreshed 10 fast-tier baselines after env drift accumulated since the May-01 lock (commit 15395dd6b). New values (rust_time): ais3_crackme 1.11, arm_le_branch 0.39, defcamp_r100 0.32, defcamp_r100__dfs 0.32, flareon2015_2 5.35, google2016_unbreakable_0 1.09, mips64_le_branch 0.56, strcpy_find 0.46, unmapped_analysis 1.07, whitehatvn2015_re400 1.52. Verified 3 consecutive green gate runs. Procedure: 5-sample isolated median via run_single.py per bench; median (not min) preserves gate headroom under concurrent gate load. Subtle finding: defcamp_r100 BFS/DFS showed isolated median = old baseline 0.27 but failed under gate load at 0.31-0.32; refreshing to gate-load value 0.32 is the right answer when isolation and gate-load diverge.

Fresh sub-2s benches (added after the 2026-05-25 angr-wnjy env-drift refresh) need their own gate-runner calibration. Symptoms: gate red on consecutive iters with 30-40% regressions on the new bench(es), local 5-sample range tight (<0.05s on the bench's scale), no perf-relevant code changes in the iters between the green and red gates. Cause: bench was baselined in isolation on a quiet machine; the gate runner is consistently slower because of contention noise. Fix tactic: set baseline to gate_max / 1.15 + ~5% safety buffer (mirrors angr-wnjy / benchmark-baseline-refresh-tactic). Concrete sizing example (angr-zzfk, 2026-06-03): cmu_binary_bomb_partial gate 1.82-1.86s -> baseline 1.65, android_arm gate 0.33-0.34s -> baseline 0.30, mips64_be_branch gate 0.62-0.63s -> baseline 0.57. SLA warns (rust slower than python) are acceptable; only SLA fail (<0.5x speedup) blocks.

Pattern for handling 'pre-existing flake' baseline drift on fast-tier benches (<0.5s): if 5-sample median is stable (range <0.03s) and bisect points to a non-revertible feature commit, REFRESH baseline to median rather than chasing diffuse perf recovery. Document bisect result + non-revertibility in commit msg. Precedent: angr-40k3 closed via 31a7e1846 (defcamp_r100 0.229 -> 0.27).

CADET_00001 CGC bench: Python 22s; Rust >90s timeout because CGC syscall ABI (transmit=2, receive=3, fdwait=4, allocate=5, deallocate=6, _terminate=1, random=7) has NO Rust impl in native/angr/src/procedures/. Every syscall trampolines to Python — hence the timeout. Binary is CGC format (magic '7f43 4743' = '.CGC'), not ELF. save_unconstrained IS supported by RustExplorationManager (added in earlier work). Added to baseline_timings.json with python_time=22.08, rust_time=null, peak_memory_mb=723.0. NOT in REGRESSION_SUITE (gate still 18/18 green). Tracked for future Rust CGC support via angr-krp1.

CADET_00001 (DECREE) under Rust engine (commit e16ca3bf3, post angr-rdgs): all seven CGC syscalls now native. Buffer-overflow phase reaches unconstrained in 3 steps / ~0.1s. Easter-egg phase (sm.explore(find=0x804833E)) still requires deeper exploration than a 60s subprocess can complete locally — the binary forks heavily on symbolic stdin bytes before reaching the easter egg. Two remaining gaps before the bench can be promoted to baseline_timings.json with rust_time=null: (1) posix.dumps(0) over the Python-mirrored state still returns b'' because Rust-side cgc_receive_* symbolic bytes are not synced into state.posix.fd[0] — needs the same stdin_symbols → posix.fd plumbing the Linux read syscall already has. (2) longer-running easter-egg exploration measurement (probably 30-60s budget) to confirm the new allocate/deallocate handlers don't introduce path-explosion regressions. Both gaps are characterization work, not correctness blockers.

angr-027h iter63: committed the FIRST half of the fix (ed600427b): break the VEX block-chain at address-based find/avoid targets. The interpreter chains blocks inside one run_until_event call; stepping.rs already forced steps_limit=1 for CALLABLE find/avoid predicates but address-based find/avoid had no guard, so a state could chain past a find target and run_loop's step-boundary filter only saw the final pc. Fix: manager maintains union stop_addrs set (rebuild_stop_addrs in set_find_addrs/set_avoid_addrs), passed to run_until_event(.., stop_addrs); loop breaks when blocks_executed>0 && stop_addrs.contains(pc), falling through to MaxBlocks return. CADET STILL NOT SOLVED by this alone (active=0 immediately — the easter-egg target 0x804833E is NOT on the main chained path; it is reached only via a deferred break-fork). THIRD issue discovered: materializing deferred_forks at the UnconstrainedJump arm (route main->unconstrained, return forks via handle_block_end) NOW bounds divergence far better than iter-28 thanks to the chain-break (active plateaus ~210 then climbs slowly vs iter-28's 493@step300) BUT still never reaches 0x804833E and active grows unbounded (351@step194) -> reverted, left as Err(Unconstrained). Conclusion: the easter-egg block is UNREACHABLE via deferred-fork resumption; the forks resume in the strlen/overflow region and re-go-unconstrained. Next investigator must find WHY no fork takes byte0==0x5e -> palindrome -> 0x804833E. Verify safely with /tmp/cadet_verify.py pattern: mgr.explore(find=0x804833E, max_steps=0) then bounded mgr.step(n=1)+stash_counts loop under systemd-run --user --scope -p MemoryMax=4G -p MemorySwapMax=0.

angr-027h CADET convergence — why per-state eager-at-UnconstrainedJump is INSUFFICIENT (iter66). Implemented: deferred forks in the block that goes unconstrained are now materialized in EAGER mode (per-state force_eager_forks flag in state.rs, honored in stepping.rs::run_interpreter_step by overriding ExecutionConfig.use_deferred_forks) and routed to active via StepError::Unconstrained(state, forks) instead of dropped. Result: forks flow (active grows ~1/step, unconstrained stays 1, pruned=0) but CADET find=0x804833E does NOT converge in 300 steps. ROOT CAUSE of the gap: handle_block_end materializes deferred forks at EVERY block end (in deferred mode) throughout the loop dive — the loop-exit forks for early iterations were already materialized as DEFERRED states long before the main chain reached the unconstrained ret. My eager fix only catches forks deferred WITHIN the final pre-unconstrained block, a tiny subset. The bulk of egg-reaching loop-exit forks are deferred-mode and re-dive loops (iter63 divergence). iter65 global-eager converged (step 38) because EVERY block forked eagerly from entry. NEXT: to match global-eager without globally regressing fast benches, the eager flag must be set on loop-exit forks materialized at block-ends too WHEN a find target is active and the fork's lineage is a deferred loop exit — i.e. seed force_eager_forks at handle_block_end for find-driven runs, scoped so non-find benches stay deferred.

angr-027h CADET easter-egg: PROVEN (iter65) that eager forking reaches the egg target 0x804833E — found=1 at step 38, active stayed bounded ~27 — so the Rust engine logic is CORRECT and the prior 'third issue / no fork takes the byte0==0x5e path' pessimism is WRONG. The convergence blocker is PURELY deferred-fork handling at the unconstrained ret (stepping.rs UnconstrainedJump arm drops deferred_forks). Experiment method: env-gated c.use_deferred_forks=false in exploration/mod.rs SymbolicExplorer ctor (reverted after), then /tmp/cadet_eager.py under systemd 4G stepping mgr.step(n=1) with find=0x804833E. WHY iter63's materialize-at-unconstrained still diverged despite reachability: resumed loop-exit forks continue in DEFERRED mode and dive the (also-symbolic) palindrome loop, re-overflow, re-go-unconstrained, re-materialize -> recursive divergence. FIX DIRECTION (revised): materialized deferred forks at unconstrained must be resumed in EAGER fork mode (per-state use_deferred_forks=false) so they BFS cleanly to the egg without recursively re-deferring. NOT viable: globally disabling deferred forks when find_addrs set (would regress all find-based fast benches; deferred forks are the perf optimization). Eager forking also exposed a latent bug fixed this iter (see cleanup-state-cache-set-mutation-bug).

CADET easter-egg (angr-027h) root cause, confirmed iter64 with full CFG disasm of sub_80481a0 (the fn containing find target 0x804833E). STRUCTURE: fn reads 0x80 bytes via receive @0x8048211 into [ebp-0x54] (overflows saved retaddr at ebp+4 -> every path that reaches the ret @0x8048352 goes UNCONSTRAINED). To reach find target 0x804833E you must: (1) terminate the symbolic strlen loop (je 0x804826f @0x804824e, guard=input[i]==0) -> FORWARD branch so deferred-fork mode TAKES fallthrough (input[i]!=0, keep looping) as MAIN and DEFERS the exit; (2) complete the palindrome compare loop (jg 0x8048301 @0x80482be); (3) hit easter-egg gate cmp eax,0x5e / jne 0x8048341 @0x804830a where input[0]==0x5e is the FALLTHROUGH -> writes egg, returns to 0x804833E. So the easter-egg is ALWAYS on the DEFERRED side of the strlen-loop exit. CONFIRMED RUNTIME (current HEAD ed600427b, /tmp/cadet_verify.py under systemd 4G): mgr.explore(find=0x804833E) goes active=1 for 3 steps then active=0/unconstrained=1 and STAYS there forever -> the main chain dives the strlen loop, overflows to the ret, goes unconstrained, and ALL deferred forks are DROPPED at the UnconstrainedJump (handoff bug #1). Net: zero exploration of egg branches. iter63's attempt to materialize deferred forks at UnconstrainedJump caused unbounded divergence (active->351) because resumed forks re-enter the loop nest and every non-egg path re-overflows to unconstrained -> reverted. FIX DIRECTION: deferred forks at the unconstrained ret must become PENDING/active manager states explored under find-driven search (not re-chained), AND find-check must fire when a resumed egg-fork's block starts at 0x804833E (chain-break already added for that). Likely needs a search-order/pending-stash change in exploration/, not just materialization.

CADET_00001_partial easter-egg explore(find=) is BIMODAL under Z3 model nondeterminism: 4 samples clustered into ~6.8s/411MB (fast) and ~8.2s/545MB (slow) modes. Added to BIMODAL_BENCHMARKS in run_regression.py; baseline_timings uses the slow mode (rust_time=8.24, peak 545) so the +15% gate tolerates both. In MEDIUM_SUITE (nightly --full only, not PR gate).

CADET_00001 (DECREE/CGC) partial-run characterization captured under run_single.py post-angr-qcsg (commit edde63e2b, 2026-06-06): Rust engine runs buffer-overflow phase in ~0.1s emitting 3 steps and 3 native simprocedures (the CGC syscalls allocate/transmit/receive — all native handlers per angr-rdgs), then solve.py crashes at sm.found[0] in the easter-egg phase with IndexError at ~4.3s. Counter snapshot at crash: steps=3, simprocedures=3, syscall_python_fallback_count=0, simprocedure_python_fallback_count=0, callback_syscall_count=0 (handled natively, never called back to Python), callback_lift_block_count=27, block_cache_hits=266. Note this REINFORCES vx8p secondary acceptance gap: even with partial-stats visibility, syscall_python_fallback_count is still 0 because all CGC syscalls already have native handlers — the criterion of 'syscall_python_fallback nonzero on at least one new bench' would only flip if we added a bench that exercised an UNIMPLEMENTED syscall path. See bd memories bench-syscall-fallback-zero-coverage and benchmark-cadet-cgc-partial-unblock.

angr-ckdy FIXED (commit 98d4fa5e1): the CADET solve.py phase-3 step-loop (while True: sm.step(); break if active.addr==0x804833E) now converges under Rust via TWO opt-in flags together. (1) set_block_granular(True) [angr-bmyx] makes the egg block observable at a step boundary. (2) set_materialize_unconstrained_forks(True) [angr-ckdy]: in deferred mode the UnconstrainedJump arm in stepping.rs DROPPED the loop-exit forks (Vec::new + deferred_forks_dropped counter), collapsing active to empty so the step-loop spun forever. New manager field materialize_unconstrained_forks (mod.rs, default false) gates the drop: the deferred-arm condition is now 'use_deferred_forks && !materialize_unconstrained_forks'; when the flag is set the else branch runs materialize_deferred_forks(force_eager=true) so forks route to active. pymethods set_materialize_unconstrained_forks/materialize_unconstrained_forks on RustExplorationManager + Python wrapper RustExplorationManager.set_materialize_unconstrained_forks(enabled=True). Validated: egg found ~step 545, active capped ~70 — does NOT explode to 310/segfault because block-granular observability lets the loop break BEFORE the fork explosion (blocker #2 in avoid-cadet-phase3-sticky-eager-retry is sidestepped, not by bounding forks but by reaching the egg first). Default off => explore() active_empty phase-2 trigger (angr-027h) and 19/19 fast-tier benches unchanged. Heavy egg-hunt regression test is env-gated behind ANGR_RUN_SLOW_CADET=1 in test_misc.py TestCadetEasterEggStepLoop; toggle plumbing test always runs.

027h CADET convergence — why ALL post-hoc/surgical eager approaches fail (iter67, definitive). In deferred-fork mode the VEX interpreter runs the ENTIRE symbolic loop within a SINGLE step_state call: at CADET step 3 active jumps 1->71 in one interpreter run (the strlen loop unrolls ~70 iterations, each deferring its loop-exit fork, then the main chain overflows the saved retaddr and goes unconstrained). So by the time the FIRST unconstrained state is observable, the wide shallow pile of ~70 deferred exit-forks ALREADY exists. Flipping those active states (or new forks) to force_eager_forks AFTER the unconstrained event (the eager_after_unconstrained_find latch, iter67) does NOT converge: active grows to 146+ with pruned=0/deadended=0 and never reaches 0x804833E in 80 steps. Same failure shape as seeding force_eager at handle_block_end (iter66/iter67). The ONLY thing that converges is eager-from-START (global use_deferred_forks=false from step 0): active stays bounded ~27 (loop explored iteration-by-iteration with loop-bound UNSAT pruning), egg found at step 38. CONSEQUENCE: a latch/flip can never work because eager must be active BEFORE the loop dive, and the dive completes in one deferred step. global-eager-when-find DOES converge but regresses whitehatvn2015_re400 by 73% time / +241% mem (fails the 0.15 CI gate) — whitehatvn produces ZERO unconstrained states (verified) and converges fine in deferred mode, so it must not pay the eager cost. CORRECT FIX = two-phase explore: phase1 deferred (fast, whitehatvn finds here, never stalls); on active_empty with found==0 and find_addrs set, phase2 re-seeds the initial states with eager forking. Needs: snapshot the active stash (or retain initial SimStates — currently NOT retained, _phase_activate consumes them) before phase1, a manager eager toggle, and Python _explore_with_addresses retry logic. Note iter66's fork-flow-at-unconstrained must be gated so the deferred phase actually EXHAUSTS (hits active_empty) instead of growing.

angr-zbpw0 (2026-07-14) re-measured the two Callable-only benches once they actually ran on Rust (before, both measured the Python engine): flareon2015_10 = 2.8s Rust vs 7.3s Python (~2.6x FASTER, old baseline said 5.4s); mma_howtouse = 6.8s Rust vs 4.3s Python (~0.63x, Rust SLOWER — per-Callable manager construction dominates: 45 Callable invocations each build a fresh RustExplorationManager, so mgr.stats only reflects the LAST one). The old 6.5s 'rust' number for mma coincidentally matched, so the 0.65x figure in benchmark-mma-howtouse-cprofile-attribution was right by accident on a Python-engine measurement. baseline_timings.json rust_time updated for both.

9w6ad.1 baseline refresh (iter55, 2026-06-19): run_regression.py --full --rust-only --update-counters is SAFE in the 6G ralph loop despite the broad 'DEFER 9w6ad — OOM' warning — each bench runs in a subprocess capped at RLIMIT_AS 4GB (run_regression.py imports the run_single subprocess runner; see _run_bench around line 297/388), so the parent collects results only. Full fast+medium sweep took ~133s wall. --update-counters refreshes baseline_counters.json ONLY (never baseline_timings.json), so it dodges the avoid-update-baseline-without-verification tight-timing trap and is the right tool for a counters-baseline refresh. Sanity-check pattern that worked: diff old vs new STRUCTURAL counters only (exclude any key containing ns/time/nano/elapsed/micros/millis — those are pure run-to-run noise and dominate the raw line diff); a clean refresh shows ~0 structural deltas >20% & >=10abs. The --full run added counter baselines for the 5 bimodal benches that --skip-bimodal omits (CADET_00001_partial, sokohashv2, unbreakable_1, angry-reverser, fairlight).

cow_fork_scaling has BIMODAL Rust timing (fast ~2.2s / slow ~2.7s, from Z3 SAT-search nondeterminism over its 256 fork-leaf constraints) but STABLE counts (steps=513). Its baseline_timings rust_time is pinned to the SLOW mode (2.7, was 2.28) so the 0.15 regression gate tolerates both modes. It is deliberately NOT in BIMODAL_BENCHMARKS: that frozenset drops a bench from the whole --skip-bimodal gate, and the steps=513 count signal is this bench's entire purpose (the O(1) CoW-fork claim). The fast-mode 2.28 baseline caused a false gate failure on iter-77's doc-only commit (2.64/2.70/2.72 across 3 retries). Same slow-mode-baseline convention as the CADET_00001_partial BIMODAL note. See angr-5cx4r.

Criterion benchmark baselines (2026-04-18, cargo bench --bench vex_engine): RustBV concrete add/sub/extract ~7ns, concat ~8.5ns, reverse ~18ns. Symbolic add/concat ~76ns, extract/reverse ~45ns. build_z3_ast (complex expr) ~3µs. SymContext fork: 150ns (2 constraints), 225ns (5), 806ns (20), 1.98µs (50) — linear scaling ~38ns/constraint. check_branch_feasibility ~110µs (dominated by Z3 check()). assume_true ~2.2µs. push/pop ~650ns. Memory concrete load 25ns, store 53ns. Symbolic load 16-range 1.6ms. Memory fork 16 pages 39ns (im::OrdMap CoW). State fork 256ns.

csgames2018 regression fixed in commit 7fe7baf79 (2026-05-09). Before: timed out >90s. After: 0.96s (baseline). The fix is one-line: live |= set(self._state_roots.values()) in _cleanup_state_cache. State 0 was being evicted from the Python _state_cache by the live filter.

fairlight benchmark progression (Rust engine): baseline 24.36s (2026-05-01) -> 10.65s after Z3 model-cache optimization (commit 44fdf6450). Profiling: z3_check went 23.4s -> 8.5s (-63%); branch_false 12.5s/15 calls -> 2.6s/2 calls; branch_true 9.8s/15 -> 4.1s/13. branch_model_hit rate: 100%. satisfiable cost grew slightly (0.98s -> 1.76s) due to get_model() call. Next bottleneck: still in branch_true at 316ms/call avg — bit-blasting on deep ITE-chains is intrinsic. LAZY_SOLVES enabled gives 0.65s upper bound (25x), so further wins possible but require correctness proofs.

fairlight 20-sample re-validation campaign (angr-hyiz.2, HEAD 607eed29e, 2026-05-18). 20/20 runs OK (no failures). Distribution: 7 fast (7.97-8.33s) / 13 slow (21.48-22.72s). Summary: min=7.97s median=21.55s max=22.72s mean=16.96s stdev=6.68s. Compared to 2026-05-13 campaign (5 fast / 15 slow, median 21.39s, max 21.53s): structurally identical — fast and slow modes intact, fast-mode rate noise-level (25%->35%), slow-mode ceiling drifted +1.2s, median +0.16s. Conclusion: fairlight is at the Z3 structural floor for this engine. Action: doc-only update to docs/advanced-topics/rust_bimodal_variance.rst with new section. Baseline 22.0s left unchanged: max 22.72s sample is within 15% threshold (25.3s), and per avoid-update-baseline-without-verification we do not tighten on single-session data. Cached python_time=15.756s gives blended-mean speedup 0.93x (warn but not fail in SLA gate). angr-hyiz.2 closed via doc characterization path; no further work without LAZY_SOLVES correctness proofs or x87/bit-blasting changes.

fairlight optimization: incremental branch assertion reduced from 15.2s to 14.0s (8%). Z3 solving dominates — exits take 56ms, 77ms, 283ms, 3806ms, 3175ms, 3978ms. The jump at exit 4 is Z3 SAT solver phase transition. check_assumptions was 2x WORSE. push/pop is O(1) and preserves Z3 bit-blasting cache.

fauxware benchmark (2026-05-11, HEAD=92e22981f, 5-run median): rust 0.28s, python 0.38s, speedup 1.36x. 1 SimProcedure callback (open), 14 lift_block callbacks, 0 memory_load/fetch_page callbacks. Was 0.385s rust before angr-3tek.2.

Per-combo cargo test counts (post-7c9j, 2026-05-01): '' = 44 unit + 2 smoke + 0 doc; 'automaton' = 59 + 3 + 0; 'vex-engine' = 346 + 3 + 0; 'vex-engine,vex-engine-z3' = 349 + 4 + 0; default (vex-engine,vex-engine-z3,automaton release) = 364 + 5 + 0. Use these as a baseline to detect missing feature-gated tests.

Witness seeding for min/max binary search (commit d8321b6d7, 2026-05-19): flareon2015_5 5.92s->3.40s (1.74x). The wall-time savings dwarfed the SAT-count reduction because the saved checks were the slow ones: z3_site_min_search dropped from 2126ms (4074 calls @~520us avg) to 122ms (2420 calls @~50us avg) — fewer calls AND each call ~10x faster. Tighter search range means each intermediate bvule/bvsle constraint is closer to feasible, making Z3's internal SAT problem easier. csaw_wyvern 3.01s->2.90s (1.04x) was much smaller because it has 10x fewer min/max calls.

Iter 4 (2026-05-23) baseline drift: flareon2015_2 rust_time was 3.62s, drifted to 4.37s isolated median (10-sample). Same pattern as commit 868be2609 (unmapped_analysis 0.789s -> 0.92s). To verify drift is pre-existing and not caused by a code change: stash, checkout HEAD1 of changed source files only (keep test changes), rebuild with 'make rebuild-cargo', sample the bench 3x in isolation. HEAD1 measurement matched HEAD measurement (~4.54s in this case), confirming drift predated the new commit. Refresh policy per gate-noise-floor-15pct: isolated 10-sample median is authoritative — refresh baseline rather than hunting for a code fault when the failing bench is sub-second or below 5s, when isolation runs match the regression, and especially when the new code is logically inert (AtomicBool field init + load is too cheap to cause 20%+ swings).

SymContext::fork criterion bench (post-w6nq, 2026-05-07): 5/20/50 constraints all ~208ns — fork is now O(1) regardless of constraint count. Pre-optimization baseline (2026-04-18): 5=225ns, 20=806ns, 50=1980ns (linear scaling ~38ns/constraint). 50-constraint case 9.5x faster. Mechanism: Mutex<Arc<Vec>> for shared, Arc::get_mut + append to drain local when uniquely owned.

Full baseline (2026-04-19, 16 benchmarks, commit a3808bda8): csaw_wyvern=15.6x(Py:16.4s,Rs:1.05s), ekoparty=7.8x(32.4s/4.1s), flareon5=5.9x(39.5s/6.7s), ais3=2.9x(2.5s/0.86s), whitehatvn=2.5x(3.1s/1.25s), codegate=2.4x(6.9s/2.8s), sym-write=2.2x(0.98s/0.45s), csgames=1.6x(1.6s/0.97s), google0=1.5x(1.4s/0.9s), defcamp=1.5x(1.0s/0.7s), flareon10=1.3x(7.5s/5.5s), flareon2=1.1x(4.1s/3.8s), fauxware=0.9x(0.38s/0.42s), google1=0.9x(3.8s/3.5s-high-variance), strcpy=0.2x(0.84s/4.0s), fairlight=0.4x(9.8s/23.7s). 16/16 correct, 12/16 faster.

FxHasher swap on 7 CallbackInterpreter per-step HashMaps (interpreter_cb/mod.rs) + PendingStoreBuffer.byte_index (pending_store.rs): state_fork 204.59 ns -> 199.66 ns (-2.1%, p<0.05). memory_fork_16pages 39.97 ns -> 39.22 ns (-2.0%, recovered last session drift). Memory store/load unchanged. The smaller win vs prior FxHash sessions (state_fork -20% on RegisterFile.symbolic+HeapMetadata.allocated) is expected: state.fork() doesn't clone the interpreter — these maps live inside CallbackInterpreter and only matter on per-step insert/lookup. The fork bench picks up only the secondary effect of FxHashMap's smaller default capacity on memory bench noise. The real wins from this swap show up under interpreter step throughput, which we don't have a microbench for. Commit 76c76c1b3 (angr-07rg).

FxHasher swap on RegisterFile.symbolic (HashMap<u32, RustBV>, arch/mod.rs:95) and HeapMetadata.allocated (HashMap<u64, u64>, state.rs:47): cargo bench state_fork dropped 256.29 ns -> 204.59 ns (-20.3%, p<0.05). memory_fork_16pages drifted +4.3% (40 ns workload, allocator/heat noise). Other benches unchanged. Why state_fork wins so much: state.fork() clones two FxHashMaps that the previous angr-75y3 swap had not yet migrated. The combined effect of the two swaps lands the headline fork win, not just one in isolation. Commit 26a9a6676 (after angr-75y3 df3a5c53e).

Benchmark gate is single-sample, so 15%-20% jitter is normal noise on the fast-tier benches. First two of three back-to-back run_regression.py runs after the angr-febn deletion (which doesn't touch the live exploration path) reported 5 then 1 regressions; the third run was 0/15. When attributing perf changes from a comments-only / dead-code-deletion / pub-method-addition commit, average 3+ samples or run 'make bench-single EXAMPLE=' for the specific bench to confirm noise vs. signal. Don't bisect a commit that has no semantic impact on the run loop.

After angr-lbze fix on 2026-05-06, the benchmark gate genuinely runs and reports passed=11 failed=1. The single real failure is 'csgames2018: Rust engine failed: list index out of range' (pre-existing, separate bug). Everything else in the FAST_SUITE passes within the timing baseline as of HEAD a6b4f6615. defcon2016quals_baby-re showed a 27% timing regression on one manual run (0.91s vs 0.71s baseline) but the orchestrator-driven run did not flag it — likely variance; watch this benchmark.

google2016_unbreakable_1 regressed from 3.3x to 0.46x speedup (Rust now slower than Python). Other significant deltas vs old CLAUDE.md table: defcamp_r100 1.5x→4.4x (improved), securityfest_fairlight 0.4x→1.0x (parity reached), ekopartyctf2016_rev250 7.8x→15.7x (improved). The unbreakable_1 regression is the most surprising — old CLAUDE.md table noted 'high variance' but flip from 3.3x faster to 2x slower is more than variance. May warrant inclusion in angr-ed7j regression investigation alongside mma_howtouse and ekopartyctf2016_sokohashv2.

angr-tlvl hypothesis (2026-05-19): the bead's premise was that hackcon2016_angry-reverser's residual 0.6x is caused by the Reverse-leaf-case emitting a Concat(extract[7:0,x], …, extract[N-1:N-8,x]) that Z3's bv_rewriter (bv_rewriter.cpp:1656-1667) cannot fold. Fixed the related semantic bug AND emission now matches claripy's canonical shape exactly. Measurement: 5-sample paired campaign on HEAD bbdb2451e → Python median 10.29s (range 10.21–10.32), Rust median 14.84s (range 14.63–15.10) → 0.69x. Pre-fix baseline from memory hackcon-regression-root-cause-fced54a07 was 3-run median 15.22s. Delta ~0.4s, within noise / stdev=1.33s from variance memory benchmark-hackcon2016-2026-05-17-variance. Hypothesis INVALIDATED. The leaf case is not reached often enough in hackcon — its constraints over the flag BVS are built from byte-wise memory loads (hitting the Concat path which was already correct) and Extract(Reverse) sites (which hit Rule 3 canonicalization at value.rs:1568). Next plausible cause: rustbv↔claripy round-trip in constraint export/import produces structurally different concat trees on the flag BVS path (per hackcon-z3-ast-structure). Reopen criteria: actual 1-sample run on hackcon shows >2s improvement (a 14% delta clears the variance band).

hackcon2016_angry-reverser 2026-05-17 variance campaign: 10/10 runs at 28.82–32.54s, median 30.79s, stdev 1.33s. UNIMODAL (not bimodal). Tool: tests/benchmarks/bimodal_variance.py --benchmarks hackcon2016_angry-reverser --runs 10. Profile: z3_check_count=1 with the single solve consuming ~28s/30s wall. Explore phase: block_exec 0.13s, lift 0.19s, expr_eval 0.003s, run_loop 0.13s, python_callback 0.18s (400 lift_block calls), solver_fork 0.12s. Python time unchanged at ~10.5s — so the regression vs 11.722s 2026-05-09 baseline is Rust-engine-specific. Baseline raised to 33.0s. Bisect filed as angr-8t45.

insomnihack_aeg solve.py at angr-examples/insomnihack_aeg/solve.py is blocked from Rust benchmarking by TWO issues, not just the historical 'uses sys.argv in main' note: (1) solve.py uses sim_options.TRACK_ACTION_HISTORY which is in angr/exploration/rust_manager.py:_RAISE_OPTION_NAMES — Rust engine raises NotImplementedError immediately; (2) even Python engine completes the AEG search logic in ~2s but stalls in ep.posix.dumps(0) over symbolic stdin (90s harness timeout exceeded). Iter 81 (2026-06-03, commit 5e87a3064) added harness argv injection so we can SEE the TRACK_ACTION_HISTORY raise — argv was a red herring blocker. Bead angr-86c4 marked blocked with three unblock paths documented. Wider lesson: when an AEG-style benchmark candidate's solve.py adds TRACK_ACTION_HISTORY 'just in case', check whether the option is actually used (insomnihack_aeg doesn't reference state.history.recent_actions anywhere, just REVERSE_MEMORY_NAME_MAP via addrs_for_name).

insomnihack_aeg bench characterized post-TRACK_ACTION_HISTORY fix (commit 974551efa, angr-fkvt). Rust path now reaches state-exploration phase but hits deterministic OOM at ~75s under 4GB RLIMIT_AS cap. Second run reproduced same outcome: gets to 'found some unconstrained states, checking exploitability' before OOM. Python completes AEG logic in ~2s but hangs >180s in posix.dumps(0) over symbolic stdin (BVS bytes get realized into the constraint solver one by one). Bench is NOT viable for the regression gate as-is: Rust exceeds the 4GB memory budget. Path forward would require either (a) Rust-side AEG memory optimization (state interning, action-history compaction, or sparse register tracking), or (b) picking a smaller AEG-pattern bench. Catalog note at run_single.py:74 updated 2026-06-06 to reflect new failure mode. Task angr-86c4 closed with this finding.

Don't expect angr-kol7 batched add_constraints to move the fast-tier bench matrix. 2026-05-20 measurement: 16/16 benches within ±0.01s of baseline. Reason: most assertion volume on those benches flows through assume_true/assume_false (one-at-a-time), not multi-constraint Python add_constraints batches. The batched path is a structural win (3 locks vs 3N for N-batches) that should help workloads that arrive as PyLists — post-callback constraint sync from Python, store-guard chains feeding RustSolverContext.add_constraints — but the existing fast-tier picks don't stress that surface.

P4-spike-B LAZY_SOLVES engagement audit (angr-9w6ad.11, commit on docs/advanced-topics/rust_engine.rst 'P4-spike-B' subsection). CONCLUSION: no eager-solve outlier, no code change. (1) LAZY_SOLVES is NOT a default SimOption (blank/entry/full_init_state all lack it), so bucket B benches (angry-reverser/fairlight/sokohashv2/unbreakable_1) run with constraint_solver.lazy_solves=false -> eager per-successor feasibility. This MATCHES Python: angr/engines/successors.py add_successor prunes via 'o.LAZY_SOLVES not in state.options and not state.satisfiable()'. (2) Every step-path .satisfiable() in native/angr/src/exploration/{run_loop,resume,stepping}.rs is guarded by 'lazy_solves || ...'; interpreter check_branch_feasibility (interpreter/statements.rs) short-circuits to (true,true) when lazy. Only UNGATED .satisfiable() are 3 explicit API queries in exploration/state_api.rs and a necessary defensive re-check on the find-successor push in run_loop.rs (straight-line find states are never interpreter-checked). (3) Rust active path solves LESS than Python: straight-line successors push via push_to_active_or_drop (helpers.rs) WITHOUT re-solve; same constraints as already-feasible parent => safe. INVARIANT: do NOT engage lazy_solves to speed up bucket B -- it would diverge from Python (unsat states survive in active) and break correctness. Z3 floor = shared eager feasibility checks at fork points, structural not engine-fixable.

Nightly RSS leak gate (tests/benchmarks/run_leak_check.py) healthy ratio measured on the RUST engine (2026-07-14, angr-pm8ha, after the engine swap was actually installed): mma_howtouse N=10 peak RSS 285MB -> 301MB, ratio 1.054, ~1.7MB/iter, 450 Callable-built RustExplorationManagers (45 per main()). Wall time creeps 6.76s -> 8.65s across the 10 iters (~28%) even though RSS is flat -- steady-state cache growth, not a leak; worth a look if it ever becomes superlinear. The 1.5x --threshold default was tuned on Python-engine numbers of the same order, so it carries over unchanged and still catches a >=2x regression with margin.

Benchmark impact of flipping libvex-ffi default-ON (angr-3trr7, commit 807bfd74d): refreshed baseline_timings.json via run_regression.py --rust-only --skip-bimodal --update -- ALL 22 fast-tier non-bimodal benches got FASTER on the native-lift build. Biggest AMD64 wins: cmu_binary_bomb_partial -44%, fauxware -41%, defcon2016quals_baby-re -38%, flareon2015_2 -32%. Non-AMD64 arch-branch benches also shifted 8-17% faster (native lift is AMD64-gated, so a chunk of that is machine variance, not libVEX) -- fine, the 15% gate absorbs it and --update sets baseline=current. --skip-bimodal left the 5 bimodal SLOW-mode pins (CADET_00001_partial 8.24, sokohashv2 16, unbreakable_1 3.5, angry-reverser 35, fairlight 22) UNTOUCHED, respecting default-flip-requires-baseline-refresh's warning against --update repinning bimodals. baseline_counters.json also refreshed (callback_count dropped to 0 on AMD64 benches now that cold blocks lift natively).

ais3_crackme dispatches 47 SimProcedures to Python (after angr-md0m perf_report). fauxware dispatches 5. defcamp_r100 dispatches 0. None of these trip Dirty or VEX-op fallback. This means the 'silent VEX op fallback' path (unop/binop/triop/qop OpError → synthesize symbolic) is rarely hit on the basic benchmark suite. If a future regression suddenly shows nonzero rust_python_vex_op_fallback_count for fauxware/ais3/defcamp, that's a strong signal a VEX op handler regressed.

mips32_le_branch added to FAST_SUITE 2026-05-14 (commit 977708da8). MIPS32 promoted Experimental -> Supported. Synthetic inline-ELF (no angr-examples binary needed). 10 MIPS instructions; sll + 2x addiu + beq comparator on symbolic a0; solution a0=42. rust_time=0.33s on local box; python_time=null in baseline so SLA gate is skipped. Total benchmark count: 23.

mips64_le_branch synthetic benchmark added 2026-05-17 (commit beba3a961) for angr-duta.4. Inline ELF64 LE (EM_MIPS=0x08, EI_CLASS=2/ELF64, EF_MIPS_ARCH_64=0x60000000), ~10 instructions: SLL t0,a0,1 + ADDIU t0,t0,0x10 + ADDIU t1,zero,100 + BEQ t0,t1,+4 + B +3 + delay-slot NOPs + FOUND/AVOID NOPs. Symbolic 64-bit $a0 driven through (a0<<1)+16==100, solver picks a0=42. PT_LOAD covers entire ELF; BASE=0x400000, ENTRY=0x400078. rust_time baseline 0.46s, peak 252MB. Wired into FAST_SUITE as rust_only=True (short program, PyO3 init tax dominates). Total benchmark count after this is 26 (22 angr-examples + 4 in-repo synthetics: ARM, MIPS32, MIPS64, AArch64). MIPS64 now at Supported in the arch matrix.

mma_howtouse post angr-9maq + angr-gra3 validation (HEAD 17ab6787a, 2026-05-17): Rust 7.32s/277MB vs Python 4.29s/226MB = 0.58x speedup. cleanup=True vs cleanup=False both 7.3s (within noise). The angr-518z cleanup hook is functionally a no-op on this workload because (a) claripy.clear_all_caches() no longer exists (WeakValueDictionary caches GC naturally), and (b) Rust-side LRU AST_CACHE (size 10000) doesn't fill in 45 Callable invocations. Gap to Python is now in some other path — not AST cache lookup overhead.

mma_howtouse 5-sample paired re-validation on HEAD 260eb4d66 (angr-hyiz.3, 2026-05-18): Python median 4.33s (range 4.20-4.42s, peak 226MB), Rust median 7.39s (range 7.25-7.45s, peak 276-277MB). Speedup 0.59x, 5/5 OK, very tight variance — not bimodal. Tracks the 2026-05-17 validated picture (benchmark-mma-howtouse-2026-05-17-final: 7.32s/4.29s=0.58x). baseline_timings.json rust_time of 6.513s pre-dates angr-9maq/gra3/8t45 wave; current Rust sits 13% above (under 15% threshold), baseline intentionally left in place per the sokohashv2/unbreakable_1/fairlight precedent. AST cache hypothesis (cleared_all_caches23% gain claim) is invalidated and the docs now say so. Re-open angr-hyiz.3 only if a per-manager AST cache scoping mechanism is being designed OR a fresh profile attributes the gap to a Rust-side hot path.

mma_howtouse after angr-b58a (commit 47bf6284b, 2026-05-20): rust median ~6.26s (5.93/6.50/6.02/6.26s samples) vs Python 4.37s = 0.70x speedup (was 0.59x at 7.23s before). Per-Callable _sync_extra_python_pages dropped from 35.26ms to 1.84ms (19x). 92,610 candidate pages × 14.65us/page any() cost was the dominant culprit; replaced with UltraPage concrete_data memcmp + bytes-deferred-until-FFI + batched add_lazy_regions_batch single-call. Peak mem unchanged (~277MB). 92,430 of 92,610 candidates are all-zero — mma's signature workload.

mma_howtouse (callback-heavy, 0.65x — Rust SLOWER) re-measured 2026-06-15 post-d1dr via cProfile in capped subprocess (tests/benchmarks/run_single.py _run_in_child + cProfile). The page-init cost is NOT a SimProcedure resume artifact: callback_count=1 and the ENTIRE callback/resume path is 8.4ms (callback_simprocedure_total_ns) of a 5.94s run; resume_after_simprocedure is no longer measurable (d1dr CoW fork likely absorbed it). The real driver: solve.py runs an angr.callable.Callable 45 times — callable.py:88 Callable.call = 88% of wall, each invocation runs factory.call_state() (factory.py:164, 5.4s cumulative) building a FRESH state whose lazy page backing re-inits ~2,060 ultra pages. Total 92,700 _initialize_page calls, 1.96s tottime concentrated in DictBackerMixin._initialize_page (page_backer_mixins.py:240) which LINEARLY SCANS _dict_memory_backer once per page. Both engines pay this identically (it is angr-core Python). Implication: 'cache Python state across SimProc round-trips' (the old dxpd/d1dr framing) was the WRONG lever. True lever = reuse call_state base or memoize CLE page backing across Callable invocations — general angr-core, shared by both engines, would NOT move the 0.65x ratio, outside rust-symex scope. angr-dxpd CLOSED with this finding (commit 64ed3c22d). Supersedes the '45 callback resumes' attribution in benchmark-mma-howtouse-cprofile-attribution.

mma_howtouse memory leak fix (commit 342df4a7f, 2026-05-02): peak_mem 1606MB -> 285MB (5.6x reduction). Wall time still ~6.5s vs Python ~4.2s (unchanged, the speedup gap is now due to AST cache lookup overhead per mma-howtouse-cache-clear-speedup memory, not memory leak). Test suite 208/208 passing. Other benchmark perf unchanged (fauxware 0.38s, ais3 0.83s, csgames 0.97s).

Initial rustbv_neon_ops criterion baseline (commit 526482ae3, on 8GB Linux host -O3). All NEON ops landed in angr-bkcs.2 (da0966893). Numbers from ./target/release/deps/vex_engine bench:

mul64_concrete_baseline: 18.6 ns (scalar Iop_Mul I64 for reference) mul8x16_concrete: 68.1 ns (~3.7x scalar — 16-lane overhead) mul8x16_symbolic: 2.0 µs (Z3 AST construction dominates) get_elem8x16_concrete_idx: 19.9 ns (bit-slice extract fast path) get_elem8x16_symbolic_idx: 1.3 µs (ITE chain over 16 lanes) set_elem8x16_concrete_idx: 21.0 ns (concrete bit-twiddle) set_elem8x16_symbolic_idx: 1.9 µs (per-lane ITE + concat_le rebuild)

Implications:

• Concrete fast paths are <2x scalar — NEON SIMD on concrete operands is essentially free.
• Symbolic path is 50-100x slower; ITE chain over 16 lanes is the cost driver, not Z3 solve.
• VSetElem is ~50% slower than VGetElem in the symbolic path because each lane must be rebuilt and concat_le'd.

Run via: ./target/release/deps/vex_engine-* rustbv_neon_ops/ Or scoped profile: tests/benchmarks/profile_rust_bench.sh --filter rustbv_neon

New benchmark examples tested (2026-04-13): google2016_unbreakable_0 works (1.0x), google2016_unbreakable_1 works (0.94x). codegate_2017-angrybird: Rust finds state (0.24s vs Py 8s) but OUTPUT IS WRONG — constraint divergence (316 vs 382), UNSAT with correct password. See codegate-constraint-divergence and codegate-session2-findings for root cause investigation. FAILS: cmu_binary_bomb (Python fails), simple_heap_overflow (Python fails libc compat), defcamp_r200 (Python fails), mma_simplehash (both fail). asisctffinals2015_fake: Rust finds state in 0.24s but output empty — symbolic memory stores lost.

5 new examples tested (2026-04-28): mma_howtouse works with Rust (Py 4.3s, Rust 6.6s = 0.65x slower, Callable-heavy with 45 calls to howtouse DLL function). defcamp_r200 broken in Python too (ManualMergepoint 'list index out of range'). CADET_00001 fails Rust: save_unconstrained=True unsupported, syscall execution errors ('ProcedureMixin.process_procedure() got multiple values for argument procedure'). ekopartyctf2015_rev100 Rust timeout: creates 30 separate sim managers with blank_state, sm.run(n=4), sm.step(size=cmp_addr-load_addr2) — Rust engine chokes on run(n=) or step(size=). whitehat_crypto400 Rust 'list index out of range': multi-stage explore with unstash(from_stash='found', to_stash='active') between stages.

Perf-campaign P1d triage table (commit 6364c171d, docs/advanced-topics/rust_engine.rst 'Perf-campaign triage table (P1d)' section) partitions all 32 baseline_timings.json benches into buckets A-E. Headline: actionable engine-overhead surface is EXACTLY 2 benches (bucket A: mma_howtouse 0.65x z3 5%, android_arm_license_validation 0.80x z3 17%) -> P2a/P2b. Bucket B (4: angry-reverser, fairlight, sokohashv2, unbreakable_1) is a structural Z3 check()/eval_upto floor (NOT engine-fixable) -> P4-spike. C=18 already faster. D=0 (low-cv baselines track 10-run medians; bimodal slow-pins are by-design, not stale). E=8 excluded (CADET_00001 >280s timeout, defcamp_r100__dfs no own solve.py, cow_fork_scaling synthetic, 5 *_branch arch smoke benches). Triage on real wallclock + z3_check_time_ns fraction, never rust_total_time_ns (zero for ~23/26 benches). Near-parity C watch-list (silent-regression risk): flareon2015_2, cmu_binary_bomb_partial, unmapped_analysis (~1.06-1.07x).

P2a (angr-9w6ad.5) mma_howtouse per-manager fixed-cost profile, HEAD 6364c171d, 2026-06-19. cProfile in capped subprocess (/tmp/prof_mma.py: factory monkeypatch -> RustExplorationManager, RLIMIT_AS 3.8G): WALL 8.36s under cProfile / 5.92s clean (run_single --counters-json), 45 managers built. Callable.call (callable.py:88) = 8.23s cum = 98.5%% of wall. Dominant cost is angr-CORE Python page init, paid by BOTH engines: state_call->state_blank (windows.py:188) 5.06s -> map_region/_map_page/_initialize_page chain ~4.29s cum; tottime hot spot page_backer_mixins.py:240 _initialize_page 1.81s (DictBackerMixin linear scan, 92.7k calls) + page_backer_mixins.py:25 getitem 0.53s (2.2M calls). Confirms benchmark-mma-howtouse-cprofile-attribution: true lever (reuse call_state base / memoize CLE page backing across Callable invocations) is angr-core, OUTSIDE rust-symex scope. Rust-attributable per-manager FIXED tax now SMALL (~0.58s total / 45 = ~12.9ms per manager): resume_after_simprocedure 0.304s (6.75ms/mgr), _sync_extra_python_pages rust_state_sync.py:599 0.106s tottime (2.36ms/mgr -- b58a fix HOLDS, was 35ms), rust_irsb_serializer _serialize_expr 0.088s (1.96ms/mgr), _extract_from_ultrapage rust_state_sync.py:1647 0.079s (1.76ms/mgr). bench_diff vs baseline_counters.json mma_howtouse entry: NO structural drift, all deltas <1%% z3 / tiny-ns noise -> bzsc loader-page cache + b58a not regressed. Feeds P3 synthesis angr-9w6ad.9: mma_howtouse 0.65x is NOT an actionable rust-symex surface.

P2b (angr-9w6ad.6) android_arm_license_validation init/state-export fixed cost, HEAD 6364c171d, 2026-06-19 (/tmp/prof_android.py, capped subprocess, single manager, NOT Callable). SURPRISE: the fixed init cost is dominated by REGISTER sync, not memory sync (opposite of mma_howtouse). Timing: proj_load ~15ms; mgr construction (init/state-export) = 64ms (stable across 3 runs); explore run = ~65ms -> INIT IS 50%% OF ACTIVE WORK on this steps=1 workload (Py 0.20s vs Rust 0.25s = 0.80x; the ~50ms gap is mostly this fixed init). perf_report breakdown of Init total 49.2ms: Register sync 41.6ms (DOMINANT), Memory sync 4.0ms (bzsc/b58a mitigations hold even for ARM), Python init 0.9ms, Register SimProcedures 0ms. ROOT CAUSE: _sync_registers_to_rust slow path (rust_state_sync.py:128) iterates _supported_register_names(arch) doing per-register getattr(angr_state.regs, reg_name) -> each materializes a claripy AST via the SimState register plugin; ARM has a large reg file (general + vector d0-d31 etc) so ~40+ getattr+eval = 41.6ms. The precomputed_regs fast path (set_registers_bulk) only fires when the disk cache populated it, which only happens for entry_state -- a blank_state at a custom addr (0x401760 here) takes the slow path every time (SAME class of miss bzsc fixed for loader pages, but for registers). ACTIONABLE rust-symex lever (bucket A): bulk-read ARM registers without per-reg AST materialization, or extend a precomputed-regs path to non-entry blank states. Follow-up bead filed. Feeds P3 synthesis angr-9w6ad.9.

P2c (angr-9w6ad.7) very-slow-tier profile: 5 never-completing benches profiled under bounded harness (one run each, run_single.py --timeout 70 --mem-limit 3500, ANGR_BENCH_SOFT_TIMEOUT=20). NONE is an actionable rust-symex engine-overhead lever — consistent with P1d (only mma/android actionable, bucket B = Z3 floor). Per-bench (faulthandler-pinned hang location): asisctffinals2015_fake=TIMEOUT, Z3-structural, blocked in single post-exploration solver.eval (rust_state_export._rust_eval -> RustSolverContext.eval, solve.py:40). tumctf2016_zwiebel=TIMEOUT(bimodal), Z3-structural, blocked in symbolic-store addr concretization eval inside libc memcpy SimProc (rust_callback_dispatch._with_extra_constraints via rust_identity.tracking_store). 0ctf_momo_3=TIMEOUT, brute-force loop rebuilding state/manager per attempt (~4s/iter, repeated sigaction/page-fill); many short managers not one solve; soft alarm fires but bench loop swallows it. sharif7_rev50=catalog argv gap (solve.py needs -f/--file, no argv_override -> argparse SystemExit, a BaseException, escapes harness except Exception and hangs pool worker -> mislabeled TIMEOUT). b01lersctf2020_little_engine=FAIL 1.6s NotImplementedError SYMBOL_FILL_UNCONSTRAINED_REGISTERS fast-fail. The 2 Z3-structural join bucket B floor; engine is not the bottleneck for any of the 5.

P2d (angr-9w6ad.8) RESOLVED: do NOT add a state_export/manager_sync criterion micro-bench. The conditional gate ('add IF none covers the mma_howtouse macro phenomenon') resolves FALSE-to-add for two independent reasons. (1) Criterion benches in native/angr/benches/vex_engine.rs are pure-Rust (rustbv_/symcontext_/memory_*/state_fork/lineage_variants); the manager-sync hot path is PYTHON (rust_state_sync.py _sync_extra_python_pages / _extract_from_ultrapage, callable.py Callable.call) and is un-callable from a Rust criterion bench by construction. (2) P2a (benchmark-p2a-mma-per-manager-cost) decomposed the mma_howtouse macro phenomenon as 98.5%% angr-core Python page-init (state_blank->_initialize_page, OUTSIDE rust-symex scope) with the Rust-attributable per-manager fixed tax now SMALL (~12.9ms/mgr: resume_after_simprocedure 6.75ms is the only pure-Rust component) and explicitly NOT an actionable rust-symex lever per P3 inputs. Adding a bench would protect no optimization target while adding suite cost (angr-vx8p coordination flag). The pure-Rust slices that COULD be benched (state.fork, sym-context push/pop, rustbv build_z3) are already covered. Conclusion feeds P3 synthesis angr-9w6ad.9: no manager-sync micro-bench gap to fill.

P3 synthesis (angr-9w6ad.9, commit on docs/advanced-topics/rust_engine.rst 'Ranked optimization candidates (P3 synthesis)' section, 2026-06-19): the perf campaign's terminal conclusion is that the Rust symex engine is NOT the bottleneck anywhere in the angr-examples corpus. All Phase-2 inputs reclassified to non-engine root causes: P2a mma_howtouse NO-OP (98.5%% angr-core Python page-init, residual Rust tax ~12.9ms/mgr; benchmark-p2a-mma-per-manager-cost), P2b android NO-OP (one-time init not per-mgr tax; benchmark-w4oo3-register-sync-noop), P2c very-slow tier = Z3-eval floor + catalog artifacts (benchmark-p2c-veryslow-profile), P2d no manager-sync micro-bench gap (benchmark-p2d-no-microbench-decision). Two UNMEASURED code candidates survive as P4 spikes, both gated on landing a counter first: PC2 = lazy export-tracking on RustSolverContext::add_constraint_ast (solver.rs) deferring claripy_to_rustbv+assumed_constraints_push via OnceCell — MUST NOT regress into pre-pinning (guardrail constraint-export-no-pre-pin); batch path add_constraints also converts per-entry so it is not a free alt; expected gain minor. PC1 = make _export_callback_bundle (pending_api.rs) the default callback-resume path to cut FFI round-trips, speculative (P2a/P2d found sync tax small). REFUTED non-candidate: 'fork replays constraints O(n*m)' — SymbolicContext Arc-shares assertions, forks O(1) via Arc::clone, lazy Z3 solver. Remaining real surface = Z3-structural floor (bucket B + 2 P2c hangs) addressed by angr-9w6ad.10 (deterministic Z3 seeding) and angr-9w6ad.11 (LAZY_SOLVES audit), NOT engine-overhead reduction.

After refresh on 2026-05-02 (commit d200d4901): peak_memory_mb baseline values jumped 19-71% for several benchmarks (ais3_crackme 237→364, defcon2016quals_baby-re 186→281, flareon2015_2 274→411, unbreakable_0 249→426, whitehatvn2015_re400 291→476). Conversely unmapped_analysis dropped 876→474 (GC leak fix 342df4a7f finally credited). Old peak_memory baselines were set at 15395dd6b — the same commit reproduces the higher peak now. Memory pattern likely reflects normal Z3-model-nondeterminism state shape variation, NOT a leak: the timings are stable across reruns, and tests/benchmarks/run_regression.py uses spawn-context subprocess so prior benchmarks cannot leak in. If a future session sees a peak_memory regression alert, this is a useful prior.

peak_memory_mb PR gate promoted to hard fail (angr-iwmp, commit e2198f86f, 2026-06-20): --memory-warn-only removed from ci.yml benchmark_regression run line. The PR fast-tier gate now hard-fails on RSS regression like nightly. Threshold stays 0.5 (50%); per angr-iwmp fallback, if RSS proves noisy in CI raise --memory-threshold rather than re-adding warn-only. Soak basis was 3 clean local runs of the gate command + elapsed time, not direct nightly observation (offline). See benchmark-peak-memory-gate for the gate internals.

angr-cudgw.2 ('peak_memory_mb gate skipped on zero-solve runs') was a phantom bug: the claimed 'if successful_solves > 0' branch never existed in tests/benchmarks/run_regression.py (git -L confirms). The peak_memory_mb check has always been an independent sibling of the timing-regression check, both inside 'if baseline_key in baseline and not args.update' — not gated on solve count or rust_stats truthiness. peak_memory_mb itself is ALWAYS set in run_single._collect_rust_diagnostics from ru_maxrss regardless of solves, so it cannot be None on an ok run. Resolved by extracting pure helper run_regression.memory_regression_pct(baseline_mem, rust_peak_mem, threshold) (signature has NO solve/stats param by design) and pinning the invariant in tests/benchmarks/test_memory_gate.py. Note: the >threshold comparison has a float-rounding edge — exactly +threshold (100MB->115MB @0.15) trips because 100*1.15=114.9999...; behavior inherited verbatim.

Per-arch benchmark tally for the rust_engine.rst arch support matrix is NOT derivable from baseline_timings.json keys or run_single.py EXAMPLE_CATALOG alone — neither records target arch. Recount by file(1)-probing each binary: synthetics in tests/benchmarks/synthetic_examples/ (aarch64_le_branch, arm_le_branch, mips32/64 branches, cmu_binary_bomb_partial=x86_64, cow_fork_scaling=x86_64), reals in ~/repos/angr-examples/examples//. Surprises: many CTF 'x86' benches are 32-bit (PE32/ELF i386) not AMD64 — flareon2015_2/5/10, mma_howtouse, ekopartyctf2016_sokohashv2, whitehatvn2015_re400, csgames2018, sym-write, CADET_00001(CGC,rust_time=null). HEAD tally (31 entries): 16 AMD64, 9 x86, 2 ARM (arm_le_branch synth + android_arm_license_validation real ELF), 1 ARM64, 1 MIPS32, 2 MIPS64. Matrix now carries a jq keys|length live-count pointer to slow drift.

Recent angr/native perf wins (likely from FxHash adoption, freeze-local-assertions, Arc-wrap fork fields) confirmed on 2026-05-09 via baseline refresh: unmapped_analysis 2.148→0.789s (-63%), google2016_unbreakable_1 3.523→~1.7s typical (-52%), csaw_wyvern 1.257→0.94s (-25%), ekopartyctf2016_sokohashv2 12.091→~9.5s typical (-21%), flareon2015_5 6.563→5.567s (-15%). securityfest_fairlight is bimodal (7.8s OR 15s) — fast mode is a real win but mode is nondeterministic.

BENCHMARK PHASE 1 ASSESSMENT (2026-04-14): wyvern=14.7x, flareon5=5.85x(was WRONG→fixed), ekoparty=8.2x, ais3=3.55x, defcamp=2.04x, flareon10=1.37x, fairlight=1.15x, fauxware=0.83x, sym-write=0.08x. 9/9 correct (all fixed). Key fix: flareon5 wrong output caused by Rust solver fallback not receiving user-added constraints post-exploration. Remaining regressions: fauxware(0.83x per-callback FFI), sym-write(0.08x eager symbolic store concretization). Phase 2 priorities: sym-write regression (native ITE stores), fauxware regression (callback overhead).

Benchmark after Phase 4.1 (2026-05-15, commit a1d9f5593): sym-write rust_only gate-off 1.62s (avg 4 runs), gate-on 1.78s (avg 4 runs). Pre-Phase 4.1 was 1.55s/1.72s per memory phase3-cache-residual-gap. Phase 4.1 did not improve gate-on wall time materially (1.72→1.78 actually slightly worse, noise-band), but it did rebalance where the cost goes: load_stmt 9ms gate-off → 10ms gate-on (was bigger fraction before; now ~1ms gap). Residual gate-on cost is now all in Z3 work (z3_check 86→158ms, z3_site_eval_upto 55→136ms). Phase 4.2 (per-byte symbolic_objects coalescing on flush) is the next lever.

sym-write rust_only after Phase 4.2 (commit 278fc5223, 2026-05-15):

• Gate OFF: 1.55s avg (matches pre-Phase-4.2; gate-default still false)
• Gate ON (Phase 4.1 baseline): 1.78s avg
• Gate ON (Phase 4.2): 1.70s avg (-5% wall) Profile deltas gate-on Phase 4.1 -> Phase 4.2: z3_check 158ms -> 147ms (-7%) z3_site_eval_upto 136ms -> 125ms (-8%) z3_check_count 310 -> 310 (unchanged — coalescing doesn't reduce eval count, just per-eval cost) z3_site_eval_upto_count 303 -> 303 (unchanged) Tests: 403/403 Python (test_rust_exploration), 71/71 Rust (memory::tests inc. 5 new phase42 tests). All pre-Phase-4.1 regression-suite failures verified pre-existing (matched gate-off run numbers before/after change).

After Phase 4.3 (commit e5c594fe1) on 8GB local host: FAST_SUITE benches (PR-time CI gate) unchanged from gate-off: fauxware 0.28s, defcamp_r100 0.27s, ais3_crackme 1.98s, strcpy_find 2.13s (vs 2.11s gate-off, +1%), defcon2016quals_baby-re 0.74s. sym-write went from 1.53-1.55s gate-off baseline to 1.70-1.74s default-on (~12% slower). Within 15% nightly threshold, fails the parent epic's 'parity-or-better' criterion. Tests: 775/775 Rust lib, 403/403 Python.

BENCHMARK POST-LAZY-SOLVER (2026-04-16): csaw_wyvern=15.0x(Py:16.4s, Rust:1.1s), flareon2015_5=5.2x(38.4s/7.3s), ais3_crackme=2.7x(2.7s/1.0s), defcamp_r100=1.5x(1.1s/0.75s), flareon2015_10=1.4x(7.8s/5.6s), fauxware=0.9x(0.4s/0.45s), flareon2015_2=0.8x(4.4s/5.2s), securityfest_fairlight=0.5x(9.7s/21.5s). 5/5 regression tests pass. Lazy solver + Arc cache changes had no measurable perf impact.

BENCHMARK POST-PHASE3 (2026-04-14): fauxware=0.48s, ais3=0.81s, defcamp=0.54s, flareon5=6.72s, flareon10=5.46s, ekoparty=3.92s, wyvern=1.09s, fairlight=11.66s, hackcon2016=36-65s(was 87s!), sym-write=12.66s. 10/10 correct. Key improvement: Z3 context sharing + fast-path Z3 AST passthrough eliminated constraint structure divergence for hackcon. hackcon eval time reduced from 81s to ~30-60s by preserving claripy's Z3 AST structure.

angr-7jv5 measurement (2026-06-06): per-callback proxy write density on the fast-tier corpus is too low to justify a within-callback write buffer. With ANGR_RUST_USE_CALLBACK_{MEMORY,REGISTER,SOLVER}_PROXY=1 on, mgr.stats now surfaces proxy_mem_concrete_writes / proxy_mem_ast_writes / proxy_reg_writes / proxy_solver_adds (instrument lives at rust_state_proxy.py:932,1336,1344,1353,1364,467 and is gated on optional python_mgr kwarg to keep low-level proxy unit tests working). Measured: fauxware 63 writes / 1 cb (58+0+3+2), csaw_wyvern 44 writes / 13 cbs (0+0+32+12), ais3_crackme & defcamp_r100 0 (both bypass the callback path entirely). Within-callback FFI batching could save at most ~12ms on fauxware (~7% wall) and is unmeasurable on csaw. Confirms the prior g7ug Option C verdict under the new write-through architecture. Reopen criteria: a single callback emitting >100 writes (none observed yet) or a future bench with sustained >10 writes/cb across hundreds of callbacks.

sym-write benchmark recovered after angr-q7r0 fix (commit c88947e65, 2026-05-07). Before: 3.22s peak_mem=196MB (regressed from 0.42s baseline). After: 0.44s peak_mem=196MB (matches baseline). Other benchmarks unaffected (regression suite all 12/12 pass, 19.7s total). The fix is a pure performance improvement; no semantics changed.

MEDIUM-tier rust_time baselines csaw_wyvern/sym-write/codegate_2017-angrybird were all set in commit cafc8e702 (2026-05-09) after a perf-win snapshot, then drifted by iter59: csaw_wyvern 0.94->2.70 (the documented SimState-materialization bottleneck, time_in_callbacks~77% of wall; the real optimization is tracked in angr-8mjd, NOT a fresh regression to bisect), sym-write 0.436->0.55 (+26%), codegate 2.84->3.82 (+29%). The latter two are gradual non-step-change drift over 800+ commits on benches that only gate under nightly --full (fast-tier PR gate never sees them). Counts are deterministic (angr-lagp). Decision pattern: refresh to measured value (15% threshold absorbs noise) rather than bisect 800+ commits for a modest non-gating drift; reserve bisect for step-changes. Same staleness pattern kvn0.1 handled for 4 fast-tier benches.

Criterion rustbv microbench, before/after Arc<[RustBV]> inlining (commit fad930315, 2026-05-07). BEFORE (Arc<[Arc]>): rustbv_symbolic/add 69.1ns, /concat 68.97ns, /extract_32 43.9ns, /reverse 44.5ns. AFTER (Arc<[RustBV]>): /add 48.1ns, /concat 48.5ns, /extract_32 37.0ns, /reverse 37.3ns. Concrete benches and rustbv_build_z3_ast within ±2% (noise). Speedup comes from eliminating N inner Arc allocations + N drops per Expression construction. Run with Z3_SYS_Z3_HEADER=/usr/include/z3.h cargo bench --bench vex_engine -- 'rustbv_(concrete|symbolic)'.

sokohashv2 first post-fix re-validation campaign (angr-hyiz.5, HEAD 260eb4d66, 2026-05-18). 10/10 OK runs after angr-ctct + angr-fv81 fixes (c6b2824cb / 13bb9f741, 2026-05-14). Distribution: min=8.63s, max=17.44s, median=11.98s, mean=12.69s, stdev=2.65s. Histogram: 1×8.63s, 5×11.82-12.06s, 2×12-13s, 2×17.18-17.44s. Historic clean bimodal (~9.5s OR ~15.4s) has softened into a trimodal-ish spread with dominant ~12s mid mode. Slow-mode max 17.44s within +15% of 16.0s baseline (18.4s threshold) so baseline kept at 16.0s. Median speedup vs python 5.833s = 0.49x; mean = 0.46x (consistent with CLAUDE.md 0.4x). Retained in BIMODAL_BENCHMARKS — slow-mode tail + structural x87+Z3 sources still warrant --skip-bimodal. Cmd: python tests/benchmarks/bimodal_variance.py --runs 10 --benchmarks ekopartyctf2016_sokohashv2 --bin-width 1.0 --timeout 60. Wall clock ~133s.

Re-baselining lesson (angr-ngver): android_arm_license_validation (0.25->0.48s) and cmu_binary_bomb_partial (1.31->1.51s) baselines in baseline_timings.json were stale/OPTIMISTIC (set on a faster/less-loaded box), NOT variance — N=5 medians on the dev box were tightly clustered (android 0.48-0.50, cmu 1.49-1.53). Distinguish: bimodal/variance benches have WIDE distributions (use BIMODAL_BENCHMARKS); a stale baseline shows a tight cluster shifted from the recorded value -> just re-measure N>=5 and update the JSON. Editing baseline_timings.json via json.dump reformats 0.90->0.9 on unrelated floats; restore them to keep the diff minimal.

Baselines rot toward stale-slow (merged from benchmark-sub1x-drift-2026-06-15): before diagnosing a 'slow' sub-1.0x bench, re-measure fresh — rust_time baselines are often never refreshed after perf gains. As of 2026-06-15 (commit 2bd351453) a 5-sample re-measure found flareon2015_2 (3.94 vs Py 4.21), cmu_binary_bomb_partial (1.31 vs 1.39), unmapped_analysis (0.91 vs 0.97) all now RUST-FASTER despite baseline_timings claiming sub-1.0x. Only android_arm_license_validation stays legitimately sub-1.0x — pure ~50ms PyO3-init/state-export tax on a ~0-step workload, same fixed cost as the *_branch synthetics, not engine throughput.

Refresh vs bisect (merged from benchmark-rust-time-baseline-drift-kdni): for MODEST non-step-change drift on non-gating (nightly --full only) benches, refresh the JSON to the measured value (the 15% threshold absorbs noise) rather than bisecting hundreds of commits; reserve bisect for step-changes. Example: csaw_wyvern 0.94->2.70 (documented SimState-materialization bottleneck, tracked in angr-8mjd — not a fresh regression), sym-write 0.436->0.55, codegate 2.84->3.82 all drifted gradually over 800+ commits. Counts stay deterministic.

Benchmark baseline staleness verification: when run_regression.py reports many consistent regressions in a stable hardware/codebase setup, they may be stale baselines (not real regressions) recorded under abnormally favorable conditions. Verify by checking out the baseline-set commit, rebuilding the .so, and running the benchmark standalone — if the 'old' code reproduces the 'new' timings, the baseline was a fluke. Pattern observed 2026-05-02 (angr-rsfv): 8 fast-suite baselines ran 19-71% slow at HEAD, all reproduced same slow timings at the original baseline-set commit 15395dd6b. Single-benchmark control like fauxware (not in regression list) matched its baseline at both commits, confirming no system-wide change. Resolution: run_regression.py --update --rust-only refresh.

state_fork criterion bench (vex_engine): 338.86 ns -> 283.80 ns (-16.4%) after Arc-wrapping FileSystem.fds, hooks, and environment in RustSimState. Verified with criterion's built-in change detection (p < 0.05). Other fork-time clones still present: RegisterFile.symbolic (2.48%, mutated frequently so Arc may regress), RegisterFile.data Vec (2.51%), SymbolicMemory's internal HashMaps (5.74% partly), HeapMetadata.allocated (1%). Date: 2026-05-07. Commit 76aef1ca2.

Native stdio shims (angr-70no, commit 05671234e) eliminated Python fallback for fwrite/fflush/setvbuf on three benches: csgames2018 fwrite 22->0 fallbacks, defcon2016quals_baby-re fflush 13->0, ekopartyctf2016_rev250 fflush+setvbuf 3->0. No measurable wall-clock change in fast-tier regression — these benches were already fast and the per-callback cost is ~100us so the absolute saving is single-digit ms. Value is in cleaner fallback counters / unblocking 'native everywhere' goal, not in this commit's wallclock impact.

Bench baseline drift is real and favorable: as of 2026-06-15 (commit 2bd351453) a 5-sample run_single.py --both re-measure found flareon2015_2 (3.94s vs Py 4.21s), cmu_binary_bomb_partial (1.31 vs 1.39), unmapped_analysis (0.91 vs 0.97) are all now RUST-FASTER despite baseline_timings.json claiming sub-1.0x — their rust_time was never refreshed after post-May perf gains. Lesson: before diagnosing a 'slow' bench, re-measure fresh; baselines rot toward stale-slow. Only android_arm_license_validation stays sub-1.0x and that's pure ~50ms PyO3-init/state-export tax (Add Rust state ~8ms) on a ~0-step workload, not engine throughput — same fixed-cost tax as the *_branch synthetics, not actionable.

sym-write (Z3-heavy 2.3x speedup) cProfile attribution (angr-trsg, 2026-06-06 commit 60bbac47d). Python engine: ~0.45s out of 2.69s wall in Z3 C API self time, spread across ~600 Z3_solver_check_assumptions calls (0.215s), Z3_solver_get_param_descrs (0.110s), Z3_solver_assert (0.075s), Z3_solver_get_model (0.031s). claripy.ast.base.new adds 0.061s self / 0.233s cumulative. Rust engine: collapses ALL exploration-loop Z3 work into RustExplorationManager.run (0.051s) + a single end-of-explore RustSolverContext.eval_upto (0.203s). The 2.3x speedup is not Z3 being faster — it's collapsing per-call Z3/Python boundary crossings into a single PyO3 call. Confirms the original hypothesis ("Python flame Z3-dominated, Rust flame Z3-dominated") for this bench class.

sym-write benchmark (2026-04-14): Python 1.04s, Rust 0.45s = 2.3x faster. Before fix: 12.18s (0.08x). Two fixes: (1) skip rustbv_to_claripy in deferred fork Exit handler (12.18s→6.46s), (2) skip _sync_exported_constraints in state export (6.46s→0.45s). Root cause was NOT the symbolic memory model — it was unnecessary Python conversions of deeply nested ITE chains.

sym-write benchmark restored after 85c59c7a0 regression: 0.42s peak_mem=193MB (matches the 4d174348c baseline before the regression). Fix: implement x86g_use_seg_selector ccall handler in Rust (commit aaaa25779). Before fix on HEAD 4fb86056: 30s+ timeout. After fix: 0.42s.

Per-callback sync_back overhead measured at HEAD 5ad385aef via run_single.py --dump-counters (angr-g7ug design pass): fauxware 13.9ms (1cb, 24%), mma_howtouse 7.2ms (1cb, 86%), unbreakable_0 7.7ms (1cb, 4%), flareon2015_10 2.4ms/cb (2cb, 21%), csaw_wyvern 0.89ms/cb (30cb, 1.3%). Range 0.9-14ms per callback; aggregate per bench tops out around 27ms (csaw_wyvern). sync_back% of cb tracks inversely with execute time: tiny procedures (CallReturn, ReturnUnconstrained) make sync the dominant phase; substantial procedures (strncpy 168ms, fauxware open 41ms) dilute it. Use this as the reference for any 'callback boundary overhead' bead — sync_back is small in absolute wall-clock terms (<=7% of wall on fauxware, <0.2% on heavy benches).

google2016_unbreakable_1 baseline refresh (commit db6ea164d): 6.18s -> 2.7s rust_time, 291.4MB -> 631.0MB peak_mem. Old baseline came from a suite-induced memory-pressure run (per benchmark-update-variance memory). After several recent perf fixes (GC leak fix 342df4a7f, SegmentList O(n) iterator 7fec7db2d, exit-cont cache fix 8964c0da3) the actual rust_time is 2.4-3.4s standalone (median 2.71s), 2.0-2.5s in suite. Note: 8 OTHER baselines (defcamp_r100, ais3_crackme, unbreakable_0, flareon2015_2, baby-re, csgames2018, whitehatvn2015_re400) consistently fail by 18-71% — same systemic baseline-too-low issue, separate task.

google2016_unbreakable_1 (Rust engine) dropped from 3.523s baseline (commit 1e33fc24d) to 2.69-2.91s after angr-8kht's range_seeded optimization. 17-23% reduction. This is the address-heavy benchmark; sym-write/csaw_wyvern/flareon2015_5 stayed within noise (none hit the >16-solution range() path often).

SUPERSEDED 2026-05-22 by benchmark-unbreakable_1-2026-05-22. The 20/20 in 2.43-2.48s unimodal window from this campaign did NOT hold — 4 days later the bench was re-bimodal in a 15-sample local run (0.91-2.65s range; gate sample 5.21s). Original text retained below for history. ---- ORIGINAL: google2016_unbreakable_1 re-validation campaign (angr-hyiz.4, HEAD f54bbba93): 20/20 runs in 2.43-2.48s (median 2.46s, stdev 0.01s) — even tighter than the 2026-05-13 campaign (3.01-3.06s). Confirms continued unimodal behavior. Removed from BIMODAL_BENCHMARKS in tests/benchmarks/run_regression.py — PR-time --skip-bimodal gate now covers it. Baseline rust_time=3.5s left as-is (42% headroom over 2.48s worst case). Cached python_time=1.602s gives ~0.65x speedup which prints SLA WARN but does not fail (default SLA fail threshold 0.5x). Committed as 607eed29e.

google2016_unbreakable_1 RE-BIMODALIZED on HEAD 14187073d (2026-05-22). Ralph iter-2 gate failed at 5.21s vs 3.5s baseline (+49%, SLA 0.31x). Local 15-sample re-validation: 11/15 in 0.91-0.99s (fast mode, well below baseline), 2/15 in 1.15s, 1/15 in 1.86s, 1/15 in 2.65s; gate sample 5.21s extends the slow tail. Contradicts 2026-05-18 angr-hyiz.4 finding (20/20 in 2.43-2.48s) that justified removal from BIMODAL_BENCHMARKS in 607eed29e. The fast mode is now ~3.6x faster than the May-18 median — post-May-18 perf gains (likely angr-b58a UltraPage memcmp + lazy-region FFI, angr-zdho z3_ast cache instr, angr-9jly proxy fast-path) appear to have widened the gap between fast and slow Z3 modes rather than collapsing them. Fix committed as e30214e88: re-added to BIMODAL_BENCHMARKS in tests/benchmarks/run_regression.py. Supersedes benchmark-unbreakable_1-2026-05-18 for the bimodal status. SLA at fast mode is 1.65x (1.602s python / 0.97s rust median) — would be a NEW BEST if not for the tail. Baseline rust_time=3.5s left as-is; not worth tightening while bimodal.

unmapped_analysis baseline drifted from 0.789s -> 0.92s between cafc8e702 (2026-05-19) and 2026-05-23. 10-sample isolated median 0.92s (range 0.90-0.95). The 0.789s baseline left only 0.118s buffer at 15% threshold, so normal noise pushed gate red on commits that had nothing to do with the bench. Refreshed at 868be2609 to 0.92s with no associated revert. Root-cause investigation deferred — the drift predates commits that triggered the failing gate; this commit only addresses the noise floor.

Running run_regression.py --update --full overwrites baseline timings, including any entries previously populated on different hardware. The 'full' suite (22 entries on 2026-05-01) takes ~120s end-to-end with significant timing variance from running benchmarks back-to-back under memory pressure (e.g. google2016_unbreakable_1: 5.82s standalone vs 17.288s in suite vs 6.18s on second pass). For stable baselines, re-run --update twice and take the second pass, or use higher regression threshold (default 0.15 = 15%).

Valgrind leak gate design (tests/benchmarks/run_valgrind_leak_check.py, angr-c4xcs.4). TWO decisions worth not re-litigating: (1) It drives a PURE-RUST probe (native/angr/examples/leak_probe.rs), NOT the Python bench -- memcheck through CPython buries findings under interpreter false-positives (pymalloc arenas, interned strings); CPython ships Misc/valgrind-python.supp and still calls the result unreliable. The Python-driven Callable path stays covered by the RSS gate (run_leak_check.py). Complementary, not redundant. (2) It gates the per-iteration SLOPE, not an absolute byte count: the probe drops everything it allocates, so a correct engine leaks a CONSTANT baseline (one-time z3 globals + Rust lazy statics) whose size varies by distro/z3 build -- gating the total would flap across CI images. Harness runs the probe at N and 10N and fails on (lost_hi-lost_lo)/(iters_hi-iters_lo) > threshold. Only 'definitely lost'+'indirectly lost' are gated; 'possibly lost' (1160 B, TLS-shaped) and 'still reachable' (171280 B, z3 globals) are reported but NOT gated -- both are byte-identical across a 10x iteration change, which is what makes them safe to ignore. Baseline 2026-07-14, valgrind 3.22: 0 B definite+indirect at BOTH N=20 and N=200, slope 0.00 B/iter; threshold 1.0 B/iter.

angr-w4oo3 register-sync 'optimization' is a NO-OP: the 41.6-47ms register sync that P2b (benchmark-p2b-android-register-sync) flagged for android_arm_license_validation is ONE-TIME process Z3/claripy/Rust-bridge init, NOT a per-manager tax. Evidence (/tmp/prof_regsync4.py, 3 managers same process): mgr#0 register sync=47.6ms, mgr#1+=1.9ms. First claripy z3 backend convert=10ms (Z3 context init); subsequent converts=0.02ms each (/tmp/prof_regsync3.py). The cost is paid by the Python engine too on its first symbolic op. Recurring per-manager cost (1.9ms for 13 symbolic-reg imports) already beats the bead's <10ms target. CANNOT skip the 13 blank_state default-fill symbolic regs (r1-r12,lr) to save the 1.9ms: Rust's register file (arch/mod.rs Registers::get) zero-defaults any unset offset by reading self.data, so skipping would make Rust read 0 where Python reads unconstrained symbolic = cat-c wrong answer. Reclassified like P2a (benchmark-p2a-mma-per-manager-cost): register sync is NOT an actionable rust-symex surface. Symbol anchors: _sync_registers_to_rust (rust_state_sync.py), Registers::get (native/angr/src/arch/mod.rs).

angr-yrhh (Native __libc_start_main, 2026-05-20) — ais3_crackme: SimProcedure callbacks 47→0, runtime 0.95s→0.87s (~8% faster). sym-write: callbacks 2→0 (runtime sub-second, noise-dominated). Fast-tier matrix neutral (16/16 pass). Mechanism: register a Rust impl with no_return=true (no args used) so the dispatcher deadends the state. Per-call cost was ~1.3ms × 47 = ~61ms saved. The bead's literal plan (allocate ctype/errno tables + jump to main) would only help on init-cache MISS (first run); the recurring cost is the after_main continuation, which the deadend approach kills.

ZeroPy bounce census (angr-gorvf.12, iter144, instrumented via ANGR_BOUNCE_TRACE=1; table at tests/benchmarks/zeropy_bounce_census.json, harness tests/benchmarks/zeropy_bounce_census.py). Across the 7 bounce-sole benches: A=1 site (no native proc), B=6 (native proc exists but did not serve the call), C=3 (irreducible user Python hook). BY BENCH: 5/7 fully reducible (fauxware=open, google2016_unbreakable_0=strncpy, securityfest_fairlight=strncpy, unmapped_analysis=strncpy+strncmp, ekopartyctf2016_rev250=get_flag); 2/7 irreducible (cmu_binary_bomb_partial=readline_hook/strtol_hook, defcon2016quals_baby-re=my_scanf x13) and those two must LEAVE the ZeroPy gate denominator, not be scored as failures. KEY: the work is NOT mostly 'write new native procs' — 6 of 7 reducible sites already HAVE a native proc. The single bucket-A site is a ReturnUnconstrained SimLibrary stub (get_flag), i.e. 'return a fresh symbol' — one generic native stub handler retires it. Bounce sets do not depend on the lift path, so this census runs on the STOCK build (no libvex-ffi .so needed, unlike run_zeropy_gate.py).

ZeroPy gate result after angr-gorvf.13 (iter148, libvex-ffi build): PASS 12/36, up from 8/36 — the 4 retired bounce-sole benches (google2016_unbreakable_0, securityfest_fairlight, ekopartyctf2016_rev250, unmapped_analysis) all reach gil_work_time_ns == 0. The lever ranking INVERTED as a result: across the 23 remaining FAIL benches, callback is now the dominant class (sole=11, 1232ms) ahead of bounce (sole=3, 772ms), and within callback the single biggest site is lift_block (appears=17, 699ms), followed by memory_store_symbolic_value (517ms, one bench: flareon2015_5). claripy_import is effectively retired (sole=0, 0ms). So the next ZeroPy increment should target lift_block (the pyvex callback lift path — note the libvex-ffi feature is exactly the fix, it just is not the default build), NOT more SimProcedure work. Remaining bounce-sole benches are bucket-C (irreducible user hooks) plus fauxware, which is blocked on angr-gorvf.15 (native read must mint symbolic content for a contentless fd).

ZeroPy .5 milestone (angr-gorvf.4) MET on the SHIPPING build as of iter19 (HEAD f5824dadc, libvex-ffi now default-ON via angr-3trr7): run_zeropy_gate.py = 13/36 benches at gil_work_time_ns==0 (>=8 acceptance). Up from iter142's 8/36 — the +5 (ekopartyctf2016_rev250, fauxware, google2016_unbreakable_0, securityfest_fairlight, unmapped_analysis) were simproc/bounce-blocked pre-proxy-flip and became Python-free after callback-memory-proxy default-on (angr-grji4) + fork-carries-model-cache + native merge landed. Full PASS set of 13 in the bd close reason. zeropy_attribution.json refreshed to this reality (angr-gorvf.17, commit 8b7926589). ekopartyctf2016_sokohashv2 ERRORs under the gate (bimodal-Z3 180s timeout, measurement artifact); CADET_00001 UNMEASURED (rust_time=null). Showcase epic angr-4n26m headline result.

[SUPERSEDED 2026-07-14 iter142 by benchmark-zeropy-pass-count-iter142 — the count is now PASS=8/36 after angr-gorvf.4.7 retired the memory_load/stack-canary crossing, and the lever ranking has flipped to bounce-first. Read that memory instead; this one is kept only as the historical record of the bounce-accounting correction.]

ZeroPy honest PASS count is 4/36 (libvex-ffi build, commit 5f8d936e4, tests/benchmarks/zeropy_attribution.json), NOT the 6/36 or 10/36 that earlier notes claim -- those predated the bounce accounting (see invariant-bounce-gil-accounting). Only ais3_crackme and strcpy_find are DURABLY Python-free. google2016_unbreakable_1 and hackcon2016_angry-reverser PASS on some runs but banked ~33us of callback GIL on others: they sit on the noise floor and flip run-to-run, so never treat a single-run PASS on those two as a retirement. angr-gorvf.4's >=8 acceptance is NOT met. Top funded lever across the FAIL set: simprocedure 95 crossings / 1639ms, then symbolic_branch 44/744ms, then lift_block 793/649ms.

ZeroPy gate result, MEASURED 2026-07-14 iter142 (commit 83adb4456, libvex-ffi FEATURE build, tests/benchmarks/run_zeropy_gate.py): PASS=8/36, FAIL=27, UNMEASURED=1 (CADET_00001, no rust baseline), ERROR=0. SUPERSEDES benchmark-zeropy-pass-count-iter136 (4/36). The >=8 acceptance on angr-gorvf.4 is MET. Passing benches: ais3_crackme, codegate_2017-angrybird, defcamp_r100, defcamp_r100__dfs, google2016_unbreakable_1, hackcon2016_angry-reverser, sharif7_rev50, strcpy_find. The +4 came from angr-gorvf.4.7 (VEXInterpreter::synthesize_unservable_load retiring the fs:[0x28] stack-canary memory_load crossing — see zeropy-memory-load-is-the-stack-canary).

CAVEAT THAT MATTERS: 8/36 holds ONLY on a libvex-ffi build. On the STOCK build that ships, lift_block fires on every bench and the honest score is 0/36 — the milestone is gated on the human policy bead angr-3trr7 (flip libvex-ffi on by default).

LEVER RANKING HAS FLIPPED — over the 27 FAIL benches, bounce 1342ms (sole=7, appears=16) now BEATS the in-loop callback class 1233ms (sole=11, appears=20); claripy_import 0ms. Within callback: lift_block 695ms (appears=18) > memory_store_symbolic_value 521ms (flareon2015_5 ONLY, angr-gorvf.7) > resolve_function 11ms > memory_load 6ms > memory_store 1ms > batch_fetch_pages 0ms (retired). By crossed surface: simprocedure count=95/1208ms, symbolic_branch count=44/740ms, lift_block count=793/643ms. CONSEQUENCE: further ZeroPy work should target the BOUNCE class (native SimProcedures + a native symbolic-branch path), not the named callback dispatch sites — those are down to a long tail plus the two policy-blocked items.