Comparing changes

ThreadSanitizer (TSAN) can now detect race conditions across the entire multi-threaded JIT pipeline with full 4GB address space emulation. This enables testing of the tier-2 LLVM compilation thread while maintaining production memory layout. Memory Layout (TSAN-compatible): - Main memory: MAP_FIXED at 0x7d0000000000 (4GB) - JIT buffer: MAP_FIXED at 0x7d1000000000 - Both allocations within TSAN app range (0x7cf-0x7ff trillion) - Prevents conflicts with TSAN shadow memory (0x02a-0x7ce trillion) ASLR Mitigation: - Added setarch -R wrapper for TSAN test execution - Disables ASLR to prevent random allocations in shadow memory - Only affects test runs, not production builds SDL Conflict Resolution: - SDL (uninstrumented system library) creates threads TSAN cannot track - Disabled SDL when TSAN enabled to focus on built-in race detection - Production builds still fully support SDL

This commit adds ThreadSanitizer (TSAN) support for ARM64/Apple Silicon and fixes critical JIT instruction cache coherency issues. ARM64 TSAN Support: - Extended TSAN-compatible memory allocation to ARM64 architecture - Main memory allocated at fixed address 0x150000000000 (21TB) - JIT buffer allocated at 0x151000000000 with MAP_JIT for Apple Silicon - Both allocations avoid TSAN shadow memory and enable race detection - Note: Requires ASLR disabled on macOS (SIP restrictions may apply) JIT Cache Coherency Fixes: 1. Fixed pthread_jit_write_protect_np() ordering in update_branch_imm 2. Added sys_icache_invalidate() in update_branch_imm 3. Added cache invalidation in resolve_jumps() for x86_64 Fix JIT regalloc conflicts in memory load After reset_reg() clears the register allocator state, load instructions (lb/lh/lw/lbu/lhu) could reallocate the same host register for both the address and destination, causing data corruption. This commit uses map_vm_reg_reserved() to prevent reusing the address register.

This commit introduces a comprehensive JIT debugging infrastructure to catch register allocation conflicts and cache coherency issues before they cause subtle runtime failures in production.

User-space emulation tests were failing because ENABLE_ELF_LOADER defaulted to 0, preventing ELF file loading. The fix automatically enables ELF_LOADER when SYSTEM=0, as user-space mode always requires it to load test ELF files.

This commit addresses multiple regressions introduced in recent changes: 1. DTB compilation regression - DTB dependencies moved outside CC_IS_EMCC conditional - Ensures DTB builds for system mode regardless of compiler - Fixes mk/wasm.mk structure for cross-platform consistency 2. Makefile syntax error in mk/toolchain.mk - Fixed TAB characters before $(warning) on lines 25, 28 - Changed to spaces for proper control flow - This was blocking all Makefile parsing 3. emcc configuration pollution - Added 'make distclean' before emcc builds in workflow - Prevents ENABLE_SYSTEM=1 from leaking between builds - Fixes "build/minimal.dtb does not exist" errors 4. Ubuntu ARM64 apt-get failures - Implemented exponential backoff retry mechanism (30s, 60s delays) - Added pipefail to preserve apt exit codes through tee - Explicit APT_EXIT capture to detect masked failures - Added InRelease to failure pattern (modern combined Release+GPG) - Ignore non-critical dep11 metadata failures - Focus on core package indices (Packages/Sources/Release/InRelease) 5. TSAN cross-compiler compatibility (fixed __has_feature issue) - Changed from defined(__has_feature) to defined(__clang__) - GCC doesn't support __has_feature, causing preprocessor errors - __has_feature only works when __clang__ is defined - Ensures __tsan_default_options() works with both GCC and clang 6. TSAN cross-platform compatibility - Guarded setarch with ifeq ($(UNAME_S),Linux) in Makefile - setarch doesn't exist on macOS, now conditionally applied - macOS TSAN builds require SIP disabled for ASLR control 7. Trace functionality regression - Reverted .log_level from LOG_INFO back to LOG_TRACE - LOG_INFO suppressed rv_log_trace() stream used by -t flag - Restores instruction trace output for debugging

Add comprehensive ThreadSanitizer output validation to prevent silent test failures. TSAN tests now explicitly check for data races and fail immediately with diagnostic output instead of masking errors. Implementation: - Capture TSAN stderr/stdout to log files for analysis - Pattern match race indicators: "ThreadSanitizer: data race", "ThreadSanitizer: race on", "WARNING: ThreadSanitizer:" - Exit with status 1 immediately upon race detection - Display race context (10 lines) for debugging - Progress indicators for 3-tier validation (Interpreter, JIT-T1, JIT-T2) Platform-Specific Handling: - Linux (x64/ARM64): Use setarch -R for ASLR mitigation (already gated in Makefile with ifeq ($(UNAME_S),Linux)) - macOS: NO setarch (not available), rely on MAP_FIXED allocations at 0x150000000000; gracefully handle SIP restrictions by distinguishing MAP_FAILED errors from actual race conditions Race Detection Patterns: ThreadSanitizer: data race # Standard race report ThreadSanitizer: race on # Race on specific object WARNING: ThreadSanitizer: # General TSAN warnings Error Handling (macOS): MAP_FAILED # mmap failure unexpected memory mapping # TSAN shadow conflict FATAL: ThreadSanitizer # Initialization failure → Skip test with warning (SIP restriction) → Still fail hard on actual races Benefits: - Immediate failure on race detection (fail-fast principle) - Clear diagnostic output in CI logs with race location/context - Platform-aware: Linux uses setarch -R, macOS handles SIP gracefully - No silent failures: Previously masked errors now cause test failure - Debugging support: Log files preserved for post-mortem analysis Validates race condition fixes from: - 2a2a5c4: TSAN with FULL4G and T2C support - f1b685e: ARM64 TSAN support and JIT cache coherency - 669efc1: Build system regressions (setarch gating, TSAN compatibility)

ARM64 CI builds fail when apt update returns exit code 100 due to dep11 metadata size mismatches during Ubuntu mirror synchronization. The fix uses '|| true' to prevent the exit code from failing the action, then manually checks for critical package index failures (Packages/Sources/ Release/InRelease). Non-critical dep11 metadata failures are ignored. Package installation uses apt-get with --fix-missing for robustness. Also fixes stale build configuration bug where 'make clean' doesn't regenerate build/.config, causing JIT extension tests to fail with 'map_file() (null) failed' errors. Changed to 'make distclean' for proper configuration reset between test runs.

ARM64 CI builds fail with "setarch: failed to set personality to (null): Operation not permitted" because Docker containers lack CAP_SYS_ADMIN capability required for setarch to modify process personality flags. Root cause: CI scripts were calling setarch -R externally before make, but the Makefile also wraps test execution with setarch via BIN_WRAPPER, causing double-wrapping and permission failures in containers. Solution: 1. Remove redundant setarch calls from CI scripts (both x64 and ARM64) 2. Add graceful fallback in Makefile's BIN_WRAPPER: sh -c 'setarch $(uname -m) -R "$@" 2>/dev/null || "$@"' -- This allows TSAN tests to run successfully in: - Native Linux (setarch works normally) - Docker containers (falls back without setarch) - macOS (already skips setarch, no change) TSAN still works correctly in containers because: - We use MAP_FIXED for 4GB allocation at fixed addresses - Container ASLR is limited compared to host systems - TSAN shadow memory layout remains valid Fixes: - host-arm64: "setarch failed" exit code 1 - host-x64: Redundant setarch removed for consistency

The run-on-arch-action captures the shell's final exit code ($?), which can remain non-zero even after successful package installation due to 'set +e' context used for error-tolerant apt operations. The install script properly handles dep11 metadata failures (retries for critical indices, ignores dep11, verifies package installation), but the shell retains the last command's exit status in the 'set +e' region. Root cause: Shell exit code propagation in 'set +e' context - 'set +e' disables errexit, allowing commands to fail without stopping - Failed commands still set $? to non-zero - Without explicit success command, shell exits with last command's $? - run-on-arch-action interprets non-zero $? as failure Fix: Re-enable errexit after verification and execute 'true' to guarantee clean exit status. This is more explicit than 'exit 0' and documents why we need the exit code reset. Alternative considered: 'exit 0' (simpler but less clear about intent) Chosen approach: 'set -e; true' (explicit about re-enabling error checking) Fixes: host-arm64 "exit code 100" despite successful package installation

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comparing changes

Open a pull request

Commits on Nov 12, 2025

This comparison is taking too long to generate.

Uh oh!