Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

JIT: the speculative optimizing tier

crates/j8jit is j8’s JIT (x86-64 Unix). It implements the executor’s Tier: hot functions and hot loops are built into a speculative SSA graph from the inline caches’ feedback, optimized, and emitted as x86-64 machine code that runs directly on the interpreter’s frames; when a speculation fails, it deoptimizes back to the interpreter.

hot function / loop ─ build (bytecode + inline caches ⇒ speculative graph) ─ opt ─ codegen ─▶ x86-64, on the interpreter's frame
a guard fails ─▶ j8_deopt rebuilds the interpreter frames; the hint it leaves shapes the next compilation

j8’s plan is to skip a baseline compiler and go straight to a speculative optimizing tier (inlining, escape analysis, deoptimization, OSR), aiming to match V8 with its JIT on common benchmarks.

Turning it on

use std::rc::Rc;
let executor = j8executor::Executor::default().with_tier(Rc::new(j8jit::Jit::default()));
  • dune --jit and J8_TEST262_JIT=1 (test262 runs its baseline with the JIT) turn it on;
  • Jit::from(n) sets the hotness threshold (default Jit::THRESHOLD = 64); 0 compiles every function on its first call;
  • debugging: J8_JIT_LOG (compilations and deopts), J8_JIT_IR (prints the graph), J8_JIT_BIN=<dir> (writes the machine code to files).

Three entries

KindWhenHow it runs
Enterthe interpreter calls a hot functionon the frame the interpreter built for it
Osr(at)loop header at has jumped back often enoughon the current frame, entering mid-loop
Native(n)compiled code calls compiled codestraight through the callee’s trampoline, no interpreter frame

Plain functions (not class constructors, not called with new, at most 3 formals) get a native entry: arguments in registers, int32 results back unboxed through a second return register. Every function has a trampoline jmp [rip + cell]: its machine code when it has some, otherwise the interpreter.

Script and eval top-level code runs once: it isn’t compiled on entry, only its hot loops are (OSR).

build: bytecode ⇒ speculative graph

build.rs is an abstract interpretation of the bytecode: for every interpreter register it keeps the IR value it holds, speculating on what the inline caches saw.

  • Numbers start as int32, guarded; a failed guard records a hint, and the next compilation uses doubles there;
  • small functions are inlined (depth at most 4, bodies of at most 80 instructions, recursion unrolled at most 2 levels), their frames rebuilt on a deopt; other calls go to compiled code or the runtime;
  • objects stay virtual: fields of objects made here are only tracked by the compiler until something needs the object itself (escape analysis);
  • context slots read and written here are kept in values and written back only when someone else could look;
  • a loop entered on-stack is peeled once: what one iteration allocates and the next replaces never exists.

The bytecode is walked region by region (Region), each region’s blocks in order: forward edges always go to blocks not walked yet, so a block’s state is the merge of its predecessors’; a loop header speculates on its back edges (phis for everything live), and the walk restarts with a hint when a back edge disagrees (at most 12 times).

cfg.rs gives the blocks, loops and liveness: for the analysis, the receiver and parameters are 0 .. P, ri is P + i, the accumulator P + R; the context register is always live (not tracked).

The IR

ir.rs: basic blocks in SSA form, each instruction one value with a representation (Rep: int32, double, word …).

Block { phis…, insts…, terminator }      a terminator is Jump / Branch / Return / Deopt
Inst  { op, rep }                        operands are value ids
FrameState { func, pc, slots…, parent }  what each interpreter register holds at a guard (virtual objects too)

Typical Ops:

KindExamples
constants and parametersI32, F64, Word, Param, Callee, Phi
arithmeticInt(op, a, b, fs) (with a frame state when it can overflow), Float, IntToFloat
boxingBoxInt, BoxFloat, UnboxInt(v, fs) (guard: an int32, not −0), UnboxFloat
guardsCheckObject, CheckShape, CheckFunc (the closure runs a given function), CheckNotHole
memoryLoadWord, LoadPtr, StoreCell (with a write barrier when needed), ContextCell
callsCall(helper, …) (runtime functions), CallJit (compiled code)

Every guard carries a FrameState: which instruction, and which register contents, the interpreter resumes with.

opt and codegen

  • opt.rs: phis that pick one value go away, and so does whatever nothing needs;
  • loop-invariant code motion, path facts across loops, constant modulus as multiplication and so on happen while building and optimizing;
  • codegen.rs: graph ⇒ x86-64 directly: linear-scan register allocation over the blocks in reverse postorder, then one pass of instruction selection; asm.rs holds the x86-64 encodings used.
rbp frame   saved rbx r12 r13 r14 r15 at −8 … −40, entry pointers at −48 … −88, spill slots below
r15         the JIT context (Ctx) the whole time
scratch     rax rcx rdx r11 xmm14 xmm15 (never allocated)
allocated   rsi rdi r8 r9 r10 (caller-saved), rbx r12 r13 r14 (callee-saved), xmm0 … xmm13

Guards jump to cold stubs that store the frame state’s values into Ctx::BUF and call j8_deopt.

The runtime

runtime.rs: what machine code calls and reads.

Ctx  +0 budget  +8 pending  +16 global  +24 script scope  +32 Object.prototype  +40 ret  +48 ret pc  +56 acc
     +64 depth  +72 depth limit  +128 ARGS (16 words)  +256 BUF (deopt values)
  • j8_helper: slow-path runtime functions (property access, allocation, generic operators);
  • j8_poll: loops check the budget and termination requests, so machine code can be interrupted like the interpreter;
  • j8_deopt: the deoptimization entry;
  • executable memory comes from mmap.

Deopts and learning

When a guard fails, j8_deopt rebuilds the interpreter frames from the frame state (inlined callees included) and the interpreter goes on. The JIT leaves a hint by reason:

ReasonHint
int32 speculation faileddoubles there next time (HINT_FLOAT)
type / shape mismatchthe generic path there next time (HINT_GENERIC)
the callee was not the one speculatedno inlining + generic (HINT_NO_INLINE)
bytecode the JIT doesn’t compilethat inlined call isn’t inlined next time; a function’s own code is given up

The code that failed is dropped and recompiled with the hints when it is hot again. After 24 deopts of one function, the JIT gives up on it, and it is interpreted from then on.

The MLIR path

j8jit also exports Compiler / Backend / MachineCode (compile.rs, x86_64.rs): MLIR (llvm dialect) through mlirpass (mem2reg, canonicalize, cse), then j8jit’s own x86-64 back end, without LLVM. That was the road of the earlier first-tier JIT; the hot path today is the graph ⇒ machine code above, and lowering V8 bytecode to MLIR is being rewritten.