Companion to the kernel walkthrough · design draft 5 · prototypes/kernel-model

The kernel in motion

One order workflow, followed through every part of the proposed kernel; then the three persistence modes side by side, what the kernel does when things go wrong, and a lab where you try to beat the fence.

Proposed design, not approved Prototype on main: prototypes/kernel-model No product code yet
01

Act one: an order, step by step

Every part of the kernel is drawn as a physical thing and named. Press Next or Play to follow one order from the HTTP request to the Studio run view. Show seams outlines the three layers and names the interface at each crossing; the glossary below can light up any part.

The kernel drawn as a machine On the left an HTTP client and an SMTP server. In the middle Node A, holding partition 17 with the instances order #1 and order #7, and Node B, holding partition 42 with order #3. Each instance card sits on its inbox tray, which sits on the partition shelf. On the right the database with the partition, instance, message and journal tables, and below it the projector and the Studio run view. The narration under the scene says what each step shows. Node A (a server process) Node B Layer 1actor runtime:where things runIActorRuntimeIScopeActivation.RunAsync (a pass) partition 17on Node A · epoch 8 lease: 10 s7.4 s left partition 42on Node B · epoch 5 lease: 10 s8.9 s left wake signal (a hint) Layer 2durable actors:every guaranteeinbox · commit · fencetimers · jobs · journalIKernelStore(the storage contract) inbox inbox inboxempty fence: epoch 8position 1204 fence: epoch 5position 311 committer: one flush at a time projectorlayer 2a background worker,off the hot path cursor (its projection row)1204 Layer 3actor types:what runsIExecutionModel.TurnAsync order #1 turn running workflow instance · Order persistence mode: Balanced native modelexecution model order #7 turn running workflow instance · Order Runningwaiting: timer same partition, so its turnsshare order #1's commits order #3 workflow instance · Order Runningwaiting: event idle: nothing is dueowner: Node B pass order #9 partition 17 turn running scene clockt = 0.0 s HTTP client SMTP server (external) Databasethe vault: one shared store behind IKernelStore Log later: log + local state; planned, not designed commits land here Studio run view order #1 · history reads committed history only POST /orders 202 Accepted · held job: email commit · one transaction REJECTEDthe fence changed 0 rows journal rows seam · IActorRuntime: partitions Node A hosts seam · IScopeActivation.RunAsync (a pass) seam · IExecutionModel.TurnAsync seam · IExecutionModel.ExecuteJobAsync seam · IKernelStore.CommitAsync seam · IKernelStore: ReadJournalAsync, ProjectAsync seam · IKernelStore.ReadViewsAsync seam · IScopeHost.ActivateAsync: the take seam · IScopeHost.ActivateAsync: the take seam · IExecutionModel.TurnAsync

02

Three persistence modes, side by side

A persistence mode chooses where a turn ends, and a turn commits where it ends; Ephemeral alone writes nothing when the run ends in its first turn and sent nothing. So the mode decides what a run costs and what a crash can lose. Three instances of one request-response workflow run at the same time, one per mode; one set of controls steps all three together, activity by activity.

Three persistence modes running the same workflow Three lanes, Ephemeral, Balanced and Durable. Each lane shows its HTTP caller, its instance card with the activity list and a pointer, a database that keeps one envelope per commit, a commit counter, a counter of database round trips on PostgreSQL, and notes. The narration under the lanes says what each step shows. persistence modeits callerthe instance, inline on Node Athe databasewhat it costsnotes
Add a Wait 5 s after Save quote:
Crash here:

The mode is chosen per workflow by its author. The deployment never changes it.

03

Act two: when things go wrong

The same machine, in three short scenarios. Each starts from its own state; the controls, narration and seams work as in act one. Ids, times, epochs and positions are illustrative.

04

The fence lab: try to make two owners both write

Act two showed one paused owner bounced by the fence. Here you choose the order of events. Node A owns partition 17 at epoch 8 and has a batch to commit; Node B would take the partition if it could. Whatever you choose, at most one of them writes, and a commit lands at most once. The lease is the decided 10 s; times and positions are illustrative.

Node A pauses:
Who reaches the row first:
The answer to A's commit is lost:
The fence lab: Node A, the partition row and Node B On the left Node A, the owner, with what its HTTP caller sees, its memory, the turns in its batch and its commit envelope. On the right Node B with its take. In the middle the database: the statement at the partition row, the row lock with its holder and its queue, the row itself (owner, epoch, lease_until, position), the lease against the database clock, and the instance, message and journal tables. A banner at the bottom gives the result. The narration under the picture says what each step shows. Node A · owns partition 17 caller sees: memory turns in this batch Node B · would take partition 17 memory Its take is a conditional update too: itlands only on a partition nobody owns, onewhose lease ran out by the database'sclock, or one Node B already owns. Databasethe partition row and the payload tablesdb clockt = 0.0 s holds the row lockfree waits in the lock queueempty id17 owner epoch lease_until position lease 0 s10 s20 s30 s a GC pause or a suspended VM: not dead A's commit · one transaction B's take · a conditional update A's take · after its reset

Why the check is its own round trip. Sent in one packet, a fence that matched nothing does not stop what follows it: a COMMIT in the same packet would still land. So the owner must see "1 row" before it sends COMMIT, which makes 2 round trips on PostgreSQL. The fence travels one of two ways: in the batch, as EF concurrency tokens on the epoch and the position (the default: 2 round trips, the lease time from the owner's clock), or as its own statement first (3 round trips, the lease time from the database's), which is SQL Server's default since finding E9. Leases decide who should write. The fence decides who can. The invariant: after a take at epoch e + 1, nothing prepared under e commits, and the new owner sees everything that did commit.

Built and measured in the prototype: both ways of sending the fence and their round trips; a response released only after its commit (decided 12 November, and built); the take; the reset after an unknown outcome or a fence that matched nothing; a simulation that injects paused owners and lost answers and checks the invariant after every action; and a mutation check showing that the position decides only when a reset is missing (FINDINGS.md, section 3). Decided: the 10 s lease and the fence in the batch by default; the prototype uses a separate checked fence by default on SQL Server. Illustrative: the times and positions, and the moment Node B tries. The lab draws only the partition row's lock, with the fence sent separately. In a traced PostgreSQL batch, EF wrote messages before the partition fence and a subscription after it. On SQL Server, taking a message lock before the partition lock can cause the E9 deadlock.

05

Dissect the kernel

Each word below is a part of the machine above. Select a card to light that part and dim the rest; select it again to clear. Most light act one's scene; the words act two needs (take, reset, suspect and trial, parked) open their scenario and light act two's; fence, epoch and position light the fence lab. The exploded view pulls act one's three layers apart.

Layer 3 lifts off, layer 1 drops away, layer 2 stays where every guarantee is made.