Phase 4: Cross-Shard Transactions & Provisions

Part 0 — The core idea (read this first)

A cross-shard Radix transaction touches multiple shards because its manifest reads or writes accounts/components on different committees. Hyperscale does not ship one shard’s live database to another. Instead:

• Each touched shard gets the same signed tx bytes (mempool + BFT).
• Each touched shard runs the full manifest when its wave is ready.
• Each shard only persistently writes substates it owns.
• For foreign-owned state, the shard receives provisions: real substate bytes + cryptographic proof they existed on the source shard at block height H — then overlays them for one VM run.

Reference scenario (keep this open while reading)

Setup: Account A + component Ca live on Shard 1. Account B + component Cb live on Shard 2.

Manifest (one transaction):

1. Take 50 USDC from account A (vault on Shard 1)
2. Call Cb-Method-1 on component Cb (Shard 2) — balance check on A, move 25 USDC into worktop bucket BUCKET_1
3. Deposit BUCKET_1 into account B’s vault (buckets cannot dangle — funds must land in a vault)
4. Deposit 10 USDC from worktop to B
5. Call Cb-Method-2 — deposit to component Ca (Shard 1)
6. Deposit LP tokens to account B

Trace in sim: cross-shard E2E lab · JMT keys: JMT deep dive

Part 1 — End-to-end flow (20 steps)

Follow these in order. They are the full pipeline from user click to mempool Completed.

A — Submit & routing (steps 1–6)

1. Submit. User posts signed bytes via RPC → SubmitTransaction on a shard loop; async Radix validation extracts declared reads/writes from manifest instructions (manifest_analysis.rs). Plain English: the node reads the manifest like a checklist of every account the tx might touch.
2. Declared sets are static. RoutableTransaction carries declared_reads / declared_writes as NodeId lists — routing, waves, and provisions all derive from these, not runtime discovery (routable.rs). Plain English: the protocol decides shards before execution, from the signed tx, not from “whatever the VM happens to read.”
3. Multi-shard hosting. One NodeHost can run multiple ShardLoops (one per hosted shard). Validator ↔ shard: each ValidatorId sits on one shard committee per epoch (TopologySnapshot), but each shard has many validators — so it is not 1 validator : 1 shard globally; it is N validators → 1 shard. Process ↔ shard is still N:M (one machine can host several loops) (host.rs, topology/snapshot.rs).
4. Admission fan-out. If the node hosts both shards, source shard AdmitAndGossipTransaction; passive shards AdmitTransaction only (mempool without submitted_locally / gossip) (host.rs). Plain English: one shard “owns” the submit; siblings still admit the tx locally but don’t re-broadcast it.
5. Mempool is per-validator, per-shard. Gossip/fetch gets tx bodies onto remote shard mempools separately from provisions; cross-shard txs may sit in quiesce until remote in_flight eases (mempool/coordinator.rs). Plain English: getting the tx into a block is separate from getting state proofs later.
6. Every touched shard executes the whole tx. all_shards_for_transaction unions reads ∪ writes; symmetric for routing (topology/snapshot.rs). Plain English: Shard 2 doesn’t run “only line 2” — it runs the entire manifest when dispatched.

B — Proposal & first commitments (steps 7–12)

7. Proposal (pre-execution). Proposer builds header from tx list; compute_waves groups txs by identical remote-shard dependency sets (wave/computation.rs).
8. Wave identity. WaveId { local_shard, block_height, remote_shards } — Shard 1 wave {2} and Shard 2 wave {1} are different ids (wave/id.rs).
9. First provision commitment — provision_tx_roots. On the source shard’s block header: for each target shard, collect every cross-shard tx in this block that needs outbound state toward that target → list of tx hashes (in strict block order) → compute_merkle_root → one ProvisionTxRoot stored as provision_tx_roots[target] (provision_tx.rs). Plain English: “Shard 1 block H promises Shard 2: you will receive state bundles for txs {TxUSDC, …}.”
10. Why “roots” (plural map, one Merkle root each)? The header field is a map target_shard → root. Each value is one Merkle root summarizing that target’s tx-hash list. Hyperscale name = provision_tx_roots; mentally: outbound_provision_tx_merkle_for_target. Not mempool — only “which committed txs’ state must ship outbound.”
11. Incoming provisions in block — provision_root. On the receiving shard: proposer embeds inbound Provisions bundles in Block::Live.provisions; header provision_root = Merkle root of those bundles’ content hashes (inbound list commitment — separate from outbound provision_tx_roots) (shard/roots/provisions.rs, block.rs).
12. BFT commit. Validators verify roots (waves, provision_tx_roots, provision_root, state_root, …) then vote; QC attests header including state_root at height H.

C — State at H & building provisions (steps 13–15)

13. “Committed state at H”. RocksDB substates + versioned JMT; state_root in header H = tree root after applying block H’s certificates, before executing txs newly committed in H. Plain English: the header’s balance sheet is “after old finished work,” not “after the txs we just added to this block.”
14. Outbound provisions (post-commit, proposer only). build_provision_requests → FetchAndBroadcastProvisions; build_provisions reads JMT at source height H, ships SubstateEntry bytes + multiproof (provisions/coordinator.rs, build.rs, execution/lookups.rs).
15. Inbound verification (where provision_tx_roots is used). Target shard receives gossip Provisions + source CertifiedBlockHeader(H). build_verify_action (verification.rs): (1) read expected_root = header.provision_tx_roots[local_shard]; (2) hash each ProvisionEntry.tx_hash in the bundle → compute_merkle_root → computed_root; (3) if computed_root ≠ expected_root, drop bundle (incomplete/tampered) and retry fetch; (4) else emit VerifyProvisions → JMT multiproof vs QC’d state_root. This is the only place the target compares against provision_tx_roots.

How the target uses provision_tx_roots

Only on inbound gossip/fetch — not when building the source block.

let expected_root = certified_header.header().provision_tx_roots().get(&local_shard)?;
let leaves: Vec<Hash> = provisions.transactions().iter().map(|t| t.tx_hash.into_raw()).collect();
let computed_root = ProvisionTxRoot::from_raw(compute_merkle_root(&leaves));
if computed_root != expected_root { return None; }  // drop → fetch retry
Some(Action::VerifyProvisions { provisions, certified_header })

provision_root on the receiver’s own block is different: Merkle of inbound bundle hashes this shard committed in body.provisions — voters recompute before voting (Part 3).

D — Execution & finish (steps 16–20)

16. Wave setup on commit. After block H commits on a shard: (1) assign_waves buckets each tx into exactly one WaveId on that shard (key = remote shard set) — a tx is not in multiple waves on the same shard in the same block; (2) for each cross-shard tx, record_required(tx, remote_shards) records which peer shards must still deliver provision bundles; (3) ConflictDetector.register_tx checks bidirectional NodeId overlap against already-committed remote provisions and may abort the lower-hash tx before execution (lookups.rs, provisioning.rs, conflict.rs).
17. Anchor at H. ExecuteCrossShardTransactions uses pending_chain.view_at(wave_start_block_hash, H) — not chain tip (action_handlers.rs, wave_state.rs, pending_chain.rs).
18. Overlay execution. ProvisionedSnapshot — provision map shadows local DB per key; Radix VM sees unified state (provisioned_snapshot.rs, executor.rs).
19. Votes → EC → wave certificate. Wave leader aggregates BLS execution votes; collect ECs from all participating shards → FinalizedWave → later block certificates (wave_state.rs, coordinator.rs).
20. Terminal. Mempool Completed when FinalizedWave commits; node locks released after wave finalization (mempool/lock_tracker.rs).

Part 2 — Provisions deep dive

What a Provisions bundle is

Provisions
├── source_shard, target_shard, block_height (H)
├── proof: JMT MultiProof → verifies ALL entries under header(H).state_root
└── transactions[]
    └── ProvisionEntry (per tx)
        ├── tx_hash
        ├── entries[]: SubstateEntry { storage_key, SBOR value | deletion }
        ├── target_nodes[]  (on target shard — conflict detection)
        └── owned_nodes[]   (vault→account map from source)

• Payload = actual substates (balances, component fields), not just roots.
• Proof = cryptographic binding to the source shard’s committed JMT at H.
• Target shard state is not in the bundle — receiver already has local DB; bundle only adds source-owned slices to overlay.

What is a JMT MultiProof?

A Merkle tree proof normally shows “this one leaf is in the tree.” A multiproof is one compact proof that proves many leaves at once belong to the same state_root — shared internal nodes are included once. The provision bundle carries one multiproof for all SubstateEntry rows in that bundle, so the target can check every balance/field byte without a separate proof per key.

Concrete example — USDC scenario (Shard 1 → Shard 2 @ H=42)

Assume our reference tx has hash TxUSDC = 0x7f3a…c901. After Shard 1 commits block 42, the proposer builds outbound provisions toward Shard 2:

One ProvisionEntry for TxUSDC:

Shard 2 verifies: Merkle of [TxUSDC] equals Shard 1 header provision_tx_roots[2]; multiproof shows 120 USDC really lived under Shard 1’s state_root@42. Symmetric bundle Shard 2 → Shard 1 carries B’s vault / Cb fields for the same tx.

Inbound vs outbound

Outbound = “I promise these txs’ state at H.” Inbound = “I received and committed (or verified) those bundles.”

Shadow base (one line)

Lookup rule: provision key → use shipped bytes; else → local view@H. Remote snapshot wins for those keys only.

Snapshot vs execution time

Provisions built after block H commit, before wave execution. Balance in provision (e.g. A = 120 USDC) is pre-this-tx at H. Later blocks on source may change live A to 200; execution still uses view@H + provisions(120) — intentional determinism.

// crates/engine/src/executor.rs
let provisioned = ProvisionedSnapshot::from_provisions(snapshot, &entry_slices);
let receipt = provisioned.execute(&executable, &vm_modules, &exec_config);

Part 3 — Block fields map (cross-shard)

Deeper field tour: Block fields deep dive

Part 4 — Why Merkle / JMT (logic + business)

Business why: Cross-shard atomicity without a single global DB — each shard BFTs its own chain; provisions bridge read-only truth; waves + ECs bridge agreement on what happened when the tx ran everywhere.

Part 5 — Q&A (sharp details)

All 37 teaching bullets. Use as self-check after Parts 1–4.

provision_tx_roots (outbound tx-list commitment)

Payload, proof, “state at H”

Inbound / outbound

Lying, voting, trust

Overlay, anchor H, staleness

Scenario: Cb-Method-1 balance check

Waves, conflicts, failure modes

Validator / mempool misc

Part 6 — One-page mental model

Submit → validate → mempool (each shard) → propose block H
  → header: waves, provision_tx_roots, provision_root, state_root
  → commit (QC)
  → outbound: source proposer builds Provisions@H → gossip/fetch
  → inbound: verify (tx-root + JMT) → absorb
  → when required ⊆ received: Execute@view(H) + overlay
  → votes → EC per shard → WaveCertificate → FinalizedWave in later block
  → mempool Completed

Three commitments, three jobs:

• provision_tx_roots = “which txs’ state must ship”
• provision_root = “which bundles we committed”
• JMT proof = “these bytes are really in source state_root at H”

Together they let Shard 2 run Cb-Method-1 on A’s real USDC at H without trusting a single peer’s gossip bytes.

Code anchors for cross-links

• crates/types/src/wave/
• crates/types/src/provisioning/
• crates/types/src/shard/header.rs
• crates/provisions/src/build.rs
• crates/execution/src/coordinator.rs
• crates/execution/src/provisioning.rs
• crates/engine/src/provisioned_snapshot.rs
• crates/node/src/host.rs