Sharding & Cross-Shard Transactions

Learning Objectives

• Understand sharding concepts and why they matter for scalability
• Learn the 2PC protocol for cross-shard atomicity
• Understand provision coordination (proofs across shards)
• Learn livelock prevention in cross-shard flow

What is Sharding?

Sharding splits the system into multiple subsets (shards), each maintaining its own subset of state and running consensus among a subset of validators. Benefits include higher throughput (parallel execution) and smaller per-shard state. The challenge is cross-shard transactions: when a single logical transaction touches state on more than one shard, the system must coordinate so that the outcome is atomic (all shards commit or all abort).

Two-Phase Commit (2PC)

In 2PC, a coordinator drives two phases:

1. Prepare: The coordinator asks each participant (here, each involved shard) whether it can commit. Each shard runs its part of the transaction (e.g. lock or reserve resources) but does not make the state change visible. It replies yes or no.
2. Commit or Abort: If all participants vote yes, the coordinator sends commit and each applies its state change. If any vote no (or timeout), the coordinator sends abort and everyone releases locks; no state change is applied.

This gives atomicity: either every shard commits or none do. The order in which shards prepare and apply updates is fixed (e.g. by shard ID) so all nodes agree on the same outcome.

Provision Coordination

Besides 2PC (prepare/commit/abort), cross-shard systems often need provisions: after a shard commits its block, it produces a signed proof (e.g. “state at height H for this tx”) and sends it to other shards. A shard that depends on remote state cannot finish its view of the transaction until it has received enough of these proofs (e.g. quorum) from the other involved shards. So there are two coordination ideas:

• 2PC – “Can everyone prepare? Commit or abort.”
• Provisions – “Do we have quorum of proofs from every other shard we need?” Only then can we consider the cross-shard tx complete on our shard.

Livelock Prevention

In cross-shard protocols, livelock can occur if shards keep retrying or re-ordering without ever committing. Common techniques:

• Deterministic order – Use a fixed order (e.g. by shard ID) for prepare and for applying commits, so all nodes agree on the same sequence.
• Timeouts and back-off – If a participant doesn’t respond in time, abort and release resources so the system can make progress elsewhere.
• Conflict detection – Detect conflicting transactions and serialize or reject them so the system doesn’t spin forever.

Next: Hyperscale-rs

This module is the conceptual intro to sharding and cross-shard transactions; Cross-Shard Transactions in Hyperscale-rs is the implementation. In the next module you’ll see how hyperscale-rs implements these ideas: 2PC in the execution crate, provision coordination in the provisions crate, and how protocol order (e.g. ShardGroupId) and livelock prevention are handled. Continue to Cross-Shard Transactions in Hyperscale-rs.