Dynamics of Node State Machine and BFT

Example: Multi-Shard Transaction

Setup: Shard 0 (component C), Shard 1 (account Bob). Distinguished transaction (Radix manifest style):

# Signer: Alice (Shard 0). C on Shard 0; Bob on Shard 1.
1. CallMethod(Component(C), "transfer_to_bob", Decimal("10"))   # C credits Bob with 10

C (Shard 0) executes the call and produces a receipt to credit Bob (Shard 1) with 10.

1. Mempool (Shard 0): The transaction above enters Shard 0 mempool.
2. Block on Shard 0: Shard 0 producer includes tx in Block_0. Execution: component C runs, produces Receipt(Shard0 → Shard1, Credit(Bob, 10)). Block_0 contains this receipt (or its hash).
3. BFT on Shard 0: Shard 0 validators validate Block_0 (including receipt output), vote BlockVote(Block_0). Quorum on Block_0 → Block_0 finalized → receipt is provisioned to Shard 1.
4. Shard 1 consumes provisions: Shard 1 receives the provision (receipt from Shard 0). Shard 1’s block producer includes “apply Receipt(Credit(Bob, 10))” in Block_1.
5. BFT on Shard 1: Shard 1 validators validate Block_1 (Bob’s balance +10), vote BlockVote(Block_1). Quorum on Block_1 → Block_1 finalized.
6. Finalization: The transaction as a whole is finalized when every block (and thus every sub-tx state) that contains a part of it is finalized. Here: when both Block_0 and Block_1 are finalized.

Message enums: BlockProposal(Block_0) (Shard 0) → BlockVote(Block_0) → Receipt(Shard0→Shard1, Credit(Bob,10)). Shard 1: BlockProposal(Block_1) containing receipt application → BlockVote(Block_1) → Block_1 finalized.

Who is involved: Shard 0 block producer + Shard 0 validators (BFT on Block_0); Shard 1 block producer + Shard 1 validators (BFT on Block_1). BFT object on each shard is that shard’s block; “vote on Shard 0 provisions” = Shard 1 voting on the Shard 1 block that applies those provisions.

Rust Deep Dive: BFT and Node State

The following sections map the flow above to the hyperscale-rs codebase: the node state machine, BftState, voting rules, data structures, and routines (genesis → request build → broadcast → QC → commit).

BFT as a Rust entity

Main object: BftState in crates/bft/src/state.rs.

• There is one BftState per node; it is held by the node’s state machine as self.bft in NodeStateMachine (crates/node/src/state.rs).
• Events are routed in NodeStateMachine::handle() to the appropriate self.bft.on_*(...) method.

So at the stages we care about, “BFT” = the BftState instance plus the BFT handlers that run outside it (e.g. build_proposal, verify_and_build_qc in crates/bft/src/handlers.rs). The handlers are pure functions invoked by the runner when it executes BFT-related Actions; BftState does not build blocks or QCs itself—it emits actions, and the runner calls the handlers.

Voting and rules

Validators vote for a specific block at a given height and round. A vote is "I support this block (hash) at this (height, round)." Votes are collected per block; when a quorum (e.g. 2f+1 out of 3f+1) of validators have voted for the same block, a Quorum Certificate (QC) can be built. The QC proves "enough of us agreed on this block" and is used to extend the chain and to justify the next proposal (parent QC).

Height and round

• Height = which slot in the chain we're deciding (block 1, block 2, …). One height corresponds to one "decision" (one block to commit).
• Round = which attempt we're on for that height. If the first proposal doesn't get a quorum in time, we advance the round (e.g. new proposer, new proposal) and try again for the same height.
• Block = one concrete proposal (header + transactions) identified by its hash. Different proposals in the same round are different blocks.

Metaphor: class voting on "what to do on Friday"

Imagine your class is voting on "what to do on Friday":

• Height = "which vote we're on" (e.g. "Friday's vote").
• Round = "which try" (first try, second try if we re-vote, etc.).
• Block = one of the options (e.g. "movie" or "park").

Rule: For one vote (same height), in one try (same round), each kid can only choose one option. If Alice says "movie" in round 1, that's fine. If Alice later also says "park" in the same round 1, that's cheating—she's saying two different things for the same vote. That's equivocation, and we treat it as Byzantine (bad behavior).

"Different rounds (revotes) are allowed" means: In round 1 Alice said "movie." Time ran out; we start round 2. Now Alice is allowed to change her mind and say "park" in round 2. That's a revote, not cheating, because it's a different round.

So:

• Same height + same round + two different blocks → we detect "this validator said two different things in the same vote" = equivocation (Byzantine).
• Same height + different round → changing your vote in a new try is allowed (revote).

Rules in the protocol

• One vote per (height, round) per validator. The protocol tracks, per validator, what they voted for at each (height, round). If we see two different block hashes from the same validator for the same (height, round), that's equivocation; the validator can be slashed or their second vote ignored.
• Quorum. A block is certified when it has a quorum (e.g. 2f+1) of valid votes for that block at that (height, round). The certificate (QC) is then used as the "parent QC" for the next height.
• Two-chain commit. A block is committed when it has a QC and its child block also has a QC (two consecutive QCs). That's the standard HotStuff-style rule: commit the parent only after the child is certified, so we know the chain didn't fork at the parent.
• Round advancement. If no QC is formed before a timeout, the round advances (view change). A new round has a new proposer (typically round-robin). Validators may then vote for a new proposal in the new round; that's a revote, not equivocation.

Where this shows up in the code

BftState keeps voted_heights: HashMap<u64, (Hash, u64)> — for each height we've voted, the (block_hash, round) we voted for — so we don't vote twice for the same height in the same round. received_votes_by_height (and similar structures) record, per (height, round), which validator voted for which block; that's how equivocation is detected (same validator, same (height, round), two different block hashes). VoteSet collects votes for a single block; when the count reaches quorum, the node can emit Action::VerifyAndBuildQuorumCertificate. Commit only happens when the two-chain condition is satisfied (on_block_ready_to_commit → commit_block_and_buffered).

Core BFT data structures

BftState (the main state machine)

Location: crates/bft/src/state.rs — pub struct BftState { ... }

Identity / config

• node_index, signing_key, topology: Arc<dyn Topology>, shard_group, config: BftConfig, now, last_proposal_time, etc.

Chain state

• view: u64 — current round
• view_at_height_start: u64 — round at start of current height (for backoff)
• committed_height: u64, committed_hash: Hash
• latest_qc: Option<QuorumCertificate>
• genesis_block: Option<Block>

Pending blocks, votes, certified

• pending_blocks: HashMap<Hash, PendingBlock> — blocks being assembled (header + txs/certs)
• pending_block_created_at: HashMap<Hash, Duration>
• vote_sets: HashMap<Hash, VoteSet> — per-block vote collection
• voted_heights: HashMap<u64, (Hash, u64)> — (height → (block_hash, round)) we voted for
• received_votes_by_height — equivocation tracking
• certified_blocks: HashMap<Hash, (Block, QuorumCertificate)> — certified but not yet committed
• pending_proposal: Option<PendingProposal> — in-flight proposal (to correlate ProposalBuilt)

Verification and buffers

• pending_qc_verifications, pending_synced_block_verifications, pending_cycle_proof_verifications
• pending_state_root_verifications, state_root_verifications_in_flight
• verified_state_roots, verified_transaction_roots, verified_cycle_proofs, verified_qcs
• pending_commits: BTreeMap<u64, (Hash, QuorumCertificate)> — buffered commits (two-chain order)
• pending_commits_awaiting_data, buffered_synced_blocks
• last_chain_committed_state_version, last_committed_jmt_state

Supporting types

• PendingBlock (crates/bft/src/pending.rs) — header + hashes + received txs/certs; builds full Block when complete.
• VoteSet (crates/bft/src/vote_set.rs) — collects votes for one block; verified vs unverified; used to decide when to call QC build.
• PendingProposal — { height, round } used to match Event::ProposalBuilt to the proposal we requested.
• RecoveredState — used when initializing from storage (voted_heights, committed_height, latest_qc, jmt_state).
• BftConfig — e.g. proposal_interval, max_transactions_per_block, min_block_interval.

Routines by stage (genesis → request build → broadcast → QC → commit)

Initialized genesis block

• Who: BftState
• Routine: BftState::initialize_genesis(&mut self, genesis: Block) -> Vec<Action>
• Called from: NodeStateMachine::initialize_genesis() → self.bft.initialize_genesis(genesis) (e.g. after the runner creates genesis).
• What it does: Sets genesis_block, committed_hash, last_chain_committed_state_version from the genesis header; logs "Initialized genesis block"; returns SetTimer for Proposal and Cleanup.

Requesting block build for proposal

• Who: BftState
• Routine: BftState::on_proposal_timer(...) -> Vec<Action>
• When: The node receives Event::ProposalTimer. In NodeStateMachine::handle() this is dispatched to BFT only after checking round timeout and gathering ready_txs (mempool), deferred (livelock), aborted (mempool), certificates (execution), commitment_proofs; then self.bft.on_proposal_timer(ready_txs, deferred, aborted, certificates, commitment_proofs) is called.
• What it does: Computes next height from latest_qc / committed_height, round from view; checks we are the proposer for that (height, round); prepares parent QC, timestamp, filters certs/deferrals/aborts, gets parent state root/version; sets pending_proposal = Some(PendingProposal { height, round }); logs "Requesting block build for proposal"; returns SetTimer and Action::BuildProposal { proposer, height, round, parent_hash, parent_qc, timestamp, is_fallback, parent_state_root, parent_state_version, retry_transactions, priority_transactions, transactions, certificates, commitment_proofs, deferred, aborted }.

So the object that does “Requesting block build for proposal” is BftState; the routine is BftState::on_proposal_timer. It does not build the block itself; it emits Action::BuildProposal, which the runner (action handler) executes.

Who actually builds the block and emits “Broadcasting proposal”

Building the block: Not BftState. The runner handles Action::BuildProposal in the action handler (crates/node/src/action_handler.rs): it calls hyperscale_bft::handlers::build_proposal(storage, ...) (crates/bft/src/handlers.rs).

• Routine: handlers::build_proposal<S>(storage, proposer, height, round, parent_hash, parent_qc, timestamp, is_fallback, parent_state_root, parent_state_version, retry_transactions, priority_transactions, transactions, certificates, commitment_proofs, deferred, aborted, local_shard) -> ProposalResult<S::PreparedCommit>.
• It uses storage (JMT) to compute state root and optional prepared commit, builds BlockHeader and Block, returns ProposalResult { block, block_hash, prepared_commit }.

The action handler then produces Event::ProposalBuilt { height, round, block, block_hash } and enqueues it.

“Broadcasting proposal”: When that event is handled, the node state machine calls self.bft.on_proposal_built(height, round, block, block_hash).

• Routine: BftState::on_proposal_built(&mut self, height, round, block, block_hash) -> Vec<Action>.
• It checks pending_proposal matches (height, round); logs "Broadcasting proposal"; builds a PendingBlock from the full block and inserts it into pending_blocks; builds BlockHeaderGossip and returns Action::BroadcastBlockHeader { shard, header } plus actions from create_vote(...) (vote for own block).

So: the object that logs “Broadcasting proposal” and does broadcast + own vote is BftState; the routine is BftState::on_proposal_built. The block value is built by handlers::build_proposal (pure function over storage and proposal parameters), not by BftState itself.

QC formed

• Who: BftState
• Routine: BftState::on_qc_formed(block_hash, qc, ready_txs, deferred, aborted, certificates) -> Vec<Action>
• When: The runner completes Action::VerifyAndBuildQuorumCertificate and produces Event::QuorumCertificateFormed (or similar); the node passes it to self.bft.on_qc_formed(...).
• What it does: Updates latest_qc, persists certified block (PersistBlock), may enqueue Event::BlockReadyToCommit (two-chain rule), may trigger the next proposal. The QC itself is built earlier by handlers::verify_and_build_qc when the runner runs VerifyAndBuildQuorumCertificate.

Committing block

• Who: BftState
• Routine: BftState::commit_block_and_buffered(&mut self, block_hash: Hash) -> Vec<Action> (internal). It is called from on_block_ready_to_commit when the two-chain rule allows committing the parent block.
• What it does: For the block at committed_height + 1, gets (block, qc) from certified_blocks (or pending); updates committed_height, committed_hash, last_chain_committed_state_version; cleans old state; logs "Committing block"; then pushes Action::EmitCommittedBlock { block, qc } and Action::EnqueueInternal { event: Event::BlockCommitted { block_hash, height, block } }. If the next height is already in pending_commits, it loops and commits that too.

Summary: BFT Rust internal flow

Flow (same style as Transaction Flow): genesis → request build → broadcast → QC → commit.

Takeaway: BFT as data structures = BftState plus PendingBlock, VoteSet, and related maps/sets. BFT as routines = the on_* methods on BftState plus the pure handlers (build_proposal, verify_and_build_qc) used by the runner when executing BFT actions.

Quiz: BFT inner workings

Answer based on the content above. Pass threshold: 70%.