How this module is scoped. Examples lean on patterns visible in the public hyperscale-rs tree (e.g. synchronous StateMachine, ShardLoop + channels, typed network registration, dispatch pools). Treat every “could improve” row as a hypothesis until you profile your branch. If time or memory deltas are lost in noise, skip the change.
Pacing. Four chunks of roughly 12–22 minutes each; use the table of contents to jump.
The consensus / node core is deliberately synchronous and allocation-conscious: StateMachine::handle(event) -> Vec<Action> (see State machines). Large enums for ProtocolEvent and Action give exhaustive pattern matching at compile time—adding a variant forces every handler to acknowledge it. That is a reliability win that in many OO languages becomes stringly-typed events or deep inheritance trees that are easy to extend incorrectly.
| Code locus (crate · pattern) | Where Rust helps (vs typical managed / OO) | Sharpen / alternatives (material only) |
|---|---|---|
crates/core — ProtocolEvent / Action as rich enums |
One type carries variants with different payloads; no common superclass; compiler checks coverage when routing BFT, mempool, execution. | If a variant group grows unwieldy, consider nested enums or small helper types—cosmetic refactors unless hot-path size affects instruction cache (measure). |
StateMachine trait — pure handle |
No hidden this mutation across layers; explicit &mut self and return value make dataflow obvious for audits. |
Other languages: linear / affine types (research features) can encode “use once” resources more strongly than Rust defaults for some protocols—not a mainstream production switch today; document invariants instead if needed. |
BFT / node composition — generic storage S: Storage |
Monomorphized paths for hot storage access; static dispatch vs pervasive virtual calls in OO designs. | Binary size trade-off: if too many monomorphizations, selective dyn Trait at cold boundaries can help—only after cargo bloaty or similar shows a real issue. |
ShardLoop owns the bridge between network threads and the pinned state-machine thread: Arc for shared storage/topology, channels for NodeInput, and careful Send / Sync bounds on what crosses threads. That maps cleanly to “one writer to the state machine” reasoning—something GC languages can do with discipline but cannot encode as strongly in the type system.
| Code locus (crate · pattern) | Where Rust helps (vs typical managed / OO) | Sharpen / alternatives (material only) |
|---|---|---|
crates/node/src/shard — event_sender + NodeInput |
Cross-thread handoff is explicit; failed send usually means shutdown—no implicit cross-thread garbage of graph edges. |
Watch for lock order when combining Mutex with channel backpressure (e.g. provision dedup); document ordering or shrink critical sections if contention shows up in prod metrics. |
Shared topology snapshot (Arc / atomic load patterns) |
Readers can use consistent configuration without copying whole topology each event—fits read-mostly validator sets. | If snapshot refresh becomes frequent, compare RCU-style swap vs finer-grained updates—only if profiling shows cache line bouncing. |
RPC / production runner → ShardLoop |
Clear boundary: HTTP/async world enqueues into the deterministic loop instead of calling handle re-entrantly. |
Other stacks (e.g. Go) unify with goroutines; Rust’s split is by design here—not a missing feature unless you need simpler mental model for a tiny service (different cost model for a full node). |
For handler names and registration timing, see Transaction flow → Typed network inbound registry.
Heavy crypto, execution, and some verification run off the hot path via dispatch pools; results return through the same channel as gossip (notify closure in dispatch_delegated_action). Simulation can run delegated work inline (SyncDispatch) for determinism—an optimization of semantics, not just speed.
| Code locus (crate · pattern) | Where Rust helps (vs typical managed / OO) | Sharpen / alternatives (material only) |
|---|---|---|
dispatch_delegated_action + pool selection per Action |
Typed actions route to known worker pools; avoids “async all the way” inside the pure state machine while still using threads for CPU-bound work. | Ensure pool sizes and queue depths are observable (production metrics already expose many queue gauges)—tune when backpressure is systemic, not preemptively. |
Tx validation batch + DispatchPool::TxValidation |
Batches amortize per-tx overhead; Rust makes batch ownership (Vec moved into closure) explicit. |
Very large batches can hurt tail latency; adaptive batch caps only if P99 regresses in traces. |
| Deterministic simulation harness | Same logical code paths without hidden scheduler reordering—easier replay than typical threaded + GC nondeterminism. | “Full language” effect systems or async colors—not proven wins for this codebase; keep determinism story in tests/sim unless requirements change. |
Typed register_*_handler methods decode SBOR once, dispatch to closures, and return GossipVerdict / responses—reducing whole classes of wire-format bugs compared to string-keyed handlers. Batch accumulators (tx gossip, validation windows) trade memory for fewer messages.
| Code locus (crate · pattern) | Where Rust helps (vs typical managed / OO) | Sharpen / alternatives (material only) |
|---|---|---|
hyperscale_network::Network trait + registry |
Per-message-type registration is type-checked; failures tend to be compile-time rather than runtime “wrong string id”. | Adding a new wire type still requires discipline in SBOR schemas—Rust cannot verify cross-repo compatibility; rely on tests and staging. |
Provision request dedup (Mutex + Condvar single-flight) |
Protects CPU from redundant Merkle work under fan-out; Drop guard pattern wakes waiters on panic paths—explicit resource pairing. |
If waiter latency spikes, revisit timeout budgets and caps (already documented in code comments)—not a language deficiency. |
Network::request + on_response + ResponseVerdict |
Integrates peer-health scoring with application-level accept/reject of payloads—typed Result on the wire path. |
Deep async/await stacks in other languages can read cleaner for chained retries; Rust uses closures here—ergonomic difference, not a proven large perf gap for this path; refactor only for clarity if a maintainer agrees. |
Language features Rust does not have (and why we barely mention them). True algebraic effects (as in some research languages) could theoretically simplify certain callback-heavy network flows; there is no standard, production-ready effects system in Rust today, and migrating a consensus node for ergonomics alone would be unjustified. JIT from VMs can win on specific micro-benchmarks; consensus hot paths here are dominated by crypto, I/O, and protocol logic—chasing a “better JIT” is not a verified lever compared to algorithmic and batching improvements (see Performance Measurement).