Hyperscale-rs Overview & Setup

Learning Objectives

By the end of this module, you will:

• Know what hyperscale-rs is and where to find the repo and docs
• Understand the best ways to trace transaction flow end to end (run, debugger, logs)
• Know what it takes to run real nodes (requirements, local cluster, Docker, or distributed) and how that differs from simulation
• Know the next step: complete the Crate Groups module and its quizzes

1. Intro to the project: what it is and links

Hyperscale (the protocol)

Hyperscale is the BFT consensus protocol designed for high-throughput, sharded blockchain networks. It defines how validators agree on blocks, handle cross-shard coordination, and achieve finality. The protocol and the Rust implementation live in the same repository.

Hyperscale-rs (the implementation)

Hyperscale-rs is the Rust implementation: a Byzantine Fault Tolerant (BFT) consensus system for high-throughput, sharded networks. It is not a full blockchain platform — it is the consensus layer that decides which transactions get included and in what order. Execution, storage, and user-facing apps are separate.

Key links: Open the repository for code and protocol docs. README, trait, and node-state paths are listed in Suggested crates / files to study at the end of this module.

Design: pure consensus (no I/O in core), state machine model, event-driven (events in → actions out), deterministic so the same code runs in simulation and production.

2. Trace flows end-to-end

The system is "events in, actions out": the node receives Events and returns Actions; the runner does I/O (network, storage). Below are three flows with step-by-step instructions: where to add tracing::debug!, the command to run the sim, and the log order you should see. Do the prerequisite once, then pick one flow at a time.

Prerequisite: Build and run (do this first)

1. Clone: git clone --recurse-submodules https://github.com/flightofthefox/hyperscale-rs.git, then cd hyperscale-rs. If you already cloned without submodules: git submodule update --init --recursive.
2. Build: cargo build --release. Wait until it finishes without errors.

Tracing: Use tracing::debug! (or debug! with use tracing::debug). Run the sim with RUST_LOG=debug so these logs show up. Remove or comment out the lines before committing.

Flow 1: Transaction submit (workload → sim runner → node → mempool)

Overview: The simulator workload submits a transaction via schedule_initial_event(..., NodeInput::SubmitTransaction { tx }); the node's ShardLoop receives it, gossips to shards, and runs the validation batch pipeline. Raw gossip arrives as NodeInput::TransactionGossipReceived (stays inside ShardLoop), valid txs surface as NodeInput::TransactionValidated, which becomes ProtocolEvent::TransactionValidated for NodeStateMachine; mempool admission happens in on_transaction_gossip. You see submit → validation → protocol event → pool.

1. Where to add debug!

2. Command to run (tx submit in simulation)

RUST_LOG=debug cargo run --release --bin hyperscale-sim -- --seed 42 -s 1 -v 4 -d 5 --tps 20

• RUST_LOG=debug — your debug! and existing debug logs appear.
• --seed 42 — reproducible run.
• -s 1 — one shard (simplest path).
• -v 4 — 4 validators (BFT quorum).
• -d 5 — 5 seconds.
• --tps 20 — low TPS so logs are easier to follow.

3. Optional shorter run

To see just a few submissions: RUST_LOG=debug cargo run --release --bin hyperscale-sim -- --seed 42 -s 1 -v 4 -d 2 --tps 5 (2 seconds, 5 TPS).

4. Summary — log order for each tx

Flow 2: Block vote (message in → BFT → vote out)

Overview: A vote for a block arrives from the network (or the node produces its own vote). The simulation runner decodes it and feeds an event into the node; the BFT layer processes it and may emit Action::SendMessage (vote); the runner sends it over the simulated network. You see the consensus vote round-trip.

1. Where to add debug!

2. Command to run

Same as Flow 1 so that proposals and votes happen: RUST_LOG=debug cargo run --release --bin hyperscale-sim -- --seed 42 -s 1 -v 4 -d 5 --tps 20. Votes occur as the sim runs; watch for your four log lines in order.

3. Summary

Flow 3: Block commit (commit → execution)

Overview: BFT agrees on a block and commits it. The node receives a "block committed" event; the state machine hands off to the execution layer, which runs the transactions in that block. You see consensus output becoming execution input.

1. Where to add debug!

2. Command to run

Same as Flow 1: RUST_LOG=debug cargo run --release --bin hyperscale-sim -- --seed 42 -s 1 -v 4 -d 5 --tps 20. Blocks commit as the sim runs; you should see "BlockCommitted, entering execution" then "execution: applying committed block" when a block commits.

3. Summary

Understanding simulator output

When you run the sim, you may see metrics like these. Hover the terms for definitions:

• Lock contention (peak) / Locked nodes — Locked nodes (peak) is the maximum number of distinct state nodes (accounts, components) held by in-flight transactions at any moment. Higher = more contention.
• In-flight — In-flight transactions are submitted but not yet in a final state (completed, rejected, retried) at the time the run stopped. “In-flight: X (at cutoff)” = X txs were still in progress when the sim ended.
• Account usage – Skew ratio — Skew ratio = max_selections / avg_selections across accounts used by the workload. 1.0 = perfectly even; higher (e.g. 2.50 or 12.50) means a few accounts were used much more than others.

Full picture: The complete end-to-end flow — from user signing a transaction to finality — is covered in a dedicated Basic module: Transaction Flow: User to Finality. Use that module for the full diagram and to map every step to crates.

3. Running real nodes (production) vs simulation

The rest of this module uses the simulator (hyperscale-sim): one process, simulated time, in-memory network. To run real validator nodes (separate processes, real libp2p, real RocksDB, real RPC), you use the production runner and launch scripts. Below: what it takes and how to do it.

Production threading (brief)

Validators use the same ShardLoop / state machine as simulation, but in production that loop runs on a dedicated std::thread (the shard-loop thread), optionally pinned to a CPU core for steadier scheduling, while Tokio runs libp2p, timers, and related async work; heavy work is often offloaded to thread pools and results return via channels. This is not a special “test-only” loop—it is how live nodes advance consensus. For a full explanation (run_shard_loop, core_affinity::set_for_current, simulation vs production), see E2E harnesses (simulation + production). A compact “threading ladder” of terms lives in State Machines & Event-Driven Architecture.

Requirements

• Build: cargo build --release (same as for sim). This builds the hyperscale-validator (or equivalent production binary) and the launch scripts use it.
• Topology: Number of shards and validators per shard. The launch scripts generate validator keys and configs for you.
• Network: Each node needs a libp2p listen address and (for distributed) reachable IPs/ports. Ports (e.g. 9000 UDP/TCP for P2P, 8080 for RPC) must be open on the host or between machines.
• Storage: Each validator writes RocksDB data to a directory; the scripts create these or use --clean to start fresh.
• Genesis: The cluster needs a shared genesis (validator set, etc.). The launch scripts generate this when they generate configs.
• Docker (optional): If you use the Docker-based cluster, Docker (and Docker Compose) must be installed. Allocate at least 8–10 GB RAM to the Docker VM/Desktop so validators have enough memory.

How to run a local cluster (one machine)

Process-based (fastest for iteration):

./scripts/launch-cluster.sh

This starts a cluster of validator nodes as background processes on your host. Options:

• --shards <N> — Number of shards (default: 2).
• --validators-per-shard <M> — Validators per shard (default: 4).
• --clean — Wipe data directories before starting.
• --monitoring — Start Prometheus and Grafana for metrics.

At the end, the script prints the RPC ports for each node (e.g. 8080, 8081, …). You need these for the spammer or for submitting transactions.

Docker-based (closer to production):

./scripts/launch-docker-compose.sh

Runs the cluster inside Docker containers. Ensure Docker has at least 8–10 GB RAM. Options include --shards, --validators-per-shard, --memory, --cpus, --latency. Stop with:

./scripts/stop-docker-compose.sh

How to run a distributed cluster (multiple machines)

To run validators across several physical hosts or VMs, use the distributed workflow (see the repo’s README_DISTRIBUTED.md):

1. Generate configs on one machine: ./scripts/generate-distributed-config.sh --hosts "IP1,IP2,..." --nodes-per-host N. This creates keys and config.toml per node under distributed-cluster-data/.
2. Copy the hyperscale-validator binary and each host’s config directory to the corresponding machine.
3. Start each validator on its host: ./hyperscale-validator --config /path/to/node-X/config.toml.

Prerequisites: Rust (for building and generating configs), SSH to all hosts, and ports 9000 (UDP/TCP) and 8080 (TCP) open as needed for P2P and RPC.

Load testing real nodes

After the cluster is up, use the hyperscale-spammer to send traffic to the RPC endpoints. Use the ports printed by the launch script (they depend on shards and validators):

./target/release/hyperscale-spammer run \
  --endpoints "http://localhost:8080,http://localhost:8081" \
  --num-shards 2 \
  --validators-per-shard 4 \
  --tps 100 \
  --duration 30s

Summary: Simulation = one binary (hyperscale-sim), no real network or disk. Real nodes = launch script or Docker/distributed + hyperscale-validator per node, real libp2p and RocksDB; then use the spammer or your own RPC client to drive load. For exact flags and troubleshooting, see the repo README.md and README_DISTRIBUTED.md.

Important crates for overall flows

When studying single-shard and multi-shard flows, focus on node, mempool, shard (per-shard consensus), beacon (validator set / topology), execution, and types; multi-shard adds provisions and ConflictDetector in crates/execution/src/conflict.rs. For simulation tracing, use simulator and simulation (NodeHost, ShardEvent queue) — paths in the study table.

See Transaction Flow for the stage diagram and crate-group table.