TECHNICAL CASE STUDY · Analytics · Full stack

ValDash / G4MM4 Scan

Making a proof-of-stake network legible.

Public application · retirement scheduled November 21, 2026

G4MM4 Scan network dashboard with validator and chain activity panels
Public application capture · November 20, 2026. Historical network data.

FROM THE IMPLEMENTATION

Prevent overlapping collectors

JavaScript · SELECTED SOURCE
export async function runSingleFlight(state, key, fn) {
  if (state.get(key)) return false;
  state.set(key, true);
  try {
    await fn();
  } finally {
    state.set(key, false);
  }
  return true;
}
valdash/server/lib/single-flight.js

Only one collector runs for a key at a time. The finally block releases the guard after success or failure so one rejected request cannot wedge future work.

THE ENGINEERING PROBLEM

What needed to work

Validator operators need to understand what a network is doing and whether their validators are performing useful work. Raw execution RPC and beacon responses do not provide that experience: they describe different timelines, arrive at different speeds and can change after a reorganization. I built the current ValDash / G4MM4 Scan application around collecting, reconciling and explaining that data.

Constraints and requirements

  • Mainnet and testnet require separate data and freshness decisions. A slow response from one network must not hold the other network’s homepage hostage.
  • Catch-up collection shares finite upstream request budgets with live queries. Restarting a worker must preserve progress, and schema changes must preserve the existing database.

ARCHITECTURE & DESIGN

How the system fits together

  1. Execution RPC + beacon APIs
  2. Budgeted collectors + reorg repair
  3. SQLite WAL + durable cursors
  4. Express APIs + caches
  5. Validator dashboards

I separated the background collector from the Express serving process. Durable cursors and relational records live in SQLite; browser pages consume application APIs rather than reconstructing the entire history themselves.

SQLite uses write-ahead logging, foreign-key checks and a busy timeout. Collector modules handle acquisition, income, participation and reconciliation, while a reorg watcher maintains observed ancestry and persists unresolved repair work.

Decisions and tradeoffs

Schedule collection explicitly instead of starting work on every timer tick.

Single-flight guards prevent overlapping scans. Fair request lanes and bounded pending work keep a long backfill from consuming every future reservation.

Tradeoff Scheduling adds state and cancellation paths, but makes resource contention a controlled part of the design rather than an accidental queue of promises.

Make reorganization repair durable and distinguish missing evidence from zero.

The watcher follows parent links, records replaced blocks and retains unfinished repair steps. Reconciliation compares served values with chain observations.

Tradeoff Honest incomplete states are less visually tidy than filled-in values, and repair needs bounded retry work, but operators can assess what the system actually knows.

Keep the relational data layer local and make freshness visible at the API.

SQLite keeps each application deployment understandable, while response caches and freshness markers reduce repeated queries without hiding stale fallbacks.

Tradeoff This requires deliberate lock and query management; it is not a claim of distributed database replication or unlimited horizontal write capacity.

PLANNING & EXECUTION

From requirements to working behavior

  1. I divided the implementation into acquisition, persistent state, derived calculations, API responses and browser presentation. Shared status rules keep lookup ordering consistent between JavaScript and SQL instead of duplicating business decisions.
  2. Operational fixes included response-body cancellation, bounded request deadlines and independent network rendering. The serial Ansible rollout enables and verifies peer forwarding before stopping a node, then checks application responsiveness and collected data before restoring direct serving.
  3. The deployment failure path removes the temporary forwarding proxy, retains maintenance already set on the failed node and stops later serial batches. This avoids stale peer proxies pointing at each other while preserving the healthy serving peer. The database remains in place through application replacement.

VALIDATION & RECOVERY

How the behavior is checked

  • Maintained unit tests exercise overlapping intervals, failed runs, fair admission, response-body deadlines, reconciliation and reorg recovery. These target failure mechanisms rather than only checking that endpoints return successful status codes.
  • Browser coverage exercises real page interactions and network states. Historical deployment records include populated page/API observations; those observations describe that release window, not a perpetual uptime guarantee.

Implementation references

Repository-relative references identify the implementation behind this case study. Selected code excerpts are also available in the portfolio.

  • valdash/server/workers/collector.js
  • valdash/server/lib/db.js: getDb
  • valdash/server/lib/single-flight.js: createSingleFlight
  • valdash/server/lib/fair-request-queue.js: FairRequestQueue
  • valdash/server/lib/beacon.js
  • valdash/server/lib/reorg-watcher.js
  • valdash/server/lib/reconciliation.js
  • valdash/server/lib/rpc.js: consumeJson
  • valdash/tests/unit/core-interval-single-flight.test.js
  • valdash/tests/unit/rpc-body-deadline.test.js
  • valdash/tests/e2e/proposer-reorg.spec.js
  • valdash/ansible/playbook.yml: Zero-downtime maintenance window