> ## Documentation Index
> Fetch the complete documentation index at: https://docs.walletwall.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Serving Architecture

> How WalletWall serves data at request time: live per-wallet provider reads, WalletWall-owned BigQuery snapshots behind fixed, bounded read contracts, and scheduled Dune caches — and how every response says which it was.

WalletWall answers two different kinds of question, and it serves them from two different kinds of
data. Keeping the two apart is the architectural rule every other data page builds on.

> [card] Live, per-wallet state
>
> What does *this* wallet hold and do right now? Request-time observations from configured on-chain and
> market-data sources, with observation time and source metadata attached.

> [card] Scheduled, analytical snapshots
>
> What do cohorts of wallets look like over a window? Answered from tables WalletWall builds on a
> schedule: WalletWall-owned BigQuery snapshots, and Dune query results warmed into a cache.

A cached or scheduled result is never presented as live. Every response names its source class and
carries the instant its data is true for, so the UI can label it and a reader can tell a snapshot from
a reading.

## Two classes of data

| Class | Examples | Freshness | What it is never used for |
| - | - | - | - |
| **Live provider reads** | Native and token balances, transaction and transfer history, contract detection, ENS, token prices | The observation time of the request; served from a short provider cache that marks stale values as stale | Cohort statistics, rankings, historical aggregates |
| **WalletWall-owned BigQuery snapshots** | Large stablecoin transfers, daily stablecoin volume, top stablecoin holders and their holdings, multi-chain signer activity, a 48-hour wallet-activity snapshot | The snapshot's watermark, set by a scheduled refresh (every six hours or daily) | Live balances or the present state of an individual wallet |
| **Scheduled Dune results** | Holder Wall cohorts, the Whale Watcher activity baseline, peg history, the quantum dormancy, value-at-risk and migration-readiness feeds | The Dune run time, warmed daily into a cache; routes read the cache, never execute a query | Anything live, anything on demand per wallet |

The [Model Layer](https://docs.walletwall.org/architecture/model-layer.md) records the class on every value as a `sourceType`
(`alchemy`, `etherscan`, `bigquery`, `dune_scheduled`, `dune_cached`, …), and the
[Wallet Evidence Model](https://docs.walletwall.org/architecture/wallet-evidence-model.md) turns provider state into per-claim
status so that an unavailable reading never becomes a confident zero.

## Why WalletWall owns its analytical data

In production, until October 2026 every scheduled analytic was served from Dune. In August 2026 WalletWall decided to
build the analytical surfaces it depends on most as **owned tables in Google BigQuery**, computed from
public blockchain datasets, and to serve them through contracts the application controls. The first
surfaces moved in October 2026, one at a time:

* **Owned schemas and lineage.** Each table has a declared schema, a watermark that says exactly how
  far it is complete, and a refresh that can be replayed and reconciled against its public source.
* **Cost that is bounded in code.** Every statement, refresh run and request-time read carries a byte
  ceiling the deployment cannot raise.
* **Independence from a single third-party query platform.** The migration moved one surface at a
  time, each behind its own switch with the previous source kept as the fallback.

The result is not "BigQuery instead of Dune". It is a layered serving model in which the surfaces with
the highest value and the highest query cost run on owned snapshots, the remaining scheduled feeds
stay on Dune, and live wallet state stays on provider reads.

## How a request is served

Every migrated route follows the same shape. The owned read is one fixed statement, bounded in bytes
and time; everything that could go wrong falls through to the route's previous source, while it still
has one.

> [!NOTE]
>
> Stable Seer's large flows and volume sparklines have finished this path. Their selector and their Dune
> fallback were removed in November 2026: the owned table is their only source, and a read that cannot
> be served answers `503`. The legacy branches below no longer apply to those two routes.

```mermaid theme={null}
flowchart TD
    accTitle: How a migrated route serves one request
    accDescr: A request reaches the route. The route reads its own source switch. In bigquery mode it runs one fixed, bounded BigQuery read of the owned snapshot, judges the result for completeness and staleness, and serves it with a source label. If the read fails, is refused for cost, is incomplete or is stale, the unchanged legacy path answers instead. In shadow mode the legacy path answers first and the owned read is compared off the response path. In legacy mode the owned path is never touched.
    req["Request"]:::input --> sw{"Route source switch"}:::decision
    sw -->|"legacy (default)"| legacy["Legacy scheduled source<br/>answers"]:::process
    sw -->|"shadow"| legacy
    sw -->|"bigquery"| read["One fixed BigQuery read<br/>64 MiB ceiling · 6 s deadline"]:::process
    read --> judge{"Whole result?<br/>Fresh watermark?"}:::decision
    judge -->|"yes"| owned["Serve the owned snapshot<br/>labelled bigquery"]:::output
    judge -->|"refused · failed · incomplete · stale"| legacy
    legacy --> resp["Response with source label<br/>and data instant"]:::output
    owned --> resp
    legacy -. "shadow mode only: compare<br/>after the response is sent" .-> cmp["Shadow comparison<br/>(numbers only, never addresses)"]:::datastore
    classDef input fill:#FAF8F3,stroke:#C9A47A,color:#1E1A14,stroke-width:1.5px;
    classDef process fill:#BF4E32,stroke:#8B3120,color:#FAF8F3,stroke-width:1.5px;
    classDef decision fill:#C9A47A,stroke:#8B6F47,color:#1E1A14,stroke-width:1.5px;
    classDef output fill:#E6DED2,stroke:#9A9186,color:#1E1A14,stroke-width:1.5px;
    classDef datastore fill:#1E1A14,stroke:#9A9186,color:#FAF8F3,stroke-width:1.5px;
```

**1. The route reads its own switch**

Each route that still has a previous source has its own selector with exactly three values:
`legacy` (the default when unset), `shadow` and `bigquery`. Nothing else selects a source — not
the presence of a credential, not another route's switch. Rolling such a route back means setting
its selector to `legacy` or removing it; no table, grant or job changes.

**2. One fixed, bounded read**

The route runs a single `SELECT` whose text is a constant in the code. Four of the five reads take
no parameters at all. The one keyed read (the quantum signature snapshot) passes the wallet as a
named query parameter, so a request value never enters the SQL text. Every read sends a 64 MiB
byte ceiling, waits at most 5 seconds for BigQuery and 6 seconds end to end.

**3. The result is judged before it is served**

A read must return its whole result in one page (the client never follows a page token and
checks the reported row total), and the snapshot's watermark must be inside the surface's
staleness tolerance. A result that fails either check is not served and is not cached.

**4. Serve with a label, or fall back**

An owned answer is labelled (`provider: "bigquery"` in the body, or a source header for the
list-shaped route) and carries the watermark as its data instant. Any failure, refusal,
incomplete or stale read falls through to the route's unchanged legacy path, which answers
exactly as it did before the migration.

## Fixed runtime-read contracts

The request-time reads are deliberately small and deliberately rigid. Tests pin the whole contract.

> [accordion] What every runtime read is
>
> * A single `SELECT` statement with an explicit `LIMIT`, embedded as a constant in its route.
> * Addressed to exactly the owned snapshot tables the runtime identity may read — seven relations
>   in five WalletWall-owned datasets, and no public dataset.
> * Sent with a 64 MiB `maximumBytesBilled` ceiling, a 5-second BigQuery wait and a 6-second
>   request deadline, all code constants.
> * Executed through one client module, the only file in the application that reaches the BigQuery
>   API, using the synchronous query endpoint only.

> [accordion] What no runtime read can do
>
> * Write, create, alter or delete anything: the runtime identity holds read-only grants on the
>   serving tables and the watermarks they are judged by, and nothing on the lease or build tables.
> * Run arbitrary SQL: there is no path from a request to a statement. The keyed read accepts one
>   named `STRING` parameter, validated for shape before the credential is even resolved.
> * Exceed its budget silently: an over-ceiling query is refused by BigQuery without charge, and the
>   client refuses before sending if the ceiling is missing or above the code-owned maximum.
> * Follow a second result page: a result larger than the page it asked for is treated as a
>   failure, not truncated. This rule exists because, on 2026-10-24, a 671-row result was once
>   served as its first 500 rows.

> [accordion] The byte ceiling has two layers
>
> The client enforces a code-owned **absolute maximum of 4 GiB** for any statement it will ever
> send; configuration can select a lower value but never a higher one. Every **serving read sends
> 64 MiB**. The gap is intentional: the 4 GiB figure bounds a runaway statement, the 64 MiB figure is
> the honest size of a snapshot read. Anything that needs more than a few tens of megabytes to answer
> is transformation work, and it belongs in the scheduled refresh, not in a request.

## Validated snapshots and cohorts

Each surface serves a small, precomputed table that a scheduled refresh built and verified. The
request-time read never aggregates history; it reads the latest validated snapshot.

| Surface | What the snapshot holds | Refresh | Not served when older than |
| - | - | - | - |
| Stable Seer large flows | Stablecoin transfers of at least \$100k at par over the last three days, newest first | Every six hours | 2 days (a warning is attached after 30 hours) |
| Stable Seer volume sparklines | Daily transfer volume per stablecoin over the last 30 complete UTC days | Daily, once the day is complete | 3 days (warning after 36 hours) |
| Wallet holdings strip | The newest daily cohort of the top Ethereum wallets by par stablecoin balance, with holdings of at least 1,000 at par; at most 15 holdings are shown and the cut is disclosed | Daily | 4 days (warning after 60 hours) |
| Quantum signature-exposure facts | One row per signer observed signing on a covered EVM chain in the last 90 complete UTC days, read by wallet | Daily | 3 days (warning after 36 hours) |
| Wallet activity (48-hour snapshot) | The most active wallets of the last 48 hours by activity rank | Every six hours | 14 hours (one missed refresh) |

A snapshot is only ever written whole: the refresh verifies the window it loaded before it advances the
watermark, and a cohort or range is published with one atomic statement. The
[refresh model](https://docs.walletwall.org/architecture/data-foundation-refresh.md) explains those guarantees.

## Safe fallback behaviour

Fallback is designed to be boring. On a route that still has a previous source, the owned path can
only make a response better; it can never make a request fail in a new way.

* **The legacy path is unchanged.** When the owned read cannot answer, the route runs exactly the code
  it ran before the migration and returns exactly that response, without the BigQuery label. What that
  answer holds depends on the legacy source's own state: the Dune stablecoin and multichain schedules
  behind the holdings and signature feeds stopped refreshing on 3 August 2026, so a fallback there
  serves an explicit empty or unknown answer — never a fresh-looking number.
* **A route with no previous source says so.** Large flows and volume sparklines answer `503` when
  their owned read cannot be served, and the page hides the section. A `200` with an empty list is a
  read that succeeded, dated by its watermark. For large flows it says only that the newest rows the
  read returns hold nothing for that symbol, not that the symbol had no large flow in the week.
* **Failures are remembered briefly.** After a failed, empty or malformed owned read the route starts
  no new owned read for five minutes, so a BigQuery problem costs at most a dozen reads an hour per
  instance. A stale snapshot does not trip that breaker, because it is already cached and costs no more
  reads.
* **Stale is not served as current.** A snapshot past its tolerance is not served at all; between the
  warning threshold and the tolerance it is served with an explicit warning.
* **Absence is not safety.** The keyed signature read answers "not observed signing on a covered chain
  in the window" for a wallet with no row, dated by the snapshot window, and never substitutes the
  legacy population for it. An absence that cannot be dated falls back.
* **Shadow mode is invisible to the client.** In `shadow`, the legacy path answers first; the owned
  read runs after the response is sent, at most once per five minutes per instance, and only
  allow-listed numbers and closed-vocabulary outcomes are logged — never an address or a hash.

## Source provenance in responses

| Route family | How an owned answer is labelled | How freshness is stated |
| - | - | - |
| Stable Seer flows and volume, wallet holdings | `provider: "bigquery"` in the response body | `queryRunAt` is the snapshot watermark: the instant the data is complete up to |
| Wallet activity (list-shaped response) | `X-WalletWall-Source: bigquery` response header; absent on every legacy answer | Unchanged list body |
| Quantum exposure | `metadata.signatureSource: "bigquery"` when the signature facts came from the owned snapshot; the value-at-risk and migration-readiness facts keep their own per-feed `queryRunAt` and freshness verdicts | `signatureQueryRunAt` and a `freshness` verdict per feed |
| Dune-backed routes | `queryRunAt` (the Dune run) is kept distinct from `cachedAt` (when WalletWall wrote the cache entry) | `warnings`, `freshness` or a staleness message per feed |

## Runtime identity, at the level of principle

* The web runtime reads BigQuery **without a stored key**. Each request's platform identity token is
  exchanged at Google's Security Token Service for a short-lived access token, and the federated
  identity itself holds the grants. A service-account key path exists only as an explicit fallback.
* The runtime identity holds **the right to run jobs and read-only access to the serving tables and
  the watermarks they are judged by**, and nothing else: no dataset-wide grant, no access to the lease
  or build tables, no permission to change IAM.
* Refresh work runs under a **separate identity** with a narrow custom role that can read and update the
  tables it maintains but cannot create, delete or alter them. The two identities never share a
  credential.
* No user value is interpolated into SQL. Request values travel as validated, typed query parameters,
  and only one read accepts any.
* Credentials, project identifiers and infrastructure names are not part of this documentation.

## What serves from BigQuery today

Status recorded on 1 November 2026 from the live routes' own source labels and the migration runbooks.
The wallet-activity row was updated after its 2 November 2026 cutover, and the two Stable Seer rows when
their fallback was removed in November 2026. A route's label is authoritative for the response you are
holding; this table is a dated snapshot.

| Surface | Serving source today | Legacy source |
| - | - | - |
| Stable Seer large flows | Owned BigQuery snapshot (`provider: "bigquery"`) | None: the Dune fallback was removed in November 2026 |
| Stable Seer volume sparklines | Owned BigQuery snapshot (`provider: "bigquery"`) | None: the Dune fallback was removed in November 2026 |
| Wallet holdings strip (Wallet Profile, Whale Watcher node detail) | Owned BigQuery cohort (`provider: "bigquery"`, `dataMode: "balance"`) | Legacy Dune query, not refreshed since 3 August 2026; answers empty for every wallet |
| Quantum signature-exposure facts | Owned 90-day signer snapshot, labelled per response by `signatureSource` | Legacy Dune signature feed, frozen since 3 August 2026; answers unknown. The value-at-risk and migration-readiness feeds stay on Dune |
| Wallet activity (48-hour list) | Owned BigQuery snapshot, primary since 2 November 2026 (`X-WalletWall-Source: bigquery`) | Scheduled Dune query, warmed daily; it answers only when the owned snapshot cannot be served, and it covers a different population, so the two lists are not equivalent |

Holder Wall, peg history, the quantum dormancy feed and the
exposure-report cohorts remain scheduled Dune surfaces warmed into a cache once a day; the Whale
Watcher 12-week baseline and the value-at-risk and migration-readiness feeds remain scheduled Dune
surfaces read cache-aside. The [Dune queries reference](https://docs.walletwall.org/development/dune-queries.md) lists them.

## Dune today

Dune remains a scheduled, cached source. Dune schedules materialise results on Dune's side; a daily
warmer copies the feeds on its list into a cache, and the remaining feeds are read cache-aside. No
public route executes a Dune query. A cold cache on a cache-aside feed triggers one result read only
when an operator has enabled result reads; otherwise the feed answers empty. The UI labels every
Dune-sourced value **Scheduled/Cached**.

## Where to go next

> [card] [Data Foundation Refresh Model](https://docs.walletwall.org/architecture/data-foundation-refresh.md)
>
> How the snapshots are built: slices, watermarks, the single-runner lease, byte budgets and atomic publication.

> [card] [BigQuery Serving & Canary](https://docs.walletwall.org/operations/bigquery-serving-and-canary.md)
>
> How a surface is promoted from shadow to primary: readiness gates, probes, rollback.

> [card] [Data Quality and Observability](https://docs.walletwall.org/architecture/data-quality.md)
>
> The provider roles, observability fields and source-confidence map behind every response.

> [card] [Dune queries](https://docs.walletwall.org/development/dune-queries.md)
>
> The scheduled Dune feeds that remain, their cadence and staleness thresholds.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.