Data Finalization

Data Finalization

The latest released datapoint can be subject to change.

Bitcoin and Ethereum blockchains

Before a block is included into the computation of a metric we wait until a block is confirmed:

• BTC: 1 block confirmation

• ETH + ERC20 Tokens: 12 blocks confirmation

The reason to wait for block confirmations is to reduce the amount of data changes, e.g. by reducing the probability of initial inclusion of orphan blocks.

Nonetheless, waiting for block confirmations leads to the fact that not always all blocks relevant to the last interval are included, and therefore the computation of the last datapoint of a metric can be initially incomplete.

For this reason we run our updaters several times at the beginning of each day, week and month for the respective metric resolutions (see our section on Data Availability). This ensures that all blocks contributing to the last datapoint are included in a timely manner.

Example for a Bitcoin metric at a 1d resolution:

At 00:00 UTC the metric updater will be triggered in order to create the last day's datapoint. Because we wait for a 1 block confirmation before the data of that block is included in the computation of the metric, at this moment there might be still a block missing for that last interval (the previous day in this case). In the next metric updater execution (in case of daily metrics that's 1h hour later) that last block (if confirmed by a subsequent block) will be finally included into the last datapoint and therefore the value of that last datapoint might slightly change. Depending on the block interval, the latest datapoint might also be only updated and complete at later metric updater executions.

Due to the probabilistic nature of mining new blocks for Bitcoin, the time until all blocks have been included in a data aggregation interval can vary a lot. While more than 50% of all datapoints settle within the first 20 minutes, it is recommended to wait for 2 hours after the initial release of a datapoint to ensure full convergence. For example, a datapoint for a metric with 1h resolution and timestamp 10:00:00 encompasses the interval from 10:00:00 to 11:00:00. Its first release is within the first few minutes after 11:00:00. Data finalization can be expected around 13:10:00.

For Ethereum the rate at which new blocks are created is much faster. Data finalization can typically be expected 10 minutes after the initial release of a datapoint.

Sidenote:

• BTC does not enforce monotonic timestamp sequences (e.g. block #10 can have a timestamp that is earlier than the timestamp of block #9). This happens only very rarely, but it means that there is a possibility that a new block is added to the blockchain with a timestamp that is earlier than the last block. If this happens, the datapoint of the interval where this timestamp resides in, will change.

All Other blockchains

For the following blockchains:

Solana, BNB, Tron, Ton, XRP, Doge, Arbitrum One, Base, Optimism

New blockchain data is appended in batches with the following ingestion frequencies:

• Solana, Doge: every 20 minutes

• BNB, Tron, Ton, XRP, Doge, Arbitrum One, Base, Optimism: approximately every 10 minutes

Due to the nature of public blockchains, raw data latency may occasionally increase e.g., when a blockchain experiences delays in block production. These situations are beyond the control of our systems.

Metric Update Cadence

• Standard metrics that are not based on individual address balances (such as total transaction volume) are updated every 10 minutes. We only publish data for time intervals where we have complete data. For example: if a 10-minute metric is scheduled to update a11:20:00, but the most recent block was produced at 11:14:00, we will only include block data up to 11:09:59. Hence, the last datapoint produced will have timestamp 11:00:00. Note that this contrasts with our current behaviour for Bitcoin and Ethereum, where we also compute “incomplete” datapoints (see above). With the approach, the data can always be considered final, because every relevant block already contributes to each published datapoint.

• Metrics based on balances currently update every 20 minutes and follow the same principle - only intervals with complete data are included.

Market Data

Timestamps reflect when we start computation of metrics, the duration of the calculation takes between 1 and 10 minutes.

Price

Price-related metrics are updated every 10 minutes. A nightly backfill reconciles gaps and finishes by 04:00:00 UTC.

Data completeness for the preceding day:

• 04:00:00 UTC for 10m and 1h resolution

• 00:00:00 UTC on the following day for 1d resolution.

Futures: Open Interest, Volume, Funding Rate, Term Structure, Basis, Liquidations, Reference Rate

These futures metrics are updated every 10 minutes. Any gaps are reconciled during a nightly backfill, which is expected to finish by 04:00:00 UTC.

Data for the preceding calendar day is expected to be complete by:

• 04:00:00 UTC for 10m and 1h resolution

• 00:00:00 UTC on the following day for 1d resolution.

CME

CME metrics are final once the times below are reached:

• 05:00:00 UTC on the calendar day after the trade date for Monday–Thursday data

• 19:00:00 UTC on the following Monday for Friday data

These are the typical cut-off points, after them the figures are not expected to change.

Futures: Liquidations

Liquidation data follows the same 10-minute update cadence. Any missing rows detected for the prior day are back-filled during the reconciliation run at 07:00:00 UTC. All resolutions (10m, 1h, 1d) reflect that back-fill in their next scheduled run.

Data completeness for the preceding day:

• 07:00:00 UTC for 10m and 1h resolution

• 00:00:00 UTC on the following day for 1d resolution.

Options

Options metrics update every 10 minutes. If an exchange delivers data late, we retroactively patch the last 12 hours of history using rolling period. Consequently, a data point is deemed final once it falls outside the 12-hour rolling backfill window.

Data completeness for the last 12-hour window, on average:

• p90 are final within 36m

• p95 stabilize within 1h 20m

• p99 are settled inside ~9h.

Other reasons for data mutability

Next to the amount of blocks that are included in the computation of a datapoint, there can be additional causes for mutable data: retrospective corrections, refined methodologies to compute a metric, or, in particular relevant for exchange-related metrics, additional address labels. In such cases metric values may change even beyond the timescales discussed above. See our documentation on Point-in-Time metrics for more details.