Data Finalization
Last updated
Last updated
The latest released datapoint can be subject to change.
Before a block is included into the computation of a metric we wait until a block is confirmed:
BTC: 1 block confirmation
ETH + ERC20 Tokens: 12 blocks confirmation
The reason to wait for block confirmations is to reduce the amount of data changes, e.g. by reducing the probability of initial inclusion of orphan blocks.
Nonetheless, waiting for block confirmations leads to the fact that not always all blocks relevant to the last interval are included, and therefore the computation of the last datapoint of a metric can be initially incomplete.
For this reason we run our updaters several times at the beginning of each day, week and month for the respective metric resolutions (see our section on ). This ensures that all blocks contributing to the last datapoint are included in a timely manner.
Example for a Bitcoin metric at a 1d resolution:
At 00:00 UTC the metric updater will be triggered in order to create the last day's datapoint. Because we wait for a 1 block confirmation before the data of that block is included in the computation of the metric, at this moment there might be still a block missing for that last interval (the previous day in this case). In the next metric updater execution (in case of daily metrics that's 1h hour later) that last block (if confirmed by a subsequent block) will be finally included into the last datapoint and therefore the value of that last datapoint might slightly change. Depending on the block interval, the latest datapoint might also be only updated and complete at later metric updater executions.
Due to the probabilistic nature of mining new blocks for Bitcoin, the time until all blocks have been included in a data aggregation interval can vary a lot. While more than 50% of all datapoints settle within the first 20 minutes, it is recommended to
wait for 2 hours after the initial release of a datapoint to ensure full convergence. For example, a datapoint for a metric with 1h resolution and timestamp 10:00:00 encompasses the interval from 10:00:00 to 11:00:00. Its first release is within the first few minutes after 11:00:00. Data finalization can be expected around 13:10:00.
For Ethereum the rate at which new blocks are created is much faster. Data finalization can typically be expected 10 minutes after the initial release of a datapoint.
Sidenote:
BTC does not enforce monotonic timestamp sequences (e.g. block #10 can have a timestamp that is earlier than the timestamp of block #9). This happens only very rarely, but it means that there is a possibility that a new block is added to the blockchain with a timestamp that is earlier than the last block. If this happens, the datapoint of the interval where this timestamp resides in, will change.
For the following blockchains:
Solana, BNB, Tron, Ton, XRP, Doge, Arbitrum One, Base, Optimism
New blockchain data is appended in batches with the following ingestion frequencies:
Solana, Doge: every 20 minutes
BNB, Tron, Ton, XRP, Doge, Arbitrum One, Base, Optimism: approximately every 10 minutes
Due to the nature of public blockchains, raw data latency may occasionally increase e.g., when a blockchain experiences delays in block production. These situations are beyond the control of our systems.
Standard metrics that are not based on individual address balances (such as total transaction volume) are updated every 10 minutes. We only publish data for time intervals where we have complete data. For example: if a 10-minute metric is scheduled to update a11:20:00, but the most recent block was produced at 11:14:00, we will only include block data up to 11:09:59. Hence, the last datapoint produced will have timestamp 11:00:00.
Note that this contrasts with our current behaviour for Bitcoin and Ethereum, where we also compute “incomplete” datapoints (see above). With the new approach, the data can always be considered final, because every relevant block already contributes to each published datapoint.
Metrics based on balances currently update every 20 minutes and follow the same principle - only intervals with complete data are included.