Data Availability

Glassnode metric updaters are triggered at the end of the resolution interval:

• 1 month resolution: runs on the first day of the calendar month at the full hour between 00:00 UTC and 03:00 UTC

• 1 week resolution: runs every Monday at the full hour between 00:00 UTC and 03:00 UTC

• 1 day resolution: runs daily at the full hour between 00:00 UTC and 03:00 UTC

• 1 hour resolution: runs once every hour at minute :00 (e.g. 9:00, 10:00, 11:00, ...)

• 10 minute resolution: runs at every 10th minute (e.g. 09:00, 09:10, 09:20, 09:30, ...)

Currently, the computation of a metric usually takes between 1 and 10 minutes.

There are exceptions to the the above:

• Updates of entity-related metrics for BTC that are available in 1d resolution only can take up to 1 hour.

When are new datapoints available?

In general it can be assumed that a new datapoint becomes available within the first 10 minutes after the end of an aggregation interval. For example, for a metric with 1h resolution the datapoint for timestamp 10:00:00 (covering the interval 10:00:00-10:59:59) will become available at around 11:10:00.

However, there are nuances to the above:

• As explained in the section on Data Finalization, raw data for blockchains other than Bitcoin and Ethereum is inserted in batches and only processed once all data for the aggregation interval is available. For some on-chain-related metrics featuring these blockchains this may delay the initial release of datapoints:

  • Arbitrum, Base, BNB, Optimism, Ton, Tron, Ripple: release of new datapoints between 5-15 minutes after the end of the aggregation interval

  • Solana: 15-35 minutes

  • Doge: 35-45 minutes

• The initial datapoints released for Bitcoin and Ethereum metrics may not be final and still be subject to change, because not all contributing blocks are available upon their first computation. For more details see our section on Data Finalization.

• Data for Bitcoin may be lagging due to its large block interval (average: 10 minutes) and probabilistic mining timestamps. We will elaborate on this special case in more detail below.

Bitcoin data availability in depth

To reduce the amount of data changes due to block reorderings and limit the probability of initial inclusion of orphan blocks, we wait for one additional confirming block before raw block data is included in the calculation of our metrics. In blockchains using a proof-of-work scheme, such as Bitcoin, the interval between two adjacent blocks is random, with only the average duration indirectly controlled via the Difficulty. In turn, the presence of a block that contributes to a datapoint as well as the subsequent confirming block are probabilistic events.

In particular for time-sensitive applications it is beneficial to know the lag $t$ between the end of an interval and the time of API request. In general, it consists of two contributions:

• The time required to compute the new datapoint. This task takes between 1-10 minutes.

• The time until a new block becomes available in the first place and is picked up by our updater. Especially for metrics in 10 min resolution this becomes an important factor, since the duration is of the same magnitude as the metric resolution. Since our metric updaters typically run at the end of an aggregation interval only, the probability for the expected lags in the following window is constant and changes only when a new updater run has finished.

Denoting the mean block interval $\langle T_b \rangle$ and the metric timestamp interval $T$ (equal to the update frequency), we can model the probability density for the second contribution to the lag $t$ as:

$$p(t) \sim \int_{\lfloor \frac t T \rfloor T}^{(\lfloor \frac t T \rfloor +1) T} t^\prime e^{-{t^\prime}/{\langle T_b\rangle}} dt^\prime \;.$$

Assuming an average runtime of 5 minutes for a metric updater, we arrive at the following combined values for data availability:

	Lag $t$
Mean	25:00 min
25th percentile	14:30 min
Median	22:10 min
75th percentile	32:30 min
90th percentile	44:10 min
95th percentile	53:10 min
99th percentile	1:12 h

In addition to the metric lag $t$, we can also describe the time difference $\tau$ between the releases of two datapoints to very good approximation via a geometric distribution. With the same notation as above, the time difference is a multiple of the metric timestamp interval and update frequency $T$, following the distribution

$$\textrm{P}(\tau=m \cdot T) = e^{- (m-1) T / \langle T_b \rangle}\left(1-e^{-T/\langle T_b \rangle} \right) \;.$$

For metrics in 10m resolution, the numbers are as follows:

Time difference $\tau$	Probability	Cumulative
10 min	0.63	0.63
20 min	0.23	0.86
30 min	0.09	0.95
40 min	0.03	0.98
50 min	0.01	0.99
60 min	0.004	0.998

Finally, we can address another variant on data availability for Bitcoin: What is the time it takes for a new block to enter our metrics, i.e. the duration between a raw block timestamp and the release of the corresponding datapoint? There are three independent contributions we have to consider:

• The time until subsequent blocks arrive, confirming the block (see above).

• A random time between 0 and 10 minutes until the block is picked up by our updaters, assuming a metric with 10m resolution.

• The time $t_\textrm{compute}$ required to compute the metric.

The probability density of the total required time evaluates to

$$p \sim \Big( t \, e^{-t/\langle T_b \rangle} \Big) \ast \Big(H(t) \cdot H(\textrm{10 min}-t) \Big) \ast \Big(\delta(t - t_\textrm{compute}) \Big)$$

where $H$ is the Heaviside step function, $\delta$ the Dirac delta function and $\ast$ denotes a convolution. Ignoring $t_\textrm{compute}$, we have:

Time until block included	Probability	Cumulative
0 min ≤ t < 10 min	0.10	0.10
10 min ≤ t < 20 min	0.33	0.43
20 min ≤ t < 30 min	0.27	0.71
30 min ≤ t < 40 min	0.15	0.86
40 min ≤ t < 50 min	0.08	0.93
50 min ≤ t < 60 min	0.04	0.97
60 min ≤ t < 90 min	0.03	0.999

Example: When are new data points added to BTC on-chain metrics?

For simplicity, we assume that broadcasting of new blocks across the globe and any processing (exporting to databases, computing the metrics) happens instantaneously.

Consider the following blocks and their block timestamps. The latter defines to which aggregation interval in the metrics the block contributes to.

Block	Block Timestamp	Corresponding metric aggregation interval
1000	00:55:00	00:50:00
1001	00:56:00	00:50:00
1002	01:01:00	01:00:00
1003	01:25:00	01:20:00
1004	01:35:00	01:30:00
1005	01:36:00	01:30:00

What happens during the different updater runs?

• 01:00:00: Blocks are available up to incl. 1001, but due to our one block confirmation period we process blocks up to 1000 only. The last block is part of the 00:50:00 aggregation interval, therefore that's the maximum timestamp for which the metric is computed.

• 01:10:00: Blocks are available up to incl. 1002, the latter one being ignored for now due to the confirmation interval. Block 1001 is in the 00:50:00 aggregation interval, therefore this data point is updated and values can change.

• 01:20:00: No new blocks available. Therefore, we don't yet append an extra data point to the metrics.

• 01:30:00: Blocks are available up to incl. 1003, we process blocks up to incl. 1002. Therefore the last data point in the metrics is 01:00:00, as this is the aggregation interval to which block 1002 contributes.

• 01:40:00: Blocks are available up to incl. 1005, we process blocks up to incl. 1004. We have a contribution to aggregation intervals 01:20:00 (block 1003) and 01:30:00 (block 1004). Since there is no block for interval 01:10:00 we forward fill the last known value (from 01:00:00) or set the value to zero (depending on what makes most sense for the particular metric). In total, this adds 3 new data points at once.

The blocks’ first contribution to the metric

Based on this, we can see when a block affects the released metrics for the first time:

Block	Block Timestamp	First contribution
1000	00:55:00	01:00:00
1001	00:56:00	01:10:00
1002	01:01:00	01:30:00
1003	01:25:00	01:40:00
1004	01:35:00	01:40:00
1005	01:36:00	-

First release of a new timestamp

Similar, we can derive when a metric timestamp is published for the first time. The delay $t$ denotes the time since the end of the aggregation interval.

Metric timestamp	First published	Delay $t$
00:50:00	01:00:00	00:00:00
01:00:00	01:30:00	00:20:00
01:10:00	01:40:00	00:20:00
01:20:00	01:40:00	00:10:00
01:30:00	01:40:00	00:00:00