v0.25.1: The Binance aggTrades Historical Limit, Measured

On 17 September we opened the app mid-morning, loaded BTCUSDT on a one-minute chart, and every candle from 00:00 UTC to the moment of launch was hollow. Price was there. Volume was there. The bid × ask breakdown — the only reason to open a footprint at all — was not. It would appear the next day, when the published daily archive landed.

This is the build log for the fix. It covers what the Binance aggTrades historical limit actually constrains, what one hour of bid × ask physically costs to rebuild over public REST, and the regression that hid behind a window which only ever asked for the seven candles you could see.

The symptom: a current day with no bid × ask

A footprint candle is hollow when the platform has the price series but not the individual executions behind it. Open, high, low, close and total volume can come from a klines endpoint in one cheap call. Bid × ask cannot: it requires every trade in the candle, each one classified by which side crossed the spread, then bucketed by price. Two different data paths, two different costs.

So a chart can look complete and be empty where it matters. The candles drew, the volume profile drew, and every cell was blank. The same class of gap we described when footprint volume does not match what the broker reports — the display is honest about what it has, and what it has is not what you assumed.

The regression, dated to 0.22.0

Startup used a helper called startupRecentFromNs. Its job was deliberate and correct: get the first useful prices on screen fast, independent of chart grain, by asking only for the current period plus the six before it. Seven candles. On a 4h chart, rebuilding seven candles over REST would otherwise block on up to 28 hours of prints before anything appeared.

The defect was not in that helper. It was that every import call ended up passing the same window — startup and catch-up both — and the Rust routine that completed the current day was no longer called by any screen. On a one-minute chart, seven candles is seven minutes. Everything before the moment you launched the software stayed without bid × ask until the next day’s archive.

The source now carries the diagnosis as a dated comment in desktop/src/lib/footprint/cryptoStartupImport.ts, so the next person to widen that window knows why it was narrow in the first place. This is the same failure shape as the order-book recorder that called 100% of its orders cancelled: nothing threw, nothing logged, every component did exactly what it was told.

The Binance aggTrades historical limit, and the ceiling we work under

We do not treat the aggregated-trades REST endpoint as an archive. Our client works inside a recent window and reads the published daily archives for anything older. That usable horizon is set at 48 hours, and the Rust side re-checks it on every recent-history call: bounds outside the last 48 hours are rejected before any network round trip (recent_start_ms in desktop/src-tauri/src/commands/crypto_archives.rs).

The client then works under a deliberately lower ceiling — RECENT_DAY_BACKFILL_MAX_HOURS = 36 in cryptoStartupImport.ts. The margin is not decoration. A long pass takes real minutes per hour of data; without it, the oldest hours of a 48-hour request would drift out of the horizon while the pass was still running, and the last requests of a job would fail for a reason that has nothing to do with the code.

The same 48-hour check applies to Spot and Perpetual alike; the retention note sits next to the archive fallback in that Rust file — “Futures REST only retains 48 h”. Unpublished parts of yesterday fit inside the window; anything older has to come from the daily archives, which is a separate path with a separate cost.

What one hour of BTCUSDT Spot aggregated trades costs

Rebuilding bid × ask for the current day on BTCUSDT Spot runs at about 1 min 40 per hour of data. That is the number this post exists for, and over one observed pass, coverage of the last 24 hours moved from 29% to 70% in eighteen minutes, with no intervention (docs/releases/v0.25.1.md). Eighteen minutes of wall clock bought roughly ten hours of footprint history on a liquid spot pair.

That is not latency and it is not our rendering. It is the shape of the endpoint: a busy hour is a paged walk, request after request, each response bounded, the whole thing rate-limited; and then every aggregated trade has to be assigned an aggressor side and folded into a price bucket before a single cell exists. The arithmetic is cheap. The acquisition is the expense.

On “how many BTCUSDT trades per hour”

We are not publishing one. A per-hour aggregated-trade count for a specific pair and a specific hour is a measurement, and it belongs in a dated report before it belongs on a public page. We have the timing; we did not write down the count. Quoting a figure we cannot point at would break the only rule that makes the rest of these numbers worth anything.

Newest hour first, one hour per call, interruptible

recentDayBackfillRanges() builds the work list newest-first and caps every range at one hour. Both choices are about what happens when the pass is long.

• Newest first — the candle under your cursor gains its bid × ask in the first pass, and the chart fills backwards behind it. Oldest-first would leave the live candle hollow for the entire job.
• One hour per range — the pass can be interrupted between hours. Change symbol, change timeframe, close the chart: nothing is stranded mid-request and nothing has to be redone from the start.
• Proven hours cost nothing — an hour already stored is skipped before a request is made. Reopening the app an hour later fetches one hour, not twenty-four.
• The current day goes first — the 180-day deep-history pass waits its turn and resumes on its own. Today’s session matters more than day −40.

How Senzoukria shows it

Each completed hour appears immediately, without reloading the chart. Before this release, a 4h candle whose data was already cached stayed empty until the page was reloaded — cached and invisible is the same thing as missing, from the reader’s side.

A candle that is still being completed shows the minutes it already has, and its remaining holes stay flagged. Nothing is interpolated to make a cell look full. The loading notice carries a live counter of hours covered rather than a spinner, the deep-history notice says it is waiting its turn rather than appearing frozen, and notices no longer fade out on a timer: they stay while their cause lasts and close on a click. A dismissed warning whose cause persists still leaves its marker.

Scope, stated plainly: crypto in Senzoukria is analysis — Binance Spot and Perpetual, Bybit — and there is no crypto order routing. The desktop app is a native Windows application; the macOS build is a universal beta that is not Apple-signed, and Linux ships as AppImage and .deb, all three on the Senzoukria download page. Market-data entitlements for futures venues are billed by the data provider and are separate from the software subscription. The full release notes are on the Senzoukria changelog, and the concepts behind the chart are in the crypto order-flow and footprint guide.

Four mistakes this class of bug punishes

• Reading a hollow candle as low activity. An empty cell means the executions have not been fetched, not that nobody traded. Unknown and zero are different values and should never render the same way.
• Assuming REST reaches as far back as the chart scrolls. It does not. Past the recent window you are reading archives, with different publication delays. Futures venues have their own limits, which is the whole subject of what Rithmic historical data will and will not give you.
• Widening the request window to go faster. Asking for 48 hours in one job puts the tail of the job outside the horizon by the time it gets there. The ceiling is 36 for that reason.
• Expecting trade history to produce a historical heatmap. It cannot. Orders that rested and were pulled never appear in a trade feed, which is why a liquidity heatmap needs recorded order-book data rather than executions.

What is still missing

The default depth is 24 hours, not the full 36 the client could ask for. A pass started late in the day still has a stretch of the early UTC morning that depends on the next archive. An hour refused by a rate limit or a network error stops the pass rather than being retried in place: it resumes the next time the chart is mounted, and what is still uncovered stays visible as a hole rather than being smoothed away.

We would rather ship a chart that tells you it is still filling than one that looks finished at 29% coverage. A gap you can see is a data question. A gap you cannot see is a trading decision made on invented data — and that is the one outcome no amount of speed is worth.