fix(remote): publish production statistics from the sync that dumped them - #235
Merged
Conversation
…them Statistics reached the server only from refreshStatsIfStale, which returns immediately when there is no drift baseline. Only a push set that baseline, and only refreshStatsIfStale pushed. A project with no snapshot on the server had nothing to seed a baseline from, so it never dumped, so it never got a snapshot. Every project stayed on synthetic statistics unless someone pushed a dump by hand. The sync already dumps those statistics and hands them to the optimizer. Record them there as well. The first sync at boot now publishes, and arms drift to keep the snapshot current. Below STATS_ROWS_THRESHOLD the sync skipped the dump. That threshold decides what the optimizer costs against, where a sequential scan on 200 rows is the right plan. It should not decide whether production's numbers get published, so the sync now dumps either way and publishes what it dumped. The optimizer still plans against the mode the threshold picked. Republishing on every boot costs nothing. The server hashes the tables, and a match only moves confirmed_at, so the timeline stays one row per change. Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.

Query Doctor — 6 successful checks
More details via MCP → get_ci_run({ runId: "019ffe21-35b9-757c-a149-05d0a71fdc77" }) · view run · docs
3 queries read against main on assumed statistics of 10,000,000 rows per table. Sync production stats for costs measured against your real data.
runner.ci.test.ts imported it from remote.test.ts, which registers that file's 11 container tests into runner.ci's suite as well. They ran twice: the full run reported 455 tests where 444 exist. test-utils.ts holds no top-level test(), so importing from there costs nothing. Co-Authored-By: Claude <noreply@anthropic.com>
…ed comments decideStatsStrategy no longer needs the row count before it can dump, so the two queries run together. The dump reads pg_class, pg_attribute and pg_statistic; the count aggregates pg_class. Neither waits on the other. The comments said the same thing in three places. StatsResult.dump keeps the part the code cannot show, which is why a static mode publishes nothing. The placement comment in syncFrom claimed an ordering constraint the position does not buy, since the dump is already taken by then; it now says what is true. A comment in seed-stats-baseline.test.ts described the old sync path and this branch made it false. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
An analyzer running against a database publishes that database's statistics on its own. Today it never does, so every project costs its queries against the 10M-row assumption for its whole life.
What
refreshStatsIfStaleis the only code path that pushes statistics to the server. It returns immediately whenstatsBaselineis unset. The only thing that setstatsBaselinewas a push, and the only other way in,seedStatsBaseline, needs a snapshot the server does not have yet. A project with no snapshot never dumps, so it never gets one.Found on veksen-enterprises/d2armory, whose analyzer has been connected since 2026-07-20. Its latest CI run reports
statisticsMode: fromAssumptionandstatisticsPayloadId: null, and the container logsStatistics refresh skipped: no drift baseline, so nothing can trigger a dump.How
syncFromalready dumps the source statistics and hands them to the optimizer. It now also passes them torecordSourceStatistics, extracted fromapplyStatistics, which sets the baseline and emitsstatsApplied. The first sync at boot publishes, and drift keeps the snapshot current after that.decideStatsStrategyused to skip the dump belowSTATS_ROWS_THRESHOLD. It now dumps either way and returns the dump alongside the mode. The threshold still decides what the optimizer costs against; it no longer decides whether production's numbers reach the server.StatsResult.dumpcarries only statistics measured from the source, so astaticmode loaded from a file still publishes nothing.Republishing on every boot costs nothing on the server:
setFromAnalyzerhashes the tables and a match only movesconfirmed_at.Tests
src/remote/publish-stats-on-sync.test.tssyncs from a container holding 200 rows, which is under the threshold, and assertsstatsAppliedcarriesreltuples: 200. It fails onmainwith zero emissions.Full suite: 455 tests across 44 files, all passing.
tsc --noEmitclean.