Faster count implementation that's still quite accurate

[brandur]

I wasn't particularly surprised to pop open our PlanetScale report this
morning and see that the count-by-state query used in River UI is now
the demo's most expensive query by cumulative time:

select state, count(*) from river_job group by state
Count: 59,386   ·   p99: 13,131 ms   ·   Cache hit: 87.5%

This has been a known problem for quite some time both in Postgres and
in River UI. The demo's now up to 1.6M completed rows, so counts are
getting slower by the day.

I was having Codex help brainstorm ways that this could be improved, and
it came up with what I think is quite a clever strategy that should be
very fast with minimum downsides:

• The count endpoint starts out with an optimistic query that tries to
  do a full count by all states, but puts a limit of 10k rows on any
  particular one.

• If only the constrained 10k+ information is available, that's what's
  shown, but we immediately try to get a full exact count of all rows
  because even if you have a lot of rows, it's still better to know that
  you have 10,001 versus 50k versus 200k, versus 5M. This longer count
  is kicked off in the background, and is refreshed every 1-30 minutes,
  depending on how long the count is taking. Its results are used when a
  reasonably fresh cache value is available so we can show users the
  best available number. Even when a cached value is available, we still
  prefer a more fresh capped count for states that don't exceed 10k.

• In Postgres, if no cached exactly count is available (most commonly
  right after startup), we use a planner estimate to find a rough
  number. This value will only be in play for a short time until an
  exact count is available.

The type of count (exact, exact_cached, estimated, lower_bound)
is communicated o the UI so that it can give context on counts in
tooltips. For example, it might show 12.3M, ≈987.7K, or 10K+ depending
on the situation, along with source and freshness.

I ran a benchmark and you can see that at large numbers doing a bounded
count stays orders of magnitude more responsive. This might seem like a
small thing, but it keeps the UI more up-to-date and responsive even for
very large users, which is very good.

Rows	Table + indexes	Existing exact count	Bounded count	Planner estimate	Bounded speedup
100K	17 MB	7.16 ms	0.93 ms	0.47 ms	7.7×
1M	174 MB	24.45 ms	1.09 ms	0.66 ms	22×
10M	1.7 GB	203.54 ms	1.01 ms	0.59 ms	201×

I'm sort of hoping that this is a nice compromise for all things -- i.e.
fast at small numbers, reasonably fast at large numbers, and still keeps
precise numbers so we don't have to get too abstract. The downside is
more code complexity, but Codex seems to have done a decent job of
implementation (and I tweaked a bunch of stuff for style) and we have
pretty good tests.

[brandur]

@bgentry Want to take a look at this one? I think it's an improvement UI-wise, but should also be some great blog post material.

[bgentry]

I'm open to an approach like this but I think there are some issues here that need to be addressed or at least explicitly considered. Aside from much of this PR belonging in the river driver layer instead of riverui, I'm concerned about some things that will show up in practical use:

1. For multi-process installs such as when embedding RiverUI into an app with many processes, each of them will independently compute and cache these values rather than sharing the value across them. This is not necessarily an issue on its own since this already happens today and this approach should generally result in fewer count queries (I think?).
2. As a result of (1), though, a UI that's querying multiple backends (i.e. behind a load balancer) will potentially see flip flopping between different cached count values on each refresh. With successful refreshes today, replicas normally differ by roughly the ten-second cache interval. This PR allows healthy exact snapshots to differ by tens of minutes. Replicas may also disagree about the result’s type, so the UI could alternate between 12.3M, ≈12.8M, and 10K+, with different observation times.

At the same time I definitely align with the goal of improving the current setup. Thoughts?

[bgentry]

I was trying to figure out why there wasn't an attached River PR here—seems like this should be a driver-level change instead of regressing to having raw SQL within riverui?

[bgentry]

Presumably this will also not be multi-driver compatible given the pg_stat_* in use 😄

[bgentry]

30 minutes feels awfully long to me, though I see the value of calculating this interval based on the duration of the query (rather than row count or something that doesn't necessarily imply it must be slow).

Table of query duration to refresh interval looks like this per GPT-5.6-Sol:

Exact query duration	Next refresh
≤600 ms	1 minute
2 s	3m 20s
10 s	16m 40s
13.1 s—the PR’s reported p99	~21m 53s
≥18 s	30 minutes
Any error or timeout	30 minutes

I wonder if we could tune this to be just a bit more aggressive?