Astro 7.4.2 turns a 60-second content sync into under a second by counting bytes differently

Astro 7.4.2 shipped on October 21, and one line in its changelog deserves its own post: the content-sync chunking step went from roughly 60 seconds to well under one second, with byte-identical output. Measured on a store of about 44,000 entries. The fix is not concurrency, not caching. It is how the code figures out how many bytes a character takes.

Where the time went

In Astro 7.x, content collections can use experimental.collectionStorage: "chunked" — instead of one giant content store, the data is split into chunks. Splitting means walking the text against a byte budget: how many bytes fit in this chunk, and at which character do we cut.

That requires knowing the UTF-8 width of every character. The old implementation ran each character through TextEncoder: take a character, encode it to a Uint8Array, read the length, accumulate. Correct, but it allocates an encoder buffer per character. Multiply that by every character in 44k entries and you get tens of seconds of pure CPU time.

Look up the code point instead

The fix computes UTF-8 width straight from the Unicode code point: depending on which range the code point falls into, the width is 1, 2, 3 or 4 bytes. That is a table lookup, not an encoding.

// roughly: return the UTF-8 byte width for a code point, no allocation
function utf8Width(cp: number): number {
  if (cp < 0x80) return 1;
  if (cp < 0x800) return 2;
  if (cp < 0x10000) return 3;
  return 4;
}

None of this is new — UTF-8’s own rules bake the width into the code point ranges. The interesting part is why nobody wrote it that way first: TextEncoder exists in both browsers and Node, it is the shortest correct code, and the performance problem only shows up at volume. Content collections are exactly the feature where you develop with a few hundred entries and deploy with tens of thousands.

Why this beats a generic “60x faster” patch

Two details make it trustworthy:

• Byte-identical output. The release note says so explicitly. The chunk boundaries did not move; only the arithmetic got faster. This kind of optimization does not need a golden-file regression test, because the golden file should not change — and if it did, the patch would be wrong.
• It only touches the experimental chunked path. The default path is untouched, so this is a ladder handed to people who already opted in, not a silent behavior change for everyone.

My own stance: I will wait until chunked storage leaves experimental. If the store format breaks, the thing that breaks is the build, not the page — a broken build is recoverable, a drifted format means regenerating everything. But 60 seconds down to 1 second says the chunked direction is right and the implementation was simply naive.

Two more fixes that day

• astro dev with @astrojs/cloudflare no longer fails with Calling require for "node-fetch" when a server dependency such as cross-fetch requires node-fetch at runtime. Bugs like this are nasty precisely because they only appear in dev; build stays green.
• @astrojs/vue@7.1.2 and create-astro@5.3.1 were released alongside for version alignment.

My take

This does not change anything for my site — a few hundred entries never made chunking slow. But it tells you something: Astro’s content layer is now designed for tens of thousands of entries, not for a personal blog with a few dozen posts. That matters for multilingual sites, where two languages plus cover images plus inline figures add up faster than you expect.

Upgrading to 7.4.2 costs nothing. The thing to watch is when chunked storage goes stable: that is the day build times fall from minutes to seconds, and that is the real dividing line for content-heavy static sites.