A URL Rewrite Is a Component Transform, Not String Surgery

Two releases in the past week fixed URL bugs that looked unrelated but shared one design error: code found meaningful structure, then edited the original string as if the delimiters had no grammar.

Astro 7.3.2, released on 8 September, fixed an internationalisation fallback that could turn /energy/en/about into /esergy/en/about. The router had already found en as an exact locale segment, but the rewrite used a broad string replacement and changed the first matching substring instead.

Gutenberg 23.9, released on 2 September, collected three corrections in @wordpress/url: preserve everything after the first = in a query pair, preserve everything after the first ? in a path's query, and contain malformed percent escapes instead of throwing during normalisation.

The common lesson is not merely “use a URL library.” These functions had different jobs: replace one route segment, parse query arguments, and derive a stable preload key. The useful rule is more exact: name the URL component and transformation contract before choosing the parser, mutation, error policy, and serializer.

The repeated angle to avoid

The ten most recent posts here covered polyglot install boundaries, privacy-safe analytics buckets, Shopify POS joins, browser-test topology, package release authority, regional search enforcement, AI review approvals, private cache encodings, product-variant graphs, and htmx state provenance. Older overlapping posts covered OAuth callback matching, multilingual cache variants, canonical URLs, and redirect revalidation.

The weak version of this article would revive the old X needs Y formula: URL handling needs better tests. It would also repeat the recent argument that callback URLs deserve exact security policy.

The sharper thesis is that different URL transformations preserve different identities. A locale fallback changes one path segment while preserving the route around it. A query parser separates each name from its complete value. A normaliser may deliberately reorder equivalent arguments to create a lookup key. A signature verifier may be required to preserve the original octets exactly. Applying one generic “clean URL” operation to all four can create broken routes, silent data loss, key collisions, or failed authorization.

The new information surplus is a review model for those distinctions: component ownership, delimiter cardinality, raw-versus-decoded form, duplicate handling, canonicalisation scope, and failure policy—plus invariants that catch bugs even when no one has thought of the exact bad URL yet.

Fresh evidence and background serve different jobs

Source	Freshness	What it contributes
Astro 7.3.2 release	8 September 2026	The shipped i18n fallback correction and its user-visible path example
Astro fallback pull request and reproduction	Merged 8 September 2026	The exact mismatch between segment-aware discovery and substring replacement, plus regression cases for two routing strategies
Gutenberg 23.9 release	2 September 2026	A fresh release containing the query-delimiter, query-pair, and malformed-percent fixes
getQueryArgs correction	Background implementation, merged 12 August 2026	Why splitting every = truncated padded values and nested URLs, including mutation through addQueryArgs
normalizePath query correction and safe-decoding correction	Background implementations, merged 18 and 14 August 2026	Why only the first ? is a delimiter and how malformed percent input reached preload normalisation
WHATWG URL Standard and RFC 3986 query syntax	Background standards	URL parsing and form-query behaviour; a query may itself contain ? and / data

The Astro release and pull request are fresh primary evidence. The Gutenberg release is a second fresh primary source; its older merged changes supply implementation mechanics. The standards define grammar and browser behaviour. They are not evidence that every application should canonicalise URLs in the same way.

One string contains several kinds of structure

Consider this reference:

https://shop.example/en/search?next=/offers/en?q=red%20shoes&tag=sale#results

It contains at least these components:

scheme:   https
host:     shop.example
path:     /en/search
query:    next=/offers/en?q=red%20shoes&tag=sale
fragment: results

Inside the path are ordered segments. Inside the query are fields. Inside the next field is another path-like value with its own ?. Whether that nested value should have been percent-encoded more aggressively is a producer-policy question; it does not make later delimiters available to the outer parser.

A string operation does not know these roles. It sees several occurrences of en, /, ?, =, %, and &. Its result can be syntactically URL-shaped while changing the wrong identity.

That gives a useful hierarchy:

Layer	Example identity	Safe operation	Dangerous shortcut
URL reference	scheme, authority, path, query, fragment	Parse according to an explicit base and URL contract	Concatenate origins, paths, and user values
Route path	ordered path segments	Select the segment by route position or matched token	Replace the first matching substring
Query sequence	ordered or unordered name-value entries, depending on contract	Parse entries while preserving duplicates when meaningful	Convert immediately to a single-value object
Query pair	first = separates name from value	Split once; later = characters belong to the value	Split on every = and destructure two fields
Percent encoding	encoded bytes interpreted under a component grammar	Decode at a named boundary with a stated error policy	Decode repeatedly until the string “looks clean”
Nested URL value	data inside one outer query field	Parse after extracting that complete field	Let inner ?, =, or & become outer delimiters
Canonical key	application-defined equivalence class	Normalise only dimensions declared equivalent	Use canonical output as proof the original request was safe

The important phrase is “depending on contract.” Query entry order is irrelevant to some caches and material to some signatures. Duplicate tag fields are meaningful to many forms but disappear when reduced to a plain object. A trailing slash can be equivalent in one router and a distinct resource in another.

Astro found a segment, then replaced a substring

The Astro reproduction shows an especially instructive two-stage bug.

The fallback code split the pathname and located a locale by exact segment equality. That part understood the route:

/energy/en/about
        ^ exact locale segment

The old rewrite then returned to the unsplit pathname and performed the equivalent of replacing the first /en substring. The first match began the energy segment, so the result became:

/esergy/en/about

The correct fallback was:

/energy/es/about

This is not a failure to detect the locale. It is a loss of provenance between detection and mutation. The parser knew which segment carried the role, but the mutator kept only its text value. Text was insufficient because the same characters appeared elsewhere.

The fix retains positional meaning: find the locale's segment index, replace or remove that array member according to the routing strategy, and join the path again. The regression tests include earlier segments such as energy, enterprise, and espresso, where a locale token appears as a substring but is not the selected segment.

The transferable rule is:

When discovery returns a structural location, pass that location to mutation. Do not downgrade it to a string and search again.

This applies beyond locales. A route might contain a tenant slug, API version, product handle, file extension, or language code whose text reappears in another segment. It also applies to HTML syntax trees, JSON paths, source-code edits, and catalog graphs: node identity should survive from match to mutation.

The first delimiter changes role; later copies may be data

The Gutenberg fixes expose a second class of bug: a delimiter's first occurrence can define structure while later occurrences belong to content.

For one query pair:

token=abc==

only the first = separates the name from the value. Splitting on every = and assigning the first two results changes abc== to abc. That can damage padded Base64, opaque cursors, signed values, or any application data that legally contains an equals sign.

For a path-like reference:

/foo/bar?redirect=/watch?v=abc

only the first ? starts the outer query. RFC 3986 permits ? and / within query data. Splitting on every question mark and retaining only the second array item silently drops v=abc.

The Gutenberg implementation reports show why this was not a read-only formatting problem:

• getQueryArgs fed addQueryArgs, so adding one field could parse, truncate, and reserialize an existing value;
• the default @wordpress/api-fetch user-locale middleware used that path while appending locale information;
• normalizePath was used to compare preload keys, so two distinct inputs could collapse to one normalised result after their differing suffixes were discarded.

A small parser bug became mutation and identity damage because downstream code trusted its output for more than display.

For a function that deliberately accepts a path reference rather than a full URL, a minimal boundary split needs to account for the fragment as well:

type ReferenceParts = {
  path: string;
  query: string | null;
  fragment: string | null;
};

function splitReference(input: string): ReferenceParts {
  const hashIndex = input.indexOf('#');
  const beforeHash = hashIndex === -1 ? input : input.slice(0, hashIndex);
  const fragment = hashIndex === -1 ? null : input.slice(hashIndex + 1);

  const queryIndex = beforeHash.indexOf('?');
  if (queryIndex === -1) {
    return { path: beforeHash, query: null, fragment };
  }

  return {
    path: beforeHash.slice(0, queryIndex),
    query: beforeHash.slice(queryIndex + 1),
    fragment,
  };
}

This is an implementation pattern, not a universal URL parser. It is appropriate only when the function's contract is a path/query/fragment reference and it must preserve the query's remaining characters. For origins, credentials, internationalised hosts, relative resolution, opaque paths, or browser-equivalent navigation, use the platform URL parser with an explicit base.

Parsing can preserve data that an object model destroys

The platform parser correctly treats the complete suffixes as values. In a small Bun probe, URLSearchParams returned /watch?v=abc for the redirect field and abc== for the token field. It also retained both entries in ?tag=a&tag=b. Bare decodeURIComponent, by contrast, threw URIError for 50%off and an incomplete UTF-8 escape.

Those observations expose two additional policy choices.

First, a query is a sequence, not automatically a dictionary:

?tag=a&tag=b

may mean two tags. Converting it immediately into { tag: 'b' } loses cardinality. Sorting duplicate keys without preserving value order can change a signature, filter, or cache key. Bracket syntax such as filter[color]=red is an application convention, not the generic URL standard's nested-object model.

Second, parsing and validation are separate. A forgiving parser can keep an interface responsive or let a preload normaliser contain one malformed value. That does not certify the value for a redirect, signature, filesystem path, outbound fetch, or authorization decision.

Use the representation that matches the next decision:

• keep URLSearchParams or an ordered entry list when duplicates and order matter;
• map to a multimap when names are keys but repeated values remain meaningful;
• map to a typed object only after cardinality, allowed names, and value formats are validated;
• retain the original reference when logging, signature verification, incident reconstruction, or lossless forwarding requires it.

Malformed percent input needs a role-specific policy

A percent sign begins a byte escape only when followed by two hexadecimal digits. Real systems still receive search text such as 50%off, incomplete escapes, mixed encodings, and hand-built request paths.

There are three defensible responses, but they are not interchangeable.

Role	Recommended failure policy	Reason
Browser-facing search or display	Preserve the raw field or encode the literal percent; show a recoverable error if needed	One malformed field should not crash the entire interface
Preload or cache-key normaliser	Contain the error, retain enough raw identity to avoid collision, and record a diagnostic	Availability matters, but two bad inputs must not become the same key silently
Redirect allowlist or outbound fetch	Parse once, resolve against the allowed base, then validate scheme, host, port, credentials, and destination	A forgiving decode is not an authorization decision
Signature or webhook verification	Verify the exact specified bytes before any lossy canonicalisation	Re-encoding can change signed content
Database or API identifier	Reject values outside the declared encoding and identifier grammar	Silent repair can select another record

Gutenberg's normalizePath moved from bare decodeURIComponent to its existing safe decoder because the function normalises preload and request paths. That choice contains a middleware-wide exception. It should not be copied blindly into a security-sensitive verifier where malformed encoding must cause rejection.

Also avoid repeated decoding. A value containing %252F becomes %2F after one pass and / after two. If different layers decode without recording ownership, data can cross a path-segment, redirect, or access-control boundary only on the second pass. Define exactly which component is decoded, once, and in which representation policy checks run.

Canonicalisation is an identity claim

normalizePath exists because some differently ordered query strings should address the same preloaded request. That is useful canonicalisation. The bug was not that it normalised; it was that it erased suffix data before deciding equivalence.

Every canonicaliser makes a claim of the form:

canonical(a) == canonical(b)
therefore a and b are equivalent for this consumer

The consumer qualifier is essential.

/products?sort=price&page=2
/products?page=2&sort=price

may be equivalent for a data-fetch cache. They may not be byte-equivalent for a signature generated over the original query. Likewise, %7E and ~, + and %20, an omitted port and a default port, or /path and /path/ can be equivalent under one layer and distinct under another.

Do not create one organisation-wide normalizeUrl() helper and use it for caches, analytics, redirects, crawler deduplication, signatures, and database keys. Name the equivalence instead:

canonical_preload_key
canonical_analytics_page
validated_redirect_destination
raw_signature_target
canonical_crawl_url

Each function should document which differences it collapses and which it preserves. A canonical URL emitted for SEO is a publishing assertion about preferred indexing, not a safe key for authenticated requests. An analytics page name may intentionally remove campaign parameters, but that does not justify removing them before attribution ingestion.

Six invariants make better tests than a list of examples

Regression examples are necessary. Invariants make the suite survive the next unfamiliar string.

1. Locality

Changing one component must not alter unrelated components.

For a locale fallback, every segment except the selected locale position remains byte-for-byte unchanged. Query and fragment remain untouched unless the contract explicitly changes them.

2. Suffix preservation

Once the structural delimiter has been consumed, later copies remain data.

Appending =tail to an existing query value should append to that value, not disappear. Adding ?inner=x inside a complete redirect value should not truncate the outer query.

3. Non-collision

Inputs that differ in meaningful data must not produce the same key.

/foo?a=1?b=2
/foo?a=1?c=3

should remain distinguishable unless the named consumer explicitly declares the second question-mark suffix irrelevant.

4. Idempotence

A canonicaliser should normally satisfy:

canonical(canonical(input)) == canonical(input)

Failure often reveals double encoding, unstable sorting, or a parser and serializer that disagree.

5. Cardinality preservation

Repeated fields survive when the contract permits them.

?tag=a&tag=b

must not quietly become one tag because an intermediate object allowed only one value per name.

6. Error containment

Malformed input follows the declared policy without widening the blast radius.

A tolerant UI parser does not throw away the entire page. A strict redirect validator does not “repair” a hostile target and continue. A normaliser does not convert two distinct malformed references into one cache key without an explicit collision rule.

These invariants can drive property-based tests. Generate path segments that contain locale tokens as prefixes, suffixes, and exact values. Generate query values containing every delimiter, repeated fields, empty names and values, Unicode, valid and invalid percent sequences, and nested references. Then assert the properties instead of merely snapshotting whichever output the current implementation returns.

A release matrix for URL transformations

Use one table per transformer. The expected result is less important than recording why it is expected.

Case	Example	Property under test	Expected policy
Token appears inside another segment	/energy/en/about	Locality	Only the locale segment changes
Token appears more than once as a segment	/en/docs/en	Structural selection	Route grammar, not first text match, chooses the locale position
Value contains equals signs	?cursor=abc==	Suffix preservation	Complete value survives parse and serialization
Value contains a question mark	?next=/watch?v=abc	Delimiter scope	Inner question mark remains field data
Repeated key	?tag=a&tag=b	Cardinality	Both values survive if repeats are allowed
Empty value	?q=	Empty-state meaning	Distinguish empty, absent, and invalid as the contract requires
Empty key	?=orphan	Validation	Reject or preserve explicitly; never promote the value into a key
Literal percent	?q=50%off	Failure policy	Tolerate, encode, or reject according to role; do not crash accidentally
Encoded delimiter	?next=%2Fwatch%3Fv%3Dabc	Decode ownership	Decode once at the query-value boundary
Fragment contains question mark	/docs#example?x=1	Component order	Fragment text does not become the outer query
Duplicate parameters reordered	?a=1&a=2	Canonical identity	Preserve duplicate order unless equivalence explicitly ignores it
Second canonicalisation	Any accepted case	Idempotence	Output remains stable

For revenue and acquisition routes, add business assertions after the syntax checks:

• locale fallback reaches an existing page and returns the intended status;
• canonical and hreflang links identify the correct locale route;
• campaign and attribution fields survive middleware that appends its own arguments;
• product filters and pagination retain repeated or opaque cursor values;
• redirect targets pass a post-resolution allowlist and cannot become open redirects;
• cache, preload, and analytics keys preserve the distinctions their consumers require;
• logs retain a safely redacted raw reference plus the parsed decision outcome.

A parser unit test can prove component behaviour. Only an end-to-end route probe can prove the customer, crawler, cache, analytics pipeline, and application all agree on the result.

A review record for one transform

A compact contract turns “URL helper” into something reviewers can reason about:

url_transform_contract:
  name: locale_fallback_path
  input_form: parsed_pathname
  target_component: path_segment
  target_selector: configured_locale_position
  mutation: replace_or_remove_one_segment
  preserves:
    - every_non_target_path_segment
    - query_sequence
    - fragment
    - trailing_slash_policy
  decoding:
    owner: upstream_url_parser
    repeat_decode: forbidden
  invalid_input:
    missing_locale_segment: no_rewrite
    malformed_percent_escape: upstream_policy
  output_use:
    - internal_rewrite
    - redirect_location
  invariants:
    - locality
    - suffix_preservation
    - non_collision
    - idempotence
  business_probes:
    - destination_exists
    - canonical_matches_destination
    - hreflang_reciprocal
    - analytics_locale_matches

For a query canonicaliser, change the target component to query_sequence, declare duplicate and ordering policy, and list the exact consumer—such as a preload lookup. For a redirect validator, include base resolution, allowed schemes and origins, credentials policy, and whether fragments are permitted. For a signature target, replace canonicalisation with exact-byte retention.

The value is not the YAML. It is forcing the function to state which identity it owns and which identities it must not change.

Stop fixing URL bugs one delimiter at a time

Astro's faulty fallback was segment-aware during discovery and substring-based during mutation. Gutenberg's query utilities treated delimiters as globally structural, dropped valid suffix data, and let one malformed escape throw from shared middleware. All produced code that looked reasonable on ordinary URLs.

The durable fix is not a longer collection of regexes. Start from the transform's role:

1. define the accepted input form and base;
2. parse only with the grammar that form promises;
3. retain structural location from match through mutation;
4. preserve raw and parsed representations when different consumers need them;
5. declare delimiter, duplicate, ordering, and percent-decoding semantics;
6. canonicalise only for a named equivalence class;
7. choose reject, preserve, or repair behaviour by operational role;
8. test locality, preservation, non-collision, idempotence, cardinality, and error containment;
9. run live route, SEO, analytics, cache, and redirect probes where the URL affects business outcomes.

A URL is serialised as text, but text is not its whole contract. Paths have positions, queries have entries, delimiters change roles, encodings have ownership, and canonical forms make identity claims. Keep those facts intact from detection through serialization, and a rewrite can change exactly one intended meaning without quietly inventing another.