Equal sitemap counts can hide a broken release: test the URL sets

AI disclosure: This article and its synthetic example were generated with AI assistance. The example was executed locally; no human review, customer experience or production outcome is claimed.

A sitemap contains three URLs before a deployment and three afterward. A count check passes. Yet an expected product URL is missing and a preview URL has taken its place. Counts summarize an inventory; they cannot establish that the inventory contains the right members.

A small release check can make that difference visible. Compare the actual additions and removals against changes explicitly approved for this release. This is useful when a publishing rule, catalog query or route generator changes, even if the resulting XML remains valid.

Make the expected change concrete

In this hypothetical release, /products/coral is retired and /products/dune replaces it. The other two product URLs should remain. The broken candidate emits /preview/draft instead of the replacement. Both candidates contain three unique URLs.

The following dependency-free example accepts decoded URL strings, as might be provided by a sitemap parser or publication pipeline. It deliberately isolates the comparison from XML parsing. Every URL uses example.test; the script makes no network requests. Save it as sitemap-set-comparison.mjs and use Node.js 20 or newer.

import assert from 'node:assert/strict';

const before = [
  'https://example.test/products/amber',
  'https://example.test/products/blue',
  'https://example.test/products/coral',
];
const broken = [
  'https://example.test/products/amber',
  'https://example.test/products/blue',
  'https://example.test/preview/draft',
];
const planned = [
  'https://example.test/products/amber',
  'https://example.test/products/blue',
  'https://example.test/products/dune',
];
const expected = {
  added: ['https://example.test/products/dune'],
  removed: ['https://example.test/products/coral'],
};

function unique(urls, label) {
  const result = new Set(urls);
  assert.equal(result.size, urls.length, `${label}: duplicate URL`);
  return result;
}

function difference(left, right) {
  return [...left].filter(url => !right.has(url)).sort();
}

const scenario = process.argv[2];
assert.ok(['broken', 'planned'].includes(scenario),
  'Usage: node sitemap-set-comparison.mjs broken|planned');
const previous = unique(before, 'before');
const candidate = unique(scenario === 'planned' ? planned : broken, 'after');

assert.equal(candidate.size, previous.size);
console.log(`Count check: PASS (${previous.size} -> ${candidate.size} URLs)`);

const actual = {
  added: difference(candidate, previous),
  removed: difference(previous, candidate),
};
console.log(`Actual: ${JSON.stringify(actual)}`);
console.log(`Expected: ${JSON.stringify(expected)}`);

try {
  assert.deepStrictEqual(actual, expected);
  console.log('Set policy: PASS');
} catch (error) {
  if (!(error instanceof assert.AssertionError)) throw error;
  console.error('Set policy: FAIL');
  process.exitCode = 1;
}

Run each scenario separately:

node sitemap-set-comparison.mjs broken
node sitemap-set-comparison.mjs planned

The first command prints Count check: PASS (3 -> 3 URLs) but finishes with Set policy: FAIL and exit code 1. Its actual addition is the draft URL. The second reports the same passing count, the approved product addition and retirement, and Set policy: PASS, exiting with code 0.

Node's built-in strict assertions let the script check both the numerical invariant and the complete change object. deepStrictEqual throws when the arrays differ; the catch retains a failing process exit status instead of merely logging a warning.

Keep the policy independent of the output

Write the expected additions and removals from the release decision. Do not automatically copy the candidate's diff into the expected object: that would approve whatever the generator happened to emit. Review an unexpected change against the publication records before deciding whether the generator or the expectation needs correction.

An exact policy catches both an unapproved addition and a missing planned addition. A numerical allowance such as “up to one removal” cannot identify which page disappeared. For an unchanged inventory, the expected arrays are both empty. For a growing catalog, remove the demonstration's equal-count assertion and retain the set policy.

The duplicate assertion runs before deduplication can hide repeated declarations. Sorting only makes reports deterministic; it does not rewrite the URLs. Keep expected lists in the same sorted order when approving multiple changes. Preserve path case, trailing slashes and query strings during comparison unless the application has a separately tested canonicalization contract. Google recommends listing the preferred canonical URLs rather than every alternate address. Sitemap construction guidance

Check the artifact you will publish

For a real pipeline, compare the previous successful sitemap artifact with the candidate artifact from the same release under review. Parse XML correctly, collect the intended page loc values, and preserve duplicates until checked. A list taken only from the source database can miss an error introduced during sitemap generation.

Retain the two artifacts, their release identifiers, the approved policy and the resulting diff. Keep an unexpected removal visible until someone can explain it. Replacing the baseline with a failed candidate erases the evidence the next run needs.

This test checks declared membership. Separate checks must cover XML validity, redirects, canonical tags, page availability and indexing directives. A passing set policy does not show that Google fetched or indexed anything: Google explicitly describes sitemaps as discovery aids without guaranteed crawling or indexing. Google's sitemap overview