ResourceSEO

Complete guide to sitemaps

XML and HTML sitemaps: what they are for, which format to choose, and how to create, maintain and troubleshoot a sitemap.

Advertisement

A sitemap is a structured file – most often in XML for search engines or in HTML for visitors – that lists the URLs you want crawled, while specifying their relationships.

Think of it as a manifest : crawlers read the XML version to find, understand, and prioritize your content. Users, meanwhile, can view an HTML version that serves as a fallback navigation map.

Who needs one ?

  • E-commerce sites with thousands of product pages
  • News publishers posting articles that are highly time-sensitive
  • Large corporate sites or knowledge bases with deep architecture
  • Sites with orphan pages or complex JavaScript navigation
  • One-page sites or showcase microsites : rarely essential

Why sitemaps matter

Now that the definition is clear, let’s see why this simple file can change your site’s visibility.

Advertisement

SEO benefits

A well-maintained XML file provides Googlebot and Bingbot with a curated list of URLs, helping optimize crawl budget and reduce guesswork.

Several case studies show that URLs included in a sitemap are discovered faster than those accessible only via internal links, especially on large or frequently updated sites. According to estimates Google presented in October 2026, a sitemap is typically processed within about 24 hours (see our article on Google indexing time).

Accessibility and UX

The Web Content Accessibility Guidelines (WCAG) 2.4.5 require “more than one way” to locate a page. An HTML sitemap meets this criterion by offering assistive technologies a linear view of the site. It also helps power users who prefer a complete index over drawer-style menus.

When is it critical ? When is it simply useful ?

It becomes essential as soon as your site exceeds 5 000 pages, updates frequently, or contains content more than three clicks from the homepage. It remains simply convenient for well-interlinked microsites or landing-page funnels.

Advertisement

Choosing the right type of sitemap

Depending on your context, one format or another – or even several – will be more relevant. Let’s take stock.

XML vs HTML : key differences

XML targets bots, supports metadata such as lastmod (in ISO YYYY-MM-DD format), priority or changefreq and is generally located at the root : /sitemap.xml. HTML is for humans, is read visually, and behaves like a standard web page. XML sitemaps are strongly recommended for large, complex, or frequently updated sites, while small, well-structured sites can sometimes do without them. The HTML version remains extra insurance for accessibility and navigation. In a browser, many XML sitemaps display as a table thanks to an XSL stylesheet, a layout Chrome stops applying on November 17, 2026, as our report on Chrome dropping XSLT explains.

Specialized XML sitemaps

Add image, video, or news variants when these media are central to your goals or when inclusion in Google News is a key KPI. Separate files make it possible to respect the 50 000 URL limit and to track each content type separately.

Size and technical limits to know

Each XML file is limited to 50 000 URLs or 50 MB once uncompressed. It can be delivered in Gzip : the compressed size is then unrestricted, but the uncompressed version must remain under 50 MB. Beyond that, create multiple files along with a sitemap index.

Advertisement

Planning your sitemap strategy

Before writing a single line of XML, determine what truly deserves to be indexed and how you will maintain the file day after day.

Determining what to include or exclude

Include only canonical pages and indexable pages. Exclude anything blocked by robots.txt, tagged noindex or returning 3xx/4xx/5xx codes : this tells bots where to invest rather than waste their crawl budget.

Single sitemap or multiple ?

Split everything into logical groups – for example /products/, /blog/, /videos/ – if you are approaching the size limit or want more readable reports. A global index will still allow a one-time submission.

Dynamic vs static generation

Dynamic sitemaps, driven by the CMS, update as soon as content changes ; they ensure accuracy without manual effort. Static files, created by hand, suit small sites that rarely change but require rigorous maintenance.

Advertisement

Creating a sitemap : step-by-step

You know what to include ; let’s move on to the actual implementation, whether handcrafted or automated.

Manual method (small sites)

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/</loc>
    <lastmod>2023-09-15</lastmod>
    <changefreq>monthly</changefreq>
    <priority>1.0</priority>
  </url>
</urlset>

Automated method (any site size)

On a CMS, simply enable built-in modules such as Yoast or Rank Math. Desktop crawlers like Screaming Frog generate the files after crawling and export them in seconds. Need a SaaS solution ? XML-Sitemaps.com or Dyno Mapper handle large inventories without local installation.

Validate, test, and deploy

Run the file through an XML validator, then place it in the root directory or in a subfolder referenced via robots.txt. Test first in pre-production to avoid exposing development URLs.

Optimize and maintain your sitemap

A published sitemap is never “finished.” Here’s how to keep it clean and useful over time.

Technical best practices

Keep URLs clean, lowercase, and only the canonical version. Update lastmod only for substantial changes : inflating it could be interpreted as a signal of low reliability. If your site is multilingual, replicate your hreflang clusters across the different sitemaps.

Submitting to search engines

In Google Search Console, open Indexing > Sitemaps, paste the path, then validate (see screenshot below). Bing Webmaster Tools and Yandex offer similar interfaces ; Baidu, for its part, primarily accepts submission via an HTTP /ping request. For a blog or an online store, the sitemap is still the route Google intends, since its Indexing API is limited to job postings and livestreams, as our article on the Google Indexing API explains.

Ongoing tracking and audits

Schedule a monthly review. In Search Console, compare the number of “Submitted” URLs with the “Indexed” count: a growing gap often signals a crawl or quality issue.

Advertisement

Tools like Semrush Site Audit or Sitebulb detect 4xx errors, non-canonical duplicates, or oversized files.

Troubleshooting common issues

Even exemplary sitemaps sometimes run into snags. Here’s how to fix them quickly.

URL-level errors

Remove or update entries that generate 404s, 301/302 loops, or unnecessary parameters, then rerun a crawl to confirm the fix.

Format and compression errors

Validate the XML structure, ensure dates follow the W3C format, and keep Gzip archives under 50 MB once uncompressed.

Pitfalls of specialized sitemaps

News files are limited to 1 000 URLs and must contain only articles published in the last 48 h ; image sitemaps must include <image:loc> (titles and captions remain optional) ; video indexing fails if robots.txt blocks the feed. Follow each protocol meticulously.

Alignment between robots.txt and sitemap

Declare your sitemap in robots.txt and avoid blocking URLs that appear in it : you’ll prevent contradictory signals that waste crawl budget and hurt rankings. For the rules that apply to AI tools’ bots, see our article on AI crawlers and robots.txt.

Recommended tools and resources

Whether you’re just getting started or looking to refine your practice, these solutions cover every need.

Advertisement

Generation and automation

  • CMS : Yoast SEO (WordPress), Rank Math
  • SaaS : Dyno Mapper, XML-Sitemaps.com
  • Desktop : Screaming Frog, Sitebulb

Audit and monitoring

  • Google Search Console, Bing Webmaster Tools
  • Semrush, OnCrawl, ContentKing

Reading and official documents

Google Search Central Guidelines, the reference for Schema.org markup and the WCAG documentation are authoritative sources.

Advertisement

Leave a Comment