Sitemap

Marketing Glossary ->

Sitemap

What is a sitemap?

 Definition
A sitemap is a page usually available on your website that acts as a collection of all pages on your site and is used to inform search engines of existence of the pages as well as the last time they were modified.

Here is an example of a sitemap. The sitemap is in the XML format and you can usually check sitemaps of most websites by browsing to the sitemap.xml file in the root of the domain name.

Sitemaps help Google, Bing and other search engines like DDG see your new pages and index them. Fresh content improves your search engine rankings and helps your brand capture more market share on search.

Sitemaps can be also be created in HTML. Alternatively Google and other search engines will repeatedly also rely on syndication feeds, TXT files and other formats in addition to looking at your sitemap in XML format.

As per the sitemap protocol

 Pro Tip
The Sitemap index file must:

  • Begin with an opening <sitemapindex> tag and end with a closing </sitemapindex> tag.
  • Include a <sitemap> entry for each Sitemap as a parent XML tag.
  • Include a <loc> child entry for each <sitemap> parent tag.<sitemapindex>

The optional <lastmod> tag is also available for sitemap index files.

While the protocol suggest that <lastmod> is optional Google has recently indicated that they use the lastmod parameter to recrawl the pages. Recently, wordpress 6.5 has introduced the lastmod natively as a part of the sitemap it generates.

Gary Illyes on Sitemap and Lastmod parameter of sitemaps

How to submit your sitemap?

Contrary to popular belief, you don’t always need to submit your sitemap to search engines but it is always useful to do so.

There are three ways of notifying any search engine or crawler of your sitemap.

  1. Submit the sitemap to the relevant search engines using their search console or similar interface
  2. Specify the location of the sitemap in the Robots.txt file
  3. Send an HTTP request to the search engine via the following format.
<searchengine_URL>/ping?sitemap=sitemap_url
 Pro Tip
Google has indicated that it will deprecate the ping functionality in early June 2023 and if you ping Google with your sitemap using the last method it will return a 404.

In fact, in the case of Google Search, the vast majority of the submissions lead to spam. To wit, we’re deprecating our support for sitemaps ping and the endpoint will stop functioning in 6 months.

Google Sitemap Ping Function Page

Do I need a sitemap?

Yes, you need a sitemap. Google & Bing both have indicated that having a sitemap is not required for a smaller site. However, if you wish your content to be found on search engines then we recommend you submit a sitemap.

In our tests of 83 websites without sitemap that made less frequent updates or posts (average of 1 post per month), the search engines failed to index all the pages on the site. On creating and submitting a sitemap, the discoverability and indexing of the pages of these sites increased 101%.

SitemapSitesAggregate Total URLsIndexed
No sitemap83 45001600
Sitemap Submitted8345003219
Experiment comparing Indexed URLS status of sites with and without sitemap

Suffice to say, whether you are a small site or a larger site, we recommend submitting a sitemap.