The meaning of the word ‘sitemap’ has changed over the past decade or so. The internet technologies evolve constantly, therefore, it is completely natural that things change over time. Sitemaps are no exception to this rule.
Back in the day, an HTML sitemap was simply a page containing links to all other pages and resources on your site. You will rarely see this today, but I have recently been working on a site with an old-fashioned sitemap. This kind of sitemap helps humans navigate the site, but that is now considered to be obsolete. The reason is that sitemaps are not intended for humans anymore. They now target a different crowd – search engines and search engine bots.
What does this change mean in practice? First off, the sitemaps are not made to be in HTML format anymore but rather stored as an XML file. The second thing is that modern sitemaps mean little to the human eye, as they are formatted to contain URLs and metadata that is useful to bots. The metadata saved in the sitemap is:
- Last modified – When was the last time the page was changed
- Priority – What priority should search engines give to a page (expressed in percentage)
- Frequency – how often the page will change
Technical aspects of a WordPress XML sitemap
There is a maximum of 50.000 URLs per WordPress sitemap.
Number of sitemaps
You don’t necessarily have to have just one WordPress sitemap. You can have multiple sitemaps with one master sitemap (which would be the sitemap of all sitemaps). This can also provide a workaround for the maximum URLs limit as you will have multiple sitemaps with 50k URLs as the limit.
How XML sitemap benefits your WordPress site
The sitemap for your WordPress site will help in the following ways:
- Search Engine Bots will crawl your website much quicker if you have a sitemap
- It will enable that bots prioritize crawling of certain pages based on the metadata
- Additional information will be taken from ‘Rich media content’ – in case you use a lot of videos and images on your posts and pages
- There’s no downside to adding a sitemap – the worst case scenario is that you will not see any improvement, but certainly, nothing bad can come out of it
The sitemap, however, does not guarantee that all items in it will be crawled and indexed. For example, search engines like Google require a good internal link structure to index an item. There would have to be internal or external links pointing to that particular item on the sitemap.
How to create a sitemap using a plugin?
Creating a sitemap using Yoast SEO
One of the most popular SEO plugins for WordPress is Yoast SEO plugin. It has a lot of features and one of them is the Sitemap Generator.
The Yoast SEO plugin usually auto-generates the XML sitemap in WordPress immediately after installation. You can check to see if the Sitemap is active by going to ‘SEO -> General -> ‘Feature’ tab’ and then scrolling down to find the ‘XML Sitemaps’ feature and turn it on if it isn’t. In case there’s any error with the sitemap, toggle this feature off and on and the map will auto-generate again.
Since the release of version 7.0, Yoast SEO authors have drastically simplified the sitemap settings, or better yet, there are none left. You can basically just turn the feature on and off. Additionally, you can set the search appearances for certain content types and decide should they be included in the sitemap or not.
Many users have welcomed this change. There are those who prefer to have more options when setting up the site map.
Google XML Sitemap Generator
Unlike the Yoast SEO, the Google XML Sitemap Generator gives more control over the sitemap settings. This plugin allows the users to specify frequency and priority parameters alongside the sitemap content settings (what to include and what to exclude).
You can find the plugin settings under ‘Settings -> XML Sitemap’. All of the plugin settings are on the same page which is very convenient in my opinion.
Some other plugins you can check out are:
Other ways to create a sitemap
Some people don’t like to pile up WordPress plugins if they can help it. This is good thinking since WordPress plugins always slow down sites to a certain extent. Luckily, there are other ways to create a sitemap, besides using a WordPress plugin. One of the tools that we can recommend is XML Sitemap Generator. The tool is available in the form of:
- Online generator
- WordPress Plugin
- Desktop program (Windows only)
We will explain how to use the Online Generator in this article.
Click on ‘Online XML Sitemap Generator’, which is on the left side of the screen. This will open a page where you need to:
- Enter your domain name
- Select the method of calculation for ‘Last modified’ date
- Choose the frequency of page changes
- Set the default priority for crawling
- Enter your email address
- Click on the ‘Generate Sitemap’ button
There’s also the ‘More Settings’ button which gives you additional options regarding specific URLs that you can include.
After you click on the ‘Generate Sitemap’ button, you will have to wait for a while. Some sites will take longer to be crawled, therefore, this process can last. Be patient.
In the end, there will be an option to download a zip file containing the generated sitemap, logs and a ‘readme’ file. All you need to do is unpack the archive and copy the sitemap to the root folder of your site.
How to submit the XML sitemap to Google
If you haven’t added your site to the Google Search console, now is the right time. One of the fastest ways is to add the site using ‘URL prefix’ option. If your site is using both ‘HTTP’ and ‘HTTPS’ you will have to add it twice, once with the HTTP prefix and once with the https prefix. This method might not be suitable for websites that have a lot of subdomains or are made available under different URLs, depending on the language (like example.com/fr/, example.com/en/). But if you have a website without a complex URL structure, this is the easiest way.
Enter your site URL (with https:// or http:// prefix) and then you will need to verify ownership. The Google Search Console will generate an HTML file, which you will need to download and upload to the root directory of your site. After you have uploaded the file, click ‘Verify’. You will get the following message:
Next, go to ‘Sitemaps’ and you will see ‘Add a new sitemap’ form. Enter the URL of your sitemap. If you are using the Yoast SEO, the map is at yourdomain.com/sitemap_index.xml. If you are using a different plugin, you will need to enter a different sitemap URL. Every plugin specifies its own sitemap URL which is visible in the plugin settings (depending on the plugin).
It is possible that you will see a ‘Couldn’t fetch’ errors after adding your sitemap. A lot of users have reported this error and it seems to be a bug in the Search Console itself. The new Search Console actually shows ‘Pending’ as ‘Couldn’t fetch’. This will disappear after a certain period which can be anywhere between 24 hours and 24 days.
You can find more information here.
Generally, SEO optimization is a gray area with a lot of obstacles. No matter what you do, it can happen that Google will not index some of your pages and often, there will not be an obvious reason. It requires a lot of trial and error and research, especially if you are new to this. If you want to save yourself some time and avoid frustration, we can install and configure Yoast SEO for you. Feel free to contact our professional WordPress support team.