External crawling and server workload

Sitemap Generator Pro uses an external service to crawl your WordPress site as a normal visitor and make an XML sitemap. The crawl runs on a server operated by the developer, while your web server responds to page requests. This can suit shared hosting, where local generation may use limited resources. A dedicated server may have enough capacity to generate a sitemap locally. The service receives your site URL and the fact that it uses WordPress. It cannot reach a local development site on your network.

Pages, documents, and crawler rules

The generator is designed to work with WordPress plugins and can include nearly all indexable file types, such as PDF, XLS, and DOC files. It respects robots.txt, including its crawl-delay directive, and excludes pages marked with a robots noindex meta element. You can use the MB-SiteCrawler visitor-agent group in robots.txt to control which pages the crawler parses and adds to the sitemap.

Free version and Pro media sitemaps

The free version works with sites with up to 500 URLs and does not include picture or video support. Pro adds image sitemaps for images in site content, plus video sitemaps for supported HTML5 video elements and embedded YouTube videos. The supplied information does not specify a separate URL limit for Pro.

Generate and check sitemap.xml

After installing and activating the plugin, open Sitemap Generator in the WordPress sidebar and start the generation process. The sitemap is saved as sitemap.xml in the WordPress root directory. An existing sitemap.xml file will be overwritten without a prompt, so back it up first. When generation finishes, open the new file and check that it is complete and correct for your site.

Requirements and hosting notes

The hosting environment needs cURL 7.18.1 or higher and OpenSSL 0.9.8f or higher. Current PHP versions supported by WordPress should provide a compatible cURL version in most hosting environments. Windows webspace has not been tested.