Release notes blog posts
📌 Improved HTML spidering (Robots, Canonical, Rel)
Friday 28 March 2014
Improved HTML spidering (Robots, Canonical, Rel) We now now parse a number of HTML elements to better understand your website and which files should be in your sitemap. Canonical urls We now detect the link rel=“canonical” tag. <link rel="canonical" href="http://xmlsitemapgenerator.org"/> Where we detect this tag and it points to another page we will not include the current page in the sitemap and will instead spider the url specified in href attribute of the tag.
📌 Improved support for character encoding and redirects
Saturday 15 February 2014
Improved support for character encoding and redirects Character encoding We’ve improved our spider so that it can cope with a wider range of character sets including Arabic and Chinese. Don’t forget that for this feature to work correctly it is important that we can understand your website encoding otherwise our spider won’t interpret it correctly and your sitemap will contain strange characters and symbols. http://www.w3schools.com/tags/ref_charactersets.asp Improved HTTP 301 redirect and 302 Moved handling Not only do we now follow HTTP 301 and HTTP 302 automatically.