sitemap generators: how to generate sitemaps for a website or blog
Sitemap is a list of URLs to webpages in your website that are to be made accessible to search engines and web crawlers. There are several different reasons to have a sitemap for the website, much of which is related to Webpage SEO.
This URL list is usually stored in an XML file, using an XML structured format. This format also allows you to specify other metadata information about each URL such as the relative importance of the page, last modified date and expected frequency of modification. You can also specify sitemap as an HTML or a text file, but XML format is the most appropriate for SEO and search engines.
You can generate sitemap in several different ways. It primarily depends on what web platform is used to design and implement your website. You might have a website which uses simple HTML with hand crafted webpages or a web blog on a platform such as WordPress or Blogger or even something in between.
If you have a website where URLs and webpages generally stays limited in number and are static, such as a product or business website then you could generate the sitemap manually or using an online sitemap generator. If your website is constantly adding new URLs such as a web blog then you might want to use some kind of an offline auto sitemap generator.
We will look at some of the commonly used sitemap generator applications first, and then delve into how you can create or modify one manually if needed.
Generate Sitemap for WordPress
If you are on a platform like WordPress which allows you to have plugins, then the quickest and easiest method is to use a sitemap generator plugin. This does have the advantage that it is automated, which means anytime you create a new page or change an URL, the sitemap is modified to include the new URL without any intervention from you. This is very beneficial if you have a blog and you post several new articles a day or week.
There are several different sitemap plugins available for WordPress. Some SEO plugins also come with sitemap generation and support capabilities built in to it, in which case you might not have to install another plugin.
Google XML Sitemaps: This is one of the highest rated plugin for WordPress and it does the job well. The plugin supports all generated pages such as posts, pages, archives etc and notifies varies search engines of the change when the sitemap is modified.
WordPress SEO by Yoast : This is a all-in-one SEO plugin that supports XML sitemaps as well. If you already use this plugin then you can enable the sitemap feature in this.
Online Sitemap Generator
If you have a website which is not developed on top of WordPress (or another blogging) framework, then you will need to generate the sitemap yourself. The easiest and quickest way to do this is to use an online sitemap generator.
You will need to have the website mostly complete and online before you can use an online generator. There are a lot of free services that can do the job. A simple Google search for "XML online sitemap generator" will pull up a lot of websites.
Unfortunately a lot of these free online generator support only a limited number of files. The limit do vary between site to site, so if you happen to have a large website with several thousands of pages then it will be wise to check beforehand. You could also use a offline sitemap generator instead.
Manual or Offline Creation of Sitemap
As mentioned the online generators are only helpful once you have the website mostly completed and online. If you are in the process of developing the website locally and want to generate a sitemap then you might have to do it either by hand or by using a generator application that can be run locally or offline.
Again offline sitemap generators unlike online ones are harder to come by as it has a very specialized set and limited audience. There are several available for Windows platform, both free as well as paid. In Linux, you might want to give SiteMap Generator from Google Webmaster tools a try. Obviously it involves some manual installation and configuration.
Another option is Google Sitemap Generator, which does not seem to have much development and maintenance during recent times. It does still work though. phpSitemapNG is another option that is available offline. You will need to have PHP to run the tool offline.
If all else fails, you can always use a text editor to manually create a xml file. This is practical if and only if your website contains very few page. It is usually a nightmare to maintain in the long run.
If an external or online sitemap generator can find your pages, then so can Google crawlers. This means that the sitemap that was generated contains URLs that could have been found by Google anyway. But sometimes your website may contain webpages that is not directly linked from other pages or the URL links are generated dynamically. This will be make it real difficult for crawlers (including online sitemap generators) to identify them correctly.
You will then need to insert these URLs manually into the generated sitemap. You can easily do this by opening the XML sitemap file in a text editor and manually editing it. You could also generate multiple XML sitemap files by using different URLs as the starting point and then merging them into one sitemap file.
A good reference point for finding the right sitemap generator is https://code.google.com/p/sitemap-generators/wiki/SitemapGenerators. This page actively maintains a list of various different sitemap generators.