Sitemap is an XML file that contains a list of URLs to webpages in your website that are to be made accessible to search engines and web crawlers. There are several different reasons to have a sitemap for the website most of which is related to SEO.
This URL list is usually stored in an XML file, using an XML structured format that allows you to specify other metadata information about each URL such as the relative importance of the page, last modified date and expected frequency of modification. You can also specify sitemap as an HTML or a text file, but XML format is the most appropriate for SEO and search engines.
You can generate sitemap in several different ways. It primarily depends on what web platform is used to design and implement your website. You might have a website which uses simple HTML with hand crafted webpages or a web blog on a platform such as WordPress or Blogger or even something in between.
If you have a website where URLs and webpages generally stays limited in number and remains mostly static, such as a product or business website then you could generate the sitemap manually or using an online sitemap generator. If your website is constantly adding new URLs such as a web blog then you might want to use some kind of an auto sitemap generator.
We will look at some of the commonly used sitemap generator applications first, and then delve into how you can create or modify one manually if needed.
Generate Sitemap for WordPress
If you are on a platform like WordPress which allows you to have plugins, then the quickest and easiest way is to use a sitemap generator plugin. This does have the advantage that it is automated, which means anytime you create a new page or change an URL, the sitemap is modified to include the new URL without any intervention from you. This is very beneficial if you have a blog and you post several new articles a day or week.
There are several different sitemap plugins available for WordPress. Some SEO plugins also come with sitemap generation and support capabilities built in to it, in which case you might not have to install yet another plugin.
Google XML Sitemaps: This is one of the highest rated plugin for WordPress and it does the job well. The plugin supports all generated pages such as posts, pages, archives etc and notifies varies search engines of the change when the sitemap is modified.
WordPress SEO by Yoast : This is a all-in-one SEO plugin that supports XML sitemaps as well. If you already use this plugin then you can enable the sitemap feature in this.
Online Sitemap Generator
If you have a website which is not developed on top of WordPress (or another blogging) framework, then you will need to generate the sitemap yourself. The easiest and quickest way to do this is to use an online sitemap generator.
You will need to have the website mostly complete and online before you can use an online generator. There are a lot of free services that can do the job. A simple Google search for “XML sitemap generator” will pull up a lot of websites.
Unfortunately a lot of these free online generator support only a limited number of files. The limit do vary between site to site, so if you happen to have a large website with several thousands of pages then it will be wise to check beforehand. You could also use a offline sitemap generator instead.
Manual or Offline Creation of Sitemap
As mentioned the online generators are only helpful once you have the website mostly completed and online. If you are in the process of developing the website locally and want to generate a sitemap then you might have to do it either by hand or by using a generator application that can be run locally or offline.
Again offline sitemap generators are harder to come by as it has a very specialized set and limited audience. There are several available for Windows platform, both free as well as paid. In Linux, you might want to give SiteMap Generator from Google Webmaster tools a try. Obviously it involves some manual installation and configuration.
Another option is Google Sitemap Generator, which does not seem to have much development and maintenance during recent times. It does still work though. phpSitemapNG is another option that is available offline. You will need to have PHP installed to run the tool.
If all else fails, you can always use a text editor to manually create a xml file. This is practical if and only if your website contains very few page. It is usually a nightmare to maintain this process in the long run.
Without a full XML sitemap
Having a XML sitemap with all the URLs in your website, though ideal is not required. You can create a small XML sitemap with just the URLs that are not linked from any other pages in the website. This will make it much easier to generate and maintain over the long run.
If an external or online sitemap generator can find your pages, then so can Google crawlers. This means that the online sitemap that was generated by online generators contains URLs that could have been found by Google any ways even without a sitemap.
But sometimes your website may contain webpages that is not directly linked from other pages or the URL links are generated dynamically. This will be make it real difficult for crawlers (including sitemap generators) to identify them correctly.
You will then need to insert these URLs manually into the generated sitemap. You can easily do this by opening the XML sitemap file in a text editor and manually editing it. You could also generate multiple XML sitemap files by using different URLs as the starting point and then merging them into one sitemap file.
A good reference point for finding the right sitemap generator is https://code.google.com/p/sitemap-generators/wiki/SitemapGenerators. This page actively maintains a list of various different sitemap generators.
Generating your own XML sitemap
If you are handy with programming or is not intimidated by it, then you can create your own program that can generate an XML sitemap for you. You can use any programming language you are comfortable with. I have a very simple XML generator that is created using wget and shell script. It gives you the entire shell script that you can use if you are up for it.