Tutorial

So we are all aware of the 3 major search engines I think by now. Google, Yahoo, and MSN. But how do we get our site listed and indexed by these?

Google eventually finds your site and over time does a fairly decent job of indexing.

Yahoo also does a really good job. I've had great luck with little interaction on my part.

MSN does a terrible job. I don't know where there bot goes, but it certainly don't do an acceptable job of finding your site let alone indexing it.



Now obviously we want to give our site a title, a good description, and some nice keywords. Which we all should have done by now in our Dolphin Admin Panel.


So what we want to do is help these Search Engines along a little, by telling them where they can find our site and all our links. This often times will help them find our links. Rather than hope they find them on their own, we can at least tell them where to find them. Whether they choose to index them or not, at least we did tell them where to find them.

Now to do this we want to create a sitemap.xml file with all our links, be sure that you do not included links that would require a log in to see or access the page. Sometimes they frown on this.

A sitemap.xml is a special format file with specific tags. After we do this we upload it to our main directory such as mysite.com/sitemap.xml.

Then we will tell these search engines, "hey come check out our sitemap.xml file" which contains ALL our websites links in one file.


So how do we create a sitemap.xml and what is the format and tags we should use?

There are a number of free online sitemap creators. You can just tell it what your domain name is, and it will scan your site and let you download a sitemap in xml format. You can then view it and add to it following the same format for any additional links you add to your site in the future. Remember to add to this file as you create and add new links to your site.

One good free online tool for this that I've used in the past is:
http://www.xml-sitemaps.com/

It is very accurate and does a really good job at finding all your links.

It basically creates a sitemap.xml with the following tags and layout:

<?xml version="1.0" encoding="UTF-8"?>
<urlset
      xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
            http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<!-- created with Free Online Sitemap Generator www.xml-sitemaps.com -->

<url>
  <loc>http://www.yourdomain.com/</loc>
  <priority>1.00</priority>
  <changefreq>daily</changefreq>
</url>
<url>
  <loc>http://www.yourdomain.com/links.php</loc>
  <priority>0.80</priority>
  <changefreq>daily</changefreq>
</url>
<url>
  <loc>http://www.yourdomain.com/articles/category/Ray</loc>
  <priority>0.64</priority>
  <changefreq>daily</changefreq>
</url>
<url>
  <loc>http://www.yourdomain.com/articles.php?action=viewarticle&amp;articleID=23</loc>
  <priority>0.51</priority>
  <changefreq>daily</changefreq>
</url>
<url>
  <loc>http://www.yourdomain.com/1/security/africa-exploits.txt</loc>
  <priority>0.41</priority>
  <changefreq>daily</changefreq>
</url>
</urlset>



Simply replace your domain/inks
priority
and freq

Or as mentioned use http://www.xml-sitemaps.com/ or just use it to get an idea of how it should look for your site. I've used it many times and it does work great.


Ok then we need to tell the search engines that we now have a sitemap.xml and where they can find it at.

So we need to get an account at google if we don't already have one here:
http://www.google.com/webmasters/tools

After doing so you can then use it to add your site and sitemap.xml. It also has many other useful features, so be sure to look around.


MSN needs all the help it can get to find your site and links. If you have a hotmail, live, or other related msn account you can use that to login and add your site and sitemap.xml here:
http://www.bing.com/toolbox/webmaster/


You can also use a sitemap validator tool such as:
http://www.validome.org/google/


Then be sure to check back over the next several days to see if the bots are visiting and indexing. Sometimes it picks them up and adds them fast. Other times it takes a while, and some links never get added. There are no promises or for sure things, but certainly anything you can do to help is worth it.

Google claims your actual page rank can be increased when another popular site links to your site. For most this is not very likely, but if you know someone that can add your link it never hurts to ask.

And drop your website in a forum signature, blog, or other post whenever you can.

Good Luck!!



MSN Update 10-20-08



According to MSN Sitemap: webmaster.live.com account, if you change, update, or add a new sitemap for your site you should notify the MSNBot you have done so.

The simplest way to do this it seems is to "ping the MSNBot" they call it.

After you have changed or update your sitemap you should do the following.

Sitemap ping service

**To notify MSNBot when you change an existing sitemap or add a new one, enter the following in your browser:

***http://webmaster.live.com/webmaster/ping.aspx?siteMap=[your sitemap web address]

Example if my site map is located at www.mycoolsite.com/sitemap.xml I would enter the following in my web browser:

http://webmaster.live.com/webmaster/ping.aspx?siteMap=www.mycoolsite.com/sitemap.xml

Be sure to make sure your sitemap.xml is valid before doing this so you don't confuse the Bot or do more harm than good.

You can do so with an online tool here:
http://www.validome.org/google/

If it tells you it's "Valid" you are good to go. If it tells you it's "Invalid" then you need to fix any errors first.

Added 11-22-08
It should also be noted that you can help the MSN Crawler by adding the following to your robots.txt file:
Sitemap: http://www.yoursite.com/sitemap.xml

 
Written By
Tutorial by: Jeremy LeSarge (AKA Ray)

I am the owner and administrator of DialMe.com. I write Tutorials for Boonex Dolphin as well as tips and resources surrounding website programming and development. I enjoy working with WordPress, SEO, and Web Hosting / Servers. I also maintain a WordPress Blog here on this site where you will find a variety of technology and webmaster resources.

Actions
Sponsored Links
Recommend