The Googlebot is a very sophisticated computer program. It is a search engine spider that scours the Internet and inspects web sites in order to have them ranked according to Google’s standards. It’s “job” is to look at your web page, make a few notes and then report back to Google, who in turn give your page a ranking.
In this article I’m going to tell you a little about how to attract the Googlebot to your website and how to make sure it’s happy with what it finds. Think of it like this: The Googlebot needs love too.
1] Read Google's Webmaster Guidelines: Most of Google’s algorithms (search rules) are really secret; we can only guess what they are.Their guidelines are very simple and precise, however. Following their guidelines can’t “hurt” your site's ranking and Googlebot will “enjoy” its time with your website. Disregarding their guidelines can and probably will hurt you and your website in the long run.
2] Make the links in your site text links: Google says: "Make a site with a clear hierarchy (structure) and text links. Every page should be reachable from at least one static text link."
The native language of the Googlebot is text; this is not to say that you cannot make your site really pretty and fill it with lashings of Java Script and Flash but you MUST have regular text and standard text links. Usually you can achieve the desired effect by having extra navigation menus based on standard text links.
3] Give every page a a complete and meaningful title: This is also directly from Google's Webmaster Guidelines. Have a look at Rule #1.
The "title" tag is supported by every web creation tool out there, and goes in the header of a web page. Make sure your title is not just a list of keywords and that it is related to the actual content of the page. Google can and will check that, before deciding on your page's 'relevance' and fate...
4] DO NOT place important text inside images: Google says: "Try to use text instead of images to display important names, content, or links. The Googlebot doesn't recognize text contained in images."
It is very tempting to create images with text inside them, for the very simple reason that you are not limited to the very few font options that basic HTML allows. Also, different browsers tend to display things differently nowadays, so it is much easier to create a text image, which will be shown consistently and not worry about styles, operating systems, etc. Unfortunately, the Googlebot doesn’t like this one bit...
5] Use descriptive "ALT" tags: The "ALT" tag is used as a text alternative for images and image links and was designed so that text browsers do not just display a generic 'Image' for every picture link you might have. If all your links say 'Image', how would a potential visitor know what they are?
Make sure that the text description is meaningful and accurate. If the "ALT" tag only says "display", that is what Googlebot will see and index. If the tag says something like "example of a tradeshow display design", that is certainly more useful to the information-hungry Googlebot.
6] Use meaningful descriptions for links: Whether you use picture links or text links, please use meaningful text inside your tags so that Googlebot can associate that text with that link.
In other words, if you intend to put a link to a set of sample coffee mugs promos, say something like "link to samples of Y.O.U.R. branded coffee mugs", not just "coffee mugs", or even worse, "click here for pictures". Never use link text like "read more" or "go here" or "download it", "click here", "don't click here". Googlebot will not understand what you mean...
7] Use a "description" tag for every page: Include a tag in your page header to summarize your site. Use a meaningful one or two sentence description and certainly do not keyword spam.
Even better, include descriptive text on the site's front page where users can actually read it. It is this text will appear as the description for your site in Google results.
You should place more important content higher in the page than less important content in a page. The Googlebot does categorize text on a page based on it's position, text at the bottom of a page is considered less relevant.
8] Use robots.txt: Google says: "Make use of the robots.txt file on your web server. This file tells crawlers which directories can or cannot be crawled."
This almost ancient and very standard mechanism for directing well-behaved robots like the Googlebot will allow you to specify places where the robot is not welcome, for whatever the reason. You might want to keep the robot away from your cgi-bin directory and other places you maybe don't want available to the entire Googling population of the world. Remember this is a guideline, not a barrier; robots that are not programmed to comply will ignore it. So, use the robots.txt to guide the noble Googlebot but not for any real security enhancing reason.
9] Make a sitemap: Google says: "Offer a site map to your users with links that point to the important parts of your site. If the site map is larger than 100 or so links, you may want to break the site map into separate pages."
A site map is just a page on your website where you guide your users through the structure of your site. The most basic form of sitemap is a page that lists all of your pages, with a brief description and a link - ALL TEXT, of course; the Googlebot cannot read siemaps that are not in plain text. When you make the sitemap, follow all the rules above and don't forget that the purpose of the sitemap is to guide your human visitor.
10] Google Webmaster Tools: You should also check out Google Webmaster Tools 101, our article on Google Webmaster tools, a great way to find out why Google is or is not listing your website in its search results.
