Lesson #02 SE Overview: Who's Updating Who?
Written By Cyndalie

Knowing where search engines get and send the information they collect, who is updating whom, and where the best places to submit are to make sure your pages get indexed, is half the battle. There is oftentimes so much conflicting information and "guesswork" going on in articles and forums that it becomes a mystery when time comes to submit.

But before submission is a thought in your mind, you need to know (or have an idea of) where search engines get their data from and what optimization factors that particular search engine is looking for when ranking pages. The following is a brief overview of what is going on in the major search engines and directories so that we have a basic background to start from. Please note that this information is subject to change (and differential opinions) and I will do my best to keep you up-to-date on those changes and how they will affect our future efforts and results.

Yahoo!
Yahoo! Directory is the largest (maintained) search directory on the Internet today and a premier place to get listed, if you have the budget. Yahoo! Directory "category" links appear below some of the sites listed in response to a normal keyword search. Since Yahoo! is a directory and not a search engine, Yahoo!, like most directories, rank its' listings based on whether the keywords or phrases appear in the title and/or description that is placed by the editor. That's right, what you submit is not always what will appear if your site is accepted, so be sure to take a look at the general length of the descriptions currently placed and how they are written before writing and submitting your own. Yahoo! costs $299US per year for the placement of non-adult commercial based web sites and $600US per year for adult sites. Paid submission does not guarantee inclusion. We will go further into Yahoo!'s rules and guidelines for acceptance in future columns.

Yahoo! Web Page Results
Yahoo uses it's own search engine spider to find and index pages. Submission is free without timely or guaranteed inclusion or pay per URL inclusion is offered through Search Submit. The engine has integrated "Sponsored Search Listings" before and after its directory matches in an advertising deal with Overture. This also includes "Content Match" contextual ad placement that appears in search results which are sold on a pay per click basis.

Over the years Yahoo! has acquired both Inktomi and Overture. See Search Marketing for more information. Yahoo! powers Altavista, AlltheWeb, Hotbot and MSN (Paid).

Google
Google is by far one of the best places to focus your SE efforts, especially for low-budget webmasters. It is a true search engine, one of the dwindling few left that crawls, indexes, scores, and maintains its own cache of pages. The Google search engine is not powered by anyone other than Google and maintains its own unique algorithm. The Google Directory has been phased back and although it is said a listing in ODP/DMOZ.org can help you get and stay indexed in Google, this tactic is becoming less prominent. Google's "Sponsored Links" powered by Google Adwords is become more well known as a secondary means to get new domains indexed and ranking. It is said that running an Adwords campaign, even with a small budget, can help to activate your domain in the primary default search results.

Google will be a place that we will focus our optimization efforts on because the highly relevant rankings and traffic offer better converting results, and well optimized pages can bring long term results, even lasting years.

Google Powers: AOL (Main & Paid), Excite (Main & Paid), Ask Jeeves (Paid), Hotbot, Teoma & Lycos (Paid), and Netscape (Main & Paid).

DMOZ.Org/ODP
Open Directory Project powers directory listings for Netscape, Google, AltaVista, Lycos, AOL, and more. ODP is also a directory that ranks its results by what appears in the title, description, and category of the listings. They best way to successfully integrate your key phrases into your title and description is to use as many keywords as possible without using commas. Here is an example of what not to do (X = Keyword): Title – X Corporation. Description – X Corporation offers X services, X programs, and X solutions for X businesses. Here is an example of what you should try to do (X = Keyword): Title – X Corporation. Description – Provides X programs and solutions for businesses needing X services. Try to avoid the "keyword, keyword, keyword" format. It will be modified by the editor, most likely to something that does not accurately describe your company with minimal ranking results.

Altavista
Owned and Powered by Yahoo! Search results (Main & Paid) with ODP directory backup. Free submission via Yahoo! is available.

Lycos
Lycos uses Google paid results, Looksmart & Open Directory search results with Yahoo! Search as backup. Paid inclusion only via Lycos.

HotBot
HotBot, owned by Lycos, uses Google paid results, Yahoo! Search results, with backup resutls from Google & AskJeeves. Paid inclusion only via Lycos.

Excite
The Excite Network is now a collection of search sites owned by Ask Jeeves.

Looksmart
As I like to say, "Looksmart" doesn't necessarily mean "is smart". Looksmart is kind of like a paid version of DMOZ/ODP directory, only much worse. Call me biased, it just does not seem to be worth the money to even spend time on it here. You can visit Looksmart to read their mumbo jumbo pitch, just be aware ahead of time that they claim to not accept adult sites in their directory. My advice is to focus on Inktomi for better bang for your buck.

AOL
AOL Search uses Google for both Main and Paid results. It also uses ODP as directory results backup.

MSN
And finally, MSN. MSN now has it's own indexing spider (MSNBOT) and algorightm and is fully self powered. Default submission is free at MSN. MNS currently uses Yahoo! for paid listing results.

I hope that the above information helps you to get a good idea of what the playing field currently looks like for search engine marketers. Recent events where many search engines and directories are going "paid only" have been somewhat disheartening, however the field is still big enough, and strong enough, to sustain us all as long as we know where and what to spend our time (and money) on by weeding out some of the mystery behind the search engines.

In the next lesson (#03) we will do an overview of terminology used to describe and measure traffic, as well as review the fine lines between terms such as index and directory, relevance and popularity, and CPM vs. CPC (Cost per impression vs. Cost per click).

If you don't know how to code HTML, start learning now! We will tackle optimization and coding tricks to help you stay ahead of the competition in future lessons.