|
|
About Search Engines
If you have an email address, you've probably received numerous spam emails promising to drive tons of traffic to your site by paying them to submit your site to thousands of search engines (one promised "360,000+"). And to do it every month!
In reality, fewer than 15 search engines and directories generate 95 percent of all Internet search. This privileged group includes
Yahoo!,
Google,
MSN Search,
AOL Search,
AltaVista,
Lycos,
AlltheWeb,
Infospace, and others.
The universe for submission is even smaller than this, because many search engines are fed results from other search engines. For example, AOL gets its results from Google, Lycos from FAST, and MSN from Open Directory and Inktomi—although some apply their own unique algorithms and thereby return unique results for any given search term. The relationships among search engines are a spider web of complexity.
The point is that there is no value to paying any of these services to submit your site to thousands of search engines. And it can actually hurt you to allow some automated piece of software to submit your site to search engines repetitiously. Especially harmful is when these programs submit a site to directories, because it's important to get a directory listing correct the first time, as there is no second chance. It may surprise you to hear that you don't even need to submit your site to search engines. You do need to submit to the directories, but the crawler-based search engines will eventually find your site, so long as some other site in their index links to your site.
Search Engines vs. Directories
Crawler-based search engines, such as Google and AltaVista, have automated bots, called spiders, that crawl the Web, from site to site, link to link, indexing all the Web pages they find, and putting them in their huge databases. When someone runs a search by plugging in a search term, the search engine refers to its database of indexed pages and returns results based on complex algorithms that determine which pages get the highest rankings for specific search terms. No search engine has ever published its algorithms, and these algorithms change frequently.
In contrast, directories such as Yahoo! and
Open Directory Project do not index Web pages; instead, they index their directory listings. A listing is comprised of a short title and a brief description. When you submit to a directory, you are asked to submit your suggested listing. However, it is highly unlikely that the listing you submit is the listing you'll get, because a human editor will visit your site and modify or completely rewrite your listing. These editors can be quite peevish about the hyperbole and salesmanship you'll want to see in your listing. Though fairly rare, your site could be rejected by the editor, even though, for example, you've already paid $299 to Yahoo! to list your site in their directory for a year. Once your site is listed in a directory, you're stuck with that listing. It's not unlike doing a media interview, where you have little power over the resulting story, which at best will disappoint you a little.
Directories, though few, are very influential, because one way or another they reach all Internet search. Yahoo! enjoys more than a 38% market share of Internet search.
LookSmart—now a hybrid directory and cost-per-click ad model— feeds its listings to MSN search, the number-three player in the search market behind Yahoo! and Google. And the Open Directory Project feeds AOL, Lycos, Hotbot, Netscape and numerous other sites. Thus, it cannot be over-emphasized how critical it is to get the best possible listing in the directories. An effective listing will be rich in keywords that are highly relevant to your business or site. The trick is, firstly, knowing which
keywords to use, and secondly, how to write a listing that will survive the human editor largely intact.
If your site is new or you've not yet listed with some of the major directories, we highly recommend you hire a professional, like Sun Hunter, to do it for you. If you want to do it yourself, please refer to our
Resources page, because you need to put in enough research to be confident of doing it correctly. A cautionary note: do not rely on your Web developer to submit your site to search engines and directories unless you're certain they know what they're doing.
Web Developers Do Not Understand Search Engines
From experience, we've found that many Web developers do not understand search engines, at least from a marketing perspective. But you'll think they do because they know more about the technology of the Internet than you do. This pitfall is hard to avoid.
The problem is that few will admit they don't understand search engines, either because they think they know what they're about—i.e., they don't know what they don't know—or for some other reason. They may even present someone on staff as their search engine specialist.
The bottom line is that the focus for most Web development firms is creating your Web site, not helping you market on the Internet. And here's the proof: they do not design search-engine friendly sites; in fact, many of their sites are impenetrable to search engines.
The Search-engine-friendly Site
We said earlier that pure search engines send spiders out to crawl the Web and index pages. What they're indexing is source code. What they're most concerned with are WORDS, which makes sense since their customers mostly search for words. Yet we see many sites with very few words, especially on the home page, which is the most important page to search engines.
And by words we mean words that search engines can read because they're in HTML code, not on images, or in Flash, which search engines cannot read. Frames are another major problem for search engines, as is JavaScript and other scripts. For these reasons you'll oftentimes find that the most effective sites with search engines are the simplest ones.
Aesthetic issues are important, of course, because style itself is a communication. A corporate Web site should not look like it was designed by a high school student. But we'd argue that, in most cases, aesthetically appropriate sites with advanced functionality can be developed that are also search-engine friendly. It is therefore ideal to understand these issues before you embark on a new Web site or major site redesign. Again, unfortunately, many Web developers don't understand these issues and never engage their clients in such discussions prior to designing their sites.
As a result, our toughest challenge in implementing
search engine optimization for clients is contending with their site's existing architecture and design. Now there are ways to get around most difficult situations. But the more impenetrable to search engines a site is, the more expensive (time intensive) or risky (risk of getting penalized or banned) the potential solutions tend to be.
This essay is not an argument for search engine optimization— a sophisticated, aggressive program for pulling traffic from search engines—rather, it is to help you understand what works and what doesn't on a fundamental level. It may explain why your site gets little to no traffic from search engines despite the fact that it has been indexed or visited by them.
If your Web site is intended to play a marketing role, and you want it to come up in the search results, do not allow your content to be enslaved in a design that's unfriendly to search engines. If your content is already enslaved, you may need to liberate it. When your content is liberated, there's a good chance you'll benefit from Internet search, at least on some level. You just need to ensure the search engines can read your content. And that your content is rich in words, because words are sweet nectar to search engines.
Importance of Links
In the early days of the Web, search engines focused primarily on page content (the words) and the meta tags (more on this to follow) in their ranking algorithms. While these are still key factors, search engines today place increasing importance on what we call "off-page" factors that are not so easily manipulated by webmasters. You can expect to see a growing trend toward emphasizing off-page factors.
Off-page factors are primarily about in-bound links: when other sites link to your site.
How search engines treat links can be very complex. Link popularity, or number of links, is not the only factor, though if your links vastly outnumber your competitor's, this can be a deciding factor. Google will give a site a high ranking based on number of links alone. But again, other factors come into play that focus on the quality of links. For example, Google pioneered a technology called PageRank™, which looks at the relative importance (based on links) of the sites that link to your site. Here's Google's own explanation:
PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important."
Important, high-quality sites receive a higher PageRank, which Google remembers each time it conducts a search. Of course, important pages mean nothing to you if they don't match your query. So, Google combines PageRank with sophisticated text-matching techniques to find pages that are both important and relevant to your search. Google goes far beyond the number of times a term appears on a page and examines all aspects of the page's content (and the content of the pages linking to it) to determine if it's a good match for your query.
In addition, search engines are now, and can be expected to increasingly look with greater sophistication at the content of the site that is linking to your site to see how relevant it is to your content. A new up-and-coming search engine,
Teoma is one example of this trend. For example, if you're an elder care site, a link from another elder care-related site will be of higher value than a link from, say, your Web development firm. A more detailed explanation of link factors is provided in our Search Engine Optimization essay.
In conclusion, if you want to do everything possible to get your site well positioned in the search engines, you'll actively seek to obtain inbound links from sites with content relevant to your own.
Role of Meta Tags
Finally, a brief word about meta tags. Meta tags are used to provide information directed solely to search engines, not to the browser (what the user sees). Numerous kinds of meta tags can be used. One, the meta robots tag, even instructs search engines not to index a page.
Meta tags are often misunderstood as playing a dominant role in optimizing a site for search engines. They do play a role, but far from dominant. Their greatest value is to ensure that your listing in the search results tells people clearly what your page is about and encourages them to click through to it. The title tag and the description meta tag are most important. In the example below, you can see that they go between the header tags in the source code.
The title tag, which is not literally a meta tag, is the most important, because this is the first part of any search result for your page. Thus you want your title tag to be highly descriptive of your content, and appealing to potential visitors. The title tag is also very important for search engine optimization, so you want it to be rich in relevant keywords.
The description meta tag is also important because some search engines will use it—up to a limited number of characters— in their search results. And some search engines will index and include it in their algorithms, so again you want it to be keyword rich.
The keywords meta tag, on the other hand, provides little benefit today. Those search engines that still index this tag place little value on it in their algorithms. Some experts say that, because some search engines index it and count it in their algorithms, it is worthwhile using the keywords meta tag. Theoretically, this makes sense, yet we've not seen any evidence that the keywords meta tag makes a real difference. We're very happy, however, that nearly every site uses the keywords meta tag vigorously, even when they do nothing else right to optimize the site. Whenever we take on a new client, we find it very helpful to look at their competitors' keywords meta tags to see what keywords they think are important.
|
