Optimizing Dynamic Pages - Part
II
Dale
Goetsch
Search Innovation
September 15, 2003
The
Widget Queen Revisited
You
have the world's finest collection of widgets.
You created the world's best widget website. You
have no traffic.
You
checked in the search engines and find that your
site does not appear at all, even though all your
competitors' sites do. Perhaps the search engine
robots cannot get to your pages to index them.
Search
Engine Robots
Search
engine robots are simple creatures. They can "read"
text to add to their databases, and they can follow
"normal" links--those links that are coded to
look like
<a
href="bluewidgets.html">blue widgets</a>
or the slight variation
<a href="bluewidgets.html><img
src="bluewidget.gif"></a>
That's
it. Search engine robots cannot select items from
lists; search engine robots cannot type text into
boxes; search engine robots cannot click "submit"
buttons. That means that no matter how important
our dynamically-generated page of blue widgets
is, if the only way to access that page is to
select it from a list or click on a button, the
robot will never be able to visit it. That, in
turn, means that it will never appear in the search
engine results.
So
how do you get your dynamic information to show
up in non-dynamic ways?
The
Painful Solution
One
of the reasons that dynamic pages exist is because
of the difficulty involved in constantly updating
-- adding and deleting -- pages from your site,
based on which widgets you are offering this season.
If you have a separate page for each make and
model of widget, each of those pages can be spidered.
They can all be reached through links that look
like
<a href="bluewidget-1.html">blue widgets style 1</a>
<a href="bluewidget-2.html">blue widgets style 2</a>
<a href="redwidget-1.html">red widgets style 1</a>
<a href="redwidget-2.html">red widgets style 2</a>
<a href="newwidget-1.html">new widgets style 1</a>
<a href="newwidget-2.html">new widgets style 2</a>
The
bad news here, of course, is that you now have
to create all of those pages. This loses the benefit
of drawing the widget information from a database.
A
Better Solution
A
better solution is to create only a "shell" of
each page, and then to dynamically populate the
page from our database. By creating a "real" file,
you can assign a fixed URL, but still use the
database to fill-in the page, using any of various
server-side techniques (HTML server-side includes,
Perl, Active Server Pages, Java Server Pages,
PHP, etc.). A simple page like this might suffice:
<html>
<head>
<title>Blue Widgets style 1</title>
</head>
<body>
<!--#exec cgi="myscript.pl?bluewidget-1"-->
</body>
</html>
Save
this page as "bluewidget-1.html" and you're good
to go, assuming that "myscript.pl" will actually
return the content you want for the body of the
page. True, you will have a discrete page for
each item in your inventory, but at least you
only need to hard-code the bare-bones of that
page.
Another
Way To Go
There
is yet another way to go. This method does not
require creating dozens of static pages, or of
having to include exotic scripts in your web pages.
It also may not work for all search engines!
Some
search engine robots just will not follow links
that include a "querystring" as part of the URL.
You have seen a querystring if you have ever looked
at the URL of a page of search results in Google.
For example, if you look for "blue widgets" on
Google, not only do you get page after page of
blue widgets, you also see that these pages have
very complicated-looking addresses
http://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=blue+widgets
In
this address, everything after the question mark
("?") is a querystring. This is used to pass additional
information to the web server. While some search
engines can follow a complicated address like
this, many simply will not follow such a link.
That means that if you use a URL like
http://www.mycompany.com/catalog.html?item=widget&color=blue&model=1
that
the robot may not be able to follow it. This is
bad.
On
the other hand, an increasing number of search
engine robots will follow such links. Usually,
links like this are created "on the fly" by filling-out
forms and clicking a "submit" button, but that
doesn't have to be the case. You can grab that
address, querystring and all, and put it into
a "normal" link, like this
<a
href="http://www.mycompany.com/catalog.html?item=widget&color=blue&model=1">
blue widgets style 1</a>
Put
several of these on a page and the search engine
robot can now visit your dynamic pages from links
that require no button-clicking. Remember that
not all robots will follow these links, so your
mileage may vary.
As
long as the link to the page exists in a form
that does not require human intervention to get
to it (pulldown menus, search results, form submits,
etc) then a bot will follow it.
Widgets
Out The Door
Using
any of these methods will help search engine robots
to find the dynamic pages on your site. This means
that the important content on those pages is more
likely to be included in the search engine databases,
and that people will be better able to find you.
That, of course, means that the Widget Queen will
reign supreme, knowing that widget customers the
world over will now be able to find you and buy
your widgets.
About the Author:
Dale Goetsch Technical Consultant for Search
Innovation Marketing , a Search Engine Promotion
company serving small businesses and non-profits.
He has over twelve years experience in software
development. Along with programming in Perl, JavaScript,
ASP and VB, he is a technical writer and editor,
with an emphasis on making technical subjects
accessible to non-technical readers.
Copyright © 2003 Search Innovation Marketing.
All Rights Reserved.
Return
to FREE articles index |
Visit Our Sponsor! WebPosition Gold