I, Robot II

Another way to prevent your Web pages from being indexed by roaming 
Web spiders is through the use of a <meta> tag. This approach offers 
three different options:

<meta name="robots" value="noindex">

This tag tells visiting robots not to index THIS page. They can, 
however, follow its links to other pages.

<meta name="robots" value="nofollow">

This tag tells robots that they may index this page, but they're not 
to follow any links on it to other pages.

<meta name="robots" value="none">

Finally, this tag tells robots neither to index this page nor to 
follow any of the links on it to other pages.

You'll need to remember to place this <meta> tag inside every page you 
wish to hide from visiting robots. As with the robots.txt file 
technique, this approach isn't guaranteed. Between the two, however, 
you'll probably have the obedience of 95 percent of the robots out on 
the Web.