VoyForums
[ Show ]
Support VoyForums
[ Shrink ]
VoyForums Announcement: Programming and providing support for this service has been a labor of love since 1997. We are one of the few services online who values our users' privacy, and have never sold your information. We have even fought hard to defend your privacy in legal cases; however, we've done it with almost no financial support -- paying out of pocket to continue providing the service. Due to the issues imposed on us by advertisers, we also stopped hosting most ads on the forums many years ago. We hope you appreciate our efforts.

Show your support by donating any amount. (Note: We are still technically a for-profit company, so your contribution is not tax-deductible.) PayPal Acct: Feedback:

Donate to VoyForums (PayPal):

[ The Zak McKracken Archive]
The Zak McKracken Forum
Login ] [ Main index ] [ Search | Check update time | Archives: [1] ]


[ Next Thread | Previous Thread | Next Message | Previous Message ]

Date Posted: Wednesday, Jan 19
Author: ken
Subject: Re: and more links
In reply to: ken 's message, "and more links" on Wednesday, Jan 19

>http://searchenginewatch.com/benefits/article.php
>middle of page about half down. expplains search
>engine bots.

Sheesh I also forgot about my most recent addition to help google and other engines. this also is a new thing.

robots.txt
======================
# Tells Scanning Robots Where They Are And Are Not Welcome
#
# User-agent: can also specify by name; "*" is for
# everyone
# Disallow: disallow if this matches first part of
# requested path
#
# For now disallow all we can modify this as needed to
# allow certain crawlers.
# the above is descriptions of the usage shown below. the # disallow does not have ALL. so it says welcome to all
# bots.


User-agent: *

Disallow:
=====================
this plain vanilla text file also goes into the root of your ftp.
ie: http://www.voy.com/186593/31.html?z=1
you'd stick robots.txt in the http://www.voy.com/
directory.

[ Next Thread | Previous Thread | Next Message | Previous Message ]


Replies:

  • Re: and more links -- ken, Wednesday, Jan 19
  • Thanks n/t -- Chris, Wednesday, Jan 19

    Forum timezone: GMT+0
    VF Version: 3.00b, ConfDB:
    Before posting please read our privacy policy.
    VoyForums(tm) is a Free Service from Voyager Info-Systems.
    Copyright © 1998-2019 Voyager Info-Systems. All Rights Reserved.