Tycoon Talk
Become a Big fish!
The number 1 forum for online business!
Post topics, ask questions, share your knowledge.
Tycoon Talk is part of Freelancer.com - find skilled workers online at a fraction of the cost.

The Other Search Engines


You are currently viewing our The Other Search Engines as a guest. Please register to participate.
Login



Reply
how how do I prevent my site from geting spidered?
Old 08-25-2004, 01:38 AM how how do I prevent my site from geting spidered?
samson's Avatar
Registered User

Posts: 107
Location: Victoria, BC
Trades: 0
User-agent: *
Disallow: /

will that work?? Is there some other preventions?
basically, I want my entire site to be unspiderable.
samson is offline
Reply With Quote
View Public Profile Visit samson's homepage!
 
 
Register now for full access!
Old 08-25-2004, 09:17 AM
Anacrusis's Avatar
Defies a Status

Posts: 2,099
Name: Adam
Location: Colchester CT
Trades: 0
That should work, if the spider obeys the robots.txt file. You should be ok if you have that and don't submit your site to any directories or anything.
Anacrusis is offline
Reply With Quote
View Public Profile
 
Old 09-05-2004, 11:14 PM
vivekar's Avatar
Webmaster Talker

Posts: 612
Trades: 0
Make your website password protected.
Use a nice authentication system.

On the home page, you can inform the human visitors to use a username and password.

Try using Website Guard. Your server should be able to run PHP.

Just my 2cents.
__________________

Please login or register to view this content. Registration is FREE
(Active since 2003) |
Please login or register to view this content. Registration is FREE

Last edited by vivekar; 09-05-2004 at 11:20 PM..
vivekar is offline
Reply With Quote
View Public Profile Visit vivekar's homepage!
 
Reply     « Reply to how how do I prevent my site from geting spidered?
 

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off





   
RSS Feed  Feeds: RSS   JS   XML
RSS Feed  Feeds for this forum: RSS   JS   XML



Page generated in 0.37752 seconds with 12 queries