Tycoon Talk
Become a Big fish!
The number 1 forum for online business!
Post topics, ask questions, share your knowledge.
Tycoon Talk is part of Freelancer.com - find skilled workers online at a fraction of the cost.

HTML Forum


You are currently viewing our HTML Forum as a guest. Please register to participate.
Login


Tycoon Talk > Web Development and Programming Support > HTML Forum > Robots

Post a Project »

Find a Professional HTML Freelancer!

Find a Freelancer to help you with your HTML projects

FREE Outsourcing eBook!

Reply
Robots
Old 06-06-2005, 07:16 AM Robots
Miz
Miz's Avatar
Average Talker

Posts: 16
Location: The Sunny South
Trades: 0
Talkupation: Miz is on a distinguished road
Hope some one can help with this, I use this line in between my <head> tags...
<meta name="robots" content="index, follow">
I want know if I can force the 'robots' to to sto a specific page on the site after my index
Miz is offline
Reply With Quote
View Public Profile Visit Miz's homepage!
 
 
Register now for full access!
Old 06-06-2005, 07:54 AM
ibbo's Avatar
Super Spam Talker

Posts: 880
Location: Leeds UK
Trades: 0
Talkupation: ibbo will become famous soon enoughibbo will become famous soon enough
robots.txt

User-agent: * # directed to all spiders

Disallow: /admin/

I didnt quite make out what your post was after but I assumed you wanted to ommit a directory from a spider crawl.

Just put the above in your robots.txt

add Disallow: /DIR/ for each one you want to be un spidered.

Ibbo
__________________

Please login or register to view this content. Registration is FREE

Please login or register to view this content. Registration is FREE

Please login or register to view this content. Registration is FREE

Please login or register to view this content. Registration is FREE

Linux user #349545 :
(GNU/Linux)iD8DBQBAzWjX+MZAIjBWXGURAmflAKCntuBbuKCWenpm XoA7LNydllVQOwCf
ibbo is offline
Reply With Quote
View Public Profile Visit ibbo's homepage!
 
Old 06-06-2005, 12:05 PM
kline11's Avatar
SearchBliss Web Tools

Latest Blog Post:
Google Alerts Users About Infected Computers
Posts: 1,788
Name: John
Location: USA
Trades: 0
Talkupation: kline11 is a splendid one to beholdkline11 is a splendid one to beholdkline11 is a splendid one to beholdkline11 is a splendid one to beholdkline11 is a splendid one to beholdkline11 is a splendid one to beholdkline11 is a splendid one to behold
You can also build a robots.txt file easily for free here:
http://www.searchbliss.com/webmaster...-generator.htm

<meta name="robots" content="index, follow"> tells the spider to index the page this is on and follow the links. The other options beside "index,follow" are "no index, follow", "index,no follow", "no follow,no index", or "all" which is the same as the one you have.
__________________

Please login or register to view this content. Registration is FREE


Please login or register to view this content. Registration is FREE


Please login or register to view this content. Registration is FREE
kline11 is offline
Reply With Quote
View Public Profile
 
Reply     « Reply to Robots
 
Previous Thread | Next Thread Tycoon Talk > Web Development and Programming Support > HTML Forum > Robots

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




   
RSS Feed  Feeds: RSS   JS   XML
RSS Feed  Feeds for this forum: RSS   JS   XML



Page generated in 0.21252 seconds with 12 queries