I have a webhosting account that has 2 domains directed to it. I have domain1 targeting the root directory and domain2 a folder in the root directory.

I was wondering the best way to stop search bots from crawling the directory that domain2 is in and associating it with domain1. I still want it to be crawled, but to be kept as its own site.

I know you can make bot instruction files, I am wondering if that would be the best way or if there is something else I should do.

Thanks

You could 301 redirect all domain1.com/domain2folder calls, to domain2.com, using .htaccess. There are lots of tutorials to be found out there, if you use a search engine.

But I dont need a redirect I dont think because I can just set up the domain to go directly to the correct directory. The problem is just that I dont want google to think that domain2folder is part of the domain1.com site because they are too totally diferent sites. But at the same time I still want it crawled as its own seperate site.

Does that make more sense?

1. don't ever link using domain1.com/folder. if you don't do it, the search engine can't know it's there.
2. just in case such a link exists, you do the 301 (permanent) redirect, which basically tells everyone (including search engine robots) that the page has permanently moved to its new domain2.com location.

Very helpful thankyou I will give it a try.

Dont make the second domain a sub directory. Make it a whole other web site. This is truly the only logical way to do what your asking. If you have your own server it wont cost a dime. Use HTTP headers.