I've been using Gsite for a few weeks now, made two successful sitemaps, but now it seems to be acting up. It doesn't heed my filter/ban url requests. I'm trying to filter our community discussion boards by banning all links with /community/, but those pages keep coming up in my url table. The crawler seems to get lost in all these pages. I've left it running overnight even and it wasn't finished in the morning.

Has anybody had these issues before, or have any suggestions? The filters have worked for me before, but it seems like I'm gambling each time I try running it - or I'm missing a preference option somewhere.

robots.txt is the place to restrict crawling, where it will be in force for all compliant bots.

A google sitemap is there to assist in crawling with google only. It certainly should not be relied on to stop some pages being accessed.

i notice that google bot is getting more efficient, compare to months ago.
submit your URL, and the site will get index very fast.

read the post, please !!!

the OP is trying to restrict the crawling

Can you post some specifics (or mail me through the form on the program's site)? I use the filters a lot, also to exclude whole sections of sites, and so far I have had few problems with them. One thing that will not work, however, is adjusting the filters on the fly while it is crawling. You will have to stop the crawlers, wait until they're empty, re-filter the URLs and optimally restart the program for the adjusted filters to take effect (once the URLs are in the crawler, in the URL table or in the crawler-cache they need to be flushed out before you continue).