Hi!

I have written a web application which crawls my domains searching all pages for broken internal/external links and what pages on my sites those are linked from.
It also looks for missing meta keywords/description, titles and h1 tags, duplicate meta keywords / description, titles and h1 tags and so on. I'm also atm developing a duplicate content checker to integrate in the application.

I'm was hoping to get some feedback from other web masters or programmers.
Would you use my website application to crawl your own sites or are you already using something similar?
What other features would you as a web master like to see in such a application if you would use it?
Are there any similar web pages or applications out there today?

I'm totally new to this forum so if this post is misplaced i would appreciate if the moderators could move it to the right location.

Yea i would use a tool like that, although there are a few tools out there like that which can crawl a site and show dead links.

Dan

That would be a useful tool, and of course people will use it. No one likes broken links.

Thank you for the replies. I will make this public i think.

I, too, would use it. It would be nice if not only is /missing.html a missing page, but what page in the 30 to 40 pages I have did that link appear on? Oh, OK, I go to /thisarea/badcall.html and fix the problem!

Thank you for your work!

Not sure if your already doing, but would be great to have a function (since the tool is looking at this stuff anyway from what you said) that would just list the pages in outline form.

Then it could display, H1 tags, title tags, meta data, etc other SEO important factors. I still sometimes find a page on my sites that I missed when updating the way my stuff is entered.

Also, something that I think would be neat too. Since you are checking which pages link where, would be neat to view the site in a web view showing all linked connections, including direction IE page 1 ---> page 2 with arrow if page two doesn't link to page one.

I am interested to test this out if you need beta testers =D

Originally Posted by patrik_swe

Thank you for the replies. I will make this public i think.

Best choice would be to make it public..At least IMO. The software industry? I think it's been dead for quite some time now.

Anyone else think this thread is a bit full of spam?

But yeah, useful tool but it has been done loads of times.

That would definitely be useful in my workflow. Send me the link when you're done!

try to pack as many features as you can in it as I am sure there are a ton of these applications out there already.