Tycoon Talk
Become a Big fish!
The number 1 forum for online business!
Post topics, ask questions, share your knowledge.
Tycoon Talk is part of Freelancer.com - find skilled workers online at a fraction of the cost.

SEO Tycoon


You are currently viewing our SEO Tycoon as a guest. Please register to participate.
Login



Freelance Jobs

Reply
Web log analyzer for multiple domains
Old 02-17-2007, 05:55 PM Web log analyzer for multiple domains
Junior Talker

Posts: 4
Trades: 0
We have a remote server hosting quite a lot of websites (over a thousand). Each site has its own logs in a different folder on the server (eg. /logs/site1.com, /logs/site2.com). We are looking for a web log analyzer which allows us to set up each site individually for separate analysis, and which will automatically download new logs from each log folder on the server on a schedule we set. The idea is we could leave it downloading overnight. Ideally it would also include the ability to summarize the stats across all domains (eg. which sites are most popular? how many visitors were there across all sites? etc)

Anyone know if such a product exists?

thanks
shedlord is offline
Reply With Quote
View Public Profile
 
 
Register now for full access!
Old 02-17-2007, 06:03 PM Re: Web log analyzer for multiple domains
ADAM Web Design's Avatar
Canadastaninianite

Posts: 5,938
Name: Adam for web page design, not program
Location: Toronto, Ontario, Canada
Trades: 0
www.deepmetrix.com (allow about 24 hours per site to import the first time)
www.awstats.org (has problems with a large number of log files, but you can always kill the log files once imported.)

I don't think you'll find one that will summarize stats across all domains. I'm not totally sure, though.
__________________

Please login or register to view this content. Registration is FREE
|
Please login or register to view this content. Registration is FREE
(my blog)


Please login or register to view this content. Registration is FREE
(with proof)
ADAM Web Design is offline
Reply With Quote
View Public Profile Visit ADAM Web Design's homepage!
 
Old 02-17-2007, 08:38 PM Re: Web log analyzer for multiple domains
Banned

Posts: 905
Name: Travel Agent
Trades: 0
Most ISP's have an analyzer already installed, if you have CPanel -- check with your host.
travelagent is offline
Reply With Quote
View Public Profile
 
Old 02-18-2007, 05:16 AM Re: Web log analyzer for multiple domains
chrishirst's Avatar
Missing! presumed drunk.

Posts: 41,520
Name: Chris Hirst
Location: Blackpool. UK
Trades: 0
Try Funnel Web

and some customisations at http://manufacturedenvironments.com/..._for_free.html
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
A foolish consistency is the hobgoblin of little minds
Thought for today:- I SEO the only industry where all the cowboys are Indians?
chrishirst is online now
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Old 02-18-2007, 01:46 PM Re: Web log analyzer for multiple domains
Junior Talker

Posts: 4
Trades: 0
Just trying Funnel Web now. Couple of things. Firstly, I don't see how to set this to look in different folders on the server for logs. Tried using wildcards for the folder names but got 550 errors.

Secondly, when I pointed it at one specific folder to test the downloading, and started it going, suddenly Norton AV went mad, popping up 'Intrusion attempt blocked' messages referring to various different trojans. I remember I had something like that hapopen years ago with ws_ftp. Either this means Norton is wrongly picking up these transfers as nasties or there's something about running this software that puts a big flashing neon sign on your connection saying "come and get me boys!"

---

travelagent, we actually own the server, but a third party hosts and maintains it for us, not a public ISP. There is no cpanel or anything like that.

---
ADAM, am I right in thinking Awstats only works on a server (with cgi & perl)? I need something that will run on an XP desktop preferably.

---
shedlord is offline
Reply With Quote
View Public Profile
 
Old 02-18-2007, 05:47 PM Bit more background on my problem.
Junior Talker

Posts: 4
Trades: 0
We have 1000 websites. Each has its own log folder on the server. Occasionally my boss asks me to tell him which of these sites have high traffic. The only way I have of doing this currently is to first decide on a figure for the filesize of a log file which suggests there are a decent number of visitors, then to manually open each log folder and note down which have logfiles that average over this figure.

1000 times!

Is there an easier way to do this?

We have FTP access only. We can't install a log analyzer on the server because it is creaking already. We can ask the 3rd party company who host it for us to send us the logs on DVD however. Even with the logs on a disk locally, not sure how to get around having to manually assess each folder.
shedlord is offline
Reply With Quote
View Public Profile
 
Old 02-18-2007, 07:43 PM Re: Web log analyzer for multiple domains
chrishirst's Avatar
Missing! presumed drunk.

Posts: 41,520
Name: Chris Hirst
Location: Blackpool. UK
Trades: 0
Bit of a tall order for any log analyser I think.
By using the "virtual" settings you can combine stats from 10 sites into one report, the paid for version had unlimited domains, unfortunately no longer available and it did cost around 1000 USD.

There is Google Analytics of course and you can buy Urchin 5 (which analytics is based on) from them if you prefer.
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
A foolish consistency is the hobgoblin of little minds
Thought for today:- I SEO the only industry where all the cowboys are Indians?
chrishirst is online now
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Old 02-19-2007, 12:51 AM Re: Web log analyzer for multiple domains
ADAM Web Design's Avatar
Canadastaninianite

Posts: 5,938
Name: Adam for web page design, not program
Location: Toronto, Ontario, Canada
Trades: 0
shedlord: to the best of my knowledge, yes. Mind you, XP Professional does come with IIS so you can use your desktop as an Intranet web server and go from there.
__________________

Please login or register to view this content. Registration is FREE
|
Please login or register to view this content. Registration is FREE
(my blog)


Please login or register to view this content. Registration is FREE
(with proof)
ADAM Web Design is offline
Reply With Quote
View Public Profile Visit ADAM Web Design's homepage!
 
Old 02-19-2007, 02:25 PM Re: Web log analyzer for multiple domains
Junior Talker

Posts: 4
Trades: 0
Thanks for the suggestions. Speaking to the company who look after our server, their opinion was that there aren't specific log analyzers for cross-site analysis, and most log analyzers are fairly basic and aimed at single sites with low traffic.

An interesting suggestion they made was Sequioa View,
http://www.win.tue.nl/sequoiaview/

...which I've never seen before.

Anyway, I'm close to a semi-decent solution to this problem. Using the ASP below I was able to list the folder structure on our server with the sizes of each folder indicated. Typically, the only folder which isn't yet playing ball is the Logs folder, because I assume it is in reality in a different location to the rest (it just looks like it is in the root when we view the server via FTP).

This script is taken from here...
http://www.brainjar.com/asp/dirlist/
...with just a long script timeout added and the target folder changed.

<%@ LANGUAGE="VBScript" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<% '************************************************* **************************
'* ASP Directory Listing *
'* *
'* Do not remove this notice. *
'* *
'* Copyright 1999, 2000 by Mike Hall. *
'* Please see http://www.brainjar.com for documentation and terms of use. *
'************************************************* **************************
%>
<% Server.ScriptTimeout=8000
%>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Directory Listing</title>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<link rel="stylesheet" type="text/css" href="/common/default.css" />
</head>
<body>

<div id="demoBox">

<h3>ASP Directory Listing</h3>

<p>All folders and subfolders listed below</p>

</div>

<ul>
<% ListFolderContents("PHYSICAL_LOG_FOLDER_PATH_HERE" ) %>
</ul>

</body>
</html>

<% sub ListFolderContents(path)

dim fs, folder, file, item, url

set fs = CreateObject("Scripting.FileSystemObject")
set folder = fs.GetFolder(path)

'Display the target folder and info.

Response.Write("<li><b>" & folder.Name & "</b> - " _
& folder.Files.Count & " files, ")
if folder.SubFolders.Count > 0 then
Response.Write(folder.SubFolders.Count & " directories, ")
end if
Response.Write(Round(folder.Size / 1024) & " KB total." _
& vbCrLf)

Response.Write("<ul>" & vbCrLf)

'Display a list of sub folders.

for each item in folder.SubFolders
ListFolderContents(item.Path)
next

'Display a list of files.

' for each item in folder.Files
' url = MapURL(item.path)
' Response.Write("<li><a href=""" & url & """>" & item.Name & "</a> - " _
' & item.Size & " bytes, " _
' & "last modified on " & item.DateLastModified & "." _
' & "</li>" & vbCrLf)
' next

Response.Write("</ul>" & vbCrLf)

Response.Write("</li>" & vbCrLf)

end sub

function MapURL(path)

dim rootPath, url

'Convert a physical file path to a URL for hypertext links.

rootPath = Server.MapPath("/")
url = Right(path, Len(path) - Len(rootPath))
MapURL = Replace(url, "\", "/")

end function %>
shedlord is offline
Reply With Quote
View Public Profile
 
Old 02-19-2007, 03:38 PM Re: Web log analyzer for multiple domains
chrishirst's Avatar
Missing! presumed drunk.

Posts: 41,520
Name: Chris Hirst
Location: Blackpool. UK
Trades: 0
Your host techs probably have it set up with FTP and virtual folders

How "on the ball" are your server techs? If they are they should know about the ODBC logging option of IIS, where the web server will write the log data to a SQL server rather than text files, from there you could write SQL scripts that could analyse and report the data in any way, shape or form you require. You could use desktop tools such as Access to connect to the Db Server or write a GUI in ASP. Or use reporting tools such as Crystal Reports, even Excel could handle this.

Alternatively if any (or all) of your sites run ASP, I have a tracking script in ASP that logs the details into a SQL database, uses Standard ANSII SQL so any Db can be used. This is just the data collection agent, reporting and analytics is up to the end user.
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
A foolish consistency is the hobgoblin of little minds
Thought for today:- I SEO the only industry where all the cowboys are Indians?
chrishirst is online now
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Reply     « Reply to Web log analyzer for multiple domains
 

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off





   
RSS Feed  Feeds: RSS   JS   XML
RSS Feed  Feeds for this forum: RSS   JS   XML



Page generated in 0.32436 seconds with 12 queries