[tor-bugs] #4463 [Website]: Set up web log analysis tool

[Tor Bug Tracker & Wiki]

#4463: Set up web log analysis tool
---------------------+------------------------------------------------------
 Reporter:  runa     |          Owner:  runa                        
     Type:  project  |         Status:  assigned                    
 Priority:  normal   |      Milestone:  Sponsor Z: December 31, 2011
Component:  Website  |        Version:                              
 Keywords:           |         Parent:                              
   Points:           |   Actualpoints:                              
---------------------+------------------------------------------------------

Comment(by karsten):

 Replying to [comment:10 runa]:
 > No static HTML as far as I know. You'd need to run apache2 on the same
 host. If you can create a tarball of the following directories (I don't
 think all of them are necessary, but hey), I'll make the stats available
 on the EC2 server:
 >
 > /var/lib/awstats
 > /usr/share/doc-base/awstats
 > /usr/share/awstats
 > /usr/share/doc/awstats
 > /etc/awstats
 > /etc/cron.d/awstats

 I'm going to send you the tarballs later today.

 > I don't see a problem with running both, so yes. A lot of people run
 both because they like something from AWStats that isn't available in
 Webalizer and vice versa.

 Sounds good.

 > Should we set up the web log analysis tools on the same VM as the one
 sanitizing the logs, or should we get a new one?

 I think we should get a new one.  The VM that has non-sanitized logs
 shouldn't run a web server.  It will also be the VM that sanitizes bridge
 descriptors.

 If you want to start setting up the VM with AWStats and Webalizer, please
 don't wait for me to set up the VM that sanitizes logs.  The 2010 logs
 should be sufficient to get something running.  The connection to the
 sanitizing VM will be a cronjob rsync'ing the sanitized logs as you find
 them in the tarballs.  Both the AWStats and the Webalizer setup should be
 able to handle adding new sanitized log files and removing files older
 than, say, one week.  We'll probably want to keep back the logs from a
 given day until that day is over (the sorting doesn't make much sense if
 we're sorting requests from just a few hours), so files shouldn't change
 once you get them.

 > > If we can avoid it, let's avoid writing something ourselves at this
 point.
 >
 > I agree.

 Okay.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4463#comment:11>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online