[tor-bugs] #20380 [Metrics/CollecTor]: Expand INSTALL.md to a more complete operator's guide

[Tor Bug Tracker & Wiki]

#20380: Expand INSTALL.md to a more complete operator's guide
-------------------------------+---------------------------------
 Reporter:  karsten            |          Owner:
     Type:  enhancement        |         Status:  needs_review
 Priority:  Medium             |      Milestone:  CollecTor 1.1.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------

Comment (by iwakeh):

 Replying to [comment:9 karsten]:
 > ...
 > A few thoughts:
 >
 >  - When you say that closer monitoring will be needed when disk space
 drops below a given number, do you mean 200G or 20G or a different number?

 I was referring to the disk space available when starting,  i.e. very
 close to 150G and logging to the same disk requires more attention than a
 terabyte setup.  Hmm, but if that is confusing just discard it.

 >
 >  - We shouldn't add new section headers easily.  The chosen section
 headers and even paragraphs in this document (will) have equivalents in
 the other operator's guides for other metrics tools.  If we want to add
 new sections, we'll also have to add those sections to the other manuals.
 The current sections are:
 >
 > {{{
 > $ grep "^#" INSTALL.md
 > # CollecTor Operator's Guide
 > ## Setting up the host
 > ## Setting up the service
 > ## Maintaining the service
 > }}}
 >

 It's important to have a consistent structure, but it would be helpful for
 readers to have sub-headings, which are application dependent.  Scrolling
 through a document with only generic headings when looking for particular
 information takes longer (of course, there is a search).
 So, maybe keep the top level consistent and allow for application
 dependent headlines below?

 >  - (continued) What other sections or even subsections should we
 include, and what instructions would go into those vs. the existing
 sections?

 I see two more sections.
 * 'Planning the Service' contrasts those sections giving a to-do list.
 People running instances will have different needs that can be better
 covered this way.
 * and even more important, a section 'Bootstrapping' or similar.  What
 data to download before a first run etc.  Again this is not a to-do list
 as it depends what data should be processed.

 >
 > > The idea behind my changes is that I think the service shouldn't be
 run from the unpacked tar
 > > folder.  The tar contains a development environment, so the jar would
 disappear after 'ant clean' or changed etc.
 > > The runtime directory should only contain files that are really
 necessary for the application or which were created by the application.
 > > Hope this doesn't make the description too complicated.
 >
 > Yes, makes sense, let's change that.  There are still a few paths left
 where we refer to files in `collector-<version>/` and where we should tell
 the user to copy those files to the working directory and run them from
 there.  I can update those places.
 >
 > > I also would like have even less description of tools from the OS,
 because such things should be decided by the operator.
 >
 > Which parts would that include?  The crontab, `@reboot`, `screen`, etc.?
 Can you make a list?

 When we avoid mentioning any such tools and methods, we avoid getting out
 of date and stay platform independent.  People operating servers have
 their favorite tools for and know what to do when told

 * run this script every three days
 * provide an http server for serving data and files in folders X, Y, Z.
 * for continuous operation ensure start-up on reboot and
 * monitoring of logs as well as running service is important
 etc.

 CollecTor does not depend on apache or crontab only the services provided
 by them.  Even the suggested install of openjdk could be left out.  Also
 apt-get.  Attempt of a list:

 * apt-get
 * apache2
 * crontab
 * gpg
 * openjdk, only Java 7
 * screen
 * ...

 Another thing would be to use `<OutPath>/recent` and similar instead of
 the default choices provided.  So, it is clear which option is referred
 to.

 The backup recommendation I would also leave out.  It depends on the setup
 and the kind of data collected.  Or, move it to 'Planning the service'?

--
Ticket URL: <https://troodi.torproject.org/projects/tor/ticket/20380#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online