[metrics-bugs] #33362 [Internal Services/Tor Sysadmin Team]: Please provision a VM for the new exit scanner

Tor Bug Tracker & Wiki blackhole at torproject.org
Mon Feb 24 20:52:01 UTC 2020


#33362: Please provision a VM for the new exit scanner
-------------------------------------------------+-------------------------
 Reporter:  irl                                  |          Owner:  anarcat
     Type:  task                                 |         Status:  closed
 Priority:  Medium                               |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Normal                               |     Resolution:  fixed
 Keywords:                                       |  Actual Points:
Parent ID:                                       |         Points:  0.5
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------
Changes (by anarcat):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 Replying to [ticket:33362 irl]:
 > * At least 2 CPU cores
 > * 1.5GB RAM
 > * 50GB disk

 done, but with:

  * 2CPU
  * 2GB RAM
  * 10GB (system) + 20GB (user) disks

 after checking with irl on IRC, because this matches our more common
 configuration.

 > This host will generate a lot of network activity, and will do a lot of
 crypto operations. I'm afraid I don't have the data to quantify "a lot"
 yet.

 i guess we'll see, and hopefully Intel's builtin hardware coprocessors can
 save our bottoms for crypto. (or backdoor us, of course.)

 > Please give this host a DNS name, and do not change any of the existing
 DNS names for the old service at this time.

 the name is `check-01.torproject.org`. previous DNS not changed.

 > Please install the following packages from Debian:
 >
 > python-dnspython
 > python-stem
 > python3-stem
 > git

 done.

 > Please allow users in the check and tordnsel groups to log in, they
 should also have home directories.

 done.

 > Please install Apache, configure a virtualhost with the name
 check.torproject.org, and an alias of the server's hostname for initial
 setup.

 done, although the latter might not have verifiable HTTPS for now. let me
 know if that's a requirement.

 > Reverse proxy all requests to port 8080 on that virtualhost, except for
 /exit-addresses which should be a symlink to
 /srv/?.torproject.org/lists/latest and /lists which should be an autoindex
 with a document root of that same lists directory that latest lives in.

 something like this, right?

 {{{
         Alias /exit-addresses /srv/check.torproject.org/lists/latest
         Alias /lists /srv/check.torproject.org/lists

         ProxyPass / http://127.0.0.1:8080/
         ProxyPassReverse / http://127.0.0.1:8080/
 }}}

 ... done. :)

 > Let me know if anything needs clarification.

 i'll note that check.torproject.org currently has the following apache
 configuration:

 {{{
 Use common-tor-vhost-https-redirect  check.torproject.org

 <VirtualHost *:443>
         ServerName check.torproject.org
         ServerAdmin torproject-admin at torproject.org

         #Use common-ssl-wildcard.tpo
         Use common-ssl-service check.torproject.org
         Use common-ssl-HSTS
         Use http-pkp-check.torproject.org

         <IfModule mod_userdir.c>
                 UserDir disabled
         </IfModule>

         ErrorLog ${APACHE_LOG_DIR}/check.torproject.org--error.log
         CustomLog ${APACHE_LOG_DIR}/check.torproject.org-access.log
 privacy

         ServerSignature On

         DocumentRoot /srv/check.torproject.org/htdocs

         <Directory /srv/check.torproject.org/htdocs>
                 Require all granted
         </Directory>

         RewriteEngine On
         RewriteRule ^/$ /torcheck/ [PT]
         RewriteRule  ^/cgi-bin/TorBulkExitList.py$ /torbulkexitlist  [PT]
         <IfModule mod_proxy.c>
                 ProxyRequests On
                 ProxyVia On
                 <Proxy *>
                         Order deny,allow
                         Allow from all
                 </Proxy>
                 ProxyPass /api http://127.0.0.1:8000/api retry=10
                 ProxyPass /torcheck/ http://127.0.0.1:8000/ retry=10
                 ProxyPass /torbulkexitlist
 http://127.0.0.1:8000/torbulkexitlist retry=10
                 ProxyPassReverse /torcheck/ http://127.0.0.1:8000/
         </IfModule>
 </VirtualHost>
 }}}

 ie.

  1. / redirects to /torcheck
  2. /cgi-bin/TorBulkExitList.py to /torbulkexitlist (go figure, but we
 might want to keep that?)
  3. /api goes to port 8000/api
  4. /torcheck goes to port 8000/
  5. /torbulkexitlist goes to port 8000/torbulkexitlist

 ... shouldn't we also have aliases for those eventually?

 > I could also review the puppet if you point me at where to find it.

 the profile looks like:

 {{{
 # rewrite of the exit scanner
 #
 # not to be confused with the old roles::check that is now deprecated.
 class profile::check {
   include apache2
   include apache2::ssl
   include apache2::proxy_http

   file { '/srv/check.torproject.org':
     ensure => directory,
     mode   => '0755',
     owner  => 'check',
     group  => 'check',
   }
   file { '/srv/check.torproject.org/home':
     ensure => directory,
     mode   => '0775',
     owner  => 'check',
     group  => 'check',
   }
   file {'/home/check':
     ensure => link,
     target => '/srv/check.torproject.org/home',
   }
   file { '/srv/check.torproject.org/htdocs':
     ensure => directory,
     mode   => '0775',
     owner  => 'check',
     group  => 'check',
   }

   $vhost_name = 'check.torproject.org'
   apache2::site { $vhost_name:
     content => template('profile/check/vhost.erb'),
   }

   ssl::service { 'check.torproject.org': notify  => Exec['service apache2
 reload'], key => true, }

   # new exit scanner dependencies, see #33362
   package { [
     'python-dnspython',
     'python-stem',
     'python3-stem',
   ]:
     ensure => installed,
   }
 }
 }}}

 the apache config file template looks like:

 {{{
 # This file is managed by puppet, all changes will be lost
 Use common-tor-vhost-https-redirect <%= @vhost_name %>
 <Macro vhost-inner-<%= @vhost_name %>>
         ServerAdmin tor-assistants at torproject.org

         ErrorLog /var/log/apache2/<%= @vhost_name %>-error.log
         CustomLog /var/log/apache2/<%= @vhost_name %>-access.log
 privacyssl
         ServerSignature On

         <IfModule mod_userdir.c>
                 UserDir disabled
         </IfModule>

         DocumentRoot /var/www/html/
         DocumentRoot /srv/check.torproject.org/htdocs

         <Directory /srv/check.torproject.org/htdocs>
                 Require all granted
         </Directory>

         Alias /exit-addresses /srv/check.torproject.org/lists/latest
         Alias /lists /srv/check.torproject.org/lists

         ProxyPass / http://127.0.0.1:8080/
         ProxyPassReverse / http://127.0.0.1:8080/
 </Macro>

 <VirtualHost *:443>
         ServerName <%= @vhost_name %>
         ServerAlias <%= @fqdn %>
         Use common-ssl-service <%= @vhost_name %>
         Use common-ssl-HSTS
         #Use http-pkp-<%= @vhost_name %>
         Use vhost-inner-<%= @vhost_name %>
 </VirtualHost>

 <% if scope.function_onion_global_service_hostname([@vhost_name]) -%>
 <Virtualhost *:80>
         ServerName <%=
 scope.function_onion_global_service_hostname([@vhost_name]) %>
         Use vhost-inner-<%= @vhost_name %>
 </VirtualHost>
 <% end -%>

 # vim:ft=apache:
 }}}

 ... and that's about it!

 i'll close this for now, let me know if you need anything else!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33362#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the metrics-bugs mailing list