- tor-commits - lists.torproject.org

[tor/maint-0.2.2] Check some more return values in unit tests
by nickm＠torproject.org 08 Jun '11

08 Jun '11

commit ff75e8b02dbf35c2ecc82ffd4d2b2112a144322e Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Wed Jun 8 21:06:01 2011 +0200 Check some more return values in unit tests --- changes/coverity_maint | 3 +++ src/test/test_addr.c | 3 +++ 2 files changed, 6 insertions(+), 0 deletions(-) diff --git a/changes/coverity_maint b/changes/coverity_maint index ec25d09..1bd1c41 100644 --- a/changes/coverity_maint +++ b/changes/coverity_maint @@ -1,3 +1,6 @@ o Code simplifications and refactoring: - Remove some dead code as indicated by coverity. + o Minor bugfixes: + - Add some forgotten return value checks during unit tests. Found + by coverity. diff --git a/src/test/test_addr.c b/src/test/test_addr.c index 6db4ee2..1dab0e0 100644 --- a/src/test/test_addr.c +++ b/src/test/test_addr.c @@ -436,13 +436,16 @@ test_addr_ip6_helpers(void) /* test tor_addr_parse_mask_ports */ test_addr_mask_ports_parse("[::f]/17:47-95", AF_INET6, 0, 0, 0, 0x0000000f, 17, 47, 95); + test_streq(p1, "::f"); //test_addr_parse("[::fefe:4.1.1.7/120]:999-1000"); //test_addr_parse_check("::fefe:401:107", 120, 999, 1000); test_addr_mask_ports_parse("[::ffff:4.1.1.7]/120:443", AF_INET6, 0, 0, 0x0000ffff, 0x04010107, 120, 443, 443); + test_streq(p1, "::ffff:4.1.1.7"); test_addr_mask_ports_parse("[abcd:2::44a:0]:2-65000", AF_INET6, 0xabcd0002, 0, 0, 0x044a0000, 128, 2, 65000); + test_streq(p1, "abcd:2::44a:0"); r=tor_addr_parse_mask_ports("[fefef::]/112", &t1, NULL, NULL, NULL); test_assert(r == -1); r=tor_addr_parse_mask_ports("efef::/112", &t1, NULL, NULL, NULL);

1 0

[tor/maint-0.2.2] Don't use signed 1-bit bitfields
by nickm＠torproject.org 08 Jun '11

08 Jun '11

commit 680646e0de29454f92d57bc3a4895d75c95e158c Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Wed Jun 8 21:27:32 2011 +0200 Don't use signed 1-bit bitfields This was harmless, we never compared it to anything but itself or 0. But Coverity complained, and it had a point. --- changes/coverity_maint | 1 + src/or/policies.c | 2 +- 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/changes/coverity_maint b/changes/coverity_maint index fd8c440..6d60355 100644 --- a/changes/coverity_maint +++ b/changes/coverity_maint @@ -4,4 +4,5 @@ o Minor bugfixes: - Add some forgotten return value checks during unit tests. Found by coverity. + - Don't use 1-bit wide signed bit fields. Found by coverity. diff --git a/src/or/policies.c b/src/or/policies.c index e48f420..c870360 100644 --- a/src/or/policies.c +++ b/src/or/policies.c @@ -45,7 +45,7 @@ typedef struct policy_summary_item_t { uint16_t prt_max; /**< Highest port number to accept/reject. */ uint64_t reject_count; /**< Number of IP-Addresses that are rejected to this port range. */ - int accepted:1; /** Has this port already been accepted */ + unsigned int accepted:1; /** Has this port already been accepted */ } policy_summary_item_t; /** Private networks. This list is used in two places, once to expand the

1 0

[tor/maint-0.2.2] Comment out some obviously dead code.
by nickm＠torproject.org 08 Jun '11

08 Jun '11

commit 9e56ac27da80969fadaad7eaf64a8384e68a778e Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Wed Jun 8 21:23:05 2011 +0200 Comment out some obviously dead code. Coverity warned about it, it's harmless to comment out. --- src/or/circuitlist.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/or/circuitlist.c b/src/or/circuitlist.c index 4ad244d..8534c38 100644 --- a/src/or/circuitlist.c +++ b/src/or/circuitlist.c @@ -775,8 +775,8 @@ circuit_get_by_circid_orconn_impl(circid_t circ_id, or_connection_t *conn) return found->circuit; return NULL; - /* The rest of this checks for bugs. Disabled by default. */ + /* We comment it out because coverity complains otherwise. { circuit_t *circ; for (circ=global_circuitlist;circ;circ = circ->next) { @@ -795,7 +795,7 @@ circuit_get_by_circid_orconn_impl(circid_t circ_id, or_connection_t *conn) } } return NULL; - } + } */ } /** Return a circ such that:

1 0

[tor/maint-0.2.2] Fix a rare memleak during stats writing
by nickm＠torproject.org 08 Jun '11

08 Jun '11

commit f30327449009a7f00b0f5c2bd09a7eff615df3dd Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Wed Jun 8 21:35:26 2011 +0200 Fix a rare memleak during stats writing If rep_hist_buffer_stats_write() was called unitinitalized, we'd leak memory. --- changes/coverity_maint | 1 + src/or/rephist.c | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/changes/coverity_maint b/changes/coverity_maint index 6d60355..e7be90a 100644 --- a/changes/coverity_maint +++ b/changes/coverity_maint @@ -5,4 +5,5 @@ - Add some forgotten return value checks during unit tests. Found by coverity. - Don't use 1-bit wide signed bit fields. Found by coverity. + - Fix a rare memory leak during stats writing. Found by coverity. diff --git a/src/or/rephist.c b/src/or/rephist.c index 242fe81..54593a0 100644 --- a/src/or/rephist.c +++ b/src/or/rephist.c @@ -2451,8 +2451,8 @@ rep_hist_buffer_stats_write(time_t now) int processed_cells[SHARES], circs_in_share[SHARES], number_of_circuits, i; double queued_cells[SHARES], time_in_queue[SHARES]; - smartlist_t *str_build = smartlist_create(); - char *str = NULL, *buf=NULL; + smartlist_t *str_build = NULL; + char *str = NULL, *buf = NULL; circuit_t *circ; if (!start_of_buffer_stats_interval) @@ -2460,6 +2460,8 @@ rep_hist_buffer_stats_write(time_t now) if (start_of_buffer_stats_interval + WRITE_STATS_INTERVAL > now) goto done; /* Not ready to write */ + str_build = smartlist_create(); + /* add current circuits to stats */ for (circ = _circuit_get_global_list(); circ; circ = circ->next) rep_hist_buffer_stats_add_circ(circ, now);

1 0

[flashproxy/master] Fix a syntax error in "data from unconnected remote" message.
by dcf＠torproject.org 08 Jun '11

08 Jun '11

commit f23dc0aa8b122c7be6b6ead300b255388041e57a Author: David Fifield <david(a)bamsoftware.com> Date: Wed Jun 8 09:49:32 2011 -0700 Fix a syntax error in "data from unconnected remote" message. --- connector.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/connector.py b/connector.py index 78db084..c2c648e 100755 --- a/connector.py +++ b/connector.py @@ -386,7 +386,7 @@ while True: remotes.remove(fd) fd.close() else: - log(u"Data from unconnected remote %s (%d bytes)." % format_addr(fd.getpeername(), len(data))) + log(u"Data from unconnected remote %s (%d bytes)." % format_addr(fd.getpeername()), len(data)) fd.buf += data if len(fd.buf) >= UNCONNECTED_BUFFER_LIMIT: log(u"Refusing to buffer more than %d bytes from local %s." % (UNCONNECTED_BUFFER_LIMIT, format_addr(fd.getpeername())))

1 0

[metrics-web/master] Put new metrics-web tarball on the website.
by karsten＠torproject.org 08 Jun '11

08 Jun '11

commit 0d6338ad730af1028903a0e76351acad08a77b6e Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Jun 8 15:14:27 2011 +0200 Put new metrics-web tarball on the website. --- ChangeLog | 2 ++ web/WEB-INF/tools.jsp | 3 +++ 2 files changed, 5 insertions(+), 0 deletions(-) diff --git a/ChangeLog b/ChangeLog index 94030d0..94884cf 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,7 @@ Metrics database and website change log: +Changes in version 0.0.2 - 2011-??-?? + Changes in version 0.0.1 - 2011-06-08 - Initial release diff --git a/web/WEB-INF/tools.jsp b/web/WEB-INF/tools.jsp index 01e5a84..56098f5 100644 --- a/web/WEB-INF/tools.jsp +++ b/web/WEB-INF/tools.jsp @@ -47,6 +47,9 @@ accesses the metrics database to generate this website. The metrics website uses R to generate custom graphs on demand.</p> <ul> + <li>Download + <a href="dist/metrics-web-0.0.1.tar">metrics-web 0.0.1</a> + (<a href="dist/metrics-web-0.0.1.tar.asc">sig</a>)</li> <li>Browse the <a href="https://gitweb.torproject.org/metrics-web.git/tree">Git repository</a></li>

1 0

[metrics-web/master] Create a ChangeLog to prepare the first release.
by karsten＠torproject.org 08 Jun '11

08 Jun '11

commit fc29ccbbfc57ffc081a26cc9c5bd3f0f4eb1ab2c Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Jun 8 14:59:41 2011 +0200 Create a ChangeLog to prepare the first release. --- ChangeLog | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/ChangeLog b/ChangeLog new file mode 100644 index 0000000..94030d0 --- /dev/null +++ b/ChangeLog @@ -0,0 +1,5 @@ +Metrics database and website change log: + +Changes in version 0.0.1 - 2011-06-08 + - Initial release +

1 0

[metrics-web/master] Fix navigation on ExoneraTor page.
by karsten＠torproject.org 08 Jun '11

08 Jun '11

commit fbc171eaf91412c2c652e5b97c44a183ae6f47a1 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Jun 8 13:19:15 2011 +0200 Fix navigation on ExoneraTor page. --- .../torproject/ernie/web/ExoneraTorServlet.java | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/src/org/torproject/ernie/web/ExoneraTorServlet.java b/src/org/torproject/ernie/web/ExoneraTorServlet.java index e501129..9f7c0cc 100644 --- a/src/org/torproject/ernie/web/ExoneraTorServlet.java +++ b/src/org/torproject/ernie/web/ExoneraTorServlet.java @@ -66,6 +66,7 @@ public class ExoneraTorServlet extends HttpServlet { + " <a href=\"status.html\">Status</a>\n" + " <br>\n" + " <font size=\"2\">\n" + + " <a href=\"networkstatus.html\">Network Status</a>\n" + " <a class=\"current\">ExoneraTor</a>\n" + " <a href=\"relay-search.html\">Relay Search</a>\n" + " <a href=\"consensus-health.html\">Consensus "

1 0

[metrics-web/master] Write an actually useful README.
by karsten＠torproject.org 08 Jun '11

08 Jun '11

commit bf314c1aa2900eced290701c6664f09cc73a8825 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Jun 8 14:43:33 2011 +0200 Write an actually useful README. --- README | 527 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- doc/rserve.pdf | Bin 99059 -> 0 bytes doc/rserve.tex | 140 --------------- 3 files changed, 523 insertions(+), 144 deletions(-) diff --git a/README b/README index 93315ee..0786852 100644 --- a/README +++ b/README @@ -1,7 +1,526 @@ -ERNIE is the Enhanced R-based tor Network Intelligence Engine - (why ERNIE? because nobody liked BIRT; sorry for misspelling Tor) +Tor Metrics Database and Website +================================ --------------------------------------------------------------------------- +The metrics database stores publicly available data about the Tor network +which are visualized by the metrics website. -Please find documentation on ERNIE in website/ernie-howto.html and doc/ . +This software package, metrics-web, contains (1) the code to import Tor +network data into a database, (2) the code to generate graphs and .CSV +output, and (3) the code for a dynamic web application. metrics-web is +based on Java, Ant, PostgreSQL, R, Apache HTTP Server, and Apache Tomcat. + +This README explains all necessary steps to install metrics-web including +the database (Section 1), the graphing engine (Section 2), and the web +application (Section 3). It is possible to install only the database part +or only the database and the graphing engine, if desired. + + +1. Installing the metrics database +================================== + +The metrics database contains data about the Tor Network coming from +different sources, including the Tor directory authorities, Torperf +performance measurement installations, the GetTor software package +delivery service, and others. + + +1.1. Preparing the operating system +----------------------------------- + +This README describes the steps for installing metrics-web on a Debian +GNU/Linux Squeeze server. Instructions for other operating systems may +vary. + +In the following it is assumed that root privileges are available. +Commands requiring root privileges will be prefixed with # below. + +Start by adding a metrics user that will be used to execute all commands +that do not require root privileges. These commands will be prefixed with +$ below. + +# adduser metrics + +The database importer and website sources will be installed in +/srv/metrics-web/ that is created as follows: + +# mkdir /srv/metrics-web/ +# chmod g+ws /srv/metrics-web/ +# chown metrics:metrics /srv/metrics-web/ + +Either extract the metrics-web source tarball... + +$ tar xf metrics-web-x.y.z.tar /srv/metrics-web/ + +... or clone the metrics-web Git repository: + +$ git clone git://git.torproject.org/metrics-web /srv/metrics-web/ + +Install Sun Java 6, Ant 1.8, and PostgreSQL 8.4 that are necessary for +setting up the metrics database (be sure to include Debian's non-free +repository in /etc/apt/sources.list). + +# apt-get install sun-java6-jdk ant postgresql-8.4 + +Make Sun's Java the default. + +# update-java-alternatives -s java-6-sun + +Check the versions of the newly installed tools. + +$ java -version +java version "1.6.0_24" +Java(TM) SE Runtime Environment (build 1.6.0_24-b07) +Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode) + +$ ant -version +Apache Ant version 1.8.0 compiled on March 11 2010 + +$ psql --version +psql (PostgreSQL) 8.4.7 +contains support for command-line editing + + +1.2. Configuring the database +============================= + +The first step in setting up the metrics database is to configure the +PostgreSQL database and import a database schema. + +Start by creating a new metrics database user. There is no need to give +the metrics user superuser privileges or allow it to create databases or +new roles. + +# sudo -u postgres createuser -P metrics + +Create a new database tordir owned by user metrics. + +# sudo -u postgres createdb -O metrics tordir + +Import the metrics database schema. + +$ psql -f /srv/metrics-web/db/tordir.sql tordir + +Confirm that the database now contains tables to hold metrics data. In +the following, => will be used as the database prompt. + +$ psql tordir +=> \dt+ +=> \q + + +1.3. Importing relay descriptor tarballs +======================================== + +In most cases it makes sense to populate the metrics database with +archived relay descriptors from the official metrics website. + +Download the relay descriptor tarballs from the metrics website at +https://metrics.torproject.org/data.html#relaydesc and extract them to +/srv/metrics-web/archives/ . The database importer can process v3 votes, +v3 consensuses, server descriptors, and extra-infos. + +Edit the config file ~/metrics-web/config (or create it if it's not there) +to contain the following five lines (be sure to remove the linebreak in +the line defining the JDBC string and insert the real password there): + +ImportDirectoryArchives 1 +DirectoryArchivesDirectory archives/ +KeepDirectoryArchiveImportHistory 1 +WriteRelayDescriptorDatabase 1 +RelayDescriptorDatabaseJDBC + jdbc:postgresql://localhost/tordir?user=metrics&password=password + +Compile and run the Java database importer. + +$ cd /srv/metrics-web/ +$ ./run.sh + +The database import will take a while. Once it's complete, check that the +database tables now contain metrics data: + +$ psql tordir +=> \dt+ +=> \q + +It's safe to delete the relay descriptor files in ~/metrics-web/archives/ +once they are imported. + +An alternative to importing relay descriptor tarballs directly into the +database is to convert them into a data format that psql's \copy command +can process. Look for the config option WriteRelayDescriptorsRawFiles in +/srv/metrics-web/config.template for more information on this experimental +feature. + + +1.4. Importing relay descriptors from a local Tor data directory +================================================================ + +WARNING: The functions described in this section are not implemented yet! + +In a future version of metrics-web, the metrics database importer will be +able to import the cached descriptors from a local Tor data directory. +(A special case of importing descriptors from a continuously updated +directory is when both metrics-db and metrics-web are run on the same +machine, but this shouldn't be the general case.) + +Configure a local Tor client to fetch all known descriptors as early as +possible by adding these config options to its torrc file: + +FetchUselessDescriptors 1 +FetchDirInfoExtraEarly 1 + +Tell the metrics database importer where to find the cached descriptor +files. One way to achieve this is to add symbolic links to +/srv/metrics-web/archives/ like this. Tor's data directory is assumed to +be /srv/tor/ here. + +$ cd /srv/metrics-web/archives/ +$ ln -s /srv/tor/cached-* . + +Add a crontab entry for the database importer to run once per hour: + +15 * * * * cd /srv/metrics-web/ && ./run.sh + +In a future version of metrics-web it may also be possible to update local +relay descriptor tarballs from the official metrics server via rsync and +import only the changes into the metrics database. The idea is to simply +rsync the data/ directory from the metrics server and have all information +available. But until this is implemented, the recommended way to keep the +metrics website up-to-date would be the one described above in this +section. + + +1.5. Importing GeoIP information +================================ + +Some of the graphs require GeoIP information to resolve IP addresses to +country codes. This information is provided in MaxMind's GeoLite City +database available at http://www.maxmind.com/app/geolitecity. + +Download and extract the two files GeoLiteCity-Location.csv and +GeoLiteCity-Blocks.csv to /srv/metrics-web/. + +Import the two files into the metrics database. + +$ ant geoipdb + +Note that there is no easy way to update the GeoIP information in the +metrics database yet. The only way to do so is to manually delete and +recreate the database table and import the new GeoIP database. + + +1.6. Pre-calculating relay statistics +===================================== + +The relay graphs on the metrics website rely on pre-calculated statistics +in the metrics database. These statistics are not calculated after every +completed import, which would usually be once per hour. In general it's +sufficient to pre-calculate statistics 2 or 4 times a day. + +Calculate statistics manually after large imports (this may take a while): + +$ psql tordir -c 'SELECT * FROM refresh_all();' + +If the metrics database gets updated automatically, write a script and add +a crontab entry for pre-calculating statistics every 6 or 12 hours. + + +1.7. Generating network status information +========================================== + +The metrics database importer can analyze the most recently parsed network +status consensus for irregularities indicating problems with the directory +authorities. There are two possible outputs: the consensus-health page +that can be found at https://metrics.torproject.org/consensus-health.html +and a local file that can be parsed by Nagios that will be written to +/srv/metrics-web/website/consensus-health . + +Edit /srv/metrics-web/config to contain either or both of the following +options: + +WriteConsensusHealth 1 +WriteNagiosStatusFile 1 + + +1.8. Importing sanitized bridge descriptors +=========================================== + +The metrics database can store aggregate statistics about running bridges +and bridge usage. These statistics are added by parsing sanitized bridge +descriptors available on the official metrics website. + +Download a sanitized bridge descriptor tarball from the metrics website at +https://metrics.torproject.org/data.html#bridgedesc and extract it to, +e.g., /srv/metrics-web/bridges/bridge-descriptors-2011-05/ . + +Edit /srv/metrics-web/config to contain the following options: + +ImportSanitizedBridges 1 +SanitizedBridgesDirectory bridges/ +KeepSanitizedBridgesImportHistory 1 +WriteBridgeStats 1 + +Note that the bridge usage statistics require parsing relay descriptors of +the same time period in order to filter bridges that have been running as +relays from the results. When parsing sanitized bridge descriptors for +the first time it may be necessary to delete the relay descriptor import +history in /srv/metrics-web/stats/archives-import-history and import all +relay descriptors once again. + +Run the database import: + +$ ./run.sh + + +1.9. Importing Torperf performance data +======================================= + +Torperf measures the performance of the Tor network as users experience +it. Torperf's measurement data are available on the metrics website and +can be imported into the metrics database, too. + +Download the Torperf measurement files from the metrics website at +https://metrics.torproject.org/data.html#performance and put them in a +subdirectory, e.g., /srv/metrics-web/torperf/ . + +Edit /srv/metrics-web/config to contain the following options: + +ImportWriteTorperfStats 1 +TorperfDirectory torperf/ + +Run the database import: + +$ ./run.sh + + +1.10. Importing GetTor statistics +================================= + +WARNING: The GetTor statistics are not available for download yet, so that +this section only applies to the official metrics website. + +GetTor is a software distribution service that allows users to fetch the +Tor software via email. GetTor produces daily statistics of requested +packages that can be imported into the metrics database. + +Put the GetTor statistics file into /srv/metrics-web/gettor/ . + +Edit /srv/metrics-web/config to contain the following options: + +ProcessGetTorStats 1 +GetTorDirectory gettor/ + +Run the database import: + +$ ./run.sh + + +2. Installing the graphing engine +================================= + +The metrics graphing engine generates custom graphs of Tor network data +based on user-provided parameters. The graphing engine requires the +metrics database to be installed as described in the previous section. + +The graphing engine uses R and Rserve to generate its graphs. Rserve is a +TCP/IP server that makes it easy for other tools to use R without spawning +their own R process. Rserve also pre-loads R code and R libraries which +saves time when processing user requests. + +In this configuration, Rserve will run in the context of the metrics user. + +Setting up the graphing engine requires installing PostgreSQL's header +files and R 2.8 or higher. R 2.8 or higher is required for the ggplot2 +library. + +# apt-get install libpq-dev r-base-dev + +Run R as user metrics and install required packages to ~/R/. In the +following, R commands will be prefixed with >. + +$ R +> install.packages("Rserve") +> install.packages("ggplot2") +> install.packages("RPostgreSQL") +> q() + +Start the Rserve daemon (the exact path of Rserve-bin.so may vary), check +that it's working by connecting via telnet, and shut it down: + +$ R CMD ~/R/x86_64-pc-linux-gnu-library/2.11/Rserve/libs/Rserve-bin.so +$ telnet 127.0.0.1 6311 +$ echo "library(Rserve); RSshutdown(RSconnect())" | R --slave + +Also check that a database connection can be established from within R +(using the actual password instead of "password"): + +$ R +> library(RPostgreSQL) +> drv <- dbDriver("PostgreSQL") +> con <- dbConnect(drv, user = "metrics", password = "password", + dbname = "tordir") +> dbDisconnect(con) +> dbUnloadDriver(drv) +> q() + +Insert the database password in the Rserve initialization script in +/srv/metrics-web/rserve/rserve-init.R. + +Update the workdir path in /srv/metrics-web/rserve/Rserv.conf . + +Start Rserve, this time with the metrics-web-specific configuration that +includes pre-loading the graph code: + +$ cd /srv/metrics-web/rserve/ && ./start.sh + +Add a crontab entry to start Rserve on reboot: + +@reboot cd /srv/metrics-web/rserve/ && ./start.sh + +Rserve will pre-load the graph code at startup. If changes are made to +the graph code, Rserve must be restarted: + +$ cd /srv/metrics-web/rserve/ +$ ./shutdown.sh && ./start.sh + + +3. Installing the metrics website +================================= + +The metrics website lets web users search parts of the metrics database +and visualizes custom graphs. Both the metrics database and the graphing +engine are required to set up the metrics website as described in this +section. + +Note that the description here has a few specific parts that only apply to +the official metrics website. These parts should be changed when setting +up a non-official metrics website. + + +3.1. Configuring Apache HTTP Server +=================================== + +The Apache HTTP Server is used as the front-end web server that serves +static resources itself and forwards requests for dynamic resources to +Apache Tomcat. + +Start by installing Apache 2: + +# apt-get install apache2 + +Disable Apache's default site. + +# a2dissite default + +Enable mod_rewrite to tell Apache where to find static resources on disk. +Also enable mod_proxy to forward requests to Tomcat. + +# a2enmod rewrite proxy_http + +Create a new virtual host configuration and store it in a new file +/etc/apache2/sites-available/metrics.torproject.org with the following +content: + +<VirtualHost *:80> + ServerName metrics.torproject.org + ServerAdmin torproject-admin(a)torproject.org + ErrorLog /var/log/apache2/error.log + CustomLog /var/log/apache2/access.log combined + ServerSignature On + <IfModule mod_rewrite.c> + RewriteEngine On + RewriteRule /(data|dist|papers)/(.*) /srv/metrics-web/$1/$2 [L] + RewriteRule /(consensus-health.html) /srv/metrics-web/website/$1 [L] + </IfModule> + <IfModule mod_proxy.c> + <Proxy *> + Order deny,allow + Allow from all + </Proxy> + ProxyPass / http://127.0.0.1:8080/ernie/ retry=15 + ProxyPassReverse / http://127.0.0.1:8080/ernie/ + ProxyPreserveHost on + </IfModule> +</VirtualHost> + +Create the directories containing static resources: /srv/metrics-web/data/ +contains the tarballs and other metrics data linked from data.html. +/srv/metrics-web/dist/ contains the software packages linked from +tools.html. /srv/metrics-web/papers/ contains the papers and technical +reports linked from papers.html. Note that there is no option not to +serve these files other than manually removing the links from the .html +pages. + +Enable the new virtual host. + +# a2ensite metrics.torproject.org + +Restart Apache just to be sure that all changes are effective. + +# /etc/init.d/apache2 restart + + +3.2. Configuring Apache Tomcat +============================== + +Apache Tomcat will process requests for dynamic resources, including web +pages and graphs. + +Install Tomcat 6: + +# apt-get install tomcat6 + +Replace Tomcat's default configuration in /etc/tomcat6/server.xml with the +following configuration: + +<Server port="8005" shutdown="SHUTDOWN"> + <Service name="Catalina"> + <Connector port="8080" maxHttpHeaderSize="8192" + maxThreads="150" minSpareThreads="25" maxSpareThreads="75" + enableLookups="false" redirectPort="8443" acceptCount="100" + connectionTimeout="20000" disableUploadTimeout="true" + compression="off" compressionMinSize="2048" + noCompressionUserAgents="gozilla, traviata" + compressableMimeType="text/html,text/xml,text/plain" /> + <Engine name="Catalina" defaultHost="yatei.torproject.org"> + <Host name="metrics.torproject.org" appBase="webapps" + unpackWARs="true" autoDeploy="true" + xmlValidation="false" xmlNamespaceAware="false"> + <Alias>yatei.torproject.org</Alias> + <Valve className="org.apache.catalina.valves.AccessLogValve" + directory="logs" prefix="metrics_access_log." suffix=".txt" + pattern="%l %u %t %r %s %b" resolveHosts="false"/> + </Host> + </Engine> + </Service> +</Server> + +Be sure to replace *.torproject.org with something else, unless this is +a re-installation of the official metrics website. + +Update the database password in /srv/metrics-web/etc/context.xml. + +Update the paths starting with /srv/metrics.torproject.org/ in +/srv/metrics-web/etc/web.xml to the correct paths in /srv/metrics-web/. +The default paths in that file are correct for the official metrics +website setup which is slightly different than the one described here. + +Now generate the web application. + +$ ant make-war + +Create a symbolic link to the ernie.war file: + +# ln -s /srv/metrics-web/ernie.war /var/lib/tomcat6/webapps/ + +Tomcat will now attempt to deploy the web application automatically. + +Whenever the metrics website needs to be redeployed, generate a new .war +file and Tomcat will reload the web application automatically. + +Restart Tomcat to make all configuration changes effective: + +# /etc/init.d/tomcat6 restart + +The metrics website should now work. diff --git a/doc/rserve.pdf b/doc/rserve.pdf deleted file mode 100644 index 2adca75..0000000 Binary files a/doc/rserve.pdf and /dev/null differ diff --git a/doc/rserve.tex b/doc/rserve.tex deleted file mode 100644 index 317b475..0000000 --- a/doc/rserve.tex +++ /dev/null @@ -1,140 +0,0 @@ -\documentclass{article} -\setlength{\parindent}{0in} -\begin{document} -\title{Tor Metrics - Rserve} -\author{by Kevin Berry \texttt{<kevin.berry(a)villanova.edu>}} -\maketitle -\section{Overview} -Rserve is a TCP/IP interface which allows other tools and languages to use -the facilities of the R language. In our case, Java, Tomcat, Postgres and -other parts of Metrics work with Rserve to generate graphs on-demand. For -more information about the Metrics website, see the sister repository -\emph{metrics-db}. Here, we will cover how to install R, Rserve, the R -Postgres driver, and ggplot2. For more information about the Postgres -setup, see manual.pdf. The database should be set up before continuing -this. - -\section{Architecture} -See the \emph{rserve} directory for the start/stop scripts and -config. The graph code is all pre-loaded when Rserve starts, so, if any -changes are made to the graph code, Rserve must be restarted. The database -name, user, and password can be configured in \emph{rserve-init.R}, as well -as the pre-loaded libraries. Rserve forks itself upon connection, so R code -can be pre-loaded to speed things up. - -\section{Setup} -\subsection{Installing R} -Before we get started, we need to have R installed. We need to have the R -dev package installed so we can use the add-ons. - -\begin{verbatim} -$ sudo apt-get install r-base-dev -\end{verbatim} - -\subsection{Installing and testing Rserve} -There are a few different ways to install Rserve. However, the easiest and -most direct way to install it is through R's built-in package manager and -package network, CRAN (\emph{The Comprehensive R Archive Network - -http://cran.r-project.org}). Unfortunately, Rserve isn't packaged currently -for many Linux distributions, so it requires a bit manual configuration and -administration. -\\ - -R needs to be started as root so its build-in package manager can access -the file system to install its own packages. Select the mirror through the -Tcl/tk or command line interface and it should install. - -\begin{verbatim} -$ sudo R -> install.packages("Rserve") -\end{verbatim} - -We want to start a bare server and see if it works correctly. - -\begin{verbatim} -$ R CMD Rserve -\end{verbatim} - -Now, test it with the built-in R connector. - -\begin{verbatim} -$ R -> library(Rserve) -> c <- RSconnect() -> RSshutdown(c) -\end{verbatim} - -If this worked, the server is listening, so it installed and started -correctly. - -\subsection{Installing ggplot2} -ggplot2 is the second necessary R package that we need for Metrics. - -\begin{verbatim} -$ R -> install.packages("ggplot2") -\end{verbatim} - -\subsection{Installing and testing the R PostgresSQL driver} -The Postgres driver installs similarly to the other R packages. - -\begin{verbatim} -$ R -> install.packages("RPostgreSQL") -\end{verbatim} - -First, make sure Postgres is started and configured correctly according to -the Metrics specifications (see manual.pdf). -\\ - -Start the R console, load the driver, and connect to the database. The -database user, password may need to be changed. -\begin{verbatim} -$ R -> library(RPostgreSQL) -> drv <- dbDriver("PostgreSQL") -> con <- dbConnect(drv, user="ernie", password="", dbname="tordir") -> dbDisconnect(con) -> dbUnloadDriver(drv) -\end{verbatim} - -\section{Administrating Rserve} -Since Rserve is not standardly packaged, a few things must be done to -ensure it runs smoothly and securely. We need to adjust permissions, add -users, and modify groups so it works nicely with Tomcat. Feel free to do -this differenty according to your system's requirements. - -\subsection{Adding users and groups} -We'll add the user 'rserve' without a shell and no home directory. - -\begin{verbatim} -$ useradd rserve -s /bin/false -U -\end{verbatim} - -Now, find the user id and group id of the rserve user, and edit them in -rserve/Rserv.conf. This is so Rserve properly forks itself and runs as the -correct user when it is started. - -\begin{verbatim} -$ id rserve -uid=1011(rserve) gid=1012(rserve) groups=1012(rserve) -\end{verbatim} - -Next, we need to add the rserve user to the 'apache' group (The default -user for Tomcat), so it can communicate correctly with Tomcat and have the -necessary permissions for writing graphs. In this case, we will add apache -to the rserve group. - -\begin{verbatim} -$ usermod -a -G rserve apache -\end{verbatim} - -\section{Start Rserve} -Now we are ready to start Rserve! Run the script rserve/start.sh as ROOT -(or else Rserve will not fork itself properly). Then, check the rserve log -file (\emph{rserve.log}). The path to this log file can be changed by -modifying the script. Do a "ps -ef | grep rserve" to see if it has started. -Now, with Rserve installed and Postgres is running, Metrics is almost ready -to start generating some graphs! - -\end{document}

1 0

r24813: {website} clean up the open-positions we don't have. (website/trunk/getinvolved/en)
by Andrew Lewman 08 Jun '11

08 Jun '11

Author: phobos Date: 2011-06-08 11:39:53 +0000 (Wed, 08 Jun 2011) New Revision: 24813 Modified: website/trunk/getinvolved/en/open-positions.wml Log: clean up the open-positions we don't have. Modified: website/trunk/getinvolved/en/open-positions.wml =================================================================== --- website/trunk/getinvolved/en/open-positions.wml 2011-06-08 11:37:46 UTC (rev 24812) +++ website/trunk/getinvolved/en/open-positions.wml 2011-06-08 11:39:53 UTC (rev 24813) @@ -14,16 +14,10 @@ <h1>Tor: Open Positions</h1> <hr> -<p>Tor is hiring! We're looking for self-motivated individuals who are -able to work independently and want to help make Tor better. These are -short-term contract positions ranging from one to three months.</p> +<p>Tor is currently not hiring. We are generally looking for +self-motivated individuals who are able to work independently and want +to help make Tor better. </p> -<p>In particular, we're looking for a few people:</p> -<ul> -<li><a href="#vidaliadev">Vidalia Developer</a></li> -<li><a href="#windowsdev">Microsoft Windows Developer</a></li> -</ul> - <p>Working on Tor is rewarding because:</p> <ul> <li>You can work your own hours in your own locations. As long as you @@ -47,50 +41,6 @@ itself as well as supporting software like Vidalia, Torbutton, etc. All of our contractors came through volunteering first.</p> -<a id="vidaliadev"></a> -<h3><a class="anchor" href="#vidaliadev">Vidalia Developer</a></h3> -<p><a href="<page projects/vidalia>">Vidalia</a> is the cross-platform graphical user interface to Tor written -in C++ using the Qt libraries. It is used by millions of people -successfully. Our current developer is a volunteer and otherwise -occupied with finishing his PhD. We're looking for someone who knows -C++, Qt, and would like to kickstart development on the next generation -of Vidalia. Some suggested next steps involve:</p> -<ul> -<li>Implementing the KDE Marble interface into Vidalia for better and -more accurate mappings of relays to countries. Plus, it just looks -cool.</li> -<li>Implementing the ability to click a country and instruct Tor to -either start or end circuits in that country.</li> -<li>Work on bug fixes and enhancements as suggested by users.</li> -<li>Give Vidalia the ability to control the http proxy included, to -start, stop, and possibly configure either privoxy, polipo, or an http -shim we ship in the combined bundles of tor, vidalia, and polipo.</li> -<li>Possibly re-design the Vidalia interface to have different modes for -different levels of user sophistication. Many users see a button or a -setting and feel they have to change it. The defaults are set to ensure -users are sufficiently anonymous. How can we better hide the complexity -of Tor configurations?</li> -</ul> - -<a id="windowsdev"></a> -<h3><a class="anchor" href="#windowsdev">Microsoft Windows Developer</a></h3> -<p>This person needs to be very familiar with Windows at a system and -network level. We're looking for someone to help make Tor on Windows -more compatible and work more reliably as a relay. -There are a number of projects relating to Windows that current need a -skilled developer:</p> -<ul> -<li>Develop a Tor network shim to redirect all traffic through Tor. -OpenVPN has a network layer device for Windows that redirects traffic -through the OpenVPN connection. Is this the model Tor should be using? -Or is there something else that may work better?</li> -<li>Tor relays under non-server editions of Windows run into WSAENOBUFS -problems after a short while. This problem is described in detail in <a -href="https://trac.torproject.org/projects/tor/ticket/98">bug ticket -98</a>. So far, there are no solutions. We think libevent 2.0 with its -bufferevents code will help solve the problem.</li> -</ul> - <p>Periodically we get new funding to work on more development projects. Your goal should be to get into the list of <a href="<page about/corepeople>">core project members</a> so we think of you when new

1 0