- tor-commits - lists.torproject.org

[metrics-web/master] Rewrite tool descriptions of metrics-db and -web.
by karsten＠torproject.org 05 Sep '11

05 Sep '11

commit 32337d75133860ccc4ee9dae89c11b9bd09625b5 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Mon Sep 5 10:56:34 2011 +0200 Rewrite tool descriptions of metrics-db and -web. --- web/WEB-INF/tools.jsp | 21 +++++++++++---------- 1 files changed, 11 insertions(+), 10 deletions(-) diff --git a/web/WEB-INF/tools.jsp b/web/WEB-INF/tools.jsp index 56098f5..f4fd539 100644 --- a/web/WEB-INF/tools.jsp +++ b/web/WEB-INF/tools.jsp @@ -17,7 +17,7 @@ page and to process the resulting files to generate <a href="graphs.html">graphs</a>.</p> <ul> - <li><a href="#metrics-db">Metrics database</a></li> + <li><a href="#metrics-db">Metrics data processor</a></li> <li><a href="#metrics-web">Metrics website</a></li> <li><a href="#dirarch">Directory-archive script</a></li> <li><a href="#torperf">Torperf</a></li> @@ -26,13 +26,13 @@ </ul> <br> <a name="metrics-db"></a> - <h3>Metrics database</h3> + <h3>Metrics data processor</h3> <br> - <p>The metrics database software is a Java application that parses - Tor's directory data and imports it into a PostgreSQL database. - This database can then be used to look up relays or perform - statistical analysis on the Tor network. The metrics database is - also used by this website.</p> + <p>The metrics data processor is a Java application that parses + Tor's directory data and the data from various other Tor services, + possibly removes sensitive parts from them, and then outputs the + <a href="data.html">data</a> in <a href="formats.html">formats</a> + that are feasible for later analysis.</p> <ul> <li>Browse the <a href="https://gitweb.torproject.org/metrics-db.git/tree">Git @@ -43,9 +43,10 @@ <a name="metrics-web"></a> <h3>Metrics website</h3> <br> - <p>The metrics website software is a Tomcat application that - accesses the metrics database to generate this website. The - metrics website uses R to generate custom graphs on demand.</p> + <p>The metrics website software consists of a Java database + importer and Tomcat application that makes Tor's directory data + easily accessible. This website is run by the metrics website + software.</p> <ul> <li>Download <a href="dist/metrics-web-0.0.1.tar">metrics-web 0.0.1</a>

1 0

[flashproxy/master] Add "make install".
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit 98bb56a52c8729164221a7251c1a0d7e1414e1a0 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 01:59:53 2011 -0400 Add "make install". This installs the local Python programs and doesn't do anything with the Flash. --- Makefile | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/Makefile b/Makefile index 20cd81b..3861c09 100644 --- a/Makefile +++ b/Makefile @@ -1,3 +1,6 @@ +PREFIX = /usr/local +BINDIR = $(PREFIX)/bin + MXMLC ?= mxmlc TARGETS = swfcat.swf @@ -9,6 +12,10 @@ swfcat.swf: *.as badge.png %.swf: %.as $(MXMLC) -output $@ -static-link-runtime-shared-libraries $< +install: + mkdir -p $(BINDIR) + cp -f connector.py crossdomaind.py facilitator.py $(BINDIR) + clean: rm -f $(TARGETS)

1 0

[flashproxy/master] Add instructions for installing the init.d scripts to README.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit 82d011fb4b7a2e29f45d76220842551d1d3a5f6f Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 02:10:22 2011 -0400 Add instructions for installing the init.d scripts to README. --- README | 19 +++++++++++++++++++ 1 files changed, 19 insertions(+), 0 deletions(-) diff --git a/README b/README index c5ab4cf..18f9934 100644 --- a/README +++ b/README @@ -90,6 +90,25 @@ try deleting the files in ~/.tor or /var/lib/tor. You will be able to see byte counts flowing in the browser window, and eventually be able to build a circuit. +== Installing so as to restart at boot + +The directory init.d contains scripts to start the facilitator and +crossdomaind server on CentOS. "make install" copies files to /usr/local/bin. + + # make install + # useradd -d /dev/null -s /bin/false flashproxy + # mkdir /var/flashproxy + # chown -R flashproxy.flashproxy /var/flashproxy/ + + # cp init.d/facilitator /etc/init.d/facilitator + -- Edit /etc/init.d/facilitator to set $RELAY. + # chkconfig --add facilitator + # service facilitator start + + # cp init.d/crossdomaind /etc/init.d/crossdomaind + # chkconfig --add crossdomaind + # service crossdomaind start + == Building the Cumulus RTMFP server There is a free RTMFP rendezvous server called Cumulus. It is a

1 0

[flashproxy/master] Move the cumulus init.d script to the init.d directory.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit 0cfc785d09b8d64009fdecce69c583ecb7cdf410 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 02:02:12 2011 -0400 Move the cumulus init.d script to the init.d directory. --- README | 2 +- cumulus | 60 -------------------------------------------------------- init.d/cumulus | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 61 insertions(+), 61 deletions(-) diff --git a/README b/README index c0c8f6f..c5ab4cf 100644 --- a/README +++ b/README @@ -118,7 +118,7 @@ Follow these steps to install the Cumulus server. You need to open port # useradd -d /dev/null -s /bin/false cumulus # mkdir /var/cumulus # chown -R cumulus.cumulus /var/cumulus - # cp flashproxy/cumulus /etc/init.d/cumulus + # cp flashproxy/init.d/cumulus /etc/init.d/cumulus # chkconfig --add cumulus # service cumulus start diff --git a/cumulus b/cumulus deleted file mode 100755 index b1c428d..0000000 --- a/cumulus +++ /dev/null @@ -1,60 +0,0 @@ -#!/bin/sh -# -# cumulus This shell script takes care of starting and stopping -# the Cumulus server. -# -# chkconfig: 2345 90 10 -# description: Cumulus RTMFP server. -# processname: CumulusService -# pidfile: /var/cumulus/cumulus.pid - -# Installation instructions: -# cp cumulus /etc/init.d/cumulus -# chkconfig --add cumulus -# service cumulus start - -# Source function library. -. /etc/rc.d/init.d/functions - -BINDIR=/usr/local/bin -VARDIR=/var/cumulus -CUMULUS=$BINDIR/CumulusService -PIDFILE=$VARDIR/cumulus.pid -USER=cumulus - -# See how we were called. -case "$1" in - start) - [ -x $CUMULUS ] || exit 1 - echo -n $"Starting Cumulus server: " - cd $VARDIR && daemon --user cumulus --pidfile $PIDFILE $CUMULUS --daemon --pidfile=$PIDFILE - RETVAL=$? - echo - [ $RETVAL -eq 0 ] && touch /var/lock/subsys/cumulus - ;; - stop) - # Stop daemon. - echo -n $"Shutting down Cumulus server: " - killproc CumulusService - RETVAL=$? - echo - [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/cumulus - ;; - status) - status cumulus - RETVAL=$? - ;; - restart|reload) - $0 stop - $0 start - ;; - condrestart) - [ -f /var/lock/subsys/cumulus ] && restart || : - ;; - *) - echo $"Usage: $0 {start|stop|status|restart}" - RETVAL=3 - ;; -esac - -exit $RETVAL diff --git a/init.d/cumulus b/init.d/cumulus new file mode 100755 index 0000000..ba4f4d8 --- /dev/null +++ b/init.d/cumulus @@ -0,0 +1,60 @@ +#!/bin/sh +# +# cumulus This shell script takes care of starting and stopping +# the Cumulus server. +# +# chkconfig: 2345 90 10 +# description: Cumulus RTMFP server. +# processname: CumulusService +# pidfile: /var/cumulus/cumulus.pid + +# Installation instructions: +# cp init.d/cumulus /etc/init.d/cumulus +# chkconfig --add cumulus +# service cumulus start + +# Source function library. +. /etc/rc.d/init.d/functions + +BINDIR=/usr/local/bin +VARDIR=/var/cumulus +CUMULUS=$BINDIR/CumulusService +PIDFILE=$VARDIR/cumulus.pid +USER=cumulus + +# See how we were called. +case "$1" in + start) + [ -x $CUMULUS ] || exit 1 + echo -n $"Starting Cumulus server: " + cd $VARDIR && daemon --user cumulus --pidfile $PIDFILE $CUMULUS --daemon --pidfile=$PIDFILE + RETVAL=$? + echo + [ $RETVAL -eq 0 ] && touch /var/lock/subsys/cumulus + ;; + stop) + # Stop daemon. + echo -n $"Shutting down Cumulus server: " + killproc CumulusService + RETVAL=$? + echo + [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/cumulus + ;; + status) + status cumulus + RETVAL=$? + ;; + restart|reload) + $0 stop + $0 start + ;; + condrestart) + [ -f /var/lock/subsys/cumulus ] && restart || : + ;; + *) + echo $"Usage: $0 {start|stop|status|restart}" + RETVAL=3 + ;; +esac + +exit $RETVAL

1 0

[flashproxy/master] Add --pidfile to connector.py.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit 2c75470d677500037746b2596c280d833ae1da19 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 02:18:03 2011 -0400 Add --pidfile to connector.py. --- connector.py | 15 ++++++++++++--- 1 files changed, 12 insertions(+), 3 deletions(-) diff --git a/connector.py b/connector.py index 5434f84..3d3fabe 100755 --- a/connector.py +++ b/connector.py @@ -29,6 +29,7 @@ class options(object): log_filename = None log_file = sys.stdout daemonize = False + pid_filename = None # We accept up to this many bytes from a socket not yet matched with a partner # before disconnecting it. @@ -51,7 +52,8 @@ FACILITATOR. -f, --facilitator=HOST[:PORT] advertise willingness to receive connections to HOST:PORT. By default PORT is %(fac_port)d. -h, --help show this help. - -l, --log FILENAME write log to FILENAME (default stdout).\ + -l, --log FILENAME write log to FILENAME (default stdout). + --pidfile FILENAME write PID to FILENAME after daemonizing.\ """ % { "progname": sys.argv[0], "local": format_addr((DEFAULT_LOCAL_ADDRESS, DEFAULT_LOCAL_PORT)), @@ -113,7 +115,7 @@ def format_addr(addr): else: return u"%s:%d" % (host, port) -opts, args = getopt.gnu_getopt(sys.argv[1:], "f:hl:", ["daemon", "facilitator=", "help", "log="]) +opts, args = getopt.gnu_getopt(sys.argv[1:], "f:hl:", ["daemon", "facilitator=", "help", "log=", "pidfile="]) for o, a in opts: if o == "--daemon": options.daemonize = True @@ -124,6 +126,8 @@ for o, a in opts: sys.exit() elif o == "-l" or o == "--log": options.log_filename = a + elif o == "--pidfile": + options.pid_filename = a if len(args) == 0: options.local_addr = (DEFAULT_LOCAL_ADDRESS, DEFAULT_LOCAL_PORT) @@ -356,7 +360,12 @@ def match_proxies(): if options.daemonize: log(u"Daemonizing.") - if os.fork() != 0: + pid = os.fork() + if pid != 0: + if options.pid_filename: + f = open(options.pid_filename, "w") + print >> f, pid + f.close() sys.exit(0) register()

1 0

[flashproxy/master] Add an init.d script for crossdomaind.py.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit ce1642b5ce8d75eef949cdefebae396ade02a4e5 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 02:07:50 2011 -0400 Add an init.d script for crossdomaind.py. --- init.d/crossdomaind | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 62 insertions(+), 0 deletions(-) diff --git a/init.d/crossdomaind b/init.d/crossdomaind new file mode 100755 index 0000000..3aecaaa --- /dev/null +++ b/init.d/crossdomaind @@ -0,0 +1,62 @@ +#!/bin/sh +# +# crossdomaind This shell script takes care of starting and stopping +# crossdomaind server. +# +# chkconfig: 2345 90 10 +# description: Crossdomain policy server. +# processname: crossdomaind.py +# pidfile: /var/flashproxy/crossdomaind.pid + +# Installation instructions: +# cp crossdomaind /etc/init.d/crossdomaind +# chkconfig --add crossdomaind +# service crossdomaind start + +# Source function library. +. /etc/rc.d/init.d/functions + +BINDIR=/usr/local/bin +VARDIR=/var/flashproxy +PROG=$BINDIR/crossdomaind.py +PIDFILE=$VARDIR/crossdomaind.pid + +# See how we were called. +case "$1" in + start) + [ -x $PROG ] || exit 1 + echo -n $"Starting flash proxy crossdomaind: " + cd $VARDIR && daemon --pidfile $PIDFILE $PROG --pidfile $PIDFILE --daemon + RETVAL=$? + echo + [ $RETVAL -eq 0 ] && touch /var/lock/subsys/crossdomaind + ;; + stop) + # Stop daemon. + echo -n $"Shutting down flash proxy crossdomaind: " + killproc -p $PIDFILE + RETVAL=$? + echo + if [ $RETVAL -eq 0 ]; then + rm -f /var/lock/subsys/crossdomaind + rm -f $PIDFILE + fi + ;; + status) + status -p $PIDFILE crossdomaind + RETVAL=$? + ;; + restart|reload) + $0 stop + $0 start + ;; + condrestart) + [ -f /var/lock/subsys/crossdomaind ] && restart || : + ;; + *) + echo $"Usage: $0 {start|stop|status|restart}" + RETVAL=3 + ;; +esac + +exit $RETVAL

1 0

[flashproxy/master] Redirect stderr to the log file to retain tracebacks.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit 794020e5bde5c49b154609a5db3b52ace572dbf5 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 02:41:44 2011 -0400 Redirect stderr to the log file to retain tracebacks. --- connector.py | 2 ++ facilitator.py | 12 +++++++----- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/connector.py b/connector.py index 3d3fabe..d763303 100755 --- a/connector.py +++ b/connector.py @@ -144,6 +144,8 @@ else: if options.log_filename: options.log_file = open(options.log_filename, "a") + # Send error tracebacks to the log. + sys.stderr = options.log_file else: options.log_file = sys.stdout diff --git a/facilitator.py b/facilitator.py index 619b8b2..ef0cb48 100755 --- a/facilitator.py +++ b/facilitator.py @@ -347,11 +347,6 @@ for o, a in opts: print >> sys.stderr, u"Can't resolve relay %s: %s" % (repr(a), str(e)) sys.exit(1) -if options.log_filename: - options.log_file = open(options.log_filename, "a") -else: - options.log_file = sys.stdout - if not options.relay_spec: print >> sys.stderr, """\ The -r option is required. Give it the relay that will be sent to proxies. @@ -373,6 +368,13 @@ else: usage(sys.stderr) sys.exit(1) +if options.log_filename: + options.log_file = open(options.log_filename, "a") + # Send error tracebacks to the log. + sys.stderr = options.log_file +else: + options.log_file = sys.stdout + addrinfo = socket.getaddrinfo(address[0], address[1], 0, socket.SOCK_STREAM, socket.IPPROTO_TCP)[0] class Server(SocketServer.ThreadingMixIn, BaseHTTPServer.HTTPServer):

1 0

[flashproxy/master] Add .PHONY Makefile target.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit c15a866c47953cc41e75fdb158a5fe6bb5755d77 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 00:12:08 2011 -0400 Add .PHONY Makefile target. --- Makefile | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/Makefile b/Makefile index ed799c6..20cd81b 100644 --- a/Makefile +++ b/Makefile @@ -11,3 +11,5 @@ swfcat.swf: *.as badge.png clean: rm -f $(TARGETS) + +.PHONY: all clean

1 0

[flashproxy/master] Add a --pidfile option to facilitator.py.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit 8e8467406e6839a25daad5e287cf676b2bfc6462 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 01:52:57 2011 -0400 Add a --pidfile option to facilitator.py. --- facilitator.py | 21 +++++++++++++++------ 1 files changed, 15 insertions(+), 6 deletions(-) diff --git a/facilitator.py b/facilitator.py index 495cd3b..619b8b2 100755 --- a/facilitator.py +++ b/facilitator.py @@ -26,6 +26,7 @@ class options(object): log_file = sys.stdout relay_spec = None daemonize = True + pid_filename = None @staticmethod def set_relay_spec(spec): @@ -40,10 +41,11 @@ Usage: %(progname)s -r RELAY <OPTIONS> [HOST] [PORT] Flash proxy facilitator: Register client addresses with HTTP POST requests and serve them out again with HTTP GET. Listen on HOST and PORT, by default %(addr)s %(port)d. - -d, --debug don't daemonize, log to stdout. - -h, --help show this help. - -l, --log FILENAME write log to FILENAME (default \"%(log)s\"). - -r, --relay RELAY send RELAY (host:port) to proxies as the relay to use.\ + -d, --debug don't daemonize, log to stdout. + -h, --help show this help. + -l, --log FILENAME write log to FILENAME (default \"%(log)s\"). + --pidfile FILENAME write PID to FILENAME after daemonizing. + -r, --relay RELAY send RELAY (host:port) to proxies as the relay to use.\ """ % { "progname": sys.argv[0], "addr": DEFAULT_ADDRESS, @@ -326,7 +328,7 @@ class Handler(BaseHTTPServer.BaseHTTPRequestHandler): REGS = RegSet() opts, args = getopt.gnu_getopt(sys.argv[1:], "dhl:r:", - ["debug", "help", "log=", "relay="]) + ["debug", "help", "log=", "pidfile=", "relay="]) for o, a in opts: if o == "-d" or o == "--debug": options.daemonize = False @@ -336,6 +338,8 @@ for o, a in opts: sys.exit() elif o == "-l" or o == "--log": options.log_filename = a + elif o == "--pidfile": + options.pid_filename = a elif o == "-r" or o == "--relay": try: options.set_relay_spec(a) @@ -382,7 +386,12 @@ log(u"using relay address %s" % options.relay_spec) if options.daemonize: log(u"daemonizing") - if os.fork() != 0: + pid = os.fork() + if pid != 0: + if options.pid_filename: + f = open(options.pid_filename, "w") + print >> f, pid + f.close() sys.exit(0) try:

1 0

[flashproxy/master] Add init.d script for facilitator.py.
by dcf＠torproject.org 05 Sep '11

05 Sep '11

commit 5065368704e341cd6aa3603cd3bf774a8d396b33 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Sep 5 01:59:06 2011 -0400 Add init.d script for facilitator.py. --- init.d/facilitator | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 68 insertions(+), 0 deletions(-) diff --git a/init.d/facilitator b/init.d/facilitator new file mode 100755 index 0000000..03be16e --- /dev/null +++ b/init.d/facilitator @@ -0,0 +1,68 @@ +#!/bin/sh +# +# facilitator This shell script takes care of starting and stopping +# the flash proxy facilitator. +# +# chkconfig: 2345 90 10 +# description: Flash proxy facilitator. +# processname: facilitator.py +# pidfile: /var/flashproxy/facilitator.pid + +# Installation instructions: +# cp facilitator /etc/init.d/facilitator +# chkconfig --add facilitator +# service facilitator start + +# Source function library. +. /etc/rc.d/init.d/functions + +# Replace this with the address of a Tor relay that allows crossdomain +# connections (i.e., is running crossdomaind.py). You can use host:port +# syntax. +RELAY=... + +BINDIR=/usr/local/bin +VARDIR=/var/flashproxy +PROG=$BINDIR/facilitator.py +PIDFILE=$VARDIR/facilitator.pid +USER=flashproxy + +# See how we were called. +case "$1" in + start) + [ -x $PROG ] || exit 1 + echo -n $"Starting flash proxy facilitator: " + cd $VARDIR && daemon --user $USER --pidfile $PIDFILE $PROG --pidfile $PIDFILE -r $RELAY + RETVAL=$? + echo + [ $RETVAL -eq 0 ] && touch /var/lock/subsys/facilitator + ;; + stop) + # Stop daemon. + echo -n $"Shutting down flash proxy facilitator: " + killproc -p $PIDFILE + RETVAL=$? + echo + if [ $RETVAL -eq 0 ]; then + rm -f /var/lock/subsys/facilitator + rm -f $PIDFILE + fi + ;; + status) + status -p $PIDFILE facilitator + RETVAL=$? + ;; + restart|reload) + $0 stop + $0 start + ;; + condrestart) + [ -f /var/lock/subsys/facilitator ] && restart || : + ;; + *) + echo $"Usage: $0 {start|stop|status|restart}" + RETVAL=3 + ;; +esac + +exit $RETVAL

1 0