March 2012 - tor-commits - lists.torproject.org

[ooni-probe/master] Merge branch 'workers'
by art＠torproject.org 21 Mar '12

21 Mar '12

commit c8cec8280f2c2eab9168a058c453e504de174545 Merge: 5d6b1e2 d5fdae9 Author: Arturo Filastò <hellais(a)gmail.com> Date: Tue Mar 20 20:33:51 2012 -0700 Merge branch 'workers' ooni-probe.conf | 3 +- plugoo/tests.py | 25 ++++++++-- plugoo/workers.py | 36 ++++++++++++++ tests/httphost.py | 132 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/tcpscan.py | 2 +- utils.py | 2 +- 6 files changed, 192 insertions(+), 8 deletions(-)

1 0

[torspec/master] Remove some MTA/MUA garbage from Proposal 197
by ioerror＠torproject.org 21 Mar '12

21 Mar '12

commit bce34f25e8d52d53b248e4c364b690ade4d50e21 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Tue Mar 20 17:44:43 2012 -0700 Remove some MTA/MUA garbage from Proposal 197 --- proposals/197-postmessage-ipc.txt | 47 +++++------------------------------- 1 files changed, 7 insertions(+), 40 deletions(-) diff --git a/proposals/197-postmessage-ipc.txt b/proposals/197-postmessage-ipc.txt index 45348c8..36bac67 100644 --- a/proposals/197-postmessage-ipc.txt +++ b/proposals/197-postmessage-ipc.txt @@ -26,7 +26,7 @@ Control Protocol Specification Changes Sent from the client to the server. The syntax is: "CONTROLLERNAME" SP ControllerID - ControllerID =3D 1*(ALNUM / "_") + ControllerID = 1*(ALNUM / "_") Server returns "250 OK" and records the ControllerID to use for this control port connection for messaging information if successful, @@ -43,7 +43,7 @@ Control Protocol Specification Changes Sent from the client to the server. The syntax is: "POSTMESSAGE" SP "@" DestControllerID SP LineItem CRLF - DestControllerID =3D "all" / 1*(ALNUM / "_") + DestControllerID = "all" / 1*(ALNUM / "_") If DestControllerID is "all", the message will be posted to all controllers that have "SETEVENTS POSTMESSAGE" set. Otherwise, the @@ -60,9 +60,9 @@ Control Protocol Specification Changes "650" SP "POSTMESSAGE" SP MessageID SP SourceControllerID SP "@" DestControllerID SP LineItem CRLF - MessageID =3D 1*DIGIT - SourceControllerID =3D "unset" / 1*(ALNUM / "_") - DestControllerID =3D "all" / 1*(ALNUM / "_") + MessageID = 1*DIGIT + SourceControllerID = "unset" / 1*(ALNUM / "_") + DestControllerID = "all" / 1*(ALNUM / "_") MessageID is an incrementing integer identifier that uniquely identifies this message to all controllers. @@ -72,8 +72,8 @@ Control Protocol Specification Changes CONTROLLERNAME command was not used or unimplemented. GETINFO commands - "recent-messages" -- Retrieves messages=20 - sent to ControllerIDs that match the current controller=20 + "recent-messages" -- Retrieves messages + sent to ControllerIDs that match the current controller in POSTMESSAGE event format. This list should be generated on the fly, to handle disconnecting controllers. @@ -97,36 +97,3 @@ Implementation plan needed. However, Torbutton does not have immediate need for this protocol. - =20 - - - ---=20 -Mike Perry - ---GID0FwUMdk1T2AWN -Content-Type: application/pgp-signature; name="signature.asc" -Content-Description: Digital signature - ------BEGIN PGP SIGNATURE----- -Version: GnuPG v1.4.11 (GNU/Linux) - -iEYEARECAAYFAk9j144ACgkQGwyjDN3GwK07dgCdEqImEOU/QeGwdAs9Nxe8vlYO -H+UAn0/NlNoEwFfRDbnH3fv+3C/MOOF8 -=gs9Z ------END PGP SIGNATURE----- - ---GID0FwUMdk1T2AWN-- - ---===============7807826766355441436== -Content-Type: text/plain; charset="us-ascii" -MIME-Version: 1.0 -Content-Transfer-Encoding: 7bit -Content-Disposition: inline - -_______________________________________________ -tor-dev mailing list -tor-dev(a)lists.torproject.org -https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev - ---===============7807826766355441436==--

1 0

[torspec/master] remove proposals/xxx-bridgefinder-integration.txt
by ioerror＠torproject.org 21 Mar '12

21 Mar '12

commit f8d4e304774c19e4cff8e918d4b4b3c5a2c34e07 Author: Jacob Appelbaum <jacob(a)appelbaum.net> Date: Tue Mar 20 20:12:08 2012 -0700 remove proposals/xxx-bridgefinder-integration.txt --- proposals/xxx-bridgefinder-integration.txt | 415 ---------------------------- 1 files changed, 0 insertions(+), 415 deletions(-) diff --git a/proposals/xxx-bridgefinder-integration.txt b/proposals/xxx-bridgefinder-integration.txt deleted file mode 100644 index c6c55ec..0000000 --- a/proposals/xxx-bridgefinder-integration.txt +++ /dev/null @@ -1,415 +0,0 @@ -Filename: xxx-bridgefinder-integration.txt -Title: Integration of BridgeFinder and BridgeFinderHelper -Author: Mike Perry -Created: 18-03-2012 -Status: Proposed -Target: 0.2.3.x+ - - -Overview - - This proposal describes how the Tor client software can interact with - an external program that performs bridge discovery based on user input - or information extracted from a web page, QR Code, online game, or - other transmission medium. - - -Scope and Audience - - This document describes how all of the components involved in bridge - discovery communicate this information to the rest of the Tor - software. The mechanisms of bridge discovery are not discussed, though - the design aims to be generalized enough to allow arbitrary new - discovery mechanisms to be added at any time. - - This document is also written with the hope that those who wish to - implement BridgeFinder components and BridgeFinderHelpers can get - started immediately after a read of this proposal, so that development - of bridge discovery mechanisms can proceed in parallel to supporting - functionality improvements in the Tor client software. - - -Components and Responsibilities - - 0. Tor Client - - The Tor Client is the piece of software that connects to the Tor - network (optionally using bridges) and provides a SOCKS proxy for - use by the user. - - In initial implementations, the Tor Client will support only - standard bridges. In later implementations, it is expected to - support pluggable transports as defined by Proposal 180. - - 1. Tor Control Port - - The Tor Control Port provides commands to perform operations, - configuration, and to obtain status information. It also optionally - provides event driven status updates. - - In initial implementations, it will be used directly by BridgeFinder - to configure bridge information via GETINFO and SETCONF. It is covered - by control-spec.txt in the tor-specs git repository. - - In later implementations, it will support the inter-controller - POSTMESSAGE IPC protocol as defined by Proposal 197 for use - in conveying bridge information to the Primary Controller. - - 2. Primary Controller - - The Primary Controller is the program that launches and configures the - Tor client, and monitors its status. - - On desktop platforms, this program is Vidalia, and it also launches - the Tor Browser. On Android, this program is Orbot. Orbot does not - launch a browser. - - On all platforms, this proposal requires that the Primary Controller - will launch one or more BridgeFinder child processes and provide - them with authentication information through the environment variables - TOR_CONTROL_PORT and TOR_CONTROL_PASSWD. - - In later implementations, the Primary Controller will be expected - to receive Bridge configuration information via the free-form - POSTMESSAGE protocol from Proposal 197, validate that information, - and hold that information for user approval. - - 3. BridgeFinder - - A BridgeFinder is a program that discovers bridges and configures - Tor to use them. - - In initial implementations, it is likely to be very dumb, and its main - purpose will be to serve as a layer of abstraction that should free - the Primary Controller from having to directly implement numerous ways - of retrieving bridges for various pluggable transports. - - In later implementations, it may perform arbitrary network operations - to discover, authenticate to, and/or verify bridges, possibly using - informational hints provided by one or more external - BridgeFinderHelpers (see next component). It could even go so far as - to download new pluggable transport plugins and/or transform - definition files from arbitrary urls. - - It will be launched by the Primary Controller and given access to the - Tor Control Port via the environment variables TOR_CONTROL_PORT and - TOR_CONTROL_PASSWD. - - Initial control port interactions can be command driven via GETINFO - and SETCONF, and do not need to subscribe to or process control port - events. Later implementations will use POSTMESSAGE as defined in - Proposal 197 to pass command requests to Vidalia, which will parse - them and ask for user confirmation before deploying them. Use of - POSTMESSAGE may or may not require event driven operation, depending - on POSTMESSAGE implementation status (POSTMESSAGE is designed to - support both command and event driven operation, but it is possible - event driven operation will happen first). - - 4. BridgeFinderHelper - - Each BridgeFinder implementation can optionally communicate with one - or more BridgeFinderHelpers. BridgeFinderHelpers are plugins to - external 3rd party applications that can inspect traffic, handle mime - types, or implement protocol handlers for accepting bridge discovery - information to pass to BridgeFinder. Example 3rd party applications - include Chrome, World of Warcraft, QR Code readers, or simple cut - and paste. - - Due to the arbitrary nature of sandboxing that may be present in - various BridgeFinderHelper host applications, we do not mandate the - exact nature of the IPC between BridgeFinder instances and external - BridgeFinderHelper addons. However, please see the "Security Concerns" - section for common pitfalls to avoid. - - 5. Tor Browser - - This is the browser the user uses with Tor. It is not useful until Tor - is properly configured to use bridges. It fails closed. - - It is not expected to run BridgeFinderHelper plugin instances, unless - those plugin instances exist to ensure the user always has a pool of - working bridges available after successfully configuring an - initial bridge. Once all bridges fail, the Tor Browser is useless. - - 6. Non-Tor Browser (aka BridgeFinderHelper host) - - This is the program the user uses for normal Internet activity to - obtain bridges via a BridgeFinderHelper plugin. It does not have to be - a browser. In advanced scenarios, this component may not be a browser - at all, but may be a program such as World of Warcraft instead. - - -Incremental Deployability - - The system is designed to be incrementally deployable: Simple designs - should be possible to develop and test immediately. The design is - flexible enough to be easily upgraded as more advanced features become - available from both Tor and new pluggable transports. - -Initial Implementation - - In the simplest possible initial implementation, BridgeFinder will - only discover Tor Bridges as they are deployed today. It will use the - Tor Control Port to configure these bridges directly via the SETCONF - command. It may or may not receive bridge information from a - BridgeFinderHelper. In an even more degenerate case, - BridgeFinderHelper may even be Vidalia or Orbot itself, acting upon - user input from cut and paste. - - Initial Implementation: BridgeFinder Launch - - In the initial implementation, the Primary Controller will launch one - or more BridgeFinders, providing control port authentication - information to them through the environment variables TOR_CONTROL_PORT - and TOR_CONTROL_PASSWD. - - BridgeFinder will then directly connect to the control port and - authenticate. Initial implementations should be able to function - without using SETEVENTS, and instead only using command-based - status inquiries and configuration (GETINFO and SETCONF). - - Initial Implementation: Obtaining Bridge Hint Information - - In the initial implementation, to test functionality, - BridgeFinderHelper can simply scrape bridges directly from - https://bridges.torproject.org. - - In slightly more advanced implementations, a BridgeFinderHelper - instance may be written for use in the user's Non-Tor Browser. This - plugin could extract bridges from images, html comments, and other - material present in ad banners and slack space on unrelated pages. - - BridgeFinderHelper would then communicate with the appropriate - BridgeFinder instance over an acceptable IPC mechanism. This proposal - does not seek to specify the nature of that IPC channel (because - BridgeFinderHelper may be arbitrarily constrained due to host - application sandboxing), but we do make several security - recommendations under the section "Security Concerns: BridgeFinder and - BridgeFinderHelper". - - Initial Implementation: Configuring New Bridges - - In the initial implementation, Bridge configuration will be done - directly though the control port using the SETCONF command. - - Initial implementations will support only retrieval and configuration - of standard Tor Bridges. These are configured using SETCONF on the Tor - Control Port as follows: - SETCONF Bridge="IP:ORPort [fingerprint]" - - -Future Implementations - - In future implementations, the system can incrementally evolve in a - few different directions. As new pluggable transports are created, it - is conceivable that BridgeFinder may want to download new plugin - binaries (and/or new transport transform definition files) and - provide them to Tor. - - Furthermore, it may prove simpler to deploy multiple concurrent - BridgeFinder+BridgeFinderHelper pairs as opposed to adding new - functionality to existing prototypes. - - Finally, it is desirable for BridgeFinder to obtain approval - from the user before updating bridge configuration, especially for - cases where BridgeFinderHelper is automatically discovering bridges - in-band during Non-Tor activity. - - The exact mechanisms for accomplishing these improvements is - described in the following subsections. - - Future Implementations: BridgeFinder Launch and POSTMESSAGE handshake - - The nature of the BridgeFinder launch and the environment variables - provided is not expected to change. However, future Primary Controller - implementations may decide to launch more than one BridgeFinder - instance side by side. - - Additionally, to negotiate the IPC channel created by Proposal 197 - for purposes of providing user confirmation, it is recommended that - BridgeFinder and the Primary Controller perform a handshake using - POSTMESSAGE upon launch, to establish that all parties properly - support the feature: - - Primary Controller: "POSTMESSAGE @all Controller wants POSTMESSAGE v1.1" - BridgeFinder: "POSTMESSAGE @all BridgeFinder has POSTMESSAGE v1.0" - Primary Controller: "POSTMESSAGE @all Controller expects POSTMESSAGE v1.0" - BridgeFinder: "POSTMESSAGE @all BridgeFinder will POSTMESSAGE v1.0" - - If this 4 step handshake proceeds with an acceptable version, - BridgeFinder must use POSTMESSAGE to transmit SETCONF Bridge lines - (see "Future Implementations: Configuring New Bridges" below). If - POSTMESSAGE support is expected, but the handshake does not complete - for any reason, BridgeFinder should either exit or go dormant. - - The exact nature of the version negotiation and exactly how much - backwards compatibility must be tolerated is unspecified. - "All-or-nothing" is a safe assumption to get started. - - Future Implementations: Obtaining Bridge Hint Information - - Future BridgeFinder implementations may download additional - information based on what is provided by BridgeFinderHelper. They - may fetch pluggable transport plugins, transformation parameters, - and other material. - - Future Implementations: Configuring New Bridges - - Future implementations will be concerned with providing two new pieces - of functionality with respect to configuring bridges: configuring - pluggable transports, and properly prompting the user before altering - Tor configuration. - - There are two ways to tell Tor clients about pluggable transports - (as defined in Proposal 180). - - On the control port, an external Proposal 180 transport will be - configured with - SETCONF ClientTransportPlugin=<method> socks5 <addr:port> [auth=X] - as in - SETCONF ClientTransportPlugin="trebuchet socks5 127.0.0.1:9999". - - A managed proxy is configured with - SETCONF ClientTransportPlugin=<methods> exec <path> [options] - as in - SETCONF ClientTransportPlugin="trebuchet exec /usr/libexec/trebuchet --managed". - - This example tells Tor to launch an external program to provide a - socks proxy for 'trebuchet' connections. The Tor client only - launches one instance of each external program with a given set of - options, even if the same executable and options are listed for - more than one method. - - Pluggable transport bridges discovered for this transport by - BridgeFinder would then be set with: - SETCONF Bridge="trebuchet 3.2.4.1:8080 keyid=09F911029D74E35BD84156C5635688C009F909F9 rocks=20 height=5.6m". - - For more information on pluggable transports and supporting Tor - configuration commands, see Proposal 180. - - Future Implementations: POSTMESSAGE and User Confirmation - - Because configuring even normal bridges alone can expose the user to - attacks, it is strongly desired to provide some mechanism to allow - the user to approve new bridges prior to their use, especially for - situations where BridgeFinderHelper is extracting them transparently - while the user performs unrelated activity. - - If BridgeFinderHelper grows to the point where it is downloading new - transform definitions or plugins, user confirmation becomes - absolutely required. - - To achieve user confirmation, we depend upon the POSTMESSAGE command - defined in Proposal 197. - - If the POSTMESSAGE handshake succeeds, instead of sending SETCONF - commands directly to the control port, the commands will be wrapped - inside a POSTMESSAGE: - POSTMESSAGE @all SETCONF Bridge="www.example.com:8284" - - Upon receiving this POSTMESSAGE, the Primary Controller will - validate it, evaluate it, store it to be later enabled by the - user, and alert the user that new bridges are available for - approval. It is only after the user has approved the new bridges - that the Primary Controller should then re-issue the SETCONF commands - to configure and deploy them in the tor client. - - Additionally, see "Security Concerns: Primary Controller" for more - discussion on potential pitfalls with POSTMESSAGE. - -Security Concerns - - While automatic bridge discovery and configuration is quite compelling - and powerful, there are several serious security concerns that warrant - extreme care. We've broken them down by component. - - Security Concerns: Primary Controller - - In the initial implementation, Orbot and Vidalia must take care to - transmit the Tor Control password to BridgeFinder in such a way that - it does not end up in system logs, process list, or viewable by other - system users. The best known strategy for doing this is by passing the - information through exported environment variables. - - Additionally, in future implementations, Orbot and Vidalia will need - to validate Proposal 197 POSTMESSAGE input before prompting the user. - POSTMESSAGE is a free-form message-passing mechanism. All sorts of - unexpected input may be passed through it by any other authenticated - Tor Controllers for their own unrelated communication purposes. - - Minimal validation includes verifying that the POSTMESSAGE data is a - valid Bridge or ClientTransportPlugin line and is acceptable input for - SETCONF. All unexpected characters should be removed through using a - whitelist, and format and structure should be checked against a - regular expression. Additionally, the POSTMESSAGE string should not be - passed through any string processing engines that automatically decode - character escape encodings, to avoid arbitrary control port execution. - - At the same time, POSTMESSAGE validation should be light. While fully - untrusted input is not expected due to the need for control port - authentication and BridgeFinder sanitation, complicated manual string - parsing techniques during validation should be avoided. Perform simple - easy-to-verify whitelist-based checks, and ignore unrecognized input. - - Beyond POSTMESSAGE validation, the manner in which the Primary - Controller achieves consent from the user is absolutely crucial to - security under this scheme. A simple "OK/Cancel" dialog is - insufficient to protect the user from the dangers of switching - bridges and running new plugins automatically. - - Newly discovered bridge lines from POSTMESSAGE should be added to a - disabled set that the user must navigate to as an independent window - apart from any confirmation dialog. The user must then explicitly - enable recently added plugins by checking them off individually. We - need the user's brain to be fully engaged and aware that it is - interacting with Tor during this step. If they get an "OK/Cancel" - popup that interrupts their online game play, they will almost - certainly simply click "OK" just to get back to the game quickly. - - The Primary Controller should transmit the POSTMESSAGE content to the - control port only after obtaining this out-of-band approval. - -Security Concerns: BridgeFinder and BridgeFinderHelper - - The unspecified nature of the IPC channel between BridgeFinder and - BridgeFinderHelper makes it difficult to make concrete security - suggestions. However, from past experience, the following best - practices must be employed to avoid security vulnerabilities: - - 1. Define a non-webby handshake and/or perform authentication - - The biggest risk is that unexpected applications will be manipulated - into posting malformed data to the BridgeFinder's IPC channel as if it - were from BridgeFinderHelper. The best way to defend against this is - to require a handshake to properly complete before accepting input. If - the handshake fails at any point, the IPC channel must be abandoned - and closed. Do not continue scanning for good input after any bad - input has been encountered. - - Additionally, if possible, it is wise to establish a shared secret - between BridgeFinder and BridgeFinderHelper through the filesystem or - any other means available for use in authentication. For an a good - example on how to use such a shared secret properly for - authentication, see Trac Ticket #5185 and/or the SafeCookie Tor - Control Port authentication mechanism. - - 2. Perform validation before parsing - - Care must be taken before converting BridgeFinderHelper data into - Bridge lines, especially for cases where the BridgeFinderHelper data - is fed directly to the control port after passing through - BridgeFinder. - - The input should be subjected to a character whitelist and possibly - also validated against a regular expression to verify format, and if - any unexpected or poorly-formed data is encountered, the IPC channel - must be closed. - - 3. Fail closed on unexpected input - - If the handshake fails, or if any other part of the BridgeFinderHelper - input is invalid, the IPC channel must be abandoned and closed. Do - *not* continue scanning for good input after any bad input has been - encountered. - -

1 0

[torspec/master] proposal 199 by mikeperry
by ioerror＠torproject.org 21 Mar '12

21 Mar '12

commit cee63cc90fd6623e1f5b4dad5f608bfd2049cc90 Author: Jacob Appelbaum <jacob(a)appelbaum.net> Date: Tue Mar 20 18:42:27 2012 -0700 proposal 199 by mikeperry --- proposals/000-index.txt | 2 + proposals/199-bridgefinder-integration.txt | 415 ++++++++++++++++++++++++++++ 2 files changed, 417 insertions(+), 0 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index d553084..86a6d5b 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -119,6 +119,7 @@ Proposals by number: 196 Extended ORPort and TransportControlPort [OPEN] 197 Message-based Inter-Controller IPC Channel [OPEN] 198 Restore semantics of TLS ClientHello [OPEN] +199 Integration of BridgeFinder and BridgeFinderHelper [OPEN] Proposals by status: @@ -158,6 +159,7 @@ Proposals by status: 196 Extended ORPort and TransportControlPort [for 0.2.4.x] 197 Message-based Inter-Controller IPC Channel [for 0.2.4.x] 198 Restore semantics of TLS ClientHello + 199 Integration of BridgeFinder and BridgeFinderHelper [for 0.2.3.x+] ACCEPTED: 117 IPv6 exits [for 0.2.3.x] 140 Provide diffs between consensuses diff --git a/proposals/199-bridgefinder-integration.txt b/proposals/199-bridgefinder-integration.txt new file mode 100644 index 0000000..d3796ad --- /dev/null +++ b/proposals/199-bridgefinder-integration.txt @@ -0,0 +1,415 @@ +Filename: 199-bridgefinder-integration.txt +Title: Integration of BridgeFinder and BridgeFinderHelper +Author: Mike Perry +Created: 18-03-2012 +Status: Open +Target: 0.2.3.x+ + + +Overview + + This proposal describes how the Tor client software can interact with + an external program that performs bridge discovery based on user input + or information extracted from a web page, QR Code, online game, or + other transmission medium. + + +Scope and Audience + + This document describes how all of the components involved in bridge + discovery communicate this information to the rest of the Tor + software. The mechanisms of bridge discovery are not discussed, though + the design aims to be generalized enough to allow arbitrary new + discovery mechanisms to be added at any time. + + This document is also written with the hope that those who wish to + implement BridgeFinder components and BridgeFinderHelpers can get + started immediately after a read of this proposal, so that development + of bridge discovery mechanisms can proceed in parallel to supporting + functionality improvements in the Tor client software. + + +Components and Responsibilities + + 0. Tor Client + + The Tor Client is the piece of software that connects to the Tor + network (optionally using bridges) and provides a SOCKS proxy for + use by the user. + + In initial implementations, the Tor Client will support only + standard bridges. In later implementations, it is expected to + support pluggable transports as defined by Proposal 180. + + 1. Tor Control Port + + The Tor Control Port provides commands to perform operations, + configuration, and to obtain status information. It also optionally + provides event driven status updates. + + In initial implementations, it will be used directly by BridgeFinder + to configure bridge information via GETINFO and SETCONF. It is covered + by control-spec.txt in the tor-specs git repository. + + In later implementations, it will support the inter-controller + POSTMESSAGE IPC protocol as defined by Proposal 197 for use + in conveying bridge information to the Primary Controller. + + 2. Primary Controller + + The Primary Controller is the program that launches and configures the + Tor client, and monitors its status. + + On desktop platforms, this program is Vidalia, and it also launches + the Tor Browser. On Android, this program is Orbot. Orbot does not + launch a browser. + + On all platforms, this proposal requires that the Primary Controller + will launch one or more BridgeFinder child processes and provide + them with authentication information through the environment variables + TOR_CONTROL_PORT and TOR_CONTROL_PASSWD. + + In later implementations, the Primary Controller will be expected + to receive Bridge configuration information via the free-form + POSTMESSAGE protocol from Proposal 197, validate that information, + and hold that information for user approval. + + 3. BridgeFinder + + A BridgeFinder is a program that discovers bridges and configures + Tor to use them. + + In initial implementations, it is likely to be very dumb, and its main + purpose will be to serve as a layer of abstraction that should free + the Primary Controller from having to directly implement numerous ways + of retrieving bridges for various pluggable transports. + + In later implementations, it may perform arbitrary network operations + to discover, authenticate to, and/or verify bridges, possibly using + informational hints provided by one or more external + BridgeFinderHelpers (see next component). It could even go so far as + to download new pluggable transport plugins and/or transform + definition files from arbitrary urls. + + It will be launched by the Primary Controller and given access to the + Tor Control Port via the environment variables TOR_CONTROL_PORT and + TOR_CONTROL_PASSWD. + + Initial control port interactions can be command driven via GETINFO + and SETCONF, and do not need to subscribe to or process control port + events. Later implementations will use POSTMESSAGE as defined in + Proposal 197 to pass command requests to Vidalia, which will parse + them and ask for user confirmation before deploying them. Use of + POSTMESSAGE may or may not require event driven operation, depending + on POSTMESSAGE implementation status (POSTMESSAGE is designed to + support both command and event driven operation, but it is possible + event driven operation will happen first). + + 4. BridgeFinderHelper + + Each BridgeFinder implementation can optionally communicate with one + or more BridgeFinderHelpers. BridgeFinderHelpers are plugins to + external 3rd party applications that can inspect traffic, handle mime + types, or implement protocol handlers for accepting bridge discovery + information to pass to BridgeFinder. Example 3rd party applications + include Chrome, World of Warcraft, QR Code readers, or simple cut + and paste. + + Due to the arbitrary nature of sandboxing that may be present in + various BridgeFinderHelper host applications, we do not mandate the + exact nature of the IPC between BridgeFinder instances and external + BridgeFinderHelper addons. However, please see the "Security Concerns" + section for common pitfalls to avoid. + + 5. Tor Browser + + This is the browser the user uses with Tor. It is not useful until Tor + is properly configured to use bridges. It fails closed. + + It is not expected to run BridgeFinderHelper plugin instances, unless + those plugin instances exist to ensure the user always has a pool of + working bridges available after successfully configuring an + initial bridge. Once all bridges fail, the Tor Browser is useless. + + 6. Non-Tor Browser (aka BridgeFinderHelper host) + + This is the program the user uses for normal Internet activity to + obtain bridges via a BridgeFinderHelper plugin. It does not have to be + a browser. In advanced scenarios, this component may not be a browser + at all, but may be a program such as World of Warcraft instead. + + +Incremental Deployability + + The system is designed to be incrementally deployable: Simple designs + should be possible to develop and test immediately. The design is + flexible enough to be easily upgraded as more advanced features become + available from both Tor and new pluggable transports. + +Initial Implementation + + In the simplest possible initial implementation, BridgeFinder will + only discover Tor Bridges as they are deployed today. It will use the + Tor Control Port to configure these bridges directly via the SETCONF + command. It may or may not receive bridge information from a + BridgeFinderHelper. In an even more degenerate case, + BridgeFinderHelper may even be Vidalia or Orbot itself, acting upon + user input from cut and paste. + + Initial Implementation: BridgeFinder Launch + + In the initial implementation, the Primary Controller will launch one + or more BridgeFinders, providing control port authentication + information to them through the environment variables TOR_CONTROL_PORT + and TOR_CONTROL_PASSWD. + + BridgeFinder will then directly connect to the control port and + authenticate. Initial implementations should be able to function + without using SETEVENTS, and instead only using command-based + status inquiries and configuration (GETINFO and SETCONF). + + Initial Implementation: Obtaining Bridge Hint Information + + In the initial implementation, to test functionality, + BridgeFinderHelper can simply scrape bridges directly from + https://bridges.torproject.org. + + In slightly more advanced implementations, a BridgeFinderHelper + instance may be written for use in the user's Non-Tor Browser. This + plugin could extract bridges from images, html comments, and other + material present in ad banners and slack space on unrelated pages. + + BridgeFinderHelper would then communicate with the appropriate + BridgeFinder instance over an acceptable IPC mechanism. This proposal + does not seek to specify the nature of that IPC channel (because + BridgeFinderHelper may be arbitrarily constrained due to host + application sandboxing), but we do make several security + recommendations under the section "Security Concerns: BridgeFinder and + BridgeFinderHelper". + + Initial Implementation: Configuring New Bridges + + In the initial implementation, Bridge configuration will be done + directly though the control port using the SETCONF command. + + Initial implementations will support only retrieval and configuration + of standard Tor Bridges. These are configured using SETCONF on the Tor + Control Port as follows: + SETCONF Bridge="IP:ORPort [fingerprint]" + + +Future Implementations + + In future implementations, the system can incrementally evolve in a + few different directions. As new pluggable transports are created, it + is conceivable that BridgeFinder may want to download new plugin + binaries (and/or new transport transform definition files) and + provide them to Tor. + + Furthermore, it may prove simpler to deploy multiple concurrent + BridgeFinder+BridgeFinderHelper pairs as opposed to adding new + functionality to existing prototypes. + + Finally, it is desirable for BridgeFinder to obtain approval + from the user before updating bridge configuration, especially for + cases where BridgeFinderHelper is automatically discovering bridges + in-band during Non-Tor activity. + + The exact mechanisms for accomplishing these improvements is + described in the following subsections. + + Future Implementations: BridgeFinder Launch and POSTMESSAGE handshake + + The nature of the BridgeFinder launch and the environment variables + provided is not expected to change. However, future Primary Controller + implementations may decide to launch more than one BridgeFinder + instance side by side. + + Additionally, to negotiate the IPC channel created by Proposal 197 + for purposes of providing user confirmation, it is recommended that + BridgeFinder and the Primary Controller perform a handshake using + POSTMESSAGE upon launch, to establish that all parties properly + support the feature: + + Primary Controller: "POSTMESSAGE @all Controller wants POSTMESSAGE v1.1" + BridgeFinder: "POSTMESSAGE @all BridgeFinder has POSTMESSAGE v1.0" + Primary Controller: "POSTMESSAGE @all Controller expects POSTMESSAGE v1.0" + BridgeFinder: "POSTMESSAGE @all BridgeFinder will POSTMESSAGE v1.0" + + If this 4 step handshake proceeds with an acceptable version, + BridgeFinder must use POSTMESSAGE to transmit SETCONF Bridge lines + (see "Future Implementations: Configuring New Bridges" below). If + POSTMESSAGE support is expected, but the handshake does not complete + for any reason, BridgeFinder should either exit or go dormant. + + The exact nature of the version negotiation and exactly how much + backwards compatibility must be tolerated is unspecified. + "All-or-nothing" is a safe assumption to get started. + + Future Implementations: Obtaining Bridge Hint Information + + Future BridgeFinder implementations may download additional + information based on what is provided by BridgeFinderHelper. They + may fetch pluggable transport plugins, transformation parameters, + and other material. + + Future Implementations: Configuring New Bridges + + Future implementations will be concerned with providing two new pieces + of functionality with respect to configuring bridges: configuring + pluggable transports, and properly prompting the user before altering + Tor configuration. + + There are two ways to tell Tor clients about pluggable transports + (as defined in Proposal 180). + + On the control port, an external Proposal 180 transport will be + configured with + SETCONF ClientTransportPlugin=<method> socks5 <addr:port> [auth=X] + as in + SETCONF ClientTransportPlugin="trebuchet socks5 127.0.0.1:9999". + + A managed proxy is configured with + SETCONF ClientTransportPlugin=<methods> exec <path> [options] + as in + SETCONF ClientTransportPlugin="trebuchet exec /usr/libexec/trebuchet --managed". + + This example tells Tor to launch an external program to provide a + socks proxy for 'trebuchet' connections. The Tor client only + launches one instance of each external program with a given set of + options, even if the same executable and options are listed for + more than one method. + + Pluggable transport bridges discovered for this transport by + BridgeFinder would then be set with: + SETCONF Bridge="trebuchet 3.2.4.1:8080 keyid=09F911029D74E35BD84156C5635688C009F909F9 rocks=20 height=5.6m". + + For more information on pluggable transports and supporting Tor + configuration commands, see Proposal 180. + + Future Implementations: POSTMESSAGE and User Confirmation + + Because configuring even normal bridges alone can expose the user to + attacks, it is strongly desired to provide some mechanism to allow + the user to approve new bridges prior to their use, especially for + situations where BridgeFinderHelper is extracting them transparently + while the user performs unrelated activity. + + If BridgeFinderHelper grows to the point where it is downloading new + transform definitions or plugins, user confirmation becomes + absolutely required. + + To achieve user confirmation, we depend upon the POSTMESSAGE command + defined in Proposal 197. + + If the POSTMESSAGE handshake succeeds, instead of sending SETCONF + commands directly to the control port, the commands will be wrapped + inside a POSTMESSAGE: + POSTMESSAGE @all SETCONF Bridge="www.example.com:8284" + + Upon receiving this POSTMESSAGE, the Primary Controller will + validate it, evaluate it, store it to be later enabled by the + user, and alert the user that new bridges are available for + approval. It is only after the user has approved the new bridges + that the Primary Controller should then re-issue the SETCONF commands + to configure and deploy them in the tor client. + + Additionally, see "Security Concerns: Primary Controller" for more + discussion on potential pitfalls with POSTMESSAGE. + +Security Concerns + + While automatic bridge discovery and configuration is quite compelling + and powerful, there are several serious security concerns that warrant + extreme care. We've broken them down by component. + + Security Concerns: Primary Controller + + In the initial implementation, Orbot and Vidalia must take care to + transmit the Tor Control password to BridgeFinder in such a way that + it does not end up in system logs, process list, or viewable by other + system users. The best known strategy for doing this is by passing the + information through exported environment variables. + + Additionally, in future implementations, Orbot and Vidalia will need + to validate Proposal 197 POSTMESSAGE input before prompting the user. + POSTMESSAGE is a free-form message-passing mechanism. All sorts of + unexpected input may be passed through it by any other authenticated + Tor Controllers for their own unrelated communication purposes. + + Minimal validation includes verifying that the POSTMESSAGE data is a + valid Bridge or ClientTransportPlugin line and is acceptable input for + SETCONF. All unexpected characters should be removed through using a + whitelist, and format and structure should be checked against a + regular expression. Additionally, the POSTMESSAGE string should not be + passed through any string processing engines that automatically decode + character escape encodings, to avoid arbitrary control port execution. + + At the same time, POSTMESSAGE validation should be light. While fully + untrusted input is not expected due to the need for control port + authentication and BridgeFinder sanitation, complicated manual string + parsing techniques during validation should be avoided. Perform simple + easy-to-verify whitelist-based checks, and ignore unrecognized input. + + Beyond POSTMESSAGE validation, the manner in which the Primary + Controller achieves consent from the user is absolutely crucial to + security under this scheme. A simple "OK/Cancel" dialog is + insufficient to protect the user from the dangers of switching + bridges and running new plugins automatically. + + Newly discovered bridge lines from POSTMESSAGE should be added to a + disabled set that the user must navigate to as an independent window + apart from any confirmation dialog. The user must then explicitly + enable recently added plugins by checking them off individually. We + need the user's brain to be fully engaged and aware that it is + interacting with Tor during this step. If they get an "OK/Cancel" + popup that interrupts their online game play, they will almost + certainly simply click "OK" just to get back to the game quickly. + + The Primary Controller should transmit the POSTMESSAGE content to the + control port only after obtaining this out-of-band approval. + +Security Concerns: BridgeFinder and BridgeFinderHelper + + The unspecified nature of the IPC channel between BridgeFinder and + BridgeFinderHelper makes it difficult to make concrete security + suggestions. However, from past experience, the following best + practices must be employed to avoid security vulnerabilities: + + 1. Define a non-webby handshake and/or perform authentication + + The biggest risk is that unexpected applications will be manipulated + into posting malformed data to the BridgeFinder's IPC channel as if it + were from BridgeFinderHelper. The best way to defend against this is + to require a handshake to properly complete before accepting input. If + the handshake fails at any point, the IPC channel must be abandoned + and closed. Do not continue scanning for good input after any bad + input has been encountered. + + Additionally, if possible, it is wise to establish a shared secret + between BridgeFinder and BridgeFinderHelper through the filesystem or + any other means available for use in authentication. For an a good + example on how to use such a shared secret properly for + authentication, see Trac Ticket #5185 and/or the SafeCookie Tor + Control Port authentication mechanism. + + 2. Perform validation before parsing + + Care must be taken before converting BridgeFinderHelper data into + Bridge lines, especially for cases where the BridgeFinderHelper data + is fed directly to the control port after passing through + BridgeFinder. + + The input should be subjected to a character whitelist and possibly + also validated against a regular expression to verify format, and if + any unexpected or poorly-formed data is encountered, the IPC channel + must be closed. + + 3. Fail closed on unexpected input + + If the handshake fails, or if any other part of the BridgeFinderHelper + input is invalid, the IPC channel must be abandoned and closed. Do + *not* continue scanning for good input after any bad input has been + encountered. + +

1 0

[torspec/master] Add BridgeFinder Integration Proposal.
by ioerror＠torproject.org 21 Mar '12

21 Mar '12

commit 1a92597e9d2af8581ff4f5229e5d962c266361bc Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Tue Mar 20 17:45:20 2012 -0700 Add BridgeFinder Integration Proposal. --- proposals/xxx-bridgefinder-integration.txt | 415 ++++++++++++++++++++++++++++ 1 files changed, 415 insertions(+), 0 deletions(-) diff --git a/proposals/xxx-bridgefinder-integration.txt b/proposals/xxx-bridgefinder-integration.txt new file mode 100644 index 0000000..c6c55ec --- /dev/null +++ b/proposals/xxx-bridgefinder-integration.txt @@ -0,0 +1,415 @@ +Filename: xxx-bridgefinder-integration.txt +Title: Integration of BridgeFinder and BridgeFinderHelper +Author: Mike Perry +Created: 18-03-2012 +Status: Proposed +Target: 0.2.3.x+ + + +Overview + + This proposal describes how the Tor client software can interact with + an external program that performs bridge discovery based on user input + or information extracted from a web page, QR Code, online game, or + other transmission medium. + + +Scope and Audience + + This document describes how all of the components involved in bridge + discovery communicate this information to the rest of the Tor + software. The mechanisms of bridge discovery are not discussed, though + the design aims to be generalized enough to allow arbitrary new + discovery mechanisms to be added at any time. + + This document is also written with the hope that those who wish to + implement BridgeFinder components and BridgeFinderHelpers can get + started immediately after a read of this proposal, so that development + of bridge discovery mechanisms can proceed in parallel to supporting + functionality improvements in the Tor client software. + + +Components and Responsibilities + + 0. Tor Client + + The Tor Client is the piece of software that connects to the Tor + network (optionally using bridges) and provides a SOCKS proxy for + use by the user. + + In initial implementations, the Tor Client will support only + standard bridges. In later implementations, it is expected to + support pluggable transports as defined by Proposal 180. + + 1. Tor Control Port + + The Tor Control Port provides commands to perform operations, + configuration, and to obtain status information. It also optionally + provides event driven status updates. + + In initial implementations, it will be used directly by BridgeFinder + to configure bridge information via GETINFO and SETCONF. It is covered + by control-spec.txt in the tor-specs git repository. + + In later implementations, it will support the inter-controller + POSTMESSAGE IPC protocol as defined by Proposal 197 for use + in conveying bridge information to the Primary Controller. + + 2. Primary Controller + + The Primary Controller is the program that launches and configures the + Tor client, and monitors its status. + + On desktop platforms, this program is Vidalia, and it also launches + the Tor Browser. On Android, this program is Orbot. Orbot does not + launch a browser. + + On all platforms, this proposal requires that the Primary Controller + will launch one or more BridgeFinder child processes and provide + them with authentication information through the environment variables + TOR_CONTROL_PORT and TOR_CONTROL_PASSWD. + + In later implementations, the Primary Controller will be expected + to receive Bridge configuration information via the free-form + POSTMESSAGE protocol from Proposal 197, validate that information, + and hold that information for user approval. + + 3. BridgeFinder + + A BridgeFinder is a program that discovers bridges and configures + Tor to use them. + + In initial implementations, it is likely to be very dumb, and its main + purpose will be to serve as a layer of abstraction that should free + the Primary Controller from having to directly implement numerous ways + of retrieving bridges for various pluggable transports. + + In later implementations, it may perform arbitrary network operations + to discover, authenticate to, and/or verify bridges, possibly using + informational hints provided by one or more external + BridgeFinderHelpers (see next component). It could even go so far as + to download new pluggable transport plugins and/or transform + definition files from arbitrary urls. + + It will be launched by the Primary Controller and given access to the + Tor Control Port via the environment variables TOR_CONTROL_PORT and + TOR_CONTROL_PASSWD. + + Initial control port interactions can be command driven via GETINFO + and SETCONF, and do not need to subscribe to or process control port + events. Later implementations will use POSTMESSAGE as defined in + Proposal 197 to pass command requests to Vidalia, which will parse + them and ask for user confirmation before deploying them. Use of + POSTMESSAGE may or may not require event driven operation, depending + on POSTMESSAGE implementation status (POSTMESSAGE is designed to + support both command and event driven operation, but it is possible + event driven operation will happen first). + + 4. BridgeFinderHelper + + Each BridgeFinder implementation can optionally communicate with one + or more BridgeFinderHelpers. BridgeFinderHelpers are plugins to + external 3rd party applications that can inspect traffic, handle mime + types, or implement protocol handlers for accepting bridge discovery + information to pass to BridgeFinder. Example 3rd party applications + include Chrome, World of Warcraft, QR Code readers, or simple cut + and paste. + + Due to the arbitrary nature of sandboxing that may be present in + various BridgeFinderHelper host applications, we do not mandate the + exact nature of the IPC between BridgeFinder instances and external + BridgeFinderHelper addons. However, please see the "Security Concerns" + section for common pitfalls to avoid. + + 5. Tor Browser + + This is the browser the user uses with Tor. It is not useful until Tor + is properly configured to use bridges. It fails closed. + + It is not expected to run BridgeFinderHelper plugin instances, unless + those plugin instances exist to ensure the user always has a pool of + working bridges available after successfully configuring an + initial bridge. Once all bridges fail, the Tor Browser is useless. + + 6. Non-Tor Browser (aka BridgeFinderHelper host) + + This is the program the user uses for normal Internet activity to + obtain bridges via a BridgeFinderHelper plugin. It does not have to be + a browser. In advanced scenarios, this component may not be a browser + at all, but may be a program such as World of Warcraft instead. + + +Incremental Deployability + + The system is designed to be incrementally deployable: Simple designs + should be possible to develop and test immediately. The design is + flexible enough to be easily upgraded as more advanced features become + available from both Tor and new pluggable transports. + +Initial Implementation + + In the simplest possible initial implementation, BridgeFinder will + only discover Tor Bridges as they are deployed today. It will use the + Tor Control Port to configure these bridges directly via the SETCONF + command. It may or may not receive bridge information from a + BridgeFinderHelper. In an even more degenerate case, + BridgeFinderHelper may even be Vidalia or Orbot itself, acting upon + user input from cut and paste. + + Initial Implementation: BridgeFinder Launch + + In the initial implementation, the Primary Controller will launch one + or more BridgeFinders, providing control port authentication + information to them through the environment variables TOR_CONTROL_PORT + and TOR_CONTROL_PASSWD. + + BridgeFinder will then directly connect to the control port and + authenticate. Initial implementations should be able to function + without using SETEVENTS, and instead only using command-based + status inquiries and configuration (GETINFO and SETCONF). + + Initial Implementation: Obtaining Bridge Hint Information + + In the initial implementation, to test functionality, + BridgeFinderHelper can simply scrape bridges directly from + https://bridges.torproject.org. + + In slightly more advanced implementations, a BridgeFinderHelper + instance may be written for use in the user's Non-Tor Browser. This + plugin could extract bridges from images, html comments, and other + material present in ad banners and slack space on unrelated pages. + + BridgeFinderHelper would then communicate with the appropriate + BridgeFinder instance over an acceptable IPC mechanism. This proposal + does not seek to specify the nature of that IPC channel (because + BridgeFinderHelper may be arbitrarily constrained due to host + application sandboxing), but we do make several security + recommendations under the section "Security Concerns: BridgeFinder and + BridgeFinderHelper". + + Initial Implementation: Configuring New Bridges + + In the initial implementation, Bridge configuration will be done + directly though the control port using the SETCONF command. + + Initial implementations will support only retrieval and configuration + of standard Tor Bridges. These are configured using SETCONF on the Tor + Control Port as follows: + SETCONF Bridge="IP:ORPort [fingerprint]" + + +Future Implementations + + In future implementations, the system can incrementally evolve in a + few different directions. As new pluggable transports are created, it + is conceivable that BridgeFinder may want to download new plugin + binaries (and/or new transport transform definition files) and + provide them to Tor. + + Furthermore, it may prove simpler to deploy multiple concurrent + BridgeFinder+BridgeFinderHelper pairs as opposed to adding new + functionality to existing prototypes. + + Finally, it is desirable for BridgeFinder to obtain approval + from the user before updating bridge configuration, especially for + cases where BridgeFinderHelper is automatically discovering bridges + in-band during Non-Tor activity. + + The exact mechanisms for accomplishing these improvements is + described in the following subsections. + + Future Implementations: BridgeFinder Launch and POSTMESSAGE handshake + + The nature of the BridgeFinder launch and the environment variables + provided is not expected to change. However, future Primary Controller + implementations may decide to launch more than one BridgeFinder + instance side by side. + + Additionally, to negotiate the IPC channel created by Proposal 197 + for purposes of providing user confirmation, it is recommended that + BridgeFinder and the Primary Controller perform a handshake using + POSTMESSAGE upon launch, to establish that all parties properly + support the feature: + + Primary Controller: "POSTMESSAGE @all Controller wants POSTMESSAGE v1.1" + BridgeFinder: "POSTMESSAGE @all BridgeFinder has POSTMESSAGE v1.0" + Primary Controller: "POSTMESSAGE @all Controller expects POSTMESSAGE v1.0" + BridgeFinder: "POSTMESSAGE @all BridgeFinder will POSTMESSAGE v1.0" + + If this 4 step handshake proceeds with an acceptable version, + BridgeFinder must use POSTMESSAGE to transmit SETCONF Bridge lines + (see "Future Implementations: Configuring New Bridges" below). If + POSTMESSAGE support is expected, but the handshake does not complete + for any reason, BridgeFinder should either exit or go dormant. + + The exact nature of the version negotiation and exactly how much + backwards compatibility must be tolerated is unspecified. + "All-or-nothing" is a safe assumption to get started. + + Future Implementations: Obtaining Bridge Hint Information + + Future BridgeFinder implementations may download additional + information based on what is provided by BridgeFinderHelper. They + may fetch pluggable transport plugins, transformation parameters, + and other material. + + Future Implementations: Configuring New Bridges + + Future implementations will be concerned with providing two new pieces + of functionality with respect to configuring bridges: configuring + pluggable transports, and properly prompting the user before altering + Tor configuration. + + There are two ways to tell Tor clients about pluggable transports + (as defined in Proposal 180). + + On the control port, an external Proposal 180 transport will be + configured with + SETCONF ClientTransportPlugin=<method> socks5 <addr:port> [auth=X] + as in + SETCONF ClientTransportPlugin="trebuchet socks5 127.0.0.1:9999". + + A managed proxy is configured with + SETCONF ClientTransportPlugin=<methods> exec <path> [options] + as in + SETCONF ClientTransportPlugin="trebuchet exec /usr/libexec/trebuchet --managed". + + This example tells Tor to launch an external program to provide a + socks proxy for 'trebuchet' connections. The Tor client only + launches one instance of each external program with a given set of + options, even if the same executable and options are listed for + more than one method. + + Pluggable transport bridges discovered for this transport by + BridgeFinder would then be set with: + SETCONF Bridge="trebuchet 3.2.4.1:8080 keyid=09F911029D74E35BD84156C5635688C009F909F9 rocks=20 height=5.6m". + + For more information on pluggable transports and supporting Tor + configuration commands, see Proposal 180. + + Future Implementations: POSTMESSAGE and User Confirmation + + Because configuring even normal bridges alone can expose the user to + attacks, it is strongly desired to provide some mechanism to allow + the user to approve new bridges prior to their use, especially for + situations where BridgeFinderHelper is extracting them transparently + while the user performs unrelated activity. + + If BridgeFinderHelper grows to the point where it is downloading new + transform definitions or plugins, user confirmation becomes + absolutely required. + + To achieve user confirmation, we depend upon the POSTMESSAGE command + defined in Proposal 197. + + If the POSTMESSAGE handshake succeeds, instead of sending SETCONF + commands directly to the control port, the commands will be wrapped + inside a POSTMESSAGE: + POSTMESSAGE @all SETCONF Bridge="www.example.com:8284" + + Upon receiving this POSTMESSAGE, the Primary Controller will + validate it, evaluate it, store it to be later enabled by the + user, and alert the user that new bridges are available for + approval. It is only after the user has approved the new bridges + that the Primary Controller should then re-issue the SETCONF commands + to configure and deploy them in the tor client. + + Additionally, see "Security Concerns: Primary Controller" for more + discussion on potential pitfalls with POSTMESSAGE. + +Security Concerns + + While automatic bridge discovery and configuration is quite compelling + and powerful, there are several serious security concerns that warrant + extreme care. We've broken them down by component. + + Security Concerns: Primary Controller + + In the initial implementation, Orbot and Vidalia must take care to + transmit the Tor Control password to BridgeFinder in such a way that + it does not end up in system logs, process list, or viewable by other + system users. The best known strategy for doing this is by passing the + information through exported environment variables. + + Additionally, in future implementations, Orbot and Vidalia will need + to validate Proposal 197 POSTMESSAGE input before prompting the user. + POSTMESSAGE is a free-form message-passing mechanism. All sorts of + unexpected input may be passed through it by any other authenticated + Tor Controllers for their own unrelated communication purposes. + + Minimal validation includes verifying that the POSTMESSAGE data is a + valid Bridge or ClientTransportPlugin line and is acceptable input for + SETCONF. All unexpected characters should be removed through using a + whitelist, and format and structure should be checked against a + regular expression. Additionally, the POSTMESSAGE string should not be + passed through any string processing engines that automatically decode + character escape encodings, to avoid arbitrary control port execution. + + At the same time, POSTMESSAGE validation should be light. While fully + untrusted input is not expected due to the need for control port + authentication and BridgeFinder sanitation, complicated manual string + parsing techniques during validation should be avoided. Perform simple + easy-to-verify whitelist-based checks, and ignore unrecognized input. + + Beyond POSTMESSAGE validation, the manner in which the Primary + Controller achieves consent from the user is absolutely crucial to + security under this scheme. A simple "OK/Cancel" dialog is + insufficient to protect the user from the dangers of switching + bridges and running new plugins automatically. + + Newly discovered bridge lines from POSTMESSAGE should be added to a + disabled set that the user must navigate to as an independent window + apart from any confirmation dialog. The user must then explicitly + enable recently added plugins by checking them off individually. We + need the user's brain to be fully engaged and aware that it is + interacting with Tor during this step. If they get an "OK/Cancel" + popup that interrupts their online game play, they will almost + certainly simply click "OK" just to get back to the game quickly. + + The Primary Controller should transmit the POSTMESSAGE content to the + control port only after obtaining this out-of-band approval. + +Security Concerns: BridgeFinder and BridgeFinderHelper + + The unspecified nature of the IPC channel between BridgeFinder and + BridgeFinderHelper makes it difficult to make concrete security + suggestions. However, from past experience, the following best + practices must be employed to avoid security vulnerabilities: + + 1. Define a non-webby handshake and/or perform authentication + + The biggest risk is that unexpected applications will be manipulated + into posting malformed data to the BridgeFinder's IPC channel as if it + were from BridgeFinderHelper. The best way to defend against this is + to require a handshake to properly complete before accepting input. If + the handshake fails at any point, the IPC channel must be abandoned + and closed. Do not continue scanning for good input after any bad + input has been encountered. + + Additionally, if possible, it is wise to establish a shared secret + between BridgeFinder and BridgeFinderHelper through the filesystem or + any other means available for use in authentication. For an a good + example on how to use such a shared secret properly for + authentication, see Trac Ticket #5185 and/or the SafeCookie Tor + Control Port authentication mechanism. + + 2. Perform validation before parsing + + Care must be taken before converting BridgeFinderHelper data into + Bridge lines, especially for cases where the BridgeFinderHelper data + is fed directly to the control port after passing through + BridgeFinder. + + The input should be subjected to a character whitelist and possibly + also validated against a regular expression to verify format, and if + any unexpected or poorly-formed data is encountered, the IPC channel + must be closed. + + 3. Fail closed on unexpected input + + If the handshake fails, or if any other part of the BridgeFinderHelper + input is invalid, the IPC channel must be abandoned and closed. Do + *not* continue scanning for good input after any bad input has been + encountered. + +

1 0

[metrics-web/master] Periodically generate R objects with default parameters.
by karsten＠torproject.org 20 Mar '12

20 Mar '12

commit 52c5c939bc8f04866fa9e21097072acc73c2ff06 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Mar 20 15:11:15 2012 +0100 Periodically generate R objects with default parameters. Fixes #3878, among other things. --- src/org/torproject/ernie/web/CsvServlet.java | 2 +- src/org/torproject/ernie/web/GraphDataServlet.java | 2 +- .../torproject/ernie/web/GraphImageServlet.java | 2 +- .../ernie/web/GraphsSubpagesServlet.java | 2 +- src/org/torproject/ernie/web/RObjectGenerator.java | 60 ++++++++++++++----- .../ernie/web/TableParameterChecker.java | 2 +- 6 files changed, 49 insertions(+), 21 deletions(-) diff --git a/src/org/torproject/ernie/web/CsvServlet.java b/src/org/torproject/ernie/web/CsvServlet.java index 1e9fbf8..3d0e06d 100644 --- a/src/org/torproject/ernie/web/CsvServlet.java +++ b/src/org/torproject/ernie/web/CsvServlet.java @@ -77,7 +77,7 @@ public class CsvServlet extends HttpServlet { /* Request CSV file from R object generator, which asks Rserve to * generate it. */ String csvFileContent = this.rObjectGenerator.generateCsv( - requestedCsvFile); + requestedCsvFile, true); /* Make sure that we have a graph to return. */ if (csvFileContent == null) { diff --git a/src/org/torproject/ernie/web/GraphDataServlet.java b/src/org/torproject/ernie/web/GraphDataServlet.java index 35df73b..b335994 100644 --- a/src/org/torproject/ernie/web/GraphDataServlet.java +++ b/src/org/torproject/ernie/web/GraphDataServlet.java @@ -112,7 +112,7 @@ public class GraphDataServlet extends HttpServlet { /* Request CSV file from R object generator, which may ask Rserve to * generate it. */ String csvFileContent = this.rObjectGenerator.generateCsv( - requestedCsvFile); + requestedCsvFile, true); /* Make sure that we have a CSV to convert into JSON. */ if (csvFileContent == null) { diff --git a/src/org/torproject/ernie/web/GraphImageServlet.java b/src/org/torproject/ernie/web/GraphImageServlet.java index 5d46cdd..20d3297 100644 --- a/src/org/torproject/ernie/web/GraphImageServlet.java +++ b/src/org/torproject/ernie/web/GraphImageServlet.java @@ -49,7 +49,7 @@ public class GraphImageServlet extends HttpServlet { /* Request graph from R object generator, which either returns it from * its cache or asks Rserve to generate it. */ byte[] graphBytes = rObjectGenerator.generateGraph(requestedGraph, - request.getParameterMap()); + request.getParameterMap(), true); /* Make sure that we have a graph to return. */ if (graphBytes == null) { diff --git a/src/org/torproject/ernie/web/GraphsSubpagesServlet.java b/src/org/torproject/ernie/web/GraphsSubpagesServlet.java index 5c2743d..d557ce5 100644 --- a/src/org/torproject/ernie/web/GraphsSubpagesServlet.java +++ b/src/org/torproject/ernie/web/GraphsSubpagesServlet.java @@ -134,7 +134,7 @@ public class GraphsSubpagesServlet extends HttpServlet { this.availableGraphsSubpageTables.get(requestedPage)) { List<Map<String, String>> tableData = rObjectGenerator. generateTable(tableName, requestedTable, - request.getParameterMap()); + request.getParameterMap(), true); request.setAttribute(tableName.replaceAll("-", "_") + "_tabledata", tableData); } diff --git a/src/org/torproject/ernie/web/RObjectGenerator.java b/src/org/torproject/ernie/web/RObjectGenerator.java index da79d43..011eccd 100644 --- a/src/org/torproject/ernie/web/RObjectGenerator.java +++ b/src/org/torproject/ernie/web/RObjectGenerator.java @@ -20,7 +20,6 @@ import java.util.TreeSet; import javax.servlet.ServletContext; import javax.servlet.ServletContextEvent; import javax.servlet.ServletContextListener; -import javax.servlet.http.HttpServletResponse; import org.rosuda.REngine.Rserve.RConnection; import org.rosuda.REngine.Rserve.RserveException; @@ -104,13 +103,40 @@ public class RObjectGenerator implements ServletContextListener { /* Register ourself, so that servlets can use us. */ servletContext.setAttribute("RObjectGenerator", this); + + /* Periodically generate R objects with default parameters. */ + new Thread() { + public void run() { + long lastUpdated = 0L, sleep; + while (true) { + while ((sleep = maxCacheAge * 1000L / 2L + lastUpdated + - System.currentTimeMillis()) > 0L) { + try { + Thread.sleep(sleep); + } catch (InterruptedException e) { + } + } + for (String csvFile : availableCsvFiles) { + generateCsv(csvFile, false); + } + for (String tableName : availableTables.keySet()) { + generateTable(tableName, tableName, new HashMap(), false); + } + for (String graphName : availableGraphs.keySet()) { + generateGraph(graphName, new HashMap(), false); + } + lastUpdated = System.currentTimeMillis(); + } + }; + }.start(); } public void contextDestroyed(ServletContextEvent event) { /* Nothing to do. */ } - public byte[] generateGraph(String requestedGraph, Map parameterMap) { + public byte[] generateGraph(String requestedGraph, Map parameterMap, + boolean checkCache) { Map<String, String[]> checkedParameters = GraphParameterChecker. getInstance().checkParameters(requestedGraph, parameterMap); if (checkedParameters == null) { @@ -144,19 +170,20 @@ public class RObjectGenerator implements ServletContextListener { String imageFilename = imageFilenameBuilder.toString(); rQueryBuilder.append("path = '%s')"); String rQuery = rQueryBuilder.toString(); - return this.generateGraph(rQuery, imageFilename); + return this.generateGraph(rQuery, imageFilename, checkCache); } /* Generate a graph using the given R query that has a placeholder for * the absolute path to the image to be created. */ - private byte[] generateGraph(String rQuery, String imageFilename) { + private byte[] generateGraph(String rQuery, String imageFilename, + boolean checkCache) { /* See if we need to generate this graph. */ File imageFile = new File(this.cachedGraphsDirectory + "/" + imageFilename); long now = System.currentTimeMillis(); - if (!imageFile.exists() || imageFile.lastModified() < now - - this.maxCacheAge * 1000L) { + if (!checkCache || !imageFile.exists() || + imageFile.lastModified() < now - this.maxCacheAge * 1000L) { /* We do. Update the R query to contain the absolute path to the * file to be generated, create a connection to Rserve, run the R @@ -202,24 +229,25 @@ public class RObjectGenerator implements ServletContextListener { return this.availableCsvFiles; } - public String generateCsv(String requestedCsvFile) { + public String generateCsv(String requestedCsvFile, boolean checkCache) { /* Prepare filename and R query string. */ String rQuery = "export_" + requestedCsvFile.replaceAll("-", "_") + "(path = '%s')"; String csvFilename = requestedCsvFile + ".csv"; - return this.generateCsv(rQuery, csvFilename); + return this.generateCsv(rQuery, csvFilename, checkCache); } /* Generate a comma-separated value file using the given R query that * has a placeholder for the absolute path to the file to be created. */ - private String generateCsv(String rQuery, String csvFilename) { + private String generateCsv(String rQuery, String csvFilename, + boolean checkCache) { /* See if we need to generate this .csv file. */ File csvFile = new File(this.cachedGraphsDirectory + "/" + csvFilename); long now = System.currentTimeMillis(); - if (!csvFile.exists() || csvFile.lastModified() < now - - this.maxCacheAge * 1000L) { + if (!checkCache || !csvFile.exists() || + csvFile.lastModified() < now - this.maxCacheAge * 1000L) { /* We do. Update the R query to contain the absolute path to the * file to be generated, create a connection to Rserve, run the R @@ -260,7 +288,7 @@ public class RObjectGenerator implements ServletContextListener { } public List<Map<String, String>> generateTable(String tableName, - String requestedTable, Map parameterMap) { + String requestedTable, Map parameterMap, boolean checkCache) { Map<String, String[]> checkedParameters = null; if (tableName.equals(requestedTable)) { @@ -303,21 +331,21 @@ public class RObjectGenerator implements ServletContextListener { String tableFilename = tableFilenameBuilder.toString(); rQueryBuilder.append("path = '%s')"); String rQuery = rQueryBuilder.toString(); - return this.generateTable(rQuery, tableFilename); + return this.generateTable(rQuery, tableFilename, checkCache); } /* Generate table data using the given R query and filename or read * previously generated table data from disk if it's not too old and * return table data. */ private List<Map<String, String>> generateTable(String rQuery, - String tableFilename) { + String tableFilename, boolean checkCache) { /* See if we need to generate this table. */ File tableFile = new File(this.cachedGraphsDirectory + "/" + tableFilename); long now = System.currentTimeMillis(); - if (!tableFile.exists() || tableFile.lastModified() < now - - this.maxCacheAge * 1000L) { + if (!checkCache || !tableFile.exists() || + tableFile.lastModified() < now - this.maxCacheAge * 1000L) { /* We do. Update the R query to contain the absolute path to the * file to be generated, create a connection to Rserve, run the R diff --git a/src/org/torproject/ernie/web/TableParameterChecker.java b/src/org/torproject/ernie/web/TableParameterChecker.java index 25bd8fa..e4b3e13 100644 --- a/src/org/torproject/ernie/web/TableParameterChecker.java +++ b/src/org/torproject/ernie/web/TableParameterChecker.java @@ -55,7 +55,7 @@ public class TableParameterChecker { public Map<String, String[]> checkParameters(String tableType, Map requestParameters) { - /* Check if the graph type exists. */ + /* Check if the table type exists. */ if (tableType == null || !this.availableTables.containsKey(tableType)) { return null;

1 0

[metrics-web/master] Re-order tables and form fields to customize tables.
by karsten＠torproject.org 20 Mar '12

20 Mar '12

commit 0ffae78626e37932019e425ea14f9505865b4bfa Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Mar 20 15:53:29 2012 +0100 Re-order tables and form fields to customize tables. Having the table caption, then the table data, and then the form to customize the table might have been confusing, in particular with table data depending on default values in the form. Maybe people are more used to reading text above tables than below tables? Let's try that. Fixes #3873. --- web/WEB-INF/users.jsp | 48 ++++++++++++++++++++++++------------------------ 1 files changed, 24 insertions(+), 24 deletions(-) diff --git a/web/WEB-INF/users.jsp b/web/WEB-INF/users.jsp index b3dfa31..b9b0914 100644 --- a/web/WEB-INF/users.jsp +++ b/web/WEB-INF/users.jsp @@ -65,19 +65,6 @@ based on the requests seen by a few dozen directory mirrors.</p> <hr> <a name="direct-users-table"></a> <p><b>Top-10 countries by directly connecting users:</b></p> -<table> - <tr> - <th>Country</th> - <th>Mean daily users</th> - </tr> - <c:forEach var="row" items="${direct_users_tabledata}"> - <tr> - <td><a href="users.html?graph=direct-users&country=${row['cc']}#direct-users">${row['country']}</a>&emsp;</td> - <td>${row['abs']} (<fmt:formatNumber type="number" minFractionDigits="2" value="${row['rel']}" /> %)</td> - </tr> - </c:forEach> -</table> -<br> <form action="users.html#direct-users-table"> <div class="formrow"> <input type="hidden" name="table" value="direct-users"> @@ -93,25 +80,23 @@ based on the requests seen by a few dozen directory mirrors.</p> </p> </div> </form> -<hr> -<a name="censorship-events"></a> -<p><b>Top-10 countries by possible censorship events (<a - href="papers/detector-2011-09-09.pdf">BETA</a>):</b></p> +<br> <table> <tr> <th>Country</th> - <th>Downturns</th> - <th>Upturns</th> + <th>Mean daily users</th> </tr> - <c:forEach var="row" items="${censorship_events_tabledata}"> + <c:forEach var="row" items="${direct_users_tabledata}"> <tr> - <td><a href="users.html?graph=direct-users&country=${row['cc']}&events=on#direct-users">${row['country']}</a>&emsp;</td> - <td>${row['downturns']}</td> - <td>${row['upturns']}</td> + <td><a href="users.html?graph=direct-users&country=${row['cc']}#direct-users">${row['country']}</a>&emsp;</td> + <td>${row['abs']} (<fmt:formatNumber type="number" minFractionDigits="2" value="${row['rel']}" /> %)</td> </tr> </c:forEach> </table> -<br> +<hr> +<a name="censorship-events"></a> +<p><b>Top-10 countries by possible censorship events (<a + href="papers/detector-2011-09-09.pdf">BETA</a>):</b></p> <form action="users.html#censorship-events"> <div class="formrow"> <input type="hidden" name="table" value="censorship-events"> @@ -127,6 +112,21 @@ based on the requests seen by a few dozen directory mirrors.</p> </p> </div> </form> +<br> +<table> + <tr> + <th>Country</th> + <th>Downturns</th> + <th>Upturns</th> + </tr> + <c:forEach var="row" items="${censorship_events_tabledata}"> + <tr> + <td><a href="users.html?graph=direct-users&country=${row['cc']}&events=on#direct-users">${row['country']}</a>&emsp;</td> + <td>${row['downturns']}</td> + <td>${row['upturns']}</td> + </tr> + </c:forEach> +</table> <hr> <p><a href="csv/direct-users.csv">CSV</a> file containing daily directly connecting users by country.</p>

1 0

[metrics-web/master] Move available R object lists to RObjectGenerator.
by karsten＠torproject.org 20 Mar '12

20 Mar '12

commit e33d4702e2d1c33a9e6c91db40bedc393c3bc7c9 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Mar 20 14:17:21 2012 +0100 Move available R object lists to RObjectGenerator. --- src/org/torproject/ernie/web/CsvServlet.java | 22 +------- .../ernie/web/GraphParameterChecker.java | 26 +------- src/org/torproject/ernie/web/RObjectGenerator.java | 61 ++++++++++++++++++++ .../ernie/web/TableParameterChecker.java | 6 +- 4 files changed, 69 insertions(+), 46 deletions(-) diff --git a/src/org/torproject/ernie/web/CsvServlet.java b/src/org/torproject/ernie/web/CsvServlet.java index 4f60eaf..1e9fbf8 100644 --- a/src/org/torproject/ernie/web/CsvServlet.java +++ b/src/org/torproject/ernie/web/CsvServlet.java @@ -33,31 +33,11 @@ public class CsvServlet extends HttpServlet { /* Initialize logger. */ this.logger = Logger.getLogger(CsvServlet.class.toString()); - /* Initialize map of available CSV files. */ - this.availableCsvFiles = new TreeSet<String>(); - this.availableCsvFiles.add("bandwidth"); - this.availableCsvFiles.add("bridge-users"); - this.availableCsvFiles.add("bwhist-flags"); - this.availableCsvFiles.add("connbidirect"); - this.availableCsvFiles.add("direct-users"); - this.availableCsvFiles.add("dirreq-stats"); - this.availableCsvFiles.add("dirbytes"); - this.availableCsvFiles.add("gettor"); - this.availableCsvFiles.add("monthly-users-average"); - this.availableCsvFiles.add("monthly-users-peak"); - this.availableCsvFiles.add("networksize"); - this.availableCsvFiles.add("platforms"); - this.availableCsvFiles.add("relaycountries"); - this.availableCsvFiles.add("relayflags"); - this.availableCsvFiles.add("relayflags-hour"); - this.availableCsvFiles.add("torperf"); - this.availableCsvFiles.add("torperf-failures"); - this.availableCsvFiles.add("versions"); - /* Get a reference to the R object generator that we need to generate * CSV files. */ this.rObjectGenerator = (RObjectGenerator) getServletContext(). getAttribute("RObjectGenerator"); + this.availableCsvFiles = rObjectGenerator.getAvailableCsvFiles(); } public void doGet(HttpServletRequest request, diff --git a/src/org/torproject/ernie/web/GraphParameterChecker.java b/src/org/torproject/ernie/web/GraphParameterChecker.java index 96efed0..19a4081 100644 --- a/src/org/torproject/ernie/web/GraphParameterChecker.java +++ b/src/org/torproject/ernie/web/GraphParameterChecker.java @@ -47,28 +47,6 @@ public class GraphParameterChecker { this.dateFormat = new SimpleDateFormat("yyyy-MM-dd"); this.dateFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - this.availableGraphs = new HashMap<String, String>(); - this.availableGraphs.put("networksize", "start,end,filename,dpi"); - this.availableGraphs.put("relaycountries", - "start,end,country,filename,dpi"); - this.availableGraphs.put("relayflags", "start,end,flag,granularity," - + "filename,dpi"); - this.availableGraphs.put("versions", "start,end,filename,dpi"); - this.availableGraphs.put("platforms", "start,end,filename,dpi"); - this.availableGraphs.put("bandwidth", "start,end,filename,dpi"); - this.availableGraphs.put("bwhist-flags", "start,end,filename,dpi"); - this.availableGraphs.put("dirbytes", "start,end,filename,dpi"); - this.availableGraphs.put("direct-users", - "start,end,country,events,filename,nocutoff,dpi"); - this.availableGraphs.put("bridge-users", - "start,end,country,filename,dpi"); - this.availableGraphs.put("gettor", "start,end,language,filename,dpi"); - this.availableGraphs.put("torperf", - "start,end,source,filesize,filename,dpi"); - this.availableGraphs.put("torperf-failures", - "start,end,source,filesize,filename,dpi"); - this.availableGraphs.put("connbidirect", "start,end,filename,dpi"); - this.knownParameterValues = new HashMap<String, String>(); this.knownParameterValues.put("flag", "Running,Exit,Guard,Fast,Stable"); @@ -86,6 +64,10 @@ public class GraphParameterChecker { this.knownParameterValues.put("dpi", "72,150,300"); } + public void setAvailableGraphs(Map<String, String> availableGraphs) { + this.availableGraphs = availableGraphs; + } + /** * Checks request parameters for the given graph type and returns a map * of recognized parameters, or null if the graph type doesn't exist or diff --git a/src/org/torproject/ernie/web/RObjectGenerator.java b/src/org/torproject/ernie/web/RObjectGenerator.java index 2fb4477..da79d43 100644 --- a/src/org/torproject/ernie/web/RObjectGenerator.java +++ b/src/org/torproject/ernie/web/RObjectGenerator.java @@ -14,6 +14,8 @@ import java.util.Arrays; import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.SortedSet; +import java.util.TreeSet; import javax.servlet.ServletContext; import javax.servlet.ServletContextEvent; @@ -33,6 +35,10 @@ public class RObjectGenerator implements ServletContextListener { private String cachedGraphsDirectory; private long maxCacheAge; + private SortedSet<String> availableCsvFiles; + private Map<String, String> availableTables; + private Map<String, String> availableGraphs; + public void contextInitialized(ServletContextEvent event) { /* Initialize using context parameters. */ @@ -45,6 +51,57 @@ public class RObjectGenerator implements ServletContextListener { this.cachedGraphsDirectory = servletContext.getInitParameter( "cachedGraphsDir"); + /* Initialize map of available CSV files. */ + this.availableCsvFiles = new TreeSet<String>(); + this.availableCsvFiles.add("bandwidth"); + this.availableCsvFiles.add("bridge-users"); + this.availableCsvFiles.add("bwhist-flags"); + this.availableCsvFiles.add("connbidirect"); + this.availableCsvFiles.add("direct-users"); + this.availableCsvFiles.add("dirreq-stats"); + this.availableCsvFiles.add("dirbytes"); + this.availableCsvFiles.add("gettor"); + this.availableCsvFiles.add("monthly-users-average"); + this.availableCsvFiles.add("monthly-users-peak"); + this.availableCsvFiles.add("networksize"); + this.availableCsvFiles.add("platforms"); + this.availableCsvFiles.add("relaycountries"); + this.availableCsvFiles.add("relayflags"); + this.availableCsvFiles.add("relayflags-hour"); + this.availableCsvFiles.add("torperf"); + this.availableCsvFiles.add("torperf-failures"); + this.availableCsvFiles.add("versions"); + + this.availableTables = new HashMap<String, String>(); + this.availableTables.put("direct-users", "start,end,filename"); + this.availableTables.put("censorship-events", "start,end,filename"); + TableParameterChecker.getInstance().setAvailableTables( + availableTables); + + this.availableGraphs = new HashMap<String, String>(); + this.availableGraphs.put("networksize", "start,end,filename,dpi"); + this.availableGraphs.put("relaycountries", + "start,end,country,filename,dpi"); + this.availableGraphs.put("relayflags", "start,end,flag,granularity," + + "filename,dpi"); + this.availableGraphs.put("versions", "start,end,filename,dpi"); + this.availableGraphs.put("platforms", "start,end,filename,dpi"); + this.availableGraphs.put("bandwidth", "start,end,filename,dpi"); + this.availableGraphs.put("bwhist-flags", "start,end,filename,dpi"); + this.availableGraphs.put("dirbytes", "start,end,filename,dpi"); + this.availableGraphs.put("direct-users", + "start,end,country,events,filename,nocutoff,dpi"); + this.availableGraphs.put("bridge-users", + "start,end,country,filename,dpi"); + this.availableGraphs.put("gettor", "start,end,language,filename,dpi"); + this.availableGraphs.put("torperf", + "start,end,source,filesize,filename,dpi"); + this.availableGraphs.put("torperf-failures", + "start,end,source,filesize,filename,dpi"); + this.availableGraphs.put("connbidirect", "start,end,filename,dpi"); + GraphParameterChecker.getInstance().setAvailableGraphs( + availableGraphs); + /* Register ourself, so that servlets can use us. */ servletContext.setAttribute("RObjectGenerator", this); } @@ -141,6 +198,10 @@ public class RObjectGenerator implements ServletContextListener { return result; } + public SortedSet<String> getAvailableCsvFiles() { + return this.availableCsvFiles; + } + public String generateCsv(String requestedCsvFile) { /* Prepare filename and R query string. */ String rQuery = "export_" + requestedCsvFile.replaceAll("-", "_") diff --git a/src/org/torproject/ernie/web/TableParameterChecker.java b/src/org/torproject/ernie/web/TableParameterChecker.java index 3209bda..25bd8fa 100644 --- a/src/org/torproject/ernie/web/TableParameterChecker.java +++ b/src/org/torproject/ernie/web/TableParameterChecker.java @@ -41,10 +41,10 @@ public class TableParameterChecker { public TableParameterChecker() { this.dateFormat = new SimpleDateFormat("yyyy-MM-dd"); this.dateFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + } - this.availableTables = new HashMap<String, String>(); - this.availableTables.put("direct-users", "start,end,filename"); - this.availableTables.put("censorship-events", "start,end,filename"); + public void setAvailableTables(Map<String, String> availableTables) { + this.availableTables = availableTables; } /**

1 0

[metrics-web/master] Move all R-specific parts to RObjectGenerator.
by karsten＠torproject.org 20 Mar '12

20 Mar '12

commit 8409797f95ce21c4d6ec93132aed101115451c1a Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Mar 20 14:00:47 2012 +0100 Move all R-specific parts to RObjectGenerator. --- src/org/torproject/ernie/web/CsvServlet.java | 11 +-- src/org/torproject/ernie/web/GraphDataServlet.java | 11 +-- .../torproject/ernie/web/GraphImageServlet.java | 43 +-------- .../ernie/web/GraphsSubpagesServlet.java | 49 +--------- src/org/torproject/ernie/web/RObjectGenerator.java | 99 +++++++++++++++++++- 5 files changed, 110 insertions(+), 103 deletions(-) diff --git a/src/org/torproject/ernie/web/CsvServlet.java b/src/org/torproject/ernie/web/CsvServlet.java index 39efc14..4f60eaf 100644 --- a/src/org/torproject/ernie/web/CsvServlet.java +++ b/src/org/torproject/ernie/web/CsvServlet.java @@ -94,15 +94,10 @@ public class CsvServlet extends HttpServlet { } logger.fine("CSV file '" + requestedCsvFile + ".csv' requested."); - /* Prepare filename and R query string. */ - String rQuery = "export_" + requestedCsvFile.replaceAll("-", "_") - + "(path = '%s')"; - String csvFilename = requestedCsvFile + ".csv"; - /* Request CSV file from R object generator, which asks Rserve to * generate it. */ - String csvFileContent = this.rObjectGenerator.generateCsv(rQuery, - csvFilename); + String csvFileContent = this.rObjectGenerator.generateCsv( + requestedCsvFile); /* Make sure that we have a graph to return. */ if (csvFileContent == null) { @@ -115,7 +110,7 @@ public class CsvServlet extends HttpServlet { response.setHeader("Content-Length", String.valueOf( csvFileContent.length())); response.setHeader("Content-Disposition", - "inline; filename=\"" + csvFilename + "\""); + "inline; filename=\"" + requestedCsvFile + ".csv\""); response.getWriter().print(csvFileContent); } } diff --git a/src/org/torproject/ernie/web/GraphDataServlet.java b/src/org/torproject/ernie/web/GraphDataServlet.java index 6bba9f0..35df73b 100644 --- a/src/org/torproject/ernie/web/GraphDataServlet.java +++ b/src/org/torproject/ernie/web/GraphDataServlet.java @@ -109,15 +109,10 @@ public class GraphDataServlet extends HttpServlet { requestedJsonFile); logger.fine("CSV file '" + requestedCsvFile + ".csv' requested."); - /* Prepare filename and R query string. */ - String rQuery = "export_" + requestedCsvFile.replaceAll("-", "_") - + "(path = '%s')"; - String csvFilename = requestedCsvFile + ".csv"; - - /* Request CSV file from R object generator, which asks Rserve to + /* Request CSV file from R object generator, which may ask Rserve to * generate it. */ - String csvFileContent = this.rObjectGenerator.generateCsv(rQuery, - csvFilename); + String csvFileContent = this.rObjectGenerator.generateCsv( + requestedCsvFile); /* Make sure that we have a CSV to convert into JSON. */ if (csvFileContent == null) { diff --git a/src/org/torproject/ernie/web/GraphImageServlet.java b/src/org/torproject/ernie/web/GraphImageServlet.java index 4ef53dd..5d46cdd 100644 --- a/src/org/torproject/ernie/web/GraphImageServlet.java +++ b/src/org/torproject/ernie/web/GraphImageServlet.java @@ -46,47 +46,10 @@ public class GraphImageServlet extends HttpServlet { lastIndexOf("/") + 1); } - /* Check parameters. */ - Map<String, String[]> checkedParameters = GraphParameterChecker. - getInstance().checkParameters(requestedGraph, - request.getParameterMap()); - if (checkedParameters == null) { - response.sendError(HttpServletResponse.SC_BAD_REQUEST); - return; - } - - /* Prepare filename and R query string. */ - StringBuilder rQueryBuilder = new StringBuilder("plot_" - + requestedGraph.replaceAll("-", "_") + "("), - imageFilenameBuilder = new StringBuilder(requestedGraph); - for (Map.Entry<String, String[]> parameter : - checkedParameters.entrySet()) { - String parameterName = parameter.getKey(); - String[] parameterValues = parameter.getValue(); - for (String param : parameterValues) { - imageFilenameBuilder.append("-" + param); - } - if (parameterValues.length < 2) { - rQueryBuilder.append(parameterName + " = '" + parameterValues[0] - + "', "); - } else { - rQueryBuilder.append(parameterName + " = c("); - for (int i = 0; i < parameterValues.length - 1; i++) { - rQueryBuilder.append("'" + parameterValues[i] + "', "); - } - rQueryBuilder.append("'" + parameterValues[ - parameterValues.length - 1] + "'), "); - } - } - imageFilenameBuilder.append(".png"); - String imageFilename = imageFilenameBuilder.toString(); - rQueryBuilder.append("path = '%s')"); - String rQuery = rQueryBuilder.toString(); - /* Request graph from R object generator, which either returns it from * its cache or asks Rserve to generate it. */ - byte[] graphBytes = rObjectGenerator.generateGraph(rQuery, - imageFilename); + byte[] graphBytes = rObjectGenerator.generateGraph(requestedGraph, + request.getParameterMap()); /* Make sure that we have a graph to return. */ if (graphBytes == null) { @@ -100,7 +63,7 @@ public class GraphImageServlet extends HttpServlet { response.setHeader("Content-Length", String.valueOf(graphBytes.length)); response.setHeader("Content-Disposition", - "inline; filename=\"" + imageFilename + "\""); + "inline; filename=\"" + requestedGraph + ".png\""); output = new BufferedOutputStream(response.getOutputStream(), 1024); output.write(graphBytes, 0, graphBytes.length); output.flush(); diff --git a/src/org/torproject/ernie/web/GraphsSubpagesServlet.java b/src/org/torproject/ernie/web/GraphsSubpagesServlet.java index 3e090eb..5c2743d 100644 --- a/src/org/torproject/ernie/web/GraphsSubpagesServlet.java +++ b/src/org/torproject/ernie/web/GraphsSubpagesServlet.java @@ -126,54 +126,15 @@ public class GraphsSubpagesServlet extends HttpServlet { } } - /* Trigger generation of table data if the graphs subpage has any - * tables, regardless of whether a table update was requested. */ + /* Generate table data if the graphs subpage has any tables, + * regardless of whether a table update was requested, and add the + * table data as request attribute. */ if (this.availableGraphsSubpageTables.containsKey(requestedPage)) { for (String tableName : this.availableGraphsSubpageTables.get(requestedPage)) { - - Map<String, String[]> checkedParameters = null; - if (tableName.equals(requestedTable)) { - checkedParameters = TableParameterChecker. - getInstance().checkParameters(requestedTable, - request.getParameterMap()); - } else { - checkedParameters = TableParameterChecker. - getInstance().checkParameters(tableName, null); - } - - /* Prepare filename and R query string. */ - StringBuilder rQueryBuilder = new StringBuilder("write_" - + tableName.replaceAll("-", "_") + "("), - tableFilenameBuilder = new StringBuilder(tableName); - - for (Map.Entry<String, String[]> parameter : - checkedParameters.entrySet()) { - String parameterName = parameter.getKey(); - String[] parameterValues = parameter.getValue(); - for (String param : parameterValues) { - tableFilenameBuilder.append("-" + param); - } - if (parameterValues.length < 2) { - rQueryBuilder.append(parameterName + " = '" - + parameterValues[0] + "', "); - } else { - rQueryBuilder.append(parameterName + " = c("); - for (int i = 0; i < parameterValues.length - 1; i++) { - rQueryBuilder.append("'" + parameterValues[i] + "', "); - } - rQueryBuilder.append("'" + parameterValues[ - parameterValues.length - 1] + "'), "); - } - } - tableFilenameBuilder.append(".tbl"); - String tableFilename = tableFilenameBuilder.toString(); - rQueryBuilder.append("path = '%s')"); - String rQuery = rQueryBuilder.toString(); - - /* Generate table data and add it as request attribute. */ List<Map<String, String>> tableData = rObjectGenerator. - generateTable(rQuery, tableFilename); + generateTable(tableName, requestedTable, + request.getParameterMap()); request.setAttribute(tableName.replaceAll("-", "_") + "_tabledata", tableData); } diff --git a/src/org/torproject/ernie/web/RObjectGenerator.java b/src/org/torproject/ernie/web/RObjectGenerator.java index 97635a2..2fb4477 100644 --- a/src/org/torproject/ernie/web/RObjectGenerator.java +++ b/src/org/torproject/ernie/web/RObjectGenerator.java @@ -18,6 +18,7 @@ import java.util.Map; import javax.servlet.ServletContext; import javax.servlet.ServletContextEvent; import javax.servlet.ServletContextListener; +import javax.servlet.http.HttpServletResponse; import org.rosuda.REngine.Rserve.RConnection; import org.rosuda.REngine.Rserve.RserveException; @@ -52,9 +53,46 @@ public class RObjectGenerator implements ServletContextListener { /* Nothing to do. */ } + public byte[] generateGraph(String requestedGraph, Map parameterMap) { + Map<String, String[]> checkedParameters = GraphParameterChecker. + getInstance().checkParameters(requestedGraph, parameterMap); + if (checkedParameters == null) { + /* TODO We're going to take the blame by sending an internal server + * error to the client, but really the user is to blame. */ + return null; + } + StringBuilder rQueryBuilder = new StringBuilder("plot_" + + requestedGraph.replaceAll("-", "_") + "("), + imageFilenameBuilder = new StringBuilder(requestedGraph); + for (Map.Entry<String, String[]> parameter : + checkedParameters.entrySet()) { + String parameterName = parameter.getKey(); + String[] parameterValues = parameter.getValue(); + for (String param : parameterValues) { + imageFilenameBuilder.append("-" + param); + } + if (parameterValues.length < 2) { + rQueryBuilder.append(parameterName + " = '" + parameterValues[0] + + "', "); + } else { + rQueryBuilder.append(parameterName + " = c("); + for (int i = 0; i < parameterValues.length - 1; i++) { + rQueryBuilder.append("'" + parameterValues[i] + "', "); + } + rQueryBuilder.append("'" + parameterValues[ + parameterValues.length - 1] + "'), "); + } + } + imageFilenameBuilder.append(".png"); + String imageFilename = imageFilenameBuilder.toString(); + rQueryBuilder.append("path = '%s')"); + String rQuery = rQueryBuilder.toString(); + return this.generateGraph(rQuery, imageFilename); + } + /* Generate a graph using the given R query that has a placeholder for * the absolute path to the image to be created. */ - public byte[] generateGraph(String rQuery, String imageFilename) { + private byte[] generateGraph(String rQuery, String imageFilename) { /* See if we need to generate this graph. */ File imageFile = new File(this.cachedGraphsDirectory + "/" @@ -103,9 +141,17 @@ public class RObjectGenerator implements ServletContextListener { return result; } + public String generateCsv(String requestedCsvFile) { + /* Prepare filename and R query string. */ + String rQuery = "export_" + requestedCsvFile.replaceAll("-", "_") + + "(path = '%s')"; + String csvFilename = requestedCsvFile + ".csv"; + return this.generateCsv(rQuery, csvFilename); + } + /* Generate a comma-separated value file using the given R query that * has a placeholder for the absolute path to the file to be created. */ - public String generateCsv(String rQuery, String csvFilename) { + private String generateCsv(String rQuery, String csvFilename) { /* See if we need to generate this .csv file. */ File csvFile = new File(this.cachedGraphsDirectory + "/" @@ -152,10 +198,57 @@ public class RObjectGenerator implements ServletContextListener { return result; } + public List<Map<String, String>> generateTable(String tableName, + String requestedTable, Map parameterMap) { + + Map<String, String[]> checkedParameters = null; + if (tableName.equals(requestedTable)) { + checkedParameters = TableParameterChecker. + getInstance().checkParameters(requestedTable, + parameterMap); + } else { + checkedParameters = TableParameterChecker. + getInstance().checkParameters(tableName, null); + } + if (checkedParameters == null) { + /* TODO We're going to take the blame by sending an internal server + * error to the client, but really the user is to blame. */ + return null; + } + StringBuilder rQueryBuilder = new StringBuilder("write_" + + tableName.replaceAll("-", "_") + "("), + tableFilenameBuilder = new StringBuilder(tableName); + + for (Map.Entry<String, String[]> parameter : + checkedParameters.entrySet()) { + String parameterName = parameter.getKey(); + String[] parameterValues = parameter.getValue(); + for (String param : parameterValues) { + tableFilenameBuilder.append("-" + param); + } + if (parameterValues.length < 2) { + rQueryBuilder.append(parameterName + " = '" + + parameterValues[0] + "', "); + } else { + rQueryBuilder.append(parameterName + " = c("); + for (int i = 0; i < parameterValues.length - 1; i++) { + rQueryBuilder.append("'" + parameterValues[i] + "', "); + } + rQueryBuilder.append("'" + parameterValues[ + parameterValues.length - 1] + "'), "); + } + } + tableFilenameBuilder.append(".tbl"); + String tableFilename = tableFilenameBuilder.toString(); + rQueryBuilder.append("path = '%s')"); + String rQuery = rQueryBuilder.toString(); + return this.generateTable(rQuery, tableFilename); + } + /* Generate table data using the given R query and filename or read * previously generated table data from disk if it's not too old and * return table data. */ - public List<Map<String, String>> generateTable(String rQuery, + private List<Map<String, String>> generateTable(String rQuery, String tableFilename) { /* See if we need to generate this table. */

1 0

[vidalia/alpha] Warn at first run if panic is enabled
by chiiph＠torproject.org 20 Mar '12

20 Mar '12

commit ffe98b13072e0b7a4f831ccdc50e32b088c8ae21 Author: Tomás Touceda <chiiph(a)torproject.org> Date: Sun Mar 18 15:38:49 2012 -0300 Warn at first run if panic is enabled --- src/vidalia/MainWindow.cpp | 12 +++++++++++- src/vidalia/config/VidaliaSettings.cpp | 14 ++++++++++++++ src/vidalia/config/VidaliaSettings.h | 5 +++++ 3 files changed, 30 insertions(+), 1 deletions(-) diff --git a/src/vidalia/MainWindow.cpp b/src/vidalia/MainWindow.cpp index c568c80..20959fc 100644 --- a/src/vidalia/MainWindow.cpp +++ b/src/vidalia/MainWindow.cpp @@ -124,8 +124,9 @@ MainWindow::MainWindow() _isVidaliaRunningTor = false; updateTorStatus(Stopped); -#if defined(Q_WS_MAC) VidaliaSettings settings; + +#if defined(Q_WS_MAC) /* Display OSX dock icon if icon preference is not set to "Tray Only" */ if (settings.getIconPref() != VidaliaSettings::Tray) { ProcessSerialNumber psn = { 0, kCurrentProcess }; @@ -134,6 +135,15 @@ MainWindow::MainWindow() /* Vidalia launched in background (LSUIElement=true). Bring to foreground. */ VidaliaWindow::setVisible(settings.showMainWindowAtStart()); #endif + + if(settings.firstRun()) { + if(settings.allowPanic()) + VMessageBox::warning(this, tr("Panic is enabled"), + tr("<b>WARNING:</b> The Panic button is enabled. Use " + "it carefully because it will remove Tor completely."), + VMessageBox::Ok|VMessageBox::Default); + settings.setFirstRun(false); + } } /** Destructor */ diff --git a/src/vidalia/config/VidaliaSettings.cpp b/src/vidalia/config/VidaliaSettings.cpp index 913b0b1..41f0a32 100644 --- a/src/vidalia/config/VidaliaSettings.cpp +++ b/src/vidalia/config/VidaliaSettings.cpp @@ -40,6 +40,7 @@ #define SETTING_SKIP_VERSION_CHECK "SkipVersionCheck" #define SETTING_ALLOW_PANIC "AllowPanic" #define SETTING_PANIC_PATH "PanicPath" +#define SETTING_FIRST_RUN "FirstRun" #if defined(Q_OS_WIN32) #define STARTUP_REG_KEY "Software\\Microsoft\\Windows\\CurrentVersion\\Run" @@ -88,6 +89,7 @@ VidaliaSettings::VidaliaSettings() setDefault(SETTING_REMEMBER_SHUTDOWN, false); setDefault(SETTING_ALLOW_PANIC, false); setDefault(SETTING_PANIC_PATH, ""); + setDefault(SETTING_FIRST_RUN, true); } /** Gets the currently preferred language code for Vidalia. */ @@ -366,3 +368,15 @@ VidaliaSettings::setPanicPath(const QString &path) { setValue(SETTING_PANIC_PATH, path); } + +bool +VidaliaSettings::firstRun() const +{ + return value(SETTING_FIRST_RUN).toBool(); +} + +void +VidaliaSettings::setFirstRun(bool val) +{ + setValue(SETTING_FIRST_RUN, val); +} diff --git a/src/vidalia/config/VidaliaSettings.h b/src/vidalia/config/VidaliaSettings.h index 6f5efb1..d088ac0 100644 --- a/src/vidalia/config/VidaliaSettings.h +++ b/src/vidalia/config/VidaliaSettings.h @@ -146,6 +146,11 @@ public: QString panicPath() const; /** Sets the path for panic */ void setPanicPath(const QString &path); + + /** Returns true if it's Vidalia's first run */ + bool firstRun() const; + /** Sets Vidalia's first run option */ + void setFirstRun(bool val); }; #endif

1 0