- tor-commits - lists.torproject.org

[torspec/master] Add proposal 182 ("Credit Bucket") by Tschorsch and Scheuermann
by nickm＠torproject.org 27 Jun '11

27 Jun '11

commit 1413093be9d575a4b0805e0328e2867319a543ae Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Jun 27 15:58:45 2011 -0400 Add proposal 182 ("Credit Bucket") by Tschorsch and Scheuermann --- proposals/000-index.txt | 2 + proposals/182-creditbucket.txt | 181 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 183 insertions(+), 0 deletions(-) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 06e97b5..43502ec 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -102,6 +102,7 @@ Proposals by number: 179 TLS certificate and parameter normalization [DRAFT] 180 Pluggable transports for circumvention [OPEN] 181 Optimistic Data for Tor: Client Side [OPEN] +182 Credit Bucket [DRAFT] Proposals by status: @@ -114,6 +115,7 @@ Proposals by status: 144 Increase the diversity of circuits by detecting nodes belonging the same provider 175 Automatically promoting Tor clients to nodes 179 TLS certificate and parameter normalization [for 0.2.3.x] + 182 Credit Bucket NEEDS-REVISION: 118 Advertising multiple ORPorts at once 131 Help users to verify they are using Tor diff --git a/proposals/182-creditbucket.txt b/proposals/182-creditbucket.txt new file mode 100644 index 0000000..8796bd1 --- /dev/null +++ b/proposals/182-creditbucket.txt @@ -0,0 +1,181 @@ +Filename: 182-creditbucket.txt +Title: Credit Bucket +Author: Florian Tschorsch and Björn Scheuermann +Created: 22 Jun 2011 +Status: Draft + +Overview: + + The following proposal targets the reduction of queuing times in onion + routers. In particular, we focus on the token bucket algorithm in Tor and + point out that current usage unnecessarily locks cells for long time spans. + We propose a non-intrusive change in Tor's design which overcomes the + deficiencies. + +Motivation and Background: + + Cell statistics from the Tor network [1] reveal that cells reside in + individual onion routers' cell queues for up to several seconds. These + queuing times increase the end-to-end delay very significantly and are + apparently the largest contributor to overall cell latency in Tor. + + In Tor there exist multiple token buckets on different logical levels. They + all work independently. They are used to limit the up- and downstream of an + onion router. All token buckets are refilled every second with a constant + amount of tokens that depends on the configured bandwidth limits. For + example, the so-called RelayedTokenBucket limits relay traffic only. All + read data of incoming connections are bound to a dedicated read token + bucket. An analogous mechanism exists for written data leaving the onion + router. We were able to identify the specific usage and implementation of + the token bucket algorithm as one cause for very high (and unnecessary) + queuing times in an onion router. + + We observe that the token buckets in Tor are (surprisingly at a first + glance) allowed to take on negative fill levels. This is justified by the + TLS connections between onion routers where whole TLS records need to be + processed. The token bucket on the incoming side (i.e., the one which + determines at which rate it is allowed to read from incoming TCP + connections) in particular often runs into non-negligible negative fill + levels. As a consequence of this behavior, sometimes slightly more data is + read than it would be admissible upon strict interpretation of the token + bucket concept. + + However, the token bucket for limiting the outgoing rate does not take on + negative fill levels equally often. Consequently, it regularly happens + that somewhat more data are read on the incoming side than the outgoing + token bucket allows to be written during the same cycle, even if their + configured data rates are the same. The respective cells will thus not be + allowed to leave the onion router immediately. They will thus necessarily + be queued for at least as long as it takes until the token bucket on the + outgoing side is refilled again. The refill interval currently is, as + mentioned before, one second -- so, these cells are delayed for a very + substantial time. In summary, one could say that the two buckets, on the + incoming and outgoing side, work like a double door system and frequently + lock cells for a full token bucket refill interval length. + +General Design: + + In order to overcome the described problem, we propose the following + changes related to the token bucket algorithm. + + We observe that the token bucket on the outgoing connections with its + current design is contra productive in the sense of queuing times. We + therefore propose modifications to the token bucket algorithm that will + eliminate the "double door effect" discussed above. + + Let us start from Tor's current approach: Thus, we have a regular token + bucket on the reading side with a certain rate and a certain burst size. + Let x denote the current amount of tokens in the bucket. On the outgoing + side we need something appropriate that monitors and constrains the + outgoing rate, but at the same time avoids holding back cells (cf. double + door effects) whenever possible. + + Here we propose something that adopts the role of a token bucket, but + realizes this functionality in a slightly different way. We call it a + "credit bucket". Like a token bucket, the credit bucket also has a current + fill level, denoted by y. However, the credit bucket is refilled in a + different way. + + To understand how it works, let us look at the possible operations: + + As said, x is the fill level of a regular token bucket on the incoming + side and thus gets incremented periodically according to the configured + rate. No changes here. + + If x<=0, we are obviously not allowed to read. If x>0, we are allowed to + read up to x bytes of incoming data. If k bytes are read (k<=x), then we + update x and y as follows: + + x = x - k (1) + y = y + k (2) + + (1) is the standard token bucket operation on the incoming side. Whenever + data is admitted in, though, an additional operation is performed: (2) + allocates the same number of bytes on the outgoing side, which will later + on allow the same number of bytes to leave the onion router without any + delays. + + If y + x > -M, we are allowed to write up to y + x + M bytes on the + outgoing side, where M is a positive constant. M specifies a burst size for + the outgoing side. M should be higher than the number of tokens that get + refilled during a refill interval, we would suggest to have M in the order + of a few seconds "worth" of data. Now if k bytes are written on the + outgoing side, we proceed as follows: + + If k <= y then y = y - k + + In this case we use "saved" credits, previously allocated on the incoming + side when incoming data has been processed. + + If k > y then y = 0 and x = x - (k-y) + + We generated additional traffic in the onion router, so that more data is + to be sent than has been read (the credit is not sufficient). We therefore + "steal" tokens from the token buffer on the incoming side to compensate for + the additionally generated data. This will result in correspondingly less + data being read on the incoming side subsequently. As a result of such an + operation, the token bucket fill level x on the incoming side may become + negative (but it can never fall below -M). + + If y + x <= -M then outgoing data will be held back. This may lead to + double-door effects, but only in extreme cases where the outgoing traffic + largely exceeds the incoming traffic, so that the outgoing bursts size M is + exceeded. + + Aside from short-term bursts of configurable size (as with every token + bucket), this procedure guarantees that the configured rate may never be + exceeded (on the application layer, that is; as with the current + implementation, an attacker may easily cause the onion router to + arbitrarily exceed the limits on the lower layers). Over time, we never + send more data than the configured rate: every sent byte needs a + corresponding token on the incoming side; this token must either have been + consumed by an incoming byte before (it then became a "credit"), or it is + "stolen" from the incoming bucket to compensate for data generated within + the onion router. + +Specific Design Changes: + + In the following we briefly point out the specific changes that need to be + done in Tor's source code. By doing so one can see how non intrusive our + modifications are. + + First we need to address the bucket increment and decrement operations. + According to the described logic above, this should be done in the methods + connection_bucket_refill and connection_buckets_decrement respectively. In + particular allocating, saving and "stealing" of tokens need to be + considered here. + + Second the rate limiting, i.e. the amount we are allowed to write + (connection_bucket_write_limit) needs to be adapted in lines of the credit + bucket logic. Meaning in order to avoid the here identified unnecessary + queuing of cells, we need to consider the new burst parameter M. Here we + also need to take non rate limited connections such as from the localhost + into account. The rate limiting on the reading side remains the same. + + At last we need to find good values/ ratios for the parameter M such that + the trade off between avoiding "double door effects" and maintaining + strict rate limits work as expected. As future work and after insights + about the performance gain of the here described proposal we need to find a + way to implement this both using bufferevent rate limiting with libevent + 2.3.x and Tor's rate limiting code. + +Conclusion: + + This proposal can be implemented with moderate effort and requires changes + only at the points where currently the token bucket operations are + performed. + + We feel that this is not the be-all and end-all solution, because it again + introduces a feedback loop between the incoming and the outgoing side. We + therefore still hope that we will be able to come to a both simpler and + more effective design in the future. However, we believe that what we + proposed here is a good compromise between avoiding double-door effects to + the furthest possible extent, strictly enforcing an application-layer data + rate, and keeping the extent of changes to the code small. + + Feedback is highly appreciated. + +References: + + [1] Karsten Loesing. Analysis of Circuit Queues in Tor. August 25, 2009. + [2] https://trac.torproject.org/projects/tor/wiki/sponsors/SponsorD/June2011

1 0

[torflow/master] Merge branch 'ratio-restrictions'
by mikeperry＠torproject.org 27 Jun '11

27 Jun '11

commit 039d5300c6d1bb9adfa7a21af0560620d620978f Merge: 07cbb7c a3fdd01 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Mon Jun 27 11:22:47 2011 -0700 Merge branch 'ratio-restrictions' NetworkScanners/BwAuthority/bwauthority_child.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)

1 0

[torflow/master] Update bwauth child to use new ratio slicing.
by mikeperry＠torproject.org 27 Jun '11

27 Jun '11

commit a3fdd013eedb3c8beb6bfb4dbffc09b80fb68c43 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Fri Jun 24 13:30:44 2011 -0700 Update bwauth child to use new ratio slicing. --- NetworkScanners/BwAuthority/bwauthority_child.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/NetworkScanners/BwAuthority/bwauthority_child.py b/NetworkScanners/BwAuthority/bwauthority_child.py index c342f2e..f842843 100755 --- a/NetworkScanners/BwAuthority/bwauthority_child.py +++ b/NetworkScanners/BwAuthority/bwauthority_child.py @@ -73,7 +73,8 @@ __selmgr = PathSupport.SelectionManager( uniform=True, use_exit=None, use_guards=False, - exit_ports=[443]) + exit_ports=[443], + order_by_ratio=True) # exit code to indicate scan completion # make sure to update this in bwauthority.py as well

1 0

[torflow/master] Submodule.
by mikeperry＠torproject.org 27 Jun '11

27 Jun '11

commit a28419da99ef8bb5ec2f51ad7109784d93530721 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Mon Jun 27 11:26:18 2011 -0700 Submodule. --- TorCtl | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/TorCtl b/TorCtl index 962d30c..25cb2bf 160000 --- a/TorCtl +++ b/TorCtl @@ -1 +1 @@ -Subproject commit 962d30c72ffc85c1df79270d6ac5b25f0336d2d2 +Subproject commit 25cb2bf77b1a98f2273b0f90c99b57e3d6e21a41

1 0

[pytorctl/master] Implement ratio ranking support.
by mikeperry＠torproject.org 27 Jun '11

27 Jun '11

commit e538547dfc2973d7bd99e2d927a1d87be5230704 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Fri Jun 24 13:32:37 2011 -0700 Implement ratio ranking support. --- PathSupport.py | 36 ++++++++++++++++++++++++++++++++---- TorCtl.py | 16 ++++++++++++++++ 2 files changed, 48 insertions(+), 4 deletions(-) diff --git a/PathSupport.py b/PathSupport.py index 4e9ad6d..d4ffbdd 100644 --- a/PathSupport.py +++ b/PathSupport.py @@ -229,6 +229,27 @@ class PercentileRestriction(NodeRestriction): def __str__(self): return self.__class__.__name__+"("+str(self.pct_skip)+","+str(self.pct_fast)+")" +class RatioPercentileRestriction(NodeRestriction): + """Restriction to cut out a percentile slice of the network by ratio of + consensus bw to descriptor bw.""" + def __init__(self, pct_skip, pct_fast, r_list): + """Constructor. Sets up the restriction such that routers in the + 'pct_skip' to 'pct_fast' percentile of bandwidth rankings are + returned from the sorted list 'r_list'""" + self.pct_fast = pct_fast + self.pct_skip = pct_skip + self.sorted_r = r_list + + def r_is_ok(self, r): + "Returns true if r is in the percentile boundaries (by rank)" + if r.ratio_rank < len(self.sorted_r)*self.pct_skip/100: return False + elif r.ratio_rank > len(self.sorted_r)*self.pct_fast/100: return False + + return True + + def __str__(self): + return self.__class__.__name__+"("+str(self.pct_skip)+","+str(self.pct_fast)+")" + class UptimeRestriction(NodeRestriction): """Restriction to filter out routers with uptimes < min_uptime or > max_uptime""" @@ -1008,7 +1029,8 @@ class SelectionManager(BaseSelectionManager): def __init__(self, pathlen, order_exits, percent_fast, percent_skip, min_bw, use_all_exits, uniform, use_exit, use_guards,geoip_config=None, - restrict_guards=False, extra_node_rstr=None, exit_ports=None): + restrict_guards=False, extra_node_rstr=None, exit_ports=None, + order_by_ratio=False): BaseSelectionManager.__init__(self) self.__ordered_exit_gen = None self.pathlen = pathlen @@ -1026,6 +1048,7 @@ class SelectionManager(BaseSelectionManager): self.consensus = None self.exit_ports = exit_ports self.extra_node_rstr=extra_node_rstr + self.order_by_ratio = order_by_ratio def reconfigure(self, consensus=None): try: @@ -1063,17 +1086,22 @@ class SelectionManager(BaseSelectionManager): nonentry_skip = self.percent_skip nonentry_fast = self.percent_fast + if self.order_by_ratio: + PctRstr = RatioPercentileRestriction + else: + PctRstr = PercentileRestriction + # XXX: sometimes we want the ability to do uniform scans # without the conserve exit restrictions.. entry_rstr = NodeRestrictionList( - [PercentileRestriction(self.percent_skip, self.percent_fast, sorted_r), + [PctRstr(self.percent_skip, self.percent_fast, sorted_r), OrNodeRestriction( [FlagsRestriction(["BadExit"]), ConserveExitsRestriction(self.exit_ports)]), FlagsRestriction(entry_flags, [])] ) mid_rstr = NodeRestrictionList( - [PercentileRestriction(nonentry_skip, nonentry_fast, sorted_r), + [PctRstr(nonentry_skip, nonentry_fast, sorted_r), OrNodeRestriction( [FlagsRestriction(["BadExit"]), ConserveExitsRestriction(self.exit_ports)]), @@ -1090,7 +1118,7 @@ class SelectionManager(BaseSelectionManager): [FlagsRestriction(["Running","Fast"], ["BadExit"])]) else: self.exit_rstr = NodeRestrictionList( - [PercentileRestriction(nonentry_skip, nonentry_fast, sorted_r), + [PctRstr(nonentry_skip, nonentry_fast, sorted_r), FlagsRestriction(["Running","Fast"], ["BadExit"])]) if self.extra_node_rstr: diff --git a/TorCtl.py b/TorCtl.py index ac49574..f0cf5c5 100755 --- a/TorCtl.py +++ b/TorCtl.py @@ -460,6 +460,7 @@ class Router: self.version = RouterVersion(version) self.os = os self.list_rank = 0 # position in a sorted list of routers. + self.ratio_rank = 0 # position in a ratio-sorted list of routers self.uptime = uptime self.published = published self.refcount = 0 # How many open circs are we currently in? @@ -1709,6 +1710,11 @@ class ConsensusTracker(EventHandler): self.sorted_r.sort(lambda x, y: cmp(y.bw, x.bw)) for i in xrange(len(self.sorted_r)): self.sorted_r[i].list_rank = i + ratio_r = copy.copy(self.sorted_r) + ratio_r.sort(lambda x, y: cmp(float(y.bw)/y.desc_bw, + float(x.bw)/x.desc_bw)) + for i in xrange(len(ratio_r)): ratio_r[i].ratio_rank = i + # XXX: Verification only. Can be removed. self._sanity_check(self.sorted_r) @@ -1792,6 +1798,11 @@ class ConsensusTracker(EventHandler): self.sorted_r = filter(lambda r: not r.down, self.routers.itervalues()) self.sorted_r.sort(lambda x, y: cmp(y.bw, x.bw)) for i in xrange(len(self.sorted_r)): self.sorted_r[i].list_rank = i + + ratio_r = copy.copy(self.sorted_r) + ratio_r.sort(lambda x, y: cmp(float(y.bw)/y.desc_bw, + float(x.bw)/x.desc_bw)) + for i in xrange(len(ratio_r)): ratio_r[i].ratio_rank = i plog("DEBUG", str(time.time()-d.arrived_at)+ " Read " + str(len(d.idlist)) +" ND => "+str(len(self.sorted_r))+" routers. Update: "+str(update)) # XXX: Verification only. Can be removed. @@ -1821,6 +1832,11 @@ class ConsensusTracker(EventHandler): self.sorted_r = filter(lambda r: not r.down, self.routers.itervalues()) self.sorted_r.sort(lambda x, y: cmp(y.bw, x.bw)) for i in xrange(len(self.sorted_r)): self.sorted_r[i].list_rank = i + + ratio_r = copy.copy(self.sorted_r) + ratio_r.sort(lambda x, y: cmp(float(y.bw)/y.desc_bw, + float(x.bw)/x.desc_bw)) + for i in xrange(len(ratio_r)): ratio_r[i].ratio_rank = i self._sanity_check(self.sorted_r) def current_consensus(self):

1 0

[pytorctl/master] Merge branch 'ratio-restrictions'
by mikeperry＠torproject.org 27 Jun '11

27 Jun '11

commit 25cb2bf77b1a98f2273b0f90c99b57e3d6e21a41 Merge: 962d30c e538547 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Mon Jun 27 11:23:39 2011 -0700 Merge branch 'ratio-restrictions' PathSupport.py | 36 ++++++++++++++++++++++++++++++++---- TorCtl.py | 16 ++++++++++++++++ 2 files changed, 48 insertions(+), 4 deletions(-)

1 0

[arm/master] Expanding config screen to allow arbitrary options
by atagar＠torproject.org 27 Jun '11

27 Jun '11

commit 83d1e367a003ec84b54527d470dde01901da6961 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jun 26 14:04:35 2011 -0700 Expanding config screen to allow arbitrary options Preping the configuration screen of the wizard to allow for other sets of options for different relay types. --- src/cli/wizard.py | 64 +++++++++++++++++++++++++++++++++++----------------- 1 files changed, 43 insertions(+), 21 deletions(-) diff --git a/src/cli/wizard.py b/src/cli/wizard.py index a3404b9..dadea16 100644 --- a/src/cli/wizard.py +++ b/src/cli/wizard.py @@ -14,12 +14,20 @@ from util import enum, uiTools RelayType = enum.Enum("RELAY", "EXIT", "BRIDGE", "CLIENT") # all options that can be configured -Options = enum.Enum("NICKNAME", "CONTACT", "NOTIFY", "BANDWIDTH", "LIMIT", "STARTUP") -RelayOptions = (Options.NICKNAME, Options.CONTACT, Options.NOTIFY, Options.BANDWIDTH, Options.LIMIT, Options.STARTUP) +Options = enum.Enum("DIVIDER", "NICKNAME", "CONTACT", "NOTIFY", "BANDWIDTH", "LIMIT", "STARTUP") +RelayOptions = {RelayType.RELAY: (Options.NICKNAME, + Options.CONTACT, + Options.NOTIFY, + Options.BANDWIDTH, + Options.DIVIDER, + Options.DIVIDER, + Options.LIMIT, + Options.STARTUP)} # other options provided in the prompts CANCEL, NEXT, BACK = "Cancel", "Next", "Back" +DESC_SIZE = 5 # height of the description field MSG_COLOR = "green" OPTION_COLOR = "yellow" @@ -97,6 +105,10 @@ def showWizard(): relayType, config = None, {} for option in Options.values(): + if option == Options.DIVIDER: + config[option] = option + continue + toggleValues = CONFIG["wizard.toggle"].get(option) default = CONFIG["wizard.default"].get(option, "") @@ -116,13 +128,10 @@ def showWizard(): if selection == CANCEL: break else: relayType = selection else: - if relayType == RelayType.RELAY: - selection = promptRelayOptions(config) - - if selection == BACK: relayType = None - elif selection == NEXT: break # TODO: implement next screen - else: - break # TODO: other catagories not yet implemented + selection = promptConfigOptions(relayType, config) + + if selection == BACK: relayType = None + elif selection == NEXT: break # TODO: implement next screen # redraws screen to clear away the dialog we just showed cli.controller.getController().requestRedraw(True) @@ -134,7 +143,7 @@ def promptRelayType(): dialog was canceled. """ - popup, _, _ = cli.popups.init(24, 58) + popup, _, _ = cli.popups.init(25, 58) if not popup: return control = cli.controller.getController() key, selection = 0, 0 @@ -181,17 +190,25 @@ def promptRelayType(): finally: cli.popups.finalize() -def promptRelayOptions(config): +def promptConfigOptions(relayType, config): """ Prompts the user for the configuration of an internal relay. """ - popup, _, _ = cli.popups.init(23, 58) - if not popup: return - control = cli.controller.getController() - options = [config[opt] for opt in RelayOptions] + # TODO: skipping section if it isn't ready yet + if not relayType in RelayOptions: return NEXT + + topContent = _splitStr(CONFIG.get("wizard.message.%s" % relayType.lower(), ""), 54) + + options = [config[opt] for opt in RelayOptions[relayType]] + options.append(Options.DIVIDER) options.append(ConfigOption(BACK, "general", "(to role selection)")) options.append(ConfigOption(NEXT, "general", "(to confirm options)")) + + popupHeight = len(topContent) + len(options) + DESC_SIZE + 5 + popup, _, _ = cli.popups.init(popupHeight, 58) + if not popup: return + control = cli.controller.getController() key, selection = 0, 0 try: @@ -202,21 +219,21 @@ def promptRelayOptions(config): popup.win.box() # provides the description for internal relays - topContent = _splitStr(CONFIG["wizard.message.relay"], 54) for i in range(len(topContent)): popup.addstr(i + 1, 2, topContent[i], curses.A_BOLD | uiTools.getColor(MSG_COLOR)) y, offset = len(topContent) + 1, 0 for i in range(len(options)): + if options[i] == Options.DIVIDER: + offset += 1 + continue + label = " %-30s%s" % (options[i].getLabel(), options[i].getDisplayValue()) optionFormat = curses.A_BOLD | uiTools.getColor(OPTION_COLOR) if i == selection: optionFormat |= curses.A_STANDOUT offset += 1 popup.addstr(y + offset, 2, uiTools.padStr(label, 54), optionFormat) - - # extra space to divide options/navigation - if i == len(options) - 3: offset += 1 # divider between the options and description offset += 2 @@ -232,8 +249,13 @@ def promptRelayOptions(config): popup.win.refresh() key = control.getScreen().getch() - if key == curses.KEY_UP: selection = (selection - 1) % len(options) - elif key == curses.KEY_DOWN: selection = (selection + 1) % len(options) + if key in (curses.KEY_UP, curses.KEY_DOWN): + posOffset = -1 if key == curses.KEY_UP else 1 + selection = (selection + posOffset) % len(options) + + # skips dividers + while options[selection] == Options.DIVIDER: + selection = (selection + posOffset) % len(options) elif uiTools.isSelectionKey(key): if selection == len(options) - 2: return BACK # selected back elif selection == len(options) - 1: return NEXT # selected next

1 0

[arm/master] Wizard config for setting up an exit
by atagar＠torproject.org 27 Jun '11

27 Jun '11

commit e8c3836e203a3e359a9b5b9a289dc73ddc3da92c Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Jun 27 08:54:07 2011 -0700 Wizard config for setting up an exit This expands the wizard to provide a config panel for setting up an exit relay. This has the same options as an internal relay plus several for DirPortFrontPage and ExitPolicy. By default this will use the reduced exit policy [1], with options like Vidalia for selecting groups of services to allow. However, unlike Vidalia it won't allow the operator to pick plaintext ports without also having their encryped counterparts. [1] https://trac.torproject.org/projects/tor/wiki/doc/ReducedExitPolicy --- src/cli/wizard.py | 155 ++++++++++++++++++++++++++++++++++++++++++++-------- src/settings.cfg | 37 ++++++++++++- 2 files changed, 167 insertions(+), 25 deletions(-) diff --git a/src/cli/wizard.py b/src/cli/wizard.py index dadea16..0d47de5 100644 --- a/src/cli/wizard.py +++ b/src/cli/wizard.py @@ -3,6 +3,7 @@ Provides user prompts for setting up a new relay. This autogenerates a torrc that's used by arm to run its own tor instance. """ +import functools import curses import cli.popups @@ -14,15 +15,30 @@ from util import enum, uiTools RelayType = enum.Enum("RELAY", "EXIT", "BRIDGE", "CLIENT") # all options that can be configured -Options = enum.Enum("DIVIDER", "NICKNAME", "CONTACT", "NOTIFY", "BANDWIDTH", "LIMIT", "STARTUP") +Options = enum.Enum("DIVIDER", "NICKNAME", "CONTACT", "NOTIFY", "BANDWIDTH", "LIMIT", "STARTUP", "NOTICE", "POLICY", "WEBSITES", "EMAIL", "IM", "MISC", "PLAINTEXT") RelayOptions = {RelayType.RELAY: (Options.NICKNAME, Options.CONTACT, Options.NOTIFY, Options.BANDWIDTH, - Options.DIVIDER, - Options.DIVIDER, Options.LIMIT, - Options.STARTUP)} + Options.STARTUP), + RelayType.EXIT: (Options.NICKNAME, + Options.CONTACT, + Options.NOTIFY, + Options.BANDWIDTH, + Options.LIMIT, + Options.STARTUP, + Options.DIVIDER, + Options.NOTICE, + Options.POLICY, + Options.WEBSITES, + Options.EMAIL, + Options.IM, + Options.MISC, + Options.PLAINTEXT)} + +# custom exit policy options +CUSTOM_POLICIES = (Options.WEBSITES, Options.EMAIL, Options.IM, Options.MISC, Options.PLAINTEXT) # other options provided in the prompts CANCEL, NEXT, BACK = "Cancel", "Next", "Back" @@ -30,10 +46,13 @@ CANCEL, NEXT, BACK = "Cancel", "Next", "Back" DESC_SIZE = 5 # height of the description field MSG_COLOR = "green" OPTION_COLOR = "yellow" +DISABLED_COLOR = "cyan" CONFIG = {"wizard.message.role": "", "wizard.message.relay": "", + "wizard.message.exit": "", "wizard.toggle": {}, + "wizard.suboptions": [], "wizard.default": {}, "wizard.label.general": {}, "wizard.label.role": {}, @@ -65,6 +84,8 @@ class ConfigOption: self.descriptionCache = None self.descriptionCacheArg = None self.value = default + self.validator = None + self._isEnabled = True def getKey(self): return self.key @@ -75,7 +96,39 @@ class ConfigOption: def getDisplayValue(self): return self.value + def getDisplayAttr(self): + myColor = OPTION_COLOR if self.isEnabled() else DISABLED_COLOR + return curses.A_BOLD | uiTools.getColor(myColor) + + def isEnabled(self): + return self._isEnabled + + def setEnabled(self, isEnabled): + self._isEnabled = isEnabled + + def setValidator(self, validator): + """ + Custom function used to check that a value is valid before setting it. + This functor should accept two arguments: this option and the value we're + attempting to set. If its invalid then a ValueError with the reason is + expected. + + Arguments: + validator - functor for checking the validitiy of values we set + """ + + self.validator = validator + def setValue(self, value): + """ + Attempts to set our value. If a validator has been set then we first check + if it's alright, raising a ValueError with the reason if not. + + Arguments: + value - value we're attempting to set + """ + + if self.validator: self.validator(self, value) self.value = value def getLabel(self, prefix = ""): @@ -90,6 +143,10 @@ class ConfigOption: return [prefix + line for line in self.descriptionCache] class ToggleConfigOption(ConfigOption): + """ + Configuration option representing a boolean. + """ + def __init__(self, key, group, default, trueLabel, falseLabel): ConfigOption.__init__(self, key, group, default) self.trueLabel = trueLabel @@ -99,11 +156,20 @@ class ToggleConfigOption(ConfigOption): return self.trueLabel if self.value else self.falseLabel def toggle(self): + # This isn't really here to validate the value (after all this is a + # boolean, the options are limited!), but rather give a method for functors + # to be triggered when selected. + + if self.validator: self.validator(self, not self.value) self.value = not self.value def showWizard(): - relayType, config = None, {} + """ + Provides a series of prompts, allowing the user to spawn a customized tor + instance. + """ + relayType, config = None, {} for option in Options.values(): if option == Options.DIVIDER: config[option] = option @@ -121,12 +187,22 @@ def showWizard(): config[option] = ToggleConfigOption(option, "opt", isSet, trueLabel.strip(), falseLabel.strip()) else: config[option] = ConfigOption(option, "opt", default) + # sets input validators + + # enables custom policies when 'custom' is selected and disables otherwise + policyOpt = config[Options.POLICY] + policyOpt.setValidator(functools.partial(_exitPolicyAction, config)) + _exitPolicyAction(config, policyOpt, policyOpt.getValue()) + + # remembers the last selection made on the type prompt page + relaySelection = RelayType.RELAY + while True: if relayType == None: - selection = promptRelayType() + selection = promptRelayType(relaySelection) if selection == CANCEL: break - else: relayType = selection + else: relayType, relaySelection = selection, selection else: selection = promptConfigOptions(relayType, config) @@ -136,7 +212,7 @@ def showWizard(): # redraws screen to clear away the dialog we just showed cli.controller.getController().requestRedraw(True) -def promptRelayType(): +def promptRelayType(initialSelection): """ Provides a prompt for selecting the general role we'd like Tor to run with. This returns a RelayType enumeration for the selection, or CANCEL if the @@ -146,9 +222,9 @@ def promptRelayType(): popup, _, _ = cli.popups.init(25, 58) if not popup: return control = cli.controller.getController() - key, selection = 0, 0 options = [ConfigOption(opt, "role", opt) for opt in RelayType.values()] options.append(ConfigOption(CANCEL, "general", CANCEL)) + selection = RelayType.indexOf(initialSelection) try: popup.win.box() @@ -162,21 +238,21 @@ def promptRelayType(): while True: y, offset = len(topContent) + 1, 0 - for i in range(len(options)): + for opt in options: optionFormat = uiTools.getColor(MSG_COLOR) - if i == selection: optionFormat |= curses.A_STANDOUT + if opt == options[selection]: optionFormat |= curses.A_STANDOUT # Curses has a weird bug where there's a one-pixel alignment # difference between bold and regular text, so it looks better # to render the whitespace here as not being bold. offset += 1 - label = options[i].getLabel(" ") + label = opt.getLabel(" ") popup.addstr(y + offset, 2, label, optionFormat | curses.A_BOLD) popup.addstr(y + offset, 2 + len(label), " " * (54 - len(label)), optionFormat) offset += 1 - for line in options[i].getDescription(52, " "): + for line in opt.getDescription(52, " "): popup.addstr(y + offset, 2, uiTools.padStr(line, 54), optionFormat) offset += 1 @@ -218,22 +294,42 @@ def promptConfigOptions(relayType, config): popup.win.erase() popup.win.box() - # provides the description for internal relays + # provides the description for the relay type for i in range(len(topContent)): popup.addstr(i + 1, 2, topContent[i], curses.A_BOLD | uiTools.getColor(MSG_COLOR)) y, offset = len(topContent) + 1, 0 - for i in range(len(options)): - if options[i] == Options.DIVIDER: + for opt in options: + if opt == Options.DIVIDER: offset += 1 continue - label = " %-30s%s" % (options[i].getLabel(), options[i].getDisplayValue()) - optionFormat = curses.A_BOLD | uiTools.getColor(OPTION_COLOR) - if i == selection: optionFormat |= curses.A_STANDOUT + optionFormat = opt.getDisplayAttr() + if opt == options[selection]: optionFormat |= curses.A_STANDOUT - offset += 1 - popup.addstr(y + offset, 2, uiTools.padStr(label, 54), optionFormat) + offset, indent = offset + 1, 0 + if opt.getKey() in CONFIG["wizard.suboptions"]: + # If the next entry is also a suboption then show a 'T', otherwise + # end the bracketing. + + bracketChar, nextIndex = curses.ACS_LLCORNER, options.index(opt) + 1 + if nextIndex < len(options) and isinstance(options[nextIndex], ConfigOption): + if options[nextIndex].getKey() in CONFIG["wizard.suboptions"]: + bracketChar = curses.ACS_LTEE + + popup.addch(y + offset, 3, bracketChar, opt.getDisplayAttr()) + popup.addch(y + offset, 4, curses.ACS_HLINE, opt.getDisplayAttr()) + + indent = 3 + + labelFormat = " %%-%is%%s" % (30 - indent) + label = labelFormat % (opt.getLabel(), opt.getDisplayValue()) + popup.addstr(y + offset, 2 + indent, uiTools.padStr(label, 54 - indent), optionFormat) + + # little hack to make "Block" policies red + if opt != options[selection] and not opt.getValue() and opt.getKey() in CUSTOM_POLICIES: + optionFormat = curses.A_BOLD | uiTools.getColor("red") + popup.addstr(y + offset, 33, opt.getDisplayValue(), optionFormat) # divider between the options and description offset += 2 @@ -253,8 +349,8 @@ def promptConfigOptions(relayType, config): posOffset = -1 if key == curses.KEY_UP else 1 selection = (selection + posOffset) % len(options) - # skips dividers - while options[selection] == Options.DIVIDER: + # skips disabled options and dividers + while options[selection] == Options.DIVIDER or not options[selection].isEnabled(): selection = (selection + posOffset) % len(options) elif uiTools.isSelectionKey(key): if selection == len(options) - 2: return BACK # selected back @@ -263,7 +359,9 @@ def promptConfigOptions(relayType, config): options[selection].toggle() else: newValue = popup.getstr(y + selection + 1, 33, options[selection].getValue(), curses.A_STANDOUT | uiTools.getColor(OPTION_COLOR), 23) - if newValue: options[selection].setValue(newValue.strip()) + if newValue: + try: options[selection].setValue(newValue.strip()) + except ValueError, exc: cli.popups.showMsg(str(exc), 3) elif key == 27: selection, key = -1, curses.KEY_ENTER # esc - cancel finally: cli.popups.finalize() @@ -280,7 +378,16 @@ def _splitStr(msg, width): results = [] while msg: msgSegment, msg = uiTools.cropStr(msg, width, None, endType = None, getRemainder = True) + if not msgSegment: break # happens if the width is less than the first word results.append(msgSegment.strip()) return results +def _exitPolicyAction(config, option, value): + """ + Enables or disables custom exit policy options based on our selection. + """ + + for opt in CUSTOM_POLICIES: + config[opt].setEnabled(not value) + diff --git a/src/settings.cfg b/src/settings.cfg index 94ff9c1..135b0e2 100644 --- a/src/settings.cfg +++ b/src/settings.cfg @@ -343,14 +343,35 @@ msg.ARM_DEBUG Unable to query process resource usage from ps # configuration option attributes used in the relay setup wizard wizard.message.role Welcome to the Tor network! This will step you through the configuration process for becoming a part of it. To start with, what role would you like to have? wizard.message.relay Internal relays provide connections within the Tor network. Since you will only be connecting to Tor users and relays this is an easy, hassle free way of helping to make the network better. +wizard.message.exit Exits connect between the Tor network and the outside Internet. This is the most vitally important role you can take, but it also needs some forethought. Please read 'http://www.atagar.com/torExitTips/' before proceeding further to avoid any nasty surprises! wizard.toggle Notify => Yes, No wizard.toggle Startup => Yes, No +wizard.toggle Notice => Yes, No +wizard.toggle Policy => Default, Custom +wizard.toggle Websites => Allow, Block +wizard.toggle Email => Allow, Block +wizard.toggle Im => Allow, Block +wizard.toggle Misc => Allow, Block +wizard.toggle Plaintext => Allow, Block + +wizard.suboptions Websites +wizard.suboptions Email +wizard.suboptions Im +wizard.suboptions Misc +wizard.suboptions Plaintext wizard.default Nickname => Unnamed wizard.default Notify => true wizard.default Bandwidth => 5 MB/s wizard.default Startup => true +wizard.default Notice => true +wizard.default Policy => true +wizard.default Websites => true +wizard.default Email => true +wizard.default Im => true +wizard.default Misc => true +wizard.default Plaintext => true wizard.label.general Cancel => Cancel wizard.label.general Back => Previous @@ -365,10 +386,17 @@ wizard.label.opt Notify => Issue Notification wizard.label.opt Bandwidth => Relay Speed wizard.label.opt Limit => Monthly Limit wizard.label.opt Startup => Run At Startup +wizard.label.opt Notice => Disclaimer Notice +wizard.label.opt Policy => Exit Policy +wizard.label.opt Websites => Web Browsing +wizard.label.opt Email => Receiving Email +wizard.label.opt Im => Instant Messaging +wizard.label.opt Misc => Other Services +wizard.label.opt Plaintext => Unencrypted Traffic wizard.description.general Cancel => Close without starting Tor. wizard.description.role Relay => Provides interconnections with other Tor relays. This is a safe and easy of making the network better. -wizard.description.role Exit => Connects between Tor an the outside Internet. This is a vital role, but can lead to abuse complaints. +wizard.description.role Exit => Connects between Tor network and the outside Internet. This is a vital role, but can lead to abuse complaints. wizard.description.role Bridge => Non-public relay specifically for helping censored users. wizard.description.role Client => Use the network without contributing to it. wizard.description.opt Nickname => Human friendly name for your relay. If this is unique then it's used instead of your fingerprint (a forty character hex string) when pages like TorStatus refer to you. @@ -377,6 +405,13 @@ wizard.description.opt Notify => Sends automated email notifications to the abov wizard.description.opt Bandwidth => Limit for the average rate at which you relay traffic. wizard.description.opt Limit => Maximum amount of traffic to relay each month. Some ISPs, like Comcast, cap their customer's Internet usage so this is an easy way of staying below that limit. wizard.description.opt Startup => Runs Tor in the background when the system starts. +wizard.description.opt Notice => Provides a disclaimer that this is an exit on port 80 (http://www.atagar.com/exitNotice) +wizard.description.opt Policy => Ports allowed to exit from your relay. The default policy allows for common services while limiting the chance of getting a DMCA takedown for torrent traffic (http://www.atagar.com/exitPolicy) +wizard.description.opt Websites => General Internet browsing including HTTP (80) and HTTPS (443). +wizard.description.opt Email => Protocols for receiving, but not sending email. This includes POP3 (110), POP3S (995), IMAP (143), and IMAPS (993). +wizard.description.opt Im => Common instant messaging protocols including Jabber, IRC, ICQ, AIM, Yahoo, MSN, SILC, GroupWise, Gadu-Gadu, Sametime, and Zephyr. +wizard.description.opt Misc => Protocols from the default policy that aren't among the above. +wizard.description.opt Plaintext => When blocked the policy will exclude ports that aren't commonly encrypted. # some config options are fetched via special values torrc.map HiddenServiceDir => HiddenServiceOptions

1 0

[arm/master] Adding option for relaying without SOCKS port
by atagar＠torproject.org 27 Jun '11

27 Jun '11

commit fd39b3a97479694ae23b7f37bd9a8e43d13890ec Author: Damian Johnson <atagar(a)torproject.org> Date: Mon Jun 27 09:59:12 2011 -0700 Adding option for relaying without SOCKS port --- src/cli/wizard.py | 4 +++- src/settings.cfg | 4 ++++ 2 files changed, 7 insertions(+), 1 deletions(-) diff --git a/src/cli/wizard.py b/src/cli/wizard.py index 0d47de5..d2fb067 100644 --- a/src/cli/wizard.py +++ b/src/cli/wizard.py @@ -15,18 +15,20 @@ from util import enum, uiTools RelayType = enum.Enum("RELAY", "EXIT", "BRIDGE", "CLIENT") # all options that can be configured -Options = enum.Enum("DIVIDER", "NICKNAME", "CONTACT", "NOTIFY", "BANDWIDTH", "LIMIT", "STARTUP", "NOTICE", "POLICY", "WEBSITES", "EMAIL", "IM", "MISC", "PLAINTEXT") +Options = enum.Enum("DIVIDER", "NICKNAME", "CONTACT", "NOTIFY", "BANDWIDTH", "LIMIT", "CLIENT", "STARTUP", "NOTICE", "POLICY", "WEBSITES", "EMAIL", "IM", "MISC", "PLAINTEXT") RelayOptions = {RelayType.RELAY: (Options.NICKNAME, Options.CONTACT, Options.NOTIFY, Options.BANDWIDTH, Options.LIMIT, + Options.CLIENT, Options.STARTUP), RelayType.EXIT: (Options.NICKNAME, Options.CONTACT, Options.NOTIFY, Options.BANDWIDTH, Options.LIMIT, + Options.CLIENT, Options.STARTUP, Options.DIVIDER, Options.NOTICE, diff --git a/src/settings.cfg b/src/settings.cfg index 135b0e2..0ab57a2 100644 --- a/src/settings.cfg +++ b/src/settings.cfg @@ -346,6 +346,7 @@ wizard.message.relay Internal relays provide connections within the Tor network. wizard.message.exit Exits connect between the Tor network and the outside Internet. This is the most vitally important role you can take, but it also needs some forethought. Please read 'http://www.atagar.com/torExitTips/' before proceeding further to avoid any nasty surprises! wizard.toggle Notify => Yes, No +wizard.toggle Client => Enabled, Disabled wizard.toggle Startup => Yes, No wizard.toggle Notice => Yes, No wizard.toggle Policy => Default, Custom @@ -365,6 +366,7 @@ wizard.default Nickname => Unnamed wizard.default Notify => true wizard.default Bandwidth => 5 MB/s wizard.default Startup => true +wizard.default Client => false wizard.default Notice => true wizard.default Policy => true wizard.default Websites => true @@ -385,6 +387,7 @@ wizard.label.opt Contact => Contact Information wizard.label.opt Notify => Issue Notification wizard.label.opt Bandwidth => Relay Speed wizard.label.opt Limit => Monthly Limit +wizard.label.opt Client => Client Usage wizard.label.opt Startup => Run At Startup wizard.label.opt Notice => Disclaimer Notice wizard.label.opt Policy => Exit Policy @@ -404,6 +407,7 @@ wizard.description.opt Contact => Address we can contact you at if there's a pro wizard.description.opt Notify => Sends automated email notifications to the above address if your relay is unreachable or out of date. This service is provided by Tor Weather (https://weather.torproject.org/) and will send you a confirmation email before it's started. wizard.description.opt Bandwidth => Limit for the average rate at which you relay traffic. wizard.description.opt Limit => Maximum amount of traffic to relay each month. Some ISPs, like Comcast, cap their customer's Internet usage so this is an easy way of staying below that limit. +wizard.description.opt Client => Enable this if you would like to use Tor yourself. This opens or closes the SOCKS port used by applications for connecting to Tor. wizard.description.opt Startup => Runs Tor in the background when the system starts. wizard.description.opt Notice => Provides a disclaimer that this is an exit on port 80 (http://www.atagar.com/exitNotice) wizard.description.opt Policy => Ports allowed to exit from your relay. The default policy allows for common services while limiting the chance of getting a DMCA takedown for torrent traffic (http://www.atagar.com/exitPolicy)

1 0

r24842: {website} Replacing hardcoded trac links with wiki tags (website/trunk/docs/en)
by Damian Johnson 26 Jun '11

26 Jun '11

Author: atagar Date: 2011-06-26 01:28:30 +0000 (Sun, 26 Jun 2011) New Revision: 24842 Modified: website/trunk/docs/en/faq-abuse.wml Log: Replacing hardcoded trac links with wiki tags Modified: website/trunk/docs/en/faq-abuse.wml =================================================================== --- website/trunk/docs/en/faq-abuse.wml 2011-06-26 01:25:35 UTC (rev 24841) +++ website/trunk/docs/en/faq-abuse.wml 2011-06-26 01:28:30 UTC (rev 24842) @@ -172,9 +172,8 @@ </ul> Some hosting providers are friendlier than others when it comes to Tor - exits. For a listing see the <a - href="https://trac.torproject.org/projects/tor/wiki/doc/GoodBadISPs">good - and bad ISPs wiki</a>. + exits. For a listing see the <a href="<wiki>doc/GoodBadISPs">good and bad + ISPs wiki</a>. For a complete set of template responses to different abuse complaint types, see <a @@ -182,8 +181,7 @@ on the Tor wiki</a>. You can also proactively reduce the amount of abuse you get by following <a href="<blog>tips-running-exit-node-minimal-harassment">these tips for running an exit node with minimal harassment</a> and <a - href="https://trac.torproject.org/projects/tor/wiki/doc/ReducedExitPolicy">running - a reduced exit policy</a>. + href="<wiki>doc/ReducedExitPolicy">running a reduced exit policy</a>. You might also find that your Tor relay's IP is blocked from accessing some Internet sites/services. This might happen regardless of your exit

1 0