tor-commits April 2011

tor-commits@lists.torproject.org

18 participants
883 discussions

[arm/master] fix: Stacktrace from unjoined thread at shutdown
by atagar＠torproject.org 28 Apr '11

28 Apr '11

commit c4454e05284d36fb6ecafc9c2f33c7b0a9ffab43 Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Apr 27 21:46:21 2011 -0700 fix: Stacktrace from unjoined thread at shutdown There's a rare (but repeated) stack trace due to an unjoined thread at shutdown: Exception in thread Thread-3 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.6/threading.py", line 525, in __bootstrap_inner File "/home/atagar/Desktop/arm/src/util/sysTools.py", line 542, in run File "/usr/lib/python2.6/threading.py", line 135, in release <type 'exceptions.TypeError'>: 'NoneType' object is not callable This isn't harmful, but it is annoying. This change is attempting to be extra careful that all resourceTracker threads have been joined, and that the controller is stopped beforehand (preventing stray BW events from triggering additional queries). The stacktrace _seems_ to be more commonly caused by long-running arm processes, but I don't have a reliable repro case. --- src/cli/controller.py | 13 ++++++------- src/cli/graphing/resourceStats.py | 4 ++-- src/util/sysTools.py | 21 ++++----------------- 3 files changed, 12 insertions(+), 26 deletions(-) diff --git a/src/cli/controller.py b/src/cli/controller.py index 22d1772..8543ccb 100644 --- a/src/cli/controller.py +++ b/src/cli/controller.py @@ -825,20 +825,19 @@ def drawTorMonitor(stdscr, startTime, loggedEvents, isBlindMode): panels["conn"].join() panels["log"].join() - # joins on utility daemon threads - this might take a moment since - # the internal threadpools being joined might be sleeping conn = torTools.getConn() - myPid = conn.getMyPid() + conn.close() # joins on TorCtl event thread - resourceTracker = sysTools.getResourceTracker(myPid) if (myPid and sysTools.isTrackerAlive(myPid)) else None + # joins on utility daemon threads - this might take a moment since + # the internal threadpools being joined might be sleeping + resourceTrackers = sysTools.RESOURCE_TRACKERS.values() resolver = connections.getResolver("tor") if connections.isResolverAlive("tor") else None - if resourceTracker: resourceTracker.stop() + for tracker in resourceTrackers: tracker.stop() if resolver: resolver.stop() # sets halt flag (returning immediately) hostnames.stop() # halts and joins on hostname worker thread pool - if resourceTracker: resourceTracker.join() + for tracker in resourceTrackers: tracker.join() if resolver: resolver.join() # joins on halted resolver - conn.close() # joins on TorCtl event thread break elif key == curses.KEY_LEFT or key == curses.KEY_RIGHT: # switch page diff --git a/src/cli/graphing/resourceStats.py b/src/cli/graphing/resourceStats.py index f26d5c1..a9a8aee 100644 --- a/src/cli/graphing/resourceStats.py +++ b/src/cli/graphing/resourceStats.py @@ -36,9 +36,9 @@ class ResourceStats(graphPanel.GraphStats): primary, secondary = 0, 0 if self.queryPid: - resourceTracker = sysTools.getResourceTracker(self.queryPid) + resourceTracker = sysTools.getResourceTracker(self.queryPid, True) - if not resourceTracker.lastQueryFailed(): + if resourceTracker and not resourceTracker.lastQueryFailed(): primary, _, secondary, _ = resourceTracker.getResourceUsage() primary *= 100 # decimal percentage to whole numbers secondary /= 1048576 # translate size to MB so axis labels are short diff --git a/src/util/sysTools.py b/src/util/sysTools.py index 8d95733..2775194 100644 --- a/src/util/sysTools.py +++ b/src/util/sysTools.py @@ -347,27 +347,13 @@ def call(command, cacheAge=0, suppressExc=False, quiet=True): return results -def isTrackerAlive(pid): - """ - Provides true if a running, singleton instance exists for the given pid, - false otherwise. - - Arguments: - pid - pid of the process being tracked - """ - - if pid in RESOURCE_TRACKERS: - if RESOURCE_TRACKERS[pid].isAlive(): return True - else: del RESOURCE_TRACKERS[pid] - - return False - -def getResourceTracker(pid): +def getResourceTracker(pid, noSpawn = False): """ Provides a running singleton ResourceTracker instance for the given pid. Arguments: - pid - pid of the process being tracked + pid - pid of the process being tracked + noSpawn - returns None rather than generating a singleton instance if True """ if pid in RESOURCE_TRACKERS: @@ -375,6 +361,7 @@ def getResourceTracker(pid): if tracker.isAlive(): return tracker else: del RESOURCE_TRACKERS[pid] + if noSpawn: return None tracker = ResourceTracker(pid, CONFIG["queries.resourceUsage.rate"]) RESOURCE_TRACKERS[pid] = tracker tracker.start()

1 0

[arm/master] fix: ns/desc lookups could crash descriptor popup
by atagar＠torproject.org 28 Apr '11

28 Apr '11

commit 4ca251a6b4be812d8fdc0468a0ae43bc5595cf27 Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Apr 27 19:48:32 2011 -0700 fix: ns/desc lookups could crash descriptor popup The exception handling for the consensus and descriptor lookups in descriptorPopup.py were expecting TorCtl exceptions rather than a None result. Caught by asn. --- src/cli/descriptorPopup.py | 25 +++++++++++-------------- 1 files changed, 11 insertions(+), 14 deletions(-) diff --git a/src/cli/descriptorPopup.py b/src/cli/descriptorPopup.py index cdc959d..d57546c 100644 --- a/src/cli/descriptorPopup.py +++ b/src/cli/descriptorPopup.py @@ -3,9 +3,7 @@ # Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html) import math -import socket import curses -from TorCtl import TorCtl import controller import connections.connEntry @@ -47,20 +45,19 @@ class PopupProperties: self.text.append(UNRESOLVED_MSG) else: conn = torTools.getConn() + self.showLineNum = True - try: - self.showLineNum = True - self.text.append("ns/id/%s" % fingerprint) - self.text += conn.getConsensusEntry(fingerprint).split("\n") - except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): - self.text = self.text + [ERROR_MSG, ""] + self.text.append("ns/id/%s" % fingerprint) + consensusEntry = conn.getConsensusEntry(fingerprint) - try: - descCommand = "desc/id/%s" % fingerprint - self.text.append("desc/id/%s" % fingerprint) - self.text += conn.getDescriptorEntry(fingerprint).split("\n") - except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): - self.text = self.text + [ERROR_MSG] + if consensusEntry: self.text += consensusEntry.split("\n") + else: self.text = self.text + [ERROR_MSG, ""] + + self.text.append("desc/id/%s" % fingerprint) + descriptorEntry = conn.getDescriptorEntry(fingerprint) + + if descriptorEntry: self.text += descriptorEntry.split("\n") + else: self.text = self.text + [ERROR_MSG] def handleKey(self, key, height): if key == curses.KEY_UP: self.scroll = max(self.scroll - 1, 0)

1 0

[arm/master] fix: Misparsing circut paths for Tor < 0.2.2.1
by atagar＠torproject.org 28 Apr '11

28 Apr '11

commit 03aa67dcb5589e626306e89ef3dc92eb9c300de8 Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Apr 27 21:13:05 2011 -0700 fix: Misparsing circut paths for Tor < 0.2.2.1 For Tor versions prior to 0.2.2.1 our circuit paths could contain nickname-only entries. These were being misparsed (expecting them to always start with fingerprints), resulting in screwed up circuit entries in the connection panel. Caught by asn. --- src/util/torTools.py | 67 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 65 insertions(+), 2 deletions(-) diff --git a/src/util/torTools.py b/src/util/torTools.py index 5d8ffd8..b527f1e 100644 --- a/src/util/torTools.py +++ b/src/util/torTools.py @@ -330,6 +330,7 @@ class Controller(TorCtl.PostEventListener): self._fingerprintLookupCache = {} # lookup cache with (ip, port) -> fingerprint mappings self._fingerprintsAttachedCache = None # cache of relays we're connected to self._nicknameLookupCache = {} # lookup cache with fingerprint -> nickname mappings + self._nicknameToFpLookupCache = {} # lookup cache with nickname -> fingerprint mappings self._consensusLookupCache = {} # lookup cache with network status entries self._descriptorLookupCache = {} # lookup cache with relay descriptors self._isReset = False # internal flag for tracking resets @@ -389,6 +390,7 @@ class Controller(TorCtl.PostEventListener): self._fingerprintLookupCache = {} self._fingerprintsAttachedCache = None self._nicknameLookupCache = {} + self._nicknameToFpLookupCache = {} self._consensusLookupCache = {} self._descriptorLookupCache = {} @@ -1164,6 +1166,44 @@ class Controller(TorCtl.PostEventListener): return result + def getNicknameFingerprint(self, relayNickname): + """ + Provides the fingerprint associated with the given relay. This provides + None if no such relay exists. + + Arguments: + relayNickname - nickname of the relay + """ + + self.connLock.acquire() + + result = None + if self.isAlive(): + # determine the nickname if it isn't yet cached + if not relayNickname in self._nicknameToFpLookupCache: + # Fingerprints are base64 encoded hex with an extra '='. For instance... + # GETINFO ns/name/torexp2 -> + # r torexp2 NPfjt8Vjr+drcbbFLQONN3KapNo LxoHteGax7ZNYh/9g/FF8I617fY 2011-04-27 15:20:35 141.161.20.50 9001 0 + # decode base64 of "NPfjt8Vjr+drcbbFLQONN3KapNo=" -> + # "4\xf7\xe3\xb7\xc5c\xaf\xe7kq\xb6\xc5-\x03\x8d7r\x9a\xa4\xda" + # encode hex of the above -> + # "34f7e3b7c563afe76b71b6c52d038d37729aa4da" + + relayFingerprint = None + consensusEntry = self.getInfo("ns/name/%s" % relayNickname) + if consensusEntry: + encodedFp = consensusEntry.split()[2] + decodedFp = (encodedFp + "=").decode('base64').encode('hex') + relayFingerprint = decodedFp.upper() + + self._nicknameToFpLookupCache[relayNickname] = relayFingerprint + + result = self._nicknameToFpLookupCache[relayNickname] + + self.connLock.release() + + return result + def addEventListener(self, listener): """ Directs further tor controller events to callback functions of the @@ -1433,6 +1473,7 @@ class Controller(TorCtl.PostEventListener): self._fingerprintLookupCache = {} self._fingerprintsAttachedCache = None self._nicknameLookupCache = {} + self._nicknameToFpLookupCache = {} self._consensusLookupCache = {} if self._fingerprintMappings != None: @@ -1870,6 +1911,12 @@ class Controller(TorCtl.PostEventListener): # 91 BUILT $E4AE6E2FE320FBBD31924E8577F3289D4BE0B4AD=Qwerty PURPOSE=GENERAL # would belong to a single hop circuit, most likely fetching the # consensus via a directory mirror. + # + # The path is made up of "$<fingerprint>[=<nickname]" entries for new + # versions of Tor, but in versions prior to 0.2.2.1-alpha this was + # just "$<fingerprint>" OR <nickname>. The dolar sign can't be used in + # nicknames so this can be used to differentiate. + circStatusResults = self.getInfo("circuit-status") if circStatusResults == "": @@ -1885,8 +1932,24 @@ class Controller(TorCtl.PostEventListener): # 5 LAUNCHED PURPOSE=TESTING if len(lineComp) < 4: continue - path = tuple([hopEntry[1:41] for hopEntry in lineComp[2].split(",")]) - result.append((int(lineComp[0]), lineComp[1], lineComp[3][8:], path)) + path = [] + for hopEntry in lineComp[2].split(","): + if hopEntry[0] == "$": path.append(hopEntry[1:41]) + else: + relayFingerprint = self.getNicknameFingerprint(hopEntry) + + # It shouldn't be possible for this lookup to fail, but we + # need to fill something (callers won't expect our own client + # paths to have unknown relays). If this turns out to be wrong + # then log a warning. + + if relayFingerprint: path.append(relayFingerprint) + else: + msg = "Unable to determine the fingerprint for a relay in our own circuit: %s" % hopEntry + log.log(log.WARN, msg) + path.append("0" * 40) + + result.append((int(lineComp[0]), lineComp[1], lineComp[3][8:], tuple(path))) elif key == "hsPorts": result = [] hsOptions = self.getOptionMap("HiddenServiceOptions")

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.1' into maint-0.2.2
by nickm＠torproject.org 28 Apr '11

28 Apr '11

commit 0130e7c9d2842ad58e1b84829aeab16a2efba3bb Merge: 7f85509 43ffd02 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Apr 27 22:14:28 2011 -0400 Merge remote-tracking branch 'origin/maint-0.2.1' into maint-0.2.2 Conflicts: src/common/torint.h changes/bug2475 | 5 +++++ src/common/crypto.c | 2 +- src/common/torint.h | 4 +++- 3 files changed, 9 insertions(+), 2 deletions(-) diff --cc src/common/torint.h index f5bebf8,d489684..0b5c29a --- a/src/common/torint.h +++ b/src/common/torint.h @@@ -329,8 -330,10 +329,10 @@@ typedef uint32_t uintptr_t #endif #endif -/* Any ssize_t larger than this amount is likely to be an underflow. */ ++/** Any ssize_t larger than this amount is likely to be an underflow. */ + #define SSIZE_T_CEILING ((ssize_t)(SSIZE_T_MAX-16)) -/* Any size_t larger than this amount is likely to be an underflow. */ +/** Any size_t larger than this amount is likely to be an underflow. */ - #define SIZE_T_CEILING (SSIZE_T_MAX-16) + #define SIZE_T_CEILING ((size_t)(SSIZE_T_MAX-16)) #endif /* __TORINT_H */

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.2'
by nickm＠torproject.org 28 Apr '11

28 Apr '11

commit 26456d33546b674d5fa4d2c0112eeec561da7279 Merge: 26c022e 0130e7c Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Apr 27 22:14:54 2011 -0400 Merge remote-tracking branch 'origin/maint-0.2.2' changes/bug2475 | 5 +++++ src/common/crypto.c | 2 +- src/common/torint.h | 4 +++- 3 files changed, 9 insertions(+), 2 deletions(-)

1 0

[tor/master] Make SIZE_T_CEILING unsigned; add a signed SSIZE_T_CEILING
by nickm＠torproject.org 28 Apr '11

28 Apr '11

commit 43ffd023e9267927539dc9c12bee86199cd1c800 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Apr 26 13:00:46 2011 -0400 Make SIZE_T_CEILING unsigned; add a signed SSIZE_T_CEILING None of the comparisons were _broken_ previously, but avoiding signed/unsigned comparisons makes everybody happier. Fixes bug2475. --- changes/bug2475 | 5 +++++ src/common/crypto.c | 2 +- src/common/torint.h | 4 +++- 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/changes/bug2475 b/changes/bug2475 new file mode 100644 index 0000000..d6f0595 --- /dev/null +++ b/changes/bug2475 @@ -0,0 +1,5 @@ + o Minor bugfixes: + - Avoid signed/unsigned comparisons by making SIZE_T_CEILING unsigned. + (None of the cases where we did this before were wrong, but by making + this change we can avoid warnings.) Fixes bug2475; bugfix on + Tor 0.2.1.28. diff --git a/src/common/crypto.c b/src/common/crypto.c index 48c8dea..838347e 100644 --- a/src/common/crypto.c +++ b/src/common/crypto.c @@ -452,7 +452,7 @@ crypto_pk_read_private_key_from_string(crypto_pk_env_t *env, tor_assert(env); tor_assert(s); - tor_assert(len < INT_MAX && len < SIZE_T_CEILING); + tor_assert(len < INT_MAX && len < SSIZE_T_CEILING); /* Create a read-only memory BIO, backed by the string 's' */ b = BIO_new_mem_buf((char*)s, (int)len); diff --git a/src/common/torint.h b/src/common/torint.h index 2a9fba6..d489684 100644 --- a/src/common/torint.h +++ b/src/common/torint.h @@ -330,8 +330,10 @@ typedef uint32_t uintptr_t; #endif #endif +/* Any ssize_t larger than this amount is likely to be an underflow. */ +#define SSIZE_T_CEILING ((ssize_t)(SSIZE_T_MAX-16)) /* Any size_t larger than this amount is likely to be an underflow. */ -#define SIZE_T_CEILING (SSIZE_T_MAX-16) +#define SIZE_T_CEILING ((size_t)(SSIZE_T_MAX-16)) #endif /* __TORINT_H */

1 0

[tor/maint-0.2.2] Merge remote-tracking branch 'origin/maint-0.2.1' into maint-0.2.2
by nickm＠torproject.org 28 Apr '11

28 Apr '11

1 0

[tor/maint-0.2.2] Make SIZE_T_CEILING unsigned; add a signed SSIZE_T_CEILING
by nickm＠torproject.org 28 Apr '11

28 Apr '11

1 0

[tor/maint-0.2.1] Make SIZE_T_CEILING unsigned; add a signed SSIZE_T_CEILING
by nickm＠torproject.org 28 Apr '11

28 Apr '11

1 0

r24687: {projects} Clarify a paragraph. (projects/articles/browser-privacy)
by Mike Perry 27 Apr '11

27 Apr '11

Author: mikeperry Date: 2011-04-27 22:12:07 +0000 (Wed, 27 Apr 2011) New Revision: 24687 Modified: projects/articles/browser-privacy/W3CIdentity.tex Log: Clarify a paragraph. Modified: projects/articles/browser-privacy/W3CIdentity.tex =================================================================== --- projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 21:33:27 UTC (rev 24686) +++ projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 22:12:07 UTC (rev 24687) @@ -131,11 +131,11 @@ utilize everything they can to build complete portraits of users' identities~\cite{tracking-identity}. -Despite what the user may believe, her actual web identity then is a -superset of all the stored identifiers and authentication tokens used by the -browser. This identity can link a user's activity in one instance to her -activity in another instance, be it across time, or even on the very same page -due to multiple content origins. +Despite what the user may believe, her actual web identity then is a superset +of all stored identifiers and authentication tokens used by the browser. This +identity can link a user's activity in one instance to her activity in another +instance, be it across time, or even on a single page due to common prevalent +third-party content origins. Therefore, instead of viewing the user's identity as the sum of her identifiers, or as her relationship to individual websites, it is best to

1 0

← Newer
1
...
11
12
13
14
15
16
17
...
89
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-commits April 2011