- tor-dev - lists.torproject.org

metrics: collecting circuit build failures from relays to detect network reachability issues and broken relays
by nusenu 20 Oct '17

20 Oct '17

Hello, In [1] David describes his preliminary results from scanning a portion of the tor network to detect connectivity problems (partitions) in the presumed tor mesh network (where every relay is expected to be able to reach every other relay). I believe this information (How far off are we from a complete mesh network?) is crucial for the anonymity properties of the tor network and collecting the data to answer that question should be an integral and continuous part of tor. Actively scanning the tor network for connectivity issues is resource intensive. What if we could collect this data without actively scanning for it? This could be achieved by collecting reachability information passively by relays themselves and uploaded via extra-info descriptors. This data could help reduce the overall scanning effort. So instead of scanning _all_ relays we can reduce the scanning to relays where some threshold of relays consistently said that they are hardly reachable. (to confirm their measurements) Due to the mutual nature of the information collection, single or a minor number of lying relays would not be a big issue. relays could: - aggregate reachability issues over the past 24 hours (or week?) per outbound destination relay (which relay failed what percentage of circuit build attempts) - if a relay failed more than - some threshold value - check if the relay in question: - had his uptime reset during that timeframe (reachability problem expected) - dropped out of consensus during that timeframe - only include it in uploaded data if it didn't drop out of consensus and did not reset his uptime (using a week instead of a day would help with reducing the amount of data that we need to process after collecting it) In addition to that passive collection, relays - when idle/not overloaded - could actively attempt to create circuits to measure their reachability to relays for which they did not collect any any passive data. (maybe limit it to fast and stable destination relays) To reduce the load on the tor network we could limit the active outbound connection tests based on relay flags: - Guard-only relays are more likely to establish outbound connections to non-exit relays - middle-only relays (no guard and exit flag) are more likely to establish connections to exits than to guards - exits are more likely to get inbound connections from non-guards (maybe skip active probes from exits since they are a bottleneck already?) Reachability issues could also be displayed to the relay operator warning them about the potential problem via log entries. So they could actively work on debugging problems themselfes. There is no doubt that this information is also valuable for (powerful) adversaries (it could help them reduce their effort when they know weak spots in the network). So you would have to decide what data you collect an what you would publish (collector.tpo) - even if that means I might never get to see that interesting data ;) This is a medium/long term goal, with the usual steps: - proposal - implementation (- deployment - data collection - the amount of data could be huge - data analysis) If there is a consensus that this makes sense and if someone would actually implement it I would be happy to work/help on a proposal. This entire idea would be an opt-in torrc setting at the beginning and a opt-out feature once we are more confident about its implications and safety. Please let me know what you think about this idea. regards, nusenu [1] https://lists.torproject.org/pipermail/tor-project/2017-October/001492.html related trac tickets: https://trac.torproject.org/projects/tor/ticket/12131 https://trac.torproject.org/projects/tor/ticket/19068 -- https://mastodon.social/@nusenu twitter: @nusenu_

3 2

Call For Testing: Capsicum Integration With Tor
by Shawn Webb 18 Oct '17

18 Oct '17

Tor Capsicum Call For Testing (CFT) =================================== Capsicum is a capabilities framework for FreeBSD and its derivatives. It's commonly used to implement sandboxing. Tor currently supports sandboxing only on Linux, via seccomp-filter. This CFT presents initial Capsicum support for Tor on FreeBSD and HardenedBSD. The work performed is sponsored by HardenedBSD. The source for the Capsicum work can be found at https://github.com/lattera/tor. The branch to use is the hardening/capsicum branch. We hope that this implementation, or a future revision of it, will be merged upstream by the Tor Project. Design and Architecture ----------------------- When capabilities mode (capmode) is entered (ie, the sandbox is enabled), the kernel will prevent Tor from accessing the global namespace. This means that once the sandbox is enabled, Tor will not be able to open files, create sockets, connect outbound, etc. Naturally, applications like Tor needs to be able to create sockets on-demand and at-will. In these situations, it is common to fork a child process which will perform these "privileged" operations. Only the parent process will enter capmode. Communication between the parent and the child is performed via a socketpair descriptor. Wrappers in the parent process call into the child process to perform these privileged operations. Thus, open(2) becomes sandbox_open(), rename(2) becomes sandbox_rename(), and so on. A full list of wrapped operations will be provided further on. When the parent process wants to open a file, it will call sandbox_open. The parent process will tell the child via the socketpair to open the file and limit its capabilities. On success, the child will return the limited file descriptor back to the parent. We prevent race conditions surrounding the limitation of file descriptors by calling cap_rights_limit(2) on the descriptor in the child before passing the descriptor back to the parent. By the time the parent can make use of the descriptor, the descriptor is fully limited to a minimal set of capabilities. Tor's existing sandbox abstraction API relies on a filtering-based approach. We've utilized parts of that API to provide a basic whitelist. Once capmode is entered, the whitelist turns read-only and cannot be modified. Efforts to apply Capsicum in the form of wrappers, such as has been done in this work and others, require ASLR and W^X to be effective. Without ASLR, an attacker could simply craft an exploit to return into the respecive wrapper and bypass any guarantees Capsicum provides. Additionally, Control Flow Integrity (CFI) from clang/llvm will provide forward-edge guarantees. SafeStack from clang/llvm will provide backward-edge guarantees. Known Issues ------------ Enabling the sandbox while having Tor configured in transparent proxy mode is currently broken. We are researching what causes the breakage. Chances are that either Tor is trying to access the global namespace in transparent proxy mode or one or more file descriptors simply need to be granted one or more extra capabilities. Future Work ----------- We plan to research and fix, if possible, the known regression regarding transparent proxying. The read-only nature of the whitelist is only enforced via a logic operation. We plan to relocate the whitelist into a memory mapping that will turn read-only upon entering capmode. The current implementation should be further abstracted in order for Tor to be able to pick at runtime the appropriate sandbox implementation. This would be similar to how Tor chooses which ed25519 implementation to use, donna or ref10. Testing and Reporting Back -------------------------- 1. Install required development tools: # pkg install libevent autoconf automake autotools libtool gmake \ git-lite 2. Clone the git repo: # git clone https://github.com/lattera/tor.git 3. Check out the hardening/capsicum branch: # cd tor # git checkout -b hardening/capsicum origin/hardening/capsicum 4. Regenerate autoconf magic: # ./autogen.sh 5. Run the configure script: # ./configure [custom arguments to configure as you'd prefer] 6. Build and install: # gmake -j$(sysctl -n hw.ncpu) && gmake install 7. Enable the sandbox in the torrc: # echo "Sandbox 1" >> [path to torrc] 8. Run Tor: # tor 9. Report results: Email shawn.webb(a)hardenedbsd.org, report back any issues you may have. If your testing succeeds, I'd like to hear about it, too. Please do not open any tickets regarding this work on the Tor Project's Trac instance. Contact me directly. A sample report back would include a sanitized torrc along with a pass/fail result. A successful result would mean that all features you rely on work as desired. A failed result would mean that enabling a feature results in Tor crashing, erroring out, or exhibiting bugs. If Tor crashes for you, please include a backtrace if possible. New Sandbox Wrappers -------------------- These APIs will need to be used regardless of operating system. On operating systems outside of FreeBSD and HardenedBSD, they are simply macros that forward to their respective native API call. * int sandbox_close(int fd): close(2) wrapper * int sandbox_rename(const char *from, const char *to): rename(2) wrapper * int sandbox_stat(const char *path, struct stat *sb): stat(2) wrapper * int sandbox_mkdir(const char *path, mode_t mode): mkdir(2) wrapper * int sandbox_connect(int sockfd, const struct sockaddr *name, socklen_t namelen): connect(2) wrapper * int sandbox_socket(int domain, int type, int protocol, cap_rights_t *rights): socket(2) wrapper * int sandbox_unlink(const char *path): unlink(2) wrapper * int sandbox_open(const char *path, int flags, mode_t mode, cap_rights_t *rights): open(2) wrapper -- Shawn Webb Cofounder and Security Engineer HardenedBSD GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE

1 1

GET request to a regular website using Tor and nodejs
by Fernando Gabrieli 18 Oct '17

18 Oct '17

Hello all, i want to tunnel a http request from my nodejs app through Tor, would you point me in the right direction to start implementing this? thanks in advance Fernando

2 1

Re: [tor-dev] User perception of onion service discovery
by dawuud 15 Oct '17

15 Oct '17

I agree with Alec. Don't block the existing tor2web stuff, that would be very rude. Instead just do not implement any kind of tor2web for v3 onion services so that tor2web will gradually fade as we migrate. > *although speaking as a geek I believe that re-engineering T2W to > support SSL via SNI-Sniffing would address this, it would be a gross > and pointless hack, complicated still further by certificate issuance, > and all reasonable use cases for which would be better addressed by > running a local copy of Tor. Ah yeah, Donncha wrote a tool to do that called oniongateway: https://github.com/DonnchaC/oniongateway Is that what you mean?

1 0

The Tor git master branch is now 0.3.3.x
by Nick Mathewson 15 Oct '17

15 Oct '17

Hello! With the opening of the new 0.3.3.x branch, the git master branch for tor is now 0.3.3.x. Feature patches should still be based on master. If you are writing any bugfix patch that should go into an earlier version, please base it on the appropriate maint branch. For example, if you're fixing a bug in Tor 0.3.2.x, you should base your git branch on the "maint-0.3.2" branch. best wishes, -- Nick

1 0

No network-team IRC meetings this week
by Nick Mathewson 15 Oct '17

15 Oct '17

Hi, all! The usual network-team IRC meeting on Monday, and the usual patch party on Tuesday, will not happen this week: almost everybody will traveling from, or recovering from, the meeting in Montreal. For more information about our regular meetings, please see: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam Remember, new contributors and external developers are always welcome! yrs, -- Nick

1 0

User perception of onion service discovery
by Philipp Winter 15 Oct '17

15 Oct '17

This is the second part of our preliminary analysis of how Tor users interact with onion services [0]. In this part, we analyse the issue of onion service discovery. Onion services are private by default, so it's the operator's responsibility to disseminate their domain if they want it to be public. Question 3.6 in our survey asked: > How do you discover new onion sites? The breakdown looks as follows. Respondents could select multiple answers, so the percentages are based on the total number of respondents. Method Percentage ----------------------------------------------------------------------- >From social networking sites such as Reddit or Twitter 50.67 I browse the list of onion site search engines such as ahmia.fi 50.50 I randomly encounter them while browsing the web 47.65 Recommendations from friends and family 19.46 Other (see below for common themes) 18.12 I am not interested in learning about new onion sites 4.19 The data shows that social networking sites, search engines, and "random encounters" are rather popular. Respondents who selected "Other" mostly brought up onion service lists and aggregators. Following up, question 3.7 then asked: > Are you satisfied with the way you discover new onion sites? 61% selected "Yes" while the remaining 39% selected "No." Some respondents who selected "Yes" brought up that they have no interest in learning about new onion services; in part because they only use Facebook's (or some other) onion service. Among the respondents who selected "No," there are a bunch of reoccurring themes, in no particular order: - The most prominent complaint was about broken links on onion site lists. There is non-trivial churn among onion sites and our respondents were frustrated that existing lists are typically not curated and contain many dead links. - Many respondents were not aware of search engines such as ahmia.fi. Among those that were, many were not satisfied with both the search results and the number of indexed onion sites. Unsurprisingly, a "Google for onion sites" was a frequent wish. - Several respondents were unhappy with existing aggregators. In addition to broken links, some distrust lists because they occasionally contain scam and phishing sites. The difficulty of telling apart two given onion domain names exacerbates this issue. - Some respondents would like aggregators to be more verbose in their description of onion sites. In particular, these respondents were trying to avoid illegal and pornographic content, which is often difficult if the description is vague and the onion domain reveals nothing about its content. - Many respondents expressed frustration about the difficulty of finding out if site X also provides a corresponding onion service. A common wish was to have site X list its onion service prominently in a footer. Ironically, some respondents were surprised that torproject.org has a corresponding onion site -- they couldn't find it on the web site. - Two respondents compared the current state of onion services with the web of the 90s: Few sites existed, they linked to each other only sparsely, and search engines were experimental at best. - Interestingly, some respondents voiced frustration about various usability issues, but mentioned in the same sentence that this is an inherent trade-off of privacy technology, suggesting that there is nothing that can be done about it. There are two potential solutions that would address some of the above issues: - Have next-gen onion services opt-in to a broadcast mechanism that automatically propagates them. Naturally, we would like such a mechanism to be censorship-resistant and built in a way that only the owner of an onion service is authorised to broadcast their service. - Websites could use an HTTP header to announce the existence of a corresponding onion site. This issue was discussed in Feb 2017 over at tor-onions. Someone brought up the Alt-Svc header as a potential solution [1]. In a subsequent survey question we asked if our respondents would appreciate an automatic redirect from a web site to its corresponding onion site. The overall tendency leaned towards "Yes," provided that the implementation is sound and users can override the redirect. Again, it's important to take these results with a grain of salt. Our data has some survivor bias: Presumably, we mostly heard from people who are Tor users despite usability issues. We likely didn't hear from many people who once experimented with Tor or onion services, decided it's not usable enough, and gave up. The above was joint work with my colleagues Marshini Chetty, Annie Edmundson, Nick Feamster, and Laura M. Roberts. [0] <https://nymity.ch/onion-services/> [1] <https://lists.torproject.org/pipermail/tor-onions/2016-February/000045.html>

5 6

non-ASCII handling in tor and onionoo
by nusenu 14 Oct '17

14 Oct '17

Hi, I'm trying to find out if non-ASCII characters are allowed in descriptor data (and what onionoo would do with non-ASCII currently displayed as "?"). According to the spec non-ASCII chars are not allowed: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n218 I searched trac because I remembered there were some ASCII related tickets: https://trac.torproject.org/projects/tor/ticket/1258 closed due to https://trac.torproject.org/projects/tor/ticket/1720 (which has nothing to do with non-ASCII) https://trac.torproject.org/projects/tor/ticket/1270 Is this summary correct? Currently non-ASCII is not compliant with the spec but your consensus is that non-ASCII chars should be supported but no one got around to patch the spec and implementation. thanks, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

2 2

Pluggable Transports 2.0 Specification, Draft 3
by Brandon Wiley 13 Oct '17

13 Oct '17

Below is a link to the third draft of the Pluggable Transport 2.0 Specification. If you have feedback on this draft, please send me your comments by October 31. Thank you! Changes in this version: - Expanded acknowledgements section - thanks Yawning! - Removed TransportConn and TransportListener in favor of net.Conn and net.Listener - Modified SOCKS authentication method to use IANA-assigned designator - Added error response codes for per-connection arguments - Many typos fixed - thanks David Fifield! - Clarified some definitions - thanks teor! Link: https://operatorfoundation.org/PluggableTransportSpecification-v2-Draft3.pdf

1 1

Pluggable Transports 2.0 Specification, Draft 2
by Brandon Wiley 10 Oct '17

10 Oct '17

Attached is the second draft of the Pluggable Transport 2.0 Specification. If you have feedback on this draft, please send me your comments by July 20.

4 11