July 2016 - tor-dev - lists.torproject.org

Tor and Namecoin
by Jeremy Rand 04 Aug '16

04 Aug '16

Hello Tor devs, Namecoin is interested in collaboration with Tor in relation to human-readable .onion names; I'm reaching out to see how open the Tor community would be to this, and to get feedback on how exactly the integration might work. The new hidden service spec is going to substantially increase the length of .onion names, which presents usability concerns. Namecoin provides a way to resolve a human-readable .bit name to a .onion name. Another benefit of Namecoin is that it provides a way to lookup TLS fingerprints for clearnet .bit sites, which reduces the risk of MITM attacks on clearnet websites from malicious or compromised CA's. I had the pleasure of meeting Mike Perry at the Decentralized Web Summit at the Internet Archive in June; I talked to him about Namecoin's rough plans and he suggested I post here. I understand that Riccardo Spagni from Monero discussed this topic as well with Roger Dingledine at the Security in Times of Surveillance conference at Ei_PSI. The two most major concerns that I expect would be brought up involve anonymity and blockchain size. Here's how we plan to deal with these issues: Namecoin already provides location-anonymity for name registrations assuming that it's routed via Tor. It's also necessary to broadcast transactions for different names to different peers, which isn't coded yet, but this is just coding work rather than an engineering challenge -- a usable workaround today is running multiple Namecoin wallets. The more interesting challenge is blockchain anonymity for registrations, due to the linkability required for blockchain validation. An important point here is that transactions for a given name are inherently linkable to each other, and that this isn't problematic. The problem would come when multiple names are linked together, or when a name is linked with currency transactions. The solution I've come up with is to use atomic cross-chain trades, which let a user buy namecoins using a cryptocurrency that is designed to provide anonymity (such as Monero or Zcash, both of which have cryptographic proofs of anonymity, given a certain anonymity set and security assumptions). The user would use an anonymous cryptocurrency to buy a small amount of namecoins (enough to register a single name and keep it renewed for a while). If the user wanted to register another name, she would perform another atomic cross-chain trade, receiving namecoins that are not linked to the namecoins obtained for the first name. As long as those namecoins are not mixed by the wallet software, the names remain unlinked. Many users won't want to download the full Namecoin blockchain (around 3 GB at the moment). I have a proof-of-concept SPV-based Namecoin name lookup client working as of early June. I just got a large part of that code upstreamed into libdohj, and I'm working on getting the rest upstreamed and released. It's in Java (based on BitcoinJ), so it's not subject to the memory safety concerns that C/C++ code are. The SPV name lookups are implemented in 3 ways, depending on the user's needs: Option A: 1. Block headers are synced over the Namecoin P2P network. (Over clearnet this takes about 5 minutes the first time it runs.) 2. An index mapping unexpired block heights to block hashes is constructed, so that lookups can be done quickly. (This occurs when the SPV client starts, after syncup has completed; it's fast enough that I haven't found a need to benchmark it.) 3. When a name lookup request is received, the client asks a remote API server for the height of the last update of the name. 4. The client looks up the block hash of that height from its index, and requests that block over the P2P network. 5. The client verifies that the received block matches the correct hash and that the block follows Namecoin rules (e.g. verifying the merkle root). 6. The client looks through the transactions in the block until it finds the one that updates the name. 7. The client retrieves the value of the name from that transaction, and returns it to the user. Option B: 1 through 3. Same as Option A. 4. The API server also provides the full content of the transaction, as well as a merkle proof of inclusion in the block. 5. The client verifies that the merkle proof links the hash of the provided transaction to the merkle root of the block header with the given height. 6. The client retrieves the value of the name from the provided transaction, and returns it to the user. Option C: 1. Block headers are synced over the Namecoin P2P network, as well as full blocks for the past year (meaning that all full blocks that contain unexpired name data will be synced). (Over clearnet this takes about 10 minutes the first time it runs.) 2. An index mapping names to transactions is constructed as the full blocks are downloaded. (This uses LevelDB.) 3. When a name lookup request is received, the client looks up the transaction in the LevelDB index. 4. The client retrieves the value of the name from that transaction, and returns it to the user. For Options A and B, if the API server is malicious, it can do any of the following: 1. Falsely claim that the name doesn't exist. 2. Provide outdated name data that is less than 36000 blocks old (the expiration period for Namecoin). (Option C is not vulnerable to either of those attacks.) If multiple API servers are consulted, and they return different results, it is easy to tell which is lying (although I haven't implemented any such logic yet). The API server cannot do any of the following: 1. Provide name data that isn't from the blockchain with the most work. 2. Provide name data that is more than 36000 blocks old (the expiration period for Namecoin). The reason an API server is used in Options A and B instead of the P2P network, is that the P2P network is unauthenticated and easy to Sybil. The P2P network is great for getting data that is independently verifiable (e.g. block headers and contents of blocks), but it's unwise to rely on the P2P network to get unverifiable data such as a block height of a name. An API server is authenticated (currently via CA-based TLS, but a cert pin or PGP signing is certainly doable), which reduces the possible points of attack. This is analogous to why Tor uses centralized directory authorities -- authenticated trust points are harder to Sybil. (We do have longer term plans to introduce a way for SPV clients to get the latest transaction associated with a name, without using an API server or needing to download any full blocks, but that's out of scope of this email.) Options A and B do reveal to the API server which name is being looked up. If mode A is used, it also reveals to a P2P peer which block height is being looked up (which narrows the set of names by a factor of ~36000). Therefore, Tor stream isolation should be used in such cases. (That's not implemented yet.) Option C doesn't generate any network traffic on lookups, so it doesn't reveal anything. In my testing, an SPV-based name lookup using Option A takes around 650 milliseconds (over clearnet). The vast majority of this is latency to the API server (the server I'm testing with is on a low-budget hosting plan). The portion consisting of a block retrieval over P2P takes around 98 milliseconds (although it varies by block size). Option C takes around 4 milliseconds. The storage overhead of Option C's LevelDB database is around 400 MB right now, although I believe it's feasible to reduce this significantly. There are a few options I can think of for integrating this with Tor for .onion naming. One would be to modify OnioNS to call the Namecoin SPV client. This would concern me because OnioNS is in C++, which introduces the risk of memory safety vulnerabilities. Another would be to use an intermediate proxy like Yawning's or-ctl-filter. A third option would be to try to get external name resolution implemented in Tor itself, which I believe Jeff Burdges has suggested in the past. If Option A or B is used, any solution would need to pass the stream isolation info to the SPV client. Integrating this with Tor Browser for TLS certificate validation might involve a Firefox patch. There are tricks that can be done with the CertDB and SiteSecurityService XPCOM interfaces that will do the job without Firefox patches, but XPCOM is being phased out by Mozilla in favor of WebExtensions, and I'm unaware of any equivalent features in WebExtensions. (Also, it's unclear to me whether CertDB and SiteSecurityService would introduce isolation issues -- I can't think of any obvious attacks, but I haven't thought very hard about it.) I'm trying to engage with Mozilla to see if we can work out a WebExtensions feature for this, but nothing conclusive has happened on that front yet. On the subject of reproducible builds, I've never tried to build Java code in Gitian, so I'm not certain how difficult it's going to be. Since Android uses Java, maybe the Guardian Project devs would have some insight into the best way to do it. One of the Namecoin developers (Joseph Bisch) is really good with reproducible builds (you probably know him since he's the author of the Debian guest support in Gitian), so I'm reasonably confident that a way to do it can be found. I'd love to hear feedback on all of this. Cheers, -Jeremy Rand Lead Application Engineer of Namecoin

4 7

Usability Improvements for Atlas (was Re: Globe is now retired)
by Iain R. Learmonth 02 Aug '16

02 Aug '16

Hi All, On 29/06/16 09:56, Karsten Loesing wrote: > tl;dr: Globe is now retired in favor of an improved Atlas. Atlas is improved, but I'd like to improve it further. I've been tackling #6787 looking to improve the homepage and make Atlas easier to use for someone that isn't already a Tor expert. The changes I'm proposing in my fix for #6787 [1] do make some noticeable UI changes and I just want to gather some opinions on whether or not Atlas users feel that this is taking it in the right direction. The "Top 10 Relays" page was added to Atlas in order to make sure that functionality was not lost when shutting down globe. (See #5430 [2]) In my proposed patch, I'm also adding an "Authorities" page and adding text to both of these pages to explain what you're seeing. These are linked from the homepage, and from the top navbar. I'm also moving the introduction and explanation of the Atlas features from the About page to the homepage and introducing a new Help page that explains the details that are found in the relay details pages. You can find a live example of this patch at: https://irl.github.io/atlas/#/ and a full list of changes are detailed in #6787. Please let me know if you can see any reason to not merge this, or if you have any suggestions for small changes before merging. Thanks, Iain. [1]: https://trac.torproject.org/projects/tor/ticket/5430 [2]: https://trac.torproject.org/projects/tor/ticket/6787

5 5

Tor Browser and Privoxy
by Paulo Roberto 02 Aug '16

02 Aug '16

Hello, By default the Tor Browser Bundle connects through SOCKS5 to the TOR daemon. Is there any advantage today to configure Privoxy to use TOR and configure the Tor Browser in the Tor Network Settings to use proxy access type HTTP/HTTPS pointing to Privoxy? Thanks in advance. Paulo Roberto.

3 3

Re: [tor-dev] prop224: Ditching key blinding for shorter onion addresses
by Nick Mathewson 30 Jul '16

30 Jul '16

whoops, didn't reply-all. On Sat, Jul 30, 2016 at 1:00 PM, Nick Mathewson <nickm(a)alum.mit.edu> wrote: > On Sat, Jul 30, 2016 at 9:32 AM, George Kadianakis <desnacked(a)riseup.net> wrote: > [...] >> >> Please let me know in what way this scheme is completely broken! > > I think it has the same problem as the last one: there does not appear > to be a way to keep an attacker from putting their own record in place > using your query_key, and thereby performing a DoS on your hidden > service. > > > -- > Nick

1 0

[GSoC] CONIKS for Tor Messenger - status report #5
by Huy Vu Quoc 30 Jul '16

30 Jul '16

Hi everyone, These are things that I have been doing in the last two weeks: - I and my mentors have had a discussions for next steps for the project. We decided to reorganize our project’s repo [1]. All repos has been merged into one [1]. This helps us maintain and keep tracking the development easier. - A lot of issues and PRs have been resolved/merged. The merkletree library is almost done for now. - Thanks for Ismail and Arlo’s suggestions, the client library for crypto verifications (including the VRF verification, Ed25519 signature verification, auth path verification and so on) is implemented in Go and compiled with `c-shared` build mode [2]. This library then will be wrapped with js-ctypes to use in the add-on. I have successfully loaded the library into the add-on in Linux machines. There’s still some problem since Tor Messenger for OS X is compiled from a Linux machine, while the library is compiled from macOS, then when loading the library into Tor Messenger for OS X, Tor Messenger may crash. We will probably dig deeper on this issue. Windows’ DLL version of the library also should be taken care. Next steps are mostly to focus on the client-side, along with finishing the server reference implementation. Here’s thing I expect in the next weeks: - Merge the server-registration and registration-bots PR. - Merge the coniks-preference PR. - Merge the PR of client library. - Submit PR for client registration and lookup. I also expect to have a working version of project before August 15, including the functions which I described in my proposal. Best, Huy — [1] https://github.com/coniks-sys/coniks-go [2] https://github.com/coniks-sys/coniks-go/tree/client-cgo

1 0

[GSOC 16] Ahmia status update #5
by Ismael R 29 Jul '16

29 Jul '16

Hi everyone, I'm working on ahmia.fi, the hidden service search engine and you're reading status update #5. During the last two weeks I worked on improving search. It required working on a new index format and updating the crawler to fill this new index. Search now has these new features: - Token search is done in addition to pure full-text search. For instance, "foxes" and "fox" are different words but are indexed as the same token. Same for "running" and "run", "tor's and tor". - Associated words are also stored. For instance, "I’m not happy I’m working" have the same words than "I’m happy I’m not working", but the meaning is different. - Meaningful words have way more impact than frequent words. - The text used with a link (called anchor text) is now stored with the page which is the target of the link. Let's say a site A is linking a site B like that: <a href="B">B is a great search engine</a>. Searching "great search engine" is going to return B instead of A. These anchors could also replace a title in the results list if the site has no title (.txt files are like that). - A word present in the title, description, anchors has way more strength than a word present in the content of a page. - An authority score is computed for each page thanks to the pagerank algorithm. Important pages (like homepages) tends to appear first in the result list. It also solves problems we have with duplicated content. - The result list now returns pages instead of sites/domains. Before updating ahmia.fi, I need to finish porting the rest of the django site (API and statistics) to the new structure. I hope to finish this next week so you can try it. After this, here are the features I plan to implement in the remaining time: - Add a language to the search engine (site:msydqstlz2kzerdg.onion to specify a domain, -term to exclude a term, filetype:txt to specify a filetype, lang:fr to specify a language, etc). - Use If-Modified-Since HTTP header to avoid crawling unmodified pages. - Improve documentation - Add unit tests Also, I have some ideas I won't have the time to implement during GSoC. Hopefully, I will do it later because I plan to continue working with the Tor community :) A small list: - Improve search for languages other than english by making one index per language - Make the crawler better understand query parameters. Some are useful (?page=3), some are useless (?printable=1). - Better tokenize the url to extract words from it. - Add highlights to each result (especially for those without meta- description) to show the context in which query terms appears. - Make a script to make sure every indexed url is still online. - Make a true RESTful API. I see you in two weeks for the next status report and maybe sooner if I push something online. Cheers! Ismael

1 0

Implementation plan for prop271 "Another algorithm for guard selection"
by George Kadianakis 29 Jul '16

29 Jul '16

Hello people, our new guard algorithm proposal (prop271) is getting finalized and we are planning to move to implementation soon. You can find the proposal here: https://gitweb.torproject.org/torspec.git/tree/proposals/271-another-guard-… You can also find the STRIKE team's implementation of proposal 259 here: https://github.com/twstrike/tor_for_patching Ideally we should borrow as much code as possible from their implementation. To move forward with engineering, here follows an initial thirteen step implementation plan for proposal 271. Where it made sense, I tried to provide links to current functions that can be used, or to code that the STRIKE team wrote and can be reused or studied. Let me know what I should make clearer or where I should go further in depth. --- Design decisions: An initial design decision here is whether we should make a new file for this feature or use the old entrynodes.c file? A related question here is whether both algos should co-exist, or should the new one replace the old? I leave both decisions to the person who writes the code. --- Implementation plan: 1) Create main guard selection state data structure (instead of using global variables) ** Ideally there should be no global variables, so that in the future we can have multiple parallel guardsets (e.g. prop247) ** Thoughtworks: guard_selection_t 2) Sample SAMPLED_GUARDS from GUARDS ** Fill in state for each sampled guard (added_on_date, etc.) ** Save/load SAMPLED_GUARDS to/from state ** Unittests for sampling/saving/loading ** Thoughtworks function: fill_sampled_guards_from_entrynodes() ** Thoughtworks unittest: test_fill_in_sampled_set() 3) Derive FILTERED_GUARDS / USABLE_FILTERED_GUARDS from SAMPLED_GUARDS ** The filtering function can be called at any point (e.g. ON_CONSENSUS) ** Write unittests ensuring correctness of filtering ** See populate_live_entry_guards() ** Thoughtworks function: filter_set() / filter_sampled() 4) Maintain set of PRIMARY_GUARDS ** Functions to extract set of (live) primary guards out of FILTERED_GUARDS ** Unittest ** Thoughtworks function: retry_primary_guards() / next_primary_guard() ** Thoughtworks unittests: test_next_primary_guard() 5) Selecting guards for circuits ** Meant to replace choose_random_entry_impl() ** See section SELECTING in prop271 ** Add new circuit states to or_circuit_t ** Implement the guard selection logic ** Unittests on circuit state machine ** Unittests on guard selection logic 6) Update guard state when a circuit fails/succeeds ** Section ON_FAIL and ON_SUCCESS ** See entry_guard_register_connect_status() ** Unittest 7) Maintain CONFIRMED_GUARDS ** Add guard to CONFIRMED_GUARDS when circuit succeeds ** Fill in state info for each confirmed guard (confirmed_on_date, etc.) ** Migration from old state format to new ** Thoughtworks unittest: test_used_guards_parse_state() 8) Retry schedule for guards (section RETRYING) ** See entry_is_time_to_retry() ** Unittest 9) Update list of waiting circuits (section UPDATE_WAITING) ** Implementation of guard "higher priority" ordering (see CONFIRMED) ** Unittest 10) Update state whenever we get a new consensus (section ON_CONSENSUS) ** Update SAMPLED_GUARDS elements (section SAMPLED) ** Remove obsolete/expired guards ** See entry_guards_compute_status() / remove_obsolete_entry_guards() ** Thoughtworks function: entry_guards_update_profiles() 11) Bridge support ** Thoughtworks function: fill_in_from_bidges() 12) Separate state instances when EntryNodes/ExcludeNodes/etc are used ** See prop271 MEANINGFUL_RESTRICTION_FRAC etc. 13) Glue things together! ** Add clean and meaningful logging so that we can *heavily* field test the feature in our machines

1 0

[GSOC16] Fingerprint Central - Status report n°5
by Pierre Laperdrix 29 Jul '16

29 Jul '16

Hi everyone, Here is my fifth status report for my GSOC project. Repository: https://github.com/plaperdr/fp-central Beta: https://fpcentral.irisa.fr 1 - Support of TBB versions for statistics through tags I have added the selection of TBB versions for the custom statistics page. In order to do that, I designed and developed the tag system that I planned to develop much later for the project. For those who do not know what tags are, it is a way to classify fingerprints and put them into different categories. By analyzing collected fingerprints, the system can assign one or more tags to them. After several iterations, I think I found a good balance between expressiveness and simplicity. You can find the documentation on how to add your own tag to the system here: https://github.com/plaperdr/fp-central/blob/master/fingerprint/tags/README.… I have added a command to refresh and recompute all tags in case a new one is added to the system or if the logic of one tag has been changed. Right now, the tag system is used to tag each fingerprint with the corresponding TBB version but it can be used later on for many different things like identifying the OS or the type of the browser. 2 - Better and more complete custom statistics page. The custom statistics page is now much more complete than before. In addition to the already-existing pie chart, I have added an HTML table which gives the complete list of data from the chosen attributes. You can sort the data and download it in several different formats. It is a handy feature if you want to put the fingerprints into a different software or if you want to copy/paste them for a thorough bug report. I've pushed the latest updates today to the beta server so that you can try them right now and send me feedback on them. My goal for the next two weeks is to iron things out in the beta and fix bugs. I still need to make sure that only TBB fingerprints are stored in the database. Feature-wise, it seems that I'm getting near the end on the list of features I wrote in my proposal. The only real big one that is left is to have a page to see if your complete TBB fingerprint is "acceptable" (right now, only several attributes are checked). Finally, for the final stretch in the second half of august, I'm planning on completing the documentation and the french localization while making sure that deployment and updates of the site are as easy as possible. Have a great week-end, Pierre

1 0

[GSoC 2016] Orfox - Report 5
by Amogh Pradeep 28 Jul '16

28 Jul '16

Hey guys, These are the things I’ve done in the past couple of weeks. 1) GetUserDataDirectoryHome in nsXREDirProvider. Looks like the blocker in nsXREDirProvider was the GetUserDataDirectoryHome function. https://trac.torproject.org/projects/tor/ticket/19706 <https://trac.torproject.org/projects/tor/ticket/19706> is the ticket opened to track updates regarding this. Once this is ready, we would be able to upstream all the Orfox patches to the tor-browser repo. 2) Commits clean up. I’ve made a couple of passes on the commits to Orfox that we have, to reorder them and merge them as much as possible. This will make it easier to maintain things in the future, and it would be easier to review code as we merge it with tor browser. 3) Disabling search widget. The search widget for Orfox has now been removed. https://dev.guardianproject.info/issues/7444 <https://dev.guardianproject.info/issues/7444> Commit: https://github.com/amoghbl1/tor-browser/commit/f415a732a8e50c2c34f43d905c36… <https://github.com/amoghbl1/tor-browser/commit/f415a732a8e50c2c34f43d905c36…> At this moment, I don’t see the advantage of having this feature and would prefer not supporting it, in order to make it easier to maintain Orfox. In other new, I am migrating this week from CET to IST! This might make it slightly harder to be online during most active hours, but I will try to be around. Hope you guys have a great weekend. Best, Amogh Pradeep amoghbl1(a)gmail.com

2 1

Further sandboxing Tor Browser (aka Tor + Firejail redux).
by Yawning Angel 22 Jul '16

22 Jul '16

I felt randomly inspired, so I spent some time poking at my firejail Tor Browser sandboxing effort, and made progress towards something more robust. In particular, I switched to using AF_LOCAL (aka AF_UNIX) sockets, via a brute force-ish approach, which seems to be working well, despite some caveats. In theory, this raises the difficulty of proxy bypass type things significantly (in that you need a sandbox escape sploit to use AF_INET/AF_INET6). In practice, who knows, for what it's worth the subterranean reptiloids who beam thoughts into my head from their satellite installations reassure me that it's safer. WARNING: * If you are not comfortable with running/configuring a system tor daemon, keeping it up to date, and keeping it in sync with the one shipped with Tor Browser when it diverges, stop reading, and wait for the Tor Browser people to come up with their own sandboxing solution. * If you expect me to provide any sort of help beyond "merging sensible patches", likewise stop reading, and wait for the Tor Browser people to come up with their own sandboxing solution. Really. Prerequisites: * A system tor daemon with the Control port and Socks port exposed as AF_LOCAL sockets, with permissions configured such that the user running Tor Browser can write to both. Due to torbutton quirks, having a SocksPort on 127.0.0.1:9050 is also a good idea, though not strictly necessary. * A recent version of firejail (nb: I did not test this with a USER_NS kernel. It may be better, it may be worse.). Setup: 1. Clone https://git.schwanenlied.me/yawning/tor-firejail 2. (Optional) Back up your Tor Browser install so you can go back to the way things used to be when you mess up. 3. Follow the instructions in the README.md, adjusting paths as appropriate. It should be self explanatory. If it's not, revert back to how things were, and wait for a more official solution. Depending on how your tor is setup, you probably need to set a bunch of env vars (At a minimum, TOR_SKIP_LAUNCH is required. Everything else depends on how the control/socks sockets are setup, and where they live, the modified start-tor-browser script has more documentation in comments.). 4. Run start-tor-browser, use lsof/netstat/whatever to verify that the AF_LOCAL sockets are being used. Caveats: * The security of this assumes that the firejail options I use to enforce address family restrictions work as advertised. * You need a system-wide tor, because if you tried to run the tor daemon inside the sandbox, it won't be able to access the network. * The firejail profile I provide disallows access to everything in the user's home directory except for the directory that actually contains Tor Browser. Edit the profile to change this if you care about it. I like the behavior for various reasons so I'm not going to change it. * The `about:tor` display falsely reports an error unless there's a SocksPort on 9050, due to torbutton. You can alternatively lie to torbutton about where the listener is if you engage in control port related tinfoil hattery. * You need to repeat parts of the installation steps after updating. How it works: * firejail's sandbox forbids all non-AF_LOCAL sockets. * A small stub is injected into the firefox process at runtime via LD_PRELOAD that fixes up socket calls to go to the system wide tor instance's AF_LOCAL sockets. Random thoughts: * The stub should be adequate for using other similar sandboxing solutions (eg: flatpak's bubblewrap, Google's thing, whatever). The code is compact, and is something anyone half way competent could write in 15 mins or so (it may have dumb bugs, dunno). Using the stub on it's own without the sandbox would be a horrible idea. * Assuming Tor Browser works as advertised, the only reason it needs control port access for this sort of use case is the circuit display (as of torbutton commit 36d849291ec0b20a58cccc2cd846fcd2540c9bbe, sending NEWNYM should be unnecessary if domain isolation is applied to everything). Regards, -- Yawning Angel

2 2