July 2016 - tor-dev - lists.torproject.org

Usability Improvements for Atlas (was Re: Globe is now retired)
by Iain R. Learmonth 02 Aug '16

02 Aug '16

Hi All, On 29/06/16 09:56, Karsten Loesing wrote: > tl;dr: Globe is now retired in favor of an improved Atlas. Atlas is improved, but I'd like to improve it further. I've been tackling #6787 looking to improve the homepage and make Atlas easier to use for someone that isn't already a Tor expert. The changes I'm proposing in my fix for #6787 [1] do make some noticeable UI changes and I just want to gather some opinions on whether or not Atlas users feel that this is taking it in the right direction. The "Top 10 Relays" page was added to Atlas in order to make sure that functionality was not lost when shutting down globe. (See #5430 [2]) In my proposed patch, I'm also adding an "Authorities" page and adding text to both of these pages to explain what you're seeing. These are linked from the homepage, and from the top navbar. I'm also moving the introduction and explanation of the Atlas features from the About page to the homepage and introducing a new Help page that explains the details that are found in the relay details pages. You can find a live example of this patch at: https://irl.github.io/atlas/#/ and a full list of changes are detailed in #6787. Please let me know if you can see any reason to not merge this, or if you have any suggestions for small changes before merging. Thanks, Iain. [1]: https://trac.torproject.org/projects/tor/ticket/5430 [2]: https://trac.torproject.org/projects/tor/ticket/6787

5 5

Tor Browser and Privoxy
by Paulo Roberto 02 Aug '16

02 Aug '16

Hello, By default the Tor Browser Bundle connects through SOCKS5 to the TOR daemon. Is there any advantage today to configure Privoxy to use TOR and configure the Tor Browser in the Tor Network Settings to use proxy access type HTTP/HTTPS pointing to Privoxy? Thanks in advance. Paulo Roberto.

3 3

Re: [tor-dev] prop224: Ditching key blinding for shorter onion addresses
by Nick Mathewson 30 Jul '16

30 Jul '16

whoops, didn't reply-all. On Sat, Jul 30, 2016 at 1:00 PM, Nick Mathewson <nickm(a)alum.mit.edu> wrote: > On Sat, Jul 30, 2016 at 9:32 AM, George Kadianakis <desnacked(a)riseup.net> wrote: > [...] >> >> Please let me know in what way this scheme is completely broken! > > I think it has the same problem as the last one: there does not appear > to be a way to keep an attacker from putting their own record in place > using your query_key, and thereby performing a DoS on your hidden > service. > > > -- > Nick

1 0

[GSoC] CONIKS for Tor Messenger - status report #5
by Huy Vu Quoc 29 Jul '16

29 Jul '16

Hi everyone, These are things that I have been doing in the last two weeks: - I and my mentors have had a discussions for next steps for the project. We decided to reorganize our project’s repo [1]. All repos has been merged into one [1]. This helps us maintain and keep tracking the development easier. - A lot of issues and PRs have been resolved/merged. The merkletree library is almost done for now. - Thanks for Ismail and Arlo’s suggestions, the client library for crypto verifications (including the VRF verification, Ed25519 signature verification, auth path verification and so on) is implemented in Go and compiled with `c-shared` build mode [2]. This library then will be wrapped with js-ctypes to use in the add-on. I have successfully loaded the library into the add-on in Linux machines. There’s still some problem since Tor Messenger for OS X is compiled from a Linux machine, while the library is compiled from macOS, then when loading the library into Tor Messenger for OS X, Tor Messenger may crash. We will probably dig deeper on this issue. Windows’ DLL version of the library also should be taken care. Next steps are mostly to focus on the client-side, along with finishing the server reference implementation. Here’s thing I expect in the next weeks: - Merge the server-registration and registration-bots PR. - Merge the coniks-preference PR. - Merge the PR of client library. - Submit PR for client registration and lookup. I also expect to have a working version of project before August 15, including the functions which I described in my proposal. Best, Huy — [1] https://github.com/coniks-sys/coniks-go [2] https://github.com/coniks-sys/coniks-go/tree/client-cgo

1 0

[GSOC 16] Ahmia status update #5
by Ismael R 29 Jul '16

29 Jul '16

Hi everyone, I'm working on ahmia.fi, the hidden service search engine and you're reading status update #5. During the last two weeks I worked on improving search. It required working on a new index format and updating the crawler to fill this new index. Search now has these new features: - Token search is done in addition to pure full-text search. For instance, "foxes" and "fox" are different words but are indexed as the same token. Same for "running" and "run", "tor's and tor". - Associated words are also stored. For instance, "I’m not happy I’m working" have the same words than "I’m happy I’m not working", but the meaning is different. - Meaningful words have way more impact than frequent words. - The text used with a link (called anchor text) is now stored with the page which is the target of the link. Let's say a site A is linking a site B like that: <a href="B">B is a great search engine</a>. Searching "great search engine" is going to return B instead of A. These anchors could also replace a title in the results list if the site has no title (.txt files are like that). - A word present in the title, description, anchors has way more strength than a word present in the content of a page. - An authority score is computed for each page thanks to the pagerank algorithm. Important pages (like homepages) tends to appear first in the result list. It also solves problems we have with duplicated content. - The result list now returns pages instead of sites/domains. Before updating ahmia.fi, I need to finish porting the rest of the django site (API and statistics) to the new structure. I hope to finish this next week so you can try it. After this, here are the features I plan to implement in the remaining time: - Add a language to the search engine (site:msydqstlz2kzerdg.onion to specify a domain, -term to exclude a term, filetype:txt to specify a filetype, lang:fr to specify a language, etc). - Use If-Modified-Since HTTP header to avoid crawling unmodified pages. - Improve documentation - Add unit tests Also, I have some ideas I won't have the time to implement during GSoC. Hopefully, I will do it later because I plan to continue working with the Tor community :) A small list: - Improve search for languages other than english by making one index per language - Make the crawler better understand query parameters. Some are useful (?page=3), some are useless (?printable=1). - Better tokenize the url to extract words from it. - Add highlights to each result (especially for those without meta- description) to show the context in which query terms appears. - Make a script to make sure every indexed url is still online. - Make a true RESTful API. I see you in two weeks for the next status report and maybe sooner if I push something online. Cheers! Ismael

1 0

Implementation plan for prop271 "Another algorithm for guard selection"
by George Kadianakis 29 Jul '16

29 Jul '16

Hello people, our new guard algorithm proposal (prop271) is getting finalized and we are planning to move to implementation soon. You can find the proposal here: https://gitweb.torproject.org/torspec.git/tree/proposals/271-another-guard-… You can also find the STRIKE team's implementation of proposal 259 here: https://github.com/twstrike/tor_for_patching Ideally we should borrow as much code as possible from their implementation. To move forward with engineering, here follows an initial thirteen step implementation plan for proposal 271. Where it made sense, I tried to provide links to current functions that can be used, or to code that the STRIKE team wrote and can be reused or studied. Let me know what I should make clearer or where I should go further in depth. --- Design decisions: An initial design decision here is whether we should make a new file for this feature or use the old entrynodes.c file? A related question here is whether both algos should co-exist, or should the new one replace the old? I leave both decisions to the person who writes the code. --- Implementation plan: 1) Create main guard selection state data structure (instead of using global variables) ** Ideally there should be no global variables, so that in the future we can have multiple parallel guardsets (e.g. prop247) ** Thoughtworks: guard_selection_t 2) Sample SAMPLED_GUARDS from GUARDS ** Fill in state for each sampled guard (added_on_date, etc.) ** Save/load SAMPLED_GUARDS to/from state ** Unittests for sampling/saving/loading ** Thoughtworks function: fill_sampled_guards_from_entrynodes() ** Thoughtworks unittest: test_fill_in_sampled_set() 3) Derive FILTERED_GUARDS / USABLE_FILTERED_GUARDS from SAMPLED_GUARDS ** The filtering function can be called at any point (e.g. ON_CONSENSUS) ** Write unittests ensuring correctness of filtering ** See populate_live_entry_guards() ** Thoughtworks function: filter_set() / filter_sampled() 4) Maintain set of PRIMARY_GUARDS ** Functions to extract set of (live) primary guards out of FILTERED_GUARDS ** Unittest ** Thoughtworks function: retry_primary_guards() / next_primary_guard() ** Thoughtworks unittests: test_next_primary_guard() 5) Selecting guards for circuits ** Meant to replace choose_random_entry_impl() ** See section SELECTING in prop271 ** Add new circuit states to or_circuit_t ** Implement the guard selection logic ** Unittests on circuit state machine ** Unittests on guard selection logic 6) Update guard state when a circuit fails/succeeds ** Section ON_FAIL and ON_SUCCESS ** See entry_guard_register_connect_status() ** Unittest 7) Maintain CONFIRMED_GUARDS ** Add guard to CONFIRMED_GUARDS when circuit succeeds ** Fill in state info for each confirmed guard (confirmed_on_date, etc.) ** Migration from old state format to new ** Thoughtworks unittest: test_used_guards_parse_state() 8) Retry schedule for guards (section RETRYING) ** See entry_is_time_to_retry() ** Unittest 9) Update list of waiting circuits (section UPDATE_WAITING) ** Implementation of guard "higher priority" ordering (see CONFIRMED) ** Unittest 10) Update state whenever we get a new consensus (section ON_CONSENSUS) ** Update SAMPLED_GUARDS elements (section SAMPLED) ** Remove obsolete/expired guards ** See entry_guards_compute_status() / remove_obsolete_entry_guards() ** Thoughtworks function: entry_guards_update_profiles() 11) Bridge support ** Thoughtworks function: fill_in_from_bidges() 12) Separate state instances when EntryNodes/ExcludeNodes/etc are used ** See prop271 MEANINGFUL_RESTRICTION_FRAC etc. 13) Glue things together! ** Add clean and meaningful logging so that we can *heavily* field test the feature in our machines

1 0

[GSOC16] Fingerprint Central - Status report n°5
by Pierre Laperdrix 29 Jul '16

29 Jul '16

Hi everyone, Here is my fifth status report for my GSOC project. Repository: https://github.com/plaperdr/fp-central Beta: https://fpcentral.irisa.fr 1 - Support of TBB versions for statistics through tags I have added the selection of TBB versions for the custom statistics page. In order to do that, I designed and developed the tag system that I planned to develop much later for the project. For those who do not know what tags are, it is a way to classify fingerprints and put them into different categories. By analyzing collected fingerprints, the system can assign one or more tags to them. After several iterations, I think I found a good balance between expressiveness and simplicity. You can find the documentation on how to add your own tag to the system here: https://github.com/plaperdr/fp-central/blob/master/fingerprint/tags/README.… I have added a command to refresh and recompute all tags in case a new one is added to the system or if the logic of one tag has been changed. Right now, the tag system is used to tag each fingerprint with the corresponding TBB version but it can be used later on for many different things like identifying the OS or the type of the browser. 2 - Better and more complete custom statistics page. The custom statistics page is now much more complete than before. In addition to the already-existing pie chart, I have added an HTML table which gives the complete list of data from the chosen attributes. You can sort the data and download it in several different formats. It is a handy feature if you want to put the fingerprints into a different software or if you want to copy/paste them for a thorough bug report. I've pushed the latest updates today to the beta server so that you can try them right now and send me feedback on them. My goal for the next two weeks is to iron things out in the beta and fix bugs. I still need to make sure that only TBB fingerprints are stored in the database. Feature-wise, it seems that I'm getting near the end on the list of features I wrote in my proposal. The only real big one that is left is to have a page to see if your complete TBB fingerprint is "acceptable" (right now, only several attributes are checked). Finally, for the final stretch in the second half of august, I'm planning on completing the documentation and the french localization while making sure that deployment and updates of the site are as easy as possible. Have a great week-end, Pierre

1 0

[GSoC 2016] Orfox - Report 5
by Amogh Pradeep 28 Jul '16

28 Jul '16

Hey guys, These are the things I’ve done in the past couple of weeks. 1) GetUserDataDirectoryHome in nsXREDirProvider. Looks like the blocker in nsXREDirProvider was the GetUserDataDirectoryHome function. https://trac.torproject.org/projects/tor/ticket/19706 <https://trac.torproject.org/projects/tor/ticket/19706> is the ticket opened to track updates regarding this. Once this is ready, we would be able to upstream all the Orfox patches to the tor-browser repo. 2) Commits clean up. I’ve made a couple of passes on the commits to Orfox that we have, to reorder them and merge them as much as possible. This will make it easier to maintain things in the future, and it would be easier to review code as we merge it with tor browser. 3) Disabling search widget. The search widget for Orfox has now been removed. https://dev.guardianproject.info/issues/7444 <https://dev.guardianproject.info/issues/7444> Commit: https://github.com/amoghbl1/tor-browser/commit/f415a732a8e50c2c34f43d905c36… <https://github.com/amoghbl1/tor-browser/commit/f415a732a8e50c2c34f43d905c36…> At this moment, I don’t see the advantage of having this feature and would prefer not supporting it, in order to make it easier to maintain Orfox. In other new, I am migrating this week from CET to IST! This might make it slightly harder to be online during most active hours, but I will try to be around. Hope you guys have a great weekend. Best, Amogh Pradeep amoghbl1(a)gmail.com

2 1

Further sandboxing Tor Browser (aka Tor + Firejail redux).
by Yawning Angel 22 Jul '16

22 Jul '16

I felt randomly inspired, so I spent some time poking at my firejail Tor Browser sandboxing effort, and made progress towards something more robust. In particular, I switched to using AF_LOCAL (aka AF_UNIX) sockets, via a brute force-ish approach, which seems to be working well, despite some caveats. In theory, this raises the difficulty of proxy bypass type things significantly (in that you need a sandbox escape sploit to use AF_INET/AF_INET6). In practice, who knows, for what it's worth the subterranean reptiloids who beam thoughts into my head from their satellite installations reassure me that it's safer. WARNING: * If you are not comfortable with running/configuring a system tor daemon, keeping it up to date, and keeping it in sync with the one shipped with Tor Browser when it diverges, stop reading, and wait for the Tor Browser people to come up with their own sandboxing solution. * If you expect me to provide any sort of help beyond "merging sensible patches", likewise stop reading, and wait for the Tor Browser people to come up with their own sandboxing solution. Really. Prerequisites: * A system tor daemon with the Control port and Socks port exposed as AF_LOCAL sockets, with permissions configured such that the user running Tor Browser can write to both. Due to torbutton quirks, having a SocksPort on 127.0.0.1:9050 is also a good idea, though not strictly necessary. * A recent version of firejail (nb: I did not test this with a USER_NS kernel. It may be better, it may be worse.). Setup: 1. Clone https://git.schwanenlied.me/yawning/tor-firejail 2. (Optional) Back up your Tor Browser install so you can go back to the way things used to be when you mess up. 3. Follow the instructions in the README.md, adjusting paths as appropriate. It should be self explanatory. If it's not, revert back to how things were, and wait for a more official solution. Depending on how your tor is setup, you probably need to set a bunch of env vars (At a minimum, TOR_SKIP_LAUNCH is required. Everything else depends on how the control/socks sockets are setup, and where they live, the modified start-tor-browser script has more documentation in comments.). 4. Run start-tor-browser, use lsof/netstat/whatever to verify that the AF_LOCAL sockets are being used. Caveats: * The security of this assumes that the firejail options I use to enforce address family restrictions work as advertised. * You need a system-wide tor, because if you tried to run the tor daemon inside the sandbox, it won't be able to access the network. * The firejail profile I provide disallows access to everything in the user's home directory except for the directory that actually contains Tor Browser. Edit the profile to change this if you care about it. I like the behavior for various reasons so I'm not going to change it. * The `about:tor` display falsely reports an error unless there's a SocksPort on 9050, due to torbutton. You can alternatively lie to torbutton about where the listener is if you engage in control port related tinfoil hattery. * You need to repeat parts of the installation steps after updating. How it works: * firejail's sandbox forbids all non-AF_LOCAL sockets. * A small stub is injected into the firefox process at runtime via LD_PRELOAD that fixes up socket calls to go to the system wide tor instance's AF_LOCAL sockets. Random thoughts: * The stub should be adequate for using other similar sandboxing solutions (eg: flatpak's bubblewrap, Google's thing, whatever). The code is compact, and is something anyone half way competent could write in 15 mins or so (it may have dumb bugs, dunno). Using the stub on it's own without the sandbox would be a horrible idea. * Assuming Tor Browser works as advertised, the only reason it needs control port access for this sort of use case is the circuit display (as of torbutton commit 36d849291ec0b20a58cccc2cd846fcd2540c9bbe, sending NEWNYM should be unnecessary if domain isolation is applied to everything). Regards, -- Yawning Angel

2 2

Tor with collective signatures
by Nicolas Gailly 22 Jul '16

22 Jul '16

Hi, First, thanks a lot both of you for your in-depth comments, I know it takes time, but they've been very helpful! Then I mean to ask you about what you said regarding the fallback directory keys. So keys are not embedded but only hashes, OK. As you pointed out, CoSi would need to download full descriptors in order to safely get the private keys of all the witnesses. This solution would need to clearly be defined as you said in order to be correct. However, there exists another solution and I'd like to know if it looks reasonable in the context of Tor . => Considering the two roles of Fallback directory mirrors are different (serving the consensus document to Tor / collectively signing on it), we thought it would make sense to create new keys for this role and embed those into the Tor source code. These keys would have to be ed25519 keys (same size as the ID), so the size overhead is minimal (32 bytes * 100 = 3.2KB vs 1.5MB). The only reason I can think of why Tor includes hashes instead of public keys is because of local storage: How does Tor consider an increase of storage need by 3.2KB ? Is it acceptable or not ? That would : * Bring down the need for exchanging the descriptors between the fallbacks / no more bandwidth / * No complex scenario such as "fallback key download mechanism" * Simplify the verification on the client side also (no need to get the keys if you already have them). * Add an additional steps for the relays operators where they would need to create that key and inform the Tor people about their public key. Maybe you thought of this solution and there's a reason why you did not mention it, or maybe you did not, then what do you think of it ? Cheers, Nicolas

4 7