- tor-dev - lists.torproject.org

Usability Improvements for Atlas (was Re: Globe is now retired)
by Iain R. Learmonth 03 Aug '16

03 Aug '16

Hi All, On 29/06/16 09:56, Karsten Loesing wrote: > tl;dr: Globe is now retired in favor of an improved Atlas. Atlas is improved, but I'd like to improve it further. I've been tackling #6787 looking to improve the homepage and make Atlas easier to use for someone that isn't already a Tor expert. The changes I'm proposing in my fix for #6787 [1] do make some noticeable UI changes and I just want to gather some opinions on whether or not Atlas users feel that this is taking it in the right direction. The "Top 10 Relays" page was added to Atlas in order to make sure that functionality was not lost when shutting down globe. (See #5430 [2]) In my proposed patch, I'm also adding an "Authorities" page and adding text to both of these pages to explain what you're seeing. These are linked from the homepage, and from the top navbar. I'm also moving the introduction and explanation of the Atlas features from the About page to the homepage and introducing a new Help page that explains the details that are found in the relay details pages. You can find a live example of this patch at: https://irl.github.io/atlas/#/ and a full list of changes are detailed in #6787. Please let me know if you can see any reason to not merge this, or if you have any suggestions for small changes before merging. Thanks, Iain. [1]: https://trac.torproject.org/projects/tor/ticket/5430 [2]: https://trac.torproject.org/projects/tor/ticket/6787

5 5

Tor Browser and Privoxy
by Paulo Roberto 02 Aug '16

02 Aug '16

Hello, By default the Tor Browser Bundle connects through SOCKS5 to the TOR daemon. Is there any advantage today to configure Privoxy to use TOR and configure the Tor Browser in the Tor Network Settings to use proxy access type HTTP/HTTPS pointing to Privoxy? Thanks in advance. Paulo Roberto.

3 3

prop224: zero bits in addresses
by teor 02 Aug '16

02 Aug '16

Hi all, At the Montreal hidden service hackfest, we discussed another scheme shortening prop224 .onion addresses. The only drawback is that it's exponentially difficult, so it can only chop off a few characters. I'm describing it here because it has other useful properties, and can be used as: 1) an address versioning scheme, and/or 2) a configurable proof-of-work threshold for generating valid addresses. The basic scheme works like this: (those of you familiar with bitcoin might recognise it) A) Valid .onion address hashes must start with a certain, hard-coded number of 0 bits, followed by a 1 bit. (We include the 1 bit, so that addresses with additional 0 bits can be used by a later version of the protocol.) B) When generating a private key, the hidden service must discard any keys that yield .onion addresses that don't fit this bit pattern. C) When creating a textual .onion address, Tor removes the hard-coded bits from the address. D) When parsing a textual .onion address, Tor adds the hard-coded bits from the address. Otherwise, the protocol proceeds as it would in prop224. This kind of scheme is mentioned as a TODO in the existing proposal: [TODO: Alternatively, we could require that the first bit of the master key always be zero, and use a 51-byte encoding. Or we could require that the first two bits be zero, and use a 51-byte encoding and reserve the first bit. Or we could require that the first nine bits, or ten bits be zero, etc.] In my very basic tests using Yawning's horse25519 on a laptop, it seems that 15 bits takes less than a second, but 20 bits can take from a few seconds to around a minute. Prop 224 .onion addresses already have 4 unused bits, so we could easily chop off 3-4 characters using this scheme, with only a small amount of computation the first time the hidden service runs. Without a scheme like this, it's possible to generate hundreds of thousands of valid .onion addresses per minute on most recent processors. I'm not sure if this is a problem or not. So we have to decide whether the added complexity is worth having addresses that are 3-4 characters shorter. I'm not sure that it is, unless we like the idea of making it harder to generate valid .onion addresses, or unless we want to allow for extensible future versioning of .onion addresses, beyond the existing 4 unused bits. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmmp: teor at torproject dot org

2 1

metrics-lib/DescripTor 1.3.1 is released
by Karsten Loesing 01 Aug '16

01 Aug '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello devs, we just released metrics-lib/DescripTor 1.3.1: https://dist.torproject.org/descriptor/1.3.1/ - From the change log: # Changes in version 1.3.1 - 2016-08-01 * Medium changes - Adapt to CollecTor's new date format to make DescriptorCollector work again. Happy descriptor collecting! All the best, Karsten -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJXn0XrAAoJEC3ESO/4X7XB+q8IAMtu1b/mhvUvS7nlETa2ejCF BDhHNMK2v23pH0usku2AaZuzc0AkG4r0d3Nyb0jDRH5J/oNu2kDCUXIbDqJxwfX7 y5UlNaZgLTcYV7eJjWAqbvAmCBQMpEYJLhhtPqO81DgZpNbMP08doAgis30TPfiE KRUrKTIJxcDVQLfvsSFOOZj3PDjkQScWtW+icBF2PdD1u62/Dhv1QEJWzBTKH2xq etmc5VsNAir/mV2/le68CsIlSNuwGH9yU4xRlZ8QEb+MOnnS76p3onnsdqmHhytJ V0hXlViWNRjFd0EzcuCp4tvkWK3//xBC19AjQkwuRCOCbbeuOpQLZI00BJV9NCg= =0qss -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] prop224: Ditching key blinding for shorter onion addresses
by Nick Mathewson 30 Jul '16

30 Jul '16

whoops, didn't reply-all. On Sat, Jul 30, 2016 at 1:00 PM, Nick Mathewson <nickm(a)alum.mit.edu> wrote: > On Sat, Jul 30, 2016 at 9:32 AM, George Kadianakis <desnacked(a)riseup.net> wrote: > [...] >> >> Please let me know in what way this scheme is completely broken! > > I think it has the same problem as the last one: there does not appear > to be a way to keep an attacker from putting their own record in place > using your query_key, and thereby performing a DoS on your hidden > service. > > > -- > Nick

1 0

[GSoC] CONIKS for Tor Messenger - status report #5
by Huy Vu Quoc 30 Jul '16

30 Jul '16

Hi everyone, These are things that I have been doing in the last two weeks: - I and my mentors have had a discussions for next steps for the project. We decided to reorganize our project’s repo [1]. All repos has been merged into one [1]. This helps us maintain and keep tracking the development easier. - A lot of issues and PRs have been resolved/merged. The merkletree library is almost done for now. - Thanks for Ismail and Arlo’s suggestions, the client library for crypto verifications (including the VRF verification, Ed25519 signature verification, auth path verification and so on) is implemented in Go and compiled with `c-shared` build mode [2]. This library then will be wrapped with js-ctypes to use in the add-on. I have successfully loaded the library into the add-on in Linux machines. There’s still some problem since Tor Messenger for OS X is compiled from a Linux machine, while the library is compiled from macOS, then when loading the library into Tor Messenger for OS X, Tor Messenger may crash. We will probably dig deeper on this issue. Windows’ DLL version of the library also should be taken care. Next steps are mostly to focus on the client-side, along with finishing the server reference implementation. Here’s thing I expect in the next weeks: - Merge the server-registration and registration-bots PR. - Merge the coniks-preference PR. - Merge the PR of client library. - Submit PR for client registration and lookup. I also expect to have a working version of project before August 15, including the functions which I described in my proposal. Best, Huy — [1] https://github.com/coniks-sys/coniks-go [2] https://github.com/coniks-sys/coniks-go/tree/client-cgo

1 0

[GSOC 16] Ahmia status update #5
by Ismael R 30 Jul '16

30 Jul '16

Hi everyone, I'm working on ahmia.fi, the hidden service search engine and you're reading status update #5. During the last two weeks I worked on improving search. It required working on a new index format and updating the crawler to fill this new index. Search now has these new features: - Token search is done in addition to pure full-text search. For instance, "foxes" and "fox" are different words but are indexed as the same token. Same for "running" and "run", "tor's and tor". - Associated words are also stored. For instance, "I’m not happy I’m working" have the same words than "I’m happy I’m not working", but the meaning is different. - Meaningful words have way more impact than frequent words. - The text used with a link (called anchor text) is now stored with the page which is the target of the link. Let's say a site A is linking a site B like that: <a href="B">B is a great search engine</a>. Searching "great search engine" is going to return B instead of A. These anchors could also replace a title in the results list if the site has no title (.txt files are like that). - A word present in the title, description, anchors has way more strength than a word present in the content of a page. - An authority score is computed for each page thanks to the pagerank algorithm. Important pages (like homepages) tends to appear first in the result list. It also solves problems we have with duplicated content. - The result list now returns pages instead of sites/domains. Before updating ahmia.fi, I need to finish porting the rest of the django site (API and statistics) to the new structure. I hope to finish this next week so you can try it. After this, here are the features I plan to implement in the remaining time: - Add a language to the search engine (site:msydqstlz2kzerdg.onion to specify a domain, -term to exclude a term, filetype:txt to specify a filetype, lang:fr to specify a language, etc). - Use If-Modified-Since HTTP header to avoid crawling unmodified pages. - Improve documentation - Add unit tests Also, I have some ideas I won't have the time to implement during GSoC. Hopefully, I will do it later because I plan to continue working with the Tor community :) A small list: - Improve search for languages other than english by making one index per language - Make the crawler better understand query parameters. Some are useful (?page=3), some are useless (?printable=1). - Better tokenize the url to extract words from it. - Add highlights to each result (especially for those without meta- description) to show the context in which query terms appears. - Make a script to make sure every indexed url is still online. - Make a true RESTful API. I see you in two weeks for the next status report and maybe sooner if I push something online. Cheers! Ismael

1 0

Implementation plan for prop271 "Another algorithm for guard selection"
by George Kadianakis 29 Jul '16

29 Jul '16

Hello people, our new guard algorithm proposal (prop271) is getting finalized and we are planning to move to implementation soon. You can find the proposal here: https://gitweb.torproject.org/torspec.git/tree/proposals/271-another-guard-… You can also find the STRIKE team's implementation of proposal 259 here: https://github.com/twstrike/tor_for_patching Ideally we should borrow as much code as possible from their implementation. To move forward with engineering, here follows an initial thirteen step implementation plan for proposal 271. Where it made sense, I tried to provide links to current functions that can be used, or to code that the STRIKE team wrote and can be reused or studied. Let me know what I should make clearer or where I should go further in depth. --- Design decisions: An initial design decision here is whether we should make a new file for this feature or use the old entrynodes.c file? A related question here is whether both algos should co-exist, or should the new one replace the old? I leave both decisions to the person who writes the code. --- Implementation plan: 1) Create main guard selection state data structure (instead of using global variables) ** Ideally there should be no global variables, so that in the future we can have multiple parallel guardsets (e.g. prop247) ** Thoughtworks: guard_selection_t 2) Sample SAMPLED_GUARDS from GUARDS ** Fill in state for each sampled guard (added_on_date, etc.) ** Save/load SAMPLED_GUARDS to/from state ** Unittests for sampling/saving/loading ** Thoughtworks function: fill_sampled_guards_from_entrynodes() ** Thoughtworks unittest: test_fill_in_sampled_set() 3) Derive FILTERED_GUARDS / USABLE_FILTERED_GUARDS from SAMPLED_GUARDS ** The filtering function can be called at any point (e.g. ON_CONSENSUS) ** Write unittests ensuring correctness of filtering ** See populate_live_entry_guards() ** Thoughtworks function: filter_set() / filter_sampled() 4) Maintain set of PRIMARY_GUARDS ** Functions to extract set of (live) primary guards out of FILTERED_GUARDS ** Unittest ** Thoughtworks function: retry_primary_guards() / next_primary_guard() ** Thoughtworks unittests: test_next_primary_guard() 5) Selecting guards for circuits ** Meant to replace choose_random_entry_impl() ** See section SELECTING in prop271 ** Add new circuit states to or_circuit_t ** Implement the guard selection logic ** Unittests on circuit state machine ** Unittests on guard selection logic 6) Update guard state when a circuit fails/succeeds ** Section ON_FAIL and ON_SUCCESS ** See entry_guard_register_connect_status() ** Unittest 7) Maintain CONFIRMED_GUARDS ** Add guard to CONFIRMED_GUARDS when circuit succeeds ** Fill in state info for each confirmed guard (confirmed_on_date, etc.) ** Migration from old state format to new ** Thoughtworks unittest: test_used_guards_parse_state() 8) Retry schedule for guards (section RETRYING) ** See entry_is_time_to_retry() ** Unittest 9) Update list of waiting circuits (section UPDATE_WAITING) ** Implementation of guard "higher priority" ordering (see CONFIRMED) ** Unittest 10) Update state whenever we get a new consensus (section ON_CONSENSUS) ** Update SAMPLED_GUARDS elements (section SAMPLED) ** Remove obsolete/expired guards ** See entry_guards_compute_status() / remove_obsolete_entry_guards() ** Thoughtworks function: entry_guards_update_profiles() 11) Bridge support ** Thoughtworks function: fill_in_from_bidges() 12) Separate state instances when EntryNodes/ExcludeNodes/etc are used ** See prop271 MEANINGFUL_RESTRICTION_FRAC etc. 13) Glue things together! ** Add clean and meaningful logging so that we can *heavily* field test the feature in our machines

1 0

[GSOC16] Fingerprint Central - Status report n°5
by Pierre Laperdrix 29 Jul '16

29 Jul '16

Hi everyone, Here is my fifth status report for my GSOC project. Repository: https://github.com/plaperdr/fp-central Beta: https://fpcentral.irisa.fr 1 - Support of TBB versions for statistics through tags I have added the selection of TBB versions for the custom statistics page. In order to do that, I designed and developed the tag system that I planned to develop much later for the project. For those who do not know what tags are, it is a way to classify fingerprints and put them into different categories. By analyzing collected fingerprints, the system can assign one or more tags to them. After several iterations, I think I found a good balance between expressiveness and simplicity. You can find the documentation on how to add your own tag to the system here: https://github.com/plaperdr/fp-central/blob/master/fingerprint/tags/README.… I have added a command to refresh and recompute all tags in case a new one is added to the system or if the logic of one tag has been changed. Right now, the tag system is used to tag each fingerprint with the corresponding TBB version but it can be used later on for many different things like identifying the OS or the type of the browser. 2 - Better and more complete custom statistics page. The custom statistics page is now much more complete than before. In addition to the already-existing pie chart, I have added an HTML table which gives the complete list of data from the chosen attributes. You can sort the data and download it in several different formats. It is a handy feature if you want to put the fingerprints into a different software or if you want to copy/paste them for a thorough bug report. I've pushed the latest updates today to the beta server so that you can try them right now and send me feedback on them. My goal for the next two weeks is to iron things out in the beta and fix bugs. I still need to make sure that only TBB fingerprints are stored in the database. Feature-wise, it seems that I'm getting near the end on the list of features I wrote in my proposal. The only real big one that is left is to have a page to see if your complete TBB fingerprint is "acceptable" (right now, only several attributes are checked). Finally, for the final stretch in the second half of august, I'm planning on completing the documentation and the french localization while making sure that deployment and updates of the site are as easy as possible. Have a great week-end, Pierre

1 0

[GSoC 2016] Orfox - Report 5
by Amogh Pradeep 28 Jul '16

28 Jul '16

Hey guys, These are the things I’ve done in the past couple of weeks. 1) GetUserDataDirectoryHome in nsXREDirProvider. Looks like the blocker in nsXREDirProvider was the GetUserDataDirectoryHome function. https://trac.torproject.org/projects/tor/ticket/19706 <https://trac.torproject.org/projects/tor/ticket/19706> is the ticket opened to track updates regarding this. Once this is ready, we would be able to upstream all the Orfox patches to the tor-browser repo. 2) Commits clean up. I’ve made a couple of passes on the commits to Orfox that we have, to reorder them and merge them as much as possible. This will make it easier to maintain things in the future, and it would be easier to review code as we merge it with tor browser. 3) Disabling search widget. The search widget for Orfox has now been removed. https://dev.guardianproject.info/issues/7444 <https://dev.guardianproject.info/issues/7444> Commit: https://github.com/amoghbl1/tor-browser/commit/f415a732a8e50c2c34f43d905c36… <https://github.com/amoghbl1/tor-browser/commit/f415a732a8e50c2c34f43d905c36…> At this moment, I don’t see the advantage of having this feature and would prefer not supporting it, in order to make it easier to maintain Orfox. In other new, I am migrating this week from CET to IST! This might make it slightly harder to be online during most active hours, but I will try to be around. Hope you guys have a great weekend. Best, Amogh Pradeep amoghbl1(a)gmail.com

2 1