- tor-dev - lists.torproject.org

User perception of onion service discovery
by Philipp Winter 15 Oct '17

15 Oct '17

This is the second part of our preliminary analysis of how Tor users interact with onion services [0]. In this part, we analyse the issue of onion service discovery. Onion services are private by default, so it's the operator's responsibility to disseminate their domain if they want it to be public. Question 3.6 in our survey asked: > How do you discover new onion sites? The breakdown looks as follows. Respondents could select multiple answers, so the percentages are based on the total number of respondents. Method Percentage ----------------------------------------------------------------------- >From social networking sites such as Reddit or Twitter 50.67 I browse the list of onion site search engines such as ahmia.fi 50.50 I randomly encounter them while browsing the web 47.65 Recommendations from friends and family 19.46 Other (see below for common themes) 18.12 I am not interested in learning about new onion sites 4.19 The data shows that social networking sites, search engines, and "random encounters" are rather popular. Respondents who selected "Other" mostly brought up onion service lists and aggregators. Following up, question 3.7 then asked: > Are you satisfied with the way you discover new onion sites? 61% selected "Yes" while the remaining 39% selected "No." Some respondents who selected "Yes" brought up that they have no interest in learning about new onion services; in part because they only use Facebook's (or some other) onion service. Among the respondents who selected "No," there are a bunch of reoccurring themes, in no particular order: - The most prominent complaint was about broken links on onion site lists. There is non-trivial churn among onion sites and our respondents were frustrated that existing lists are typically not curated and contain many dead links. - Many respondents were not aware of search engines such as ahmia.fi. Among those that were, many were not satisfied with both the search results and the number of indexed onion sites. Unsurprisingly, a "Google for onion sites" was a frequent wish. - Several respondents were unhappy with existing aggregators. In addition to broken links, some distrust lists because they occasionally contain scam and phishing sites. The difficulty of telling apart two given onion domain names exacerbates this issue. - Some respondents would like aggregators to be more verbose in their description of onion sites. In particular, these respondents were trying to avoid illegal and pornographic content, which is often difficult if the description is vague and the onion domain reveals nothing about its content. - Many respondents expressed frustration about the difficulty of finding out if site X also provides a corresponding onion service. A common wish was to have site X list its onion service prominently in a footer. Ironically, some respondents were surprised that torproject.org has a corresponding onion site -- they couldn't find it on the web site. - Two respondents compared the current state of onion services with the web of the 90s: Few sites existed, they linked to each other only sparsely, and search engines were experimental at best. - Interestingly, some respondents voiced frustration about various usability issues, but mentioned in the same sentence that this is an inherent trade-off of privacy technology, suggesting that there is nothing that can be done about it. There are two potential solutions that would address some of the above issues: - Have next-gen onion services opt-in to a broadcast mechanism that automatically propagates them. Naturally, we would like such a mechanism to be censorship-resistant and built in a way that only the owner of an onion service is authorised to broadcast their service. - Websites could use an HTTP header to announce the existence of a corresponding onion site. This issue was discussed in Feb 2017 over at tor-onions. Someone brought up the Alt-Svc header as a potential solution [1]. In a subsequent survey question we asked if our respondents would appreciate an automatic redirect from a web site to its corresponding onion site. The overall tendency leaned towards "Yes," provided that the implementation is sound and users can override the redirect. Again, it's important to take these results with a grain of salt. Our data has some survivor bias: Presumably, we mostly heard from people who are Tor users despite usability issues. We likely didn't hear from many people who once experimented with Tor or onion services, decided it's not usable enough, and gave up. The above was joint work with my colleagues Marshini Chetty, Annie Edmundson, Nick Feamster, and Laura M. Roberts. [0] <https://nymity.ch/onion-services/> [1] <https://lists.torproject.org/pipermail/tor-onions/2016-February/000045.html>

5 6

non-ASCII handling in tor and onionoo
by nusenu 14 Oct '17

14 Oct '17

Hi, I'm trying to find out if non-ASCII characters are allowed in descriptor data (and what onionoo would do with non-ASCII currently displayed as "?"). According to the spec non-ASCII chars are not allowed: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n218 I searched trac because I remembered there were some ASCII related tickets: https://trac.torproject.org/projects/tor/ticket/1258 closed due to https://trac.torproject.org/projects/tor/ticket/1720 (which has nothing to do with non-ASCII) https://trac.torproject.org/projects/tor/ticket/1270 Is this summary correct? Currently non-ASCII is not compliant with the spec but your consensus is that non-ASCII chars should be supported but no one got around to patch the spec and implementation. thanks, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

2 2

Pluggable Transports 2.0 Specification, Draft 3
by Brandon Wiley 13 Oct '17

13 Oct '17

Below is a link to the third draft of the Pluggable Transport 2.0 Specification. If you have feedback on this draft, please send me your comments by October 31. Thank you! Changes in this version: - Expanded acknowledgements section - thanks Yawning! - Removed TransportConn and TransportListener in favor of net.Conn and net.Listener - Modified SOCKS authentication method to use IANA-assigned designator - Added error response codes for per-connection arguments - Many typos fixed - thanks David Fifield! - Clarified some definitions - thanks teor! Link: https://operatorfoundation.org/PluggableTransportSpecification-v2-Draft3.pdf

1 1

Pluggable Transports 2.0 Specification, Draft 2
by Brandon Wiley 10 Oct '17

10 Oct '17

Attached is the second draft of the Pluggable Transport 2.0 Specification. If you have feedback on this draft, please send me your comments by July 20.

4 11

Your input on the Tor Metrics Roadmap 2017/18
by Karsten Loesing 10 Oct '17

10 Oct '17

Hello everyone, we, the Tor Metrics Team, are currently drafting a roadmap for the work we'd like to do on in the upcoming 12 months until September 2018. If you believe that you're affected by these plans and want your ideas to be included in this roadmap, please read our current draft and send your feedback either to this list, to the metrics-team@ mailing list, or to iwakeh or irl and/or me directly. https://people.torproject.org/~karsten/volatile/metrics-team-roadmap-2017-1… In particular, we're interested in: - tasks we're missing or that we're listing as long-term goals (Q4/2018 or later) that you think should have higher priority over the tasks we picked for the time until Q3/2018, - tasks that we picked that you think we put in for you or users with similar interests and that you think we could de-prioritize, and - tasks that you'd want to contribute to in any way, in which case we might reach out to you when we start working on them. Thanks for helping to make this roadmap document reflect what really needs to be done to make Tor Metrics better. All the best, Karsten

4 3

[release] CollecTor 1.4.0, metrics-lib 2.1.1, Onionoo 4.2-1.6.0
by iwakeh 10 Oct '17

10 Oct '17

Hi there! three new releases are available: https://dist.torproject.org/collector/1.4.0/ https://dist.torproject.org/onionoo/4.2-1.6.0/ https://dist.torproject.org/metrics-lib/2.1.1/ These releases publish the build revision of the software running on a particular Onionoo or Collector instance. For Onionoo this means an additional field "build_revision" in documents (cf. protocol changes [0]), and CollecTor shows ths field in its index.json [1]. All these changes are already deployed on all tp.o instances. CollecTor now enables sync-ing of OnionPerf measuremnts between instances. As there were changes to CollecTor's configuration file, all operators of CollecTor instances should also update collector.properties on their instance. INSTALL.md gives all details about how to create a new collector.properties configuration file [2]. Other details can be found in the respective change logs [3], [4], [5]. Please direct comments and questions to the metrics-team mailing list [6]. Cheers, iwakeh [0] https://metrics.torproject.org/onionoo.html#versions [1] https://metrics.torproject.org/collector.html#index-json [2] https://gitweb.torproject.org/collector.git/tree/INSTALL.md#n103 [3] https://gitweb.torproject.org/collector.git/tree/CHANGELOG.md?h=collector-1… [4] https://gitweb.torproject.org/metrics-lib.git/tree/CHANGELOG.md?h=metrics-l… [5] https://gitweb.torproject.org/onionoo.git/tree/CHANGELOG.md?h=onionoo-4.2-1… [6] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

PrivCount - Draft of secret-sharing specification
by Carolin Zöbelein 02 Oct '17

02 Oct '17

Hi (PrivCount) folks! I just finished the first draft of the k,n-secret-sharing thing for PrivCount. :) Sorry that it need some time, but I'm sadly a bit slowly with reading and writing. You can find it here at github: https://github.com/Samdney/28X-k-of-n-secret-sharing and also below in this email. You will see a lot of "=> TODO". This belongs of course not to the specification ;). There are still a lot of open details and questions left (or stuff which still have to be written, I think) and hence it is time for some help from you, now. I looked a bit around how a specification has to look like, before I started to write it, but this was more confusing like helpful, hehe. I'm a completely newbie with writing this kind of documents, hence I'm pretty nervous for your feedback :) Bye, Carolin ========== Draft v1 ========== Filename: 28X-k-of-n-secret-sharing.txt Title: k-of-n Secret Sharing Author: Carolin Zöbelein Created: XX-Sept-2017 Status: Draft 0. Motivation The implementation of schemes for collecting statistic data within a high sensitive network like Tor for preserving anonymity, is a hard challenge. Over the years the Tor network has grown but its usage and operation is not well-understood and already existing ways [1] leads to some open issues [maybe add also a reference here]. For doing this better like the current state of the art, we discuss to switch to PrivCount [2][3], a system for measuring the Tor network created with a high attention on user privacy. PrivCount consists of a system of Data Collectors (DC) which forward their blinded measure counter results to a number of, so-called, Tally Reporters (TR) which are only together be able to reconstruct the original data. In the context of the implementation of the mentioned system, we decided to use a secret sharing algorithm for forwarding the blinded counter values. This gives us the chance of reconstructing the data also with a particular minimum amount of secret share holders and hence a failure handling possibility of Tally Reporters. => TODO: References [maybe add also a reference here] 1. Introduction Assume, we have a given secret s which we want to share with a particular number N of participants who are only together be able to reconstruct it. To realize this, we can split our secret in n parts s_i. Our secret will be then the sum over all s_i. This is the simplest secret sharing scheme at all. Since it needs all participants for the reconstruction, it is called a (N,N) treshold secret sharing algorithm. But we also see that it has weaknesses. With every leaked share s_i, an adversary can reduce the number of possible soulutions for our secret very easily. This leads to the group of more efficient secret sharing algorithms. In [4], Adi Shamir introduced a (K,N) secret sharing scheme which is named after him and offers more security. Additionally, on the contrary to our scheme above, we only need a minimum amount of k out of n participants to reconstruct the secret. Our specification will be based on this scheme. 3. Overview and preliminaries In this section, we make some preparations for the protocol specification itself. 3.1. Scope In this document we describe the protocol specification for a Shamir Secret Sharing scheme on a finite field of size p > s and p > N, with p be prime number. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. 3.2. Notation We will use for public, non-secret, values UPPER CASE and for private, secret, values lower case. We write: "a", type: b, c, d "a" gives the name of the parameter. type: b be the type of the parameter a. c be the amount of this parameters. d be the mathematical definition set for a. Mathematical assignments: Let "a := b" be the assignment of the value of b to the variable a. Let "a mod b" be the modulus calculation of a with respect to b. Let "a != b" be that a is unequal to b. In our document "natural numbers" are defined as the set of all integers greater than zero. g[X] describes a polynomial with respect to X. SUM(a_i) gives the sum over a_i for all known i. SUM(i=a,b,c_i) gives the sum over all c_i for all a <= i <= b. PRODUCT(a,b,c_i) gives the product over all c_i for all a <= i <= b. The secret sharing protocol has three participating parties which we will call as follows: Secret Keeper (SK) knows the secret, does the initial setup and determines the secret shares. Share Holders (SH) receive the secret shares from the SK. Secret Reconstructor (SR) takes a particular number of secret shares from the SHs and reconstruct the secret. 4. Protocol outline We give a raw protocol overview. 0. Preparation: The parties negotiate an appropriate handshake and communication way for forwarding the secret shares between SK to the SHs and between the SHs to SR. [This is not part of that specifiation] => TODO: Do we need a more detailed definition of "appropriate"? 1. The SK knows the secret s. Additionally, given are the amount N of participating SHs and the threshold K for the minimum number of necessary shares for the reconstruction. [see sec. 5.1.] 2. The SK generate a random prime number p, with p > s AND p > N. [see sec. 5.3.] 3. The SK determines the secret polynomial coefficients a_j, 1 <= j <= K-1. With this, the secret keeping polynomial is given by g[X] := s + SUM(a_j*X^j). [see sec. 5.4.] 4. The SK determines the secret shares parts x_i, 1 <= i <= N. [see sec. 5.5.] 5. The SK computes the secret shares parts y_i := g[x_i]. [see sec. 5.5.] 6. The SK forward the secret shares to the SHs. Each SH_i MUST receive exactly one secret share pair (x_i,y_i). [see sec. 5.6.] 7. The SR receives K secret share pairs (x_h,y_h) from the SHs and p from the SK, 1 <= h <= K. [see sec. 5.7] 8. The SR compute the Lagrange basis polynomials l_h[X]. [see sec. 5.8.] 9. The SR reconstruct the original polynomial with g[x] = SUM(h=1, K, y_h*l_h[X] mod p). [see sec. 5.8.] 10. The SR computes the secret s = g[0]. [see sec. 5.8.] 5. Specification Now we will give more details to the raw outline above. 5.1. Given constants "s", type: int, exactly one, integer The given secret. "N", type: int, exactly one, natural number The number of participating SHs. It MUST to be N >= N_min. => TODO: Which value should N_min has? Default: N_min = 2? "K", type: int, exactly one, natural number The threshold of the minimum number of necessary shares for the reconstruction. It MUST to be K <= N. => TODO: Are more (size) information necessary? E.g. amount of bits/bytes? I think so. 5.2. Parties Secret Keeper (SK) It MUST exists exactly one SK. Share Holders (SH) It MUST exists exactly N SHs. Secret Reconstructor (SR) It SHOULD exists exactly one SR. [=> TODO: SHOULD since one is necessary but more could be used for checking the result. But I would prefere MUST.] => TODO: Which additional information do we need to know/to give about the parties? 5.3. Prime number Since we are using a secret sharing scheme on a finite field, we need a random prime number. "p", type: int, exactly one, prime number It MUST to be p > s AND p > N AND it MUST to be the secret s element of Z/pZ. => TODO: I'm not sure how exactly p should be handled. When and from where, is it given to whom? => TODO: Do we need to write anything about the necessary "random" characteristic? The "quality" of the randomly generation of the number? => TODO: Minimum size of p? Which value should p has, at least, caused by security reasons? => TODO: Are more (size) information necessary? E.g. amount of bits/bytes? I think so. 5.4. The secret keeping polynomial "a_j", type: int, K-1 values, Z/pZ "g[X]", type: polynomial with order K-1, exactly one, (Z/pZ)[X] The SK determines the final secret keeping polynomial, which is given by g[X] := s + SUM(a_j*X^j) and hence our secret for g[0] = s. Its random coefficients are a_j, 1 <= j <= K-1 which MUST be element of Z/pZ. => TODO: Which constraints exists for this a_j values? Size? Relative, pairwise, distance a_j - a_m between for all a_j,a_m with j != m? Is this relevant? References for this? => TODO: Are more (size) information necessary? E.g. amount of bits/bytes? I think so. 5.5. The secret shares "x_i", type: int, N values, Z/pZ "y_i", type: int, N values, Z/pZ "(x_i,y_i)", type: (int,int), N value pairs, Z/pZ -> Z/pZ The SK determines the random secret shares parts x_i, i <= N which MUST be element of Z/pZ and MUST be pairwise different from zero. With this x_i, SK computes the secret shares parts y_i := g[x_i] and hence the finally secret share pairs (x_i,y_i). => TODO: How should this x_i be generated? Distribution? E.g. the smallest, non negative, representatives? => TODO: Which constraints exists for this x_i values? Size? Relative, pairwise, distance x_i - x_m between for all x_i,x_m with i != m? Is this relevant? References for this? => TODO: Are more (size) information necessary? E.g. amount of bits/bytes? I think so. 5.6. Sending secret shares from SK to SHs The SK sends the secret share pairs to the SHs. Each SH_i MUST receive exactly one secret share pair (x_i,y_i). => TODO: How exactly has to look the data which has to be send and which size has it (bits/bytes) to be? => TODO: Which data has also to be send apart from (x_i,y_i)? => TODO: How should looks like a possible answer of the SHs for the SK to confirm the receipt? [Is this necessary, at all? I think so.] => TODO: I'm not sure how exactly p should be handled. When and from where, is it given to whom? => TODO: I need a helping hand for this specification section :) 5.7. SR receives secret shares from the SHs The SR MUST receive K secret share pairs from the SHs and p from the SK. The SR MUST receive exactly one secret share pair (x_,y_h), 1 <= h <= k, from each SH_h => TODO: How exactly has to look the data which has to be send as response to the SHs? What, which additionally data, has to be send? And which size has it (bits/bytes) to be? => TODO: I'm not sure how exactly p should be handled. When and from where, is it given to whom? => TODO: From where comes the information about N and K? (and p?) => TODO: Where has to be decided, from which K out of N SHs has this (x_h,y_h) to come from? And how (randomly)? And in which way has this to be comunicated to the given parties? !!! => TODO: I need a helping hand for this specification section :) 5.8. Reconstruction "l_h[x]", type: polynomial with order K-1, K, (Z/pZ)[X] The SR compute the Lagrange basis polynomials l_h[x] with the secret share pairs (x_h,y_h), 1 <= h <= K, which it received from the SHs. The SR MUST receive exactly K pairs from exactly K SHs. I MUST be exactly one secret share pair from each, of this K, SH. The Lagrange basis polynomials are given by l_h[X]:= PRODUCT(1 <= m <= K AND m != h, (X - x_m)/(x_h - x_m)) with 1 <= j <= K. This leads to our original secret keeping polynomial g[X] := SUM(h=1, K, y_h*l_h[x] mod p) and the given secret by s = g[0]. => TODO: From which K out of N SHs come this secret shares? => TODO: Are more (size) information necessary? E.g. amount of bits/bytes? I think so. 6. Security discussion => TODO: Write important points about the security aspects of this scheme. :) 7. References [1] https://www.cypherpunks.ca/~iang/pubs/privex-ccs14.pdf [maybe add also a reference here] [2] http://www.robgjansen.com/publications/privcount-ccs2016.pdf [3] https://github.com/privcount/privcount [4] Shamir A., "How to share a secret", Communications of the ACM. 22, 1979, S. 612–613. => TODO: References => TODO: Correct references for regular citation => TODO: Add missing references A. Lemma => TODO: Still to write. The Lemma (why this Shamir thing works :) proof ========== TODO: RESEARCH AND EXTENSION OF SPECIFICATION!!! => TODO: Investigate more some very interesting papers! :) => TODO: Multi-Secret Sharing Schemes!!! TODO: MISC: => TODO: Notation stuff checking => TODO: Check my English for language mistakes :) => TODO: I used: scheme, algorithm, protocol, ... what is the best word in what context? ========== -- ----------------------------------------------------------------------- Carolin Zöbelein / Nick: Samdney PGP: D4A7 35E8 D47F 801F 2CF6 2BA7 927A FD3C DE47 E13B -----------------------------------------------------------------------

4 7

User perception of the prop224 domain format
by Philipp Winter 27 Sep '17

27 Sep '17

We recently ran a survey on the usability of Tor and onion services [0]. I had a closer look at how our respondents perceive the prop224 domain format and wanted to share some early insights. The original survey question was: > The Tor Project is currently working on the next generation of onion > services. The new onion domain format will consist of 52 characters, > for example: > a1uik0w1gmfq3i5ievxdm9ceu27e88g6o7pe0rffdw9jmntwkdsd.onion > Do you expect this to change your browsing habits? 591 users answered this question. 95 (16%) selected that prop224 domains will change their habits while the remaining 496 (84%) selected that their habits won't be affected. Respondents who believe that their habits will change (16%) gave the following reasons: - Several users memorise a number of onion domains -- most prominently Facebook's onion domain and self-hosted domains. They write that memorising domains will no longer be possible, and they will look into bookmarking tools. Several users voiced concern about the confidentiality of their bookmarks, so they are looking into ways to encrypt them. - Similarly but less commonly, users voice concerns that communicating, typing, and writing down prop224 domains will no longer be feasible. - A small number of users write that it will be harder to recognise onion domains. Alarmingly, one user mentioned that the lack of a discernible prefix will make it hard to recognise genuine domains, suggesting that they rely on an onion domain's easy-to-spoof vanity prefix. - A user suggested to add spaces to prop224 domains to "make the address more visually appealing." Respondents who believe that their habits will *not* change (84%) gave the following reasons: - The majority of this crowd never bothered to memorise onion domains and uses bookmarks. A bunch of users store domains in text files and an even smaller bunch uses search engines to rediscover domains. In general, most people in this category treat onion domains as an opaque identifier. - Some users write that the additional inconvenience is likely worth the extra security and anonymity. - Some users mention Reddit as their primary way of discovering onion domains. Judging by the above, I believe that the new domain format is among the minor usability issues surrounding onion services. In fact, an easy-to-remember domain format ranks last among the six criteria whose importance we asked users about. On a five-point Likert scale ranging from "not at all important" to "very important," we got the following results: - 77% think that quality of content is at least somewhat important. - 70% think that a search engine (like Google) for onion services is at least somewhat important. - 66% think that diversity of content is at least somewhat important. - 62% think that page load time is at least somewhat important. - 43% think that having an onion service version of popular services such as Facebook is at least somewhat important. - 26% think that an easy-to-remember domain format is at least somewhat important. However, our survey data is likely biased towards a particularly young and educated crowd that's presumably less bothered by technological hurdles, which may be why they can afford to care more about content. [0] <https://blog.torproject.org/take-part-study-help-improve-onion-services> Cheers, Philipp

3 4

Tor 0.3.1 -> 0.3.2 coverage update
by Nick Mathewson 26 Sep '17

26 Sep '17

So, here's some good news about unit test coverage (as measured by 'make check') in 0.3.2: Our (line) test coverage rate is up, from 52.88% to 54.72%. But here is some bad news: The number of uncovered lines has still increased (from 30860 to 31608), even though covered lines have increased more rapidly. If you're interested in tracking down where we need coverage, I have used our "cov-diff" tool to generate a cleaned diff from the old coverage to the new coverage. You can download it (it's big!) from https://people.torproject.org/~nickm/volatile/coverage-diff.xz The format is: it is a diff between normalized gcov output files. To find new or modified un-covered lines, look for lines that begin with the regex /^\+ *\#/ ("starts with a plus, then some spaces, then a #"). Any lines like this are good candidates for additional testing IMO. peace, -- Nick

1 0

Does "ar: `u' modifier ignored since `D' is the default (see `U')" warning qualify as a bug?
by Patrick Durusau 25 Sep '17

25 Sep '17

Hello, Running Ubuntu 16.04, using: https://gitweb.torproject.org/tor.git/tag/?h=tor-0.3.2.1-alpha Following: https://lists.torproject.org/pipermail/tor-project/2017-September/001449.ht… and I get this warning 17 times from make: ar: `u' modifier ignored since `D' is the default (see `U') I traced this back to line 187 in Makefile.in ARFLAGS = cru Correcting to: ARFLAGS = cr src compiles sans the warnings. Is that a bug meriting a ticket? My primary strength is documentation and making mistakes in following it. ;-) Hope everyone is looking forward to a great weekend! Patrick -- Patrick Durusau patrick(a)durusau.net Technical Advisory Board, OASIS (TAB) Editor, OpenDocument Format TC (OASIS), Project Editor ISO/IEC 26300 Co-Editor, ISO/IEC 13250-1, 13250-5 (Topic Maps) Another Word For It (blog): http://tm.durusau.net Homepage: http://www.durusau.net Twitter: patrickDurusau

2 1