- tor-dev - lists.torproject.org

Re: [tor-dev] Standardizing the 'onion service' name
by Paul Syverson 26 Apr '18

26 Apr '18

Thanks teor for spelling things out in moderate detail and Steph for commenting. My tl;dr is to basically concur with everything they both said: The further from user-facing and the more embedded down in the codebase, variables, code comments, controller commands, etc. the less important to spend effort eliminating such vestiges from existing text. Going forward, certainly any code comments and e.g. any commands that won't break things should use current terminology. aloha, Paul On Thu, Apr 26, 2018 at 11:07:27AM -0400, Stephanie A. Whited wrote: > Hi! > > Thanks for adding me to the thread. > > > On 4/26/18 3:34 AM, teor wrote: > > Hi All, > > > > There seems to be some confusion in this thread about the > > current state of the hidden service to onion service name transition. > > > > So I'm going to describe the state of things as they are, then try to > > describe what would need to be done. > > > > I'd also appreciate feedback from Steph and others on our priorities for > > transitioning to the onion service name. I think we have been prioritising > > user-facing text. (The blog, website, Tor Browser, metrics, etc.) > Yes, user, funder, and press facing text has consistently been using > onion services at least since I've been on board. When I first joined, > the message to me was that this was changed a couple years prior but > changeover had been slow. I realize now a lot of folks didn't get that > message. > > We have already, with the help of Kat, updated hidden to onion services > where possible on the website. I think the exception to this is older > blog posts. > > I realize it is not as easy to change some of the other elements, like > the directory protocol, but I think it's important to make what changes > are possible. I think it'd have a very positive impact to see these > through. > > Being aligned will help us build trust with the press, new users, etc. > There are several reasons the name changed, if it is helpful to share > more about that lmk. > > > > > Is this a sensible way of prioritising things? > > > > On 26 Apr 2018, at 16:42, Paul Syverson <paul.syverson(a)nrl.navy.mil > > <mailto:paul.syverson@nrl.navy.mil>> wrote: > > > >>> Have OnionService aliases for controller commands, events, > > > > These are current called "hidden service" or an abbreviation. > > > > Tor could add an alias mechanism for controller commands, events, > > and fields, and use it to do the rename: > > > > https://trac.torproject.org/projects/tor/ticket/25922 > > <https://trac.torproject.org/projects/tor/ticket/25922#ticket> > > > > I don't think they are as high a priority as the torrc options and man > > page. > > > >>> descriptor > >>> fields > > > > These are currently called "hidden service", or an abbreviation. > > > > Descriptor fields are part of the directory specification and > > implementation, and they are highly technical. So I'm not sure we gain > > much from aliasing them or renaming them. > > > > Similar arguments might apply to other codebases: > > * Onionoo > > * stem > > * consensus health > > * Tor (network daemon) > > > > But the following user-facing applications should add documentation or > > change names, if they haven't already: > > * Relay Search / metrics website > > * uses HSDir for relay search, because that's what it's called in the > > directory protocol > > * uses "onion service" for statistics > > * Tor Browser > > * uses "onion site" > > * the Tor website > > * new tor blog posts > Website and new posts are covered. > > > >>> and anything else referencing 'HS' or 'HiddenService'. > > > > We considered adding OnionService* torrc option aliases for every > > HiddenService* option in 0.2.9. But we deferred that change because we > > ran out of time. > > > > All we need to do is add some new entries in the alias table, then do a > > search and replace in the tor man page: > > https://trac.torproject.org/projects/tor/ticket/17343 > > > >>> Speaking of which, how do we plan to replace abbreviations? Having an > >>> 'OSFETCH' or 'OS_CREATED' event doesn't exactly have the same ring as > >>> their HS counterparts. ;P > > > > That's a good question. > > > > OS conflicts with "operating system", so we could use: > > * Onion > > * OnSrv > > * no abbreviations > > Or any other colour you want to paint the bikeshed. > > > > To avoid an endless discussion, let's leave the decision to > > the people who write, review, and merge the code. > > > >>> Adjust all our docs to be consistent about the name. > >> > >> Right, anything v3 should be consistently calling these 'onion > >> services'. Variable names, etc. particularly those still in use in > >> code shared with v2, don't need to be changed. It is OK if such > >> vestiges of older usage remain in abbreviations, as long as the > >> description of them, e.g., in any new Tor proposal, describes them > >> with appropriately current terminology. > > > > torrc options, the tor man page, and the v3 onion service code > > mainly use "hidden service", and sometimes use "onion service". > > > > I don't see much value in changing the code. > > > > If we decide there is value in changing the torrc options and man page, > > we need to allocate a few days of work on the roadmap to make it > > happen. > > > >>> Renaming takes work. Lesson I learned from Nyx is that it works best > >>> if you draw a line in the sand and stand by it. With Nyx, version 2.0 > >>> is called Nyx (you won't find any docs saying otherwise) and version > >>> 1.x is the legacy 'arm' project. > >>> > >>> If I was in your shoes I'd opt for the same. Either prioritize the > >>> aliases and be firm that v3 are 'Onion Services' or abort the rename. > >>> Otherwise this will live in a confusing dual-named limbo land > >>> indefinitely. ;P > >> > >> I'm prettty sure that v3 being 'onion services' has been the official > >> Tor Project position since at least half a year. We wouldn't be > >> aborting the rename, because 'abort' would imply it is not > >> complete. Anything now not using the current name is not part of an > >> incomplete process, it is simply wrong and outdated. Steph correct me > >> if I am wrong about that. > >> > >> So I think you've answered your own question. Nothing in v3 should be > >> called 'hidden services'. > Right. In our communications, all are onion services, there are just > different versions and configurations possible. > > If I've missed something I can respond to more specifically in this > thread, please lmk. > -Steph > > > > That's not the current state of the tor network daemon v3 onion service > > code, specs, options, and man page. They use a mix of terminology > > (see above). > > > >> And anything new in code and documentation > >> should be called 'onion services'. If you want to think of v2 and > >> earlier as 'hidden services' for purposes of understanding legacy > >> component and variable names, e.g., HSDir that's fine. And as such, > >> variable names, etc. in code that continues to be used for both v2 and > >> v3 can can persist. But again, any new specs, documentation, etc. > >> should call them 'onion services'. > > > > We have gradually been using onion services in new documentation and > > specs, since "single onion services". But we haven't changed existing > > code and documentation. > > > >> This acceptance of existing v2 documentation calling them 'hidden > >> services' while refusing this for anything v3 is a little misleading > >> about when and why the naming transition happened, but its close > >> enough to serve as your line in the sand if you feel one is needed. > >> I actually argued the value of essentially such a line-in-the-sand > >> position to Steph a while ago. This doesn't preclude also calling v2 > >> and earlier 'onion services'. Indeed, it's not just OK but preferable, > >> for the above mentioned reasons. > > > > It looks like we are doing a gradual transition. I think we prioritised > > the things that are seen by the most users (Tor Browser, statistics, > > blog, website). > > > > We did not take a line in the sand position in the past, but we could > > adopt one for the remaining changes. We could decide on a particular > > Tor release (and releases of other codebases). But we need to schedule > > the work on the relevant roadmaps. > > > > And maybe there are some obscure technical things (code, comments, > > descriptor fields) that just aren't worth changing. > > > > T >

1 0

sbws meeting notes 26 April 2018
by Matt Traudt 26 Apr '18

26 Apr '18

Next meeting is 3 May 2018 at 0930 UTC. http://meetbot.debian.net/tor-meeting/2018/tor-meeting.2018-04-26-09.59.html https://pad.riseup.net/p/6292yqhUOuHC (copy/pasted below) Simple Bandwidth Scanner Meeting 25 April 2018 ============================================== This pad: https://pad.riseup.net/p/6292yqhUOuHC Previous week's notes: https://lists.torproject.org/pipermail/tor-dev/2018-April/013084.html Witty quote - Unknown Updates =========================================== pastly: last 7 days: - sbws is now open source https://github.com/pastly/simple-bw-scanner https://sbws.readthedocs.io/en/latest/ - fix some bad bugs in 'sbws cleanup' command while writing tests for it - integrate with Travis CI and automatically test against python 3.4, 3.5, and 3.6 - integrate with Read the Docs - gather intel on the environments our users will be running sbws in https://github.com/pastly/simple-bw-scanner/wiki/Information-regarding-sbws's-target-user-base - did some research regarding library support for various distros. incomplete - remvoe filelock dependency - brainstorm what I think a switch to http(s) should look like https://sbws.readthedocs.io/en/latest/proposals/001-switchtohttp.html next 7 days: - start implementing the switch to http(s) - verify sbws works with library versions that aren't the absolute newest misc notes section (not udpates) juga's tor tickets: https://trac.torproject.org/projects/tor/query?status=!closed&cc=%7Ejuga juga: Not sure which week was (since we changed from Monday to Thursday): - review, comment on debug option not being used #85 - add logging conf to log to file/system #87 - review Periodically test reachability of helpers #93 - create issue Include file where to write ``generate`` results in the config #91 suggest having options both in config and cli and cli overriding conf - report about problem with init and helpers #95 - create issue about adding software to header #96 - review using kilobytes #97, #98 Past week: - fix conf issues locally and in testnet after "megacommit" - more changes on the spec, after new teor review https://github.com/torproject/torspec/pull/4/commits/54ef4d4355d2b475466bf1… - create improve deployability #99 - create issue Debian package #101 - Work on issue software and sofware_version in v3bw files #112 - create issue Include coverage in travis #117 - create issue Include travis and coverage badges in README enhancement #118 Next week: - debug what tor does to solve https://github.com/torproject/torspec/pull/4/#issuecomment-384555243 - review the software/version ticket according to results from previous (https://github.com/pastly/simple-bw-scanner/issues/96) - review spec according to results from previous - add a default output file for generate that can be overriden by cli args (https://github.com/pastly/simple-bw-scanner/issues/91) - improve deployability, this should also help to update instances running in testnet (https://github.com/pastly/simple-bw-scanner/issues/99) - update version sbws running in testnet Things to communicate: - i put as date of sbws release 1.0.0 (milestone 1) 1st May, but i'd postpone to later so that we've more time testing and fixing last details - teor, can i make public the list of tasks you've proposed for SoP?, any place you would propose? - pastly: It's fine to close issues/PRs you think aren't apropiate for any reasons, but please say so before closing (#111, #87) - i stop attending net-team meetings until we need to disscuss something there that we don't discuss in this meeting List of tasks for 3 months proposed by T. (19th Apr) - implement features we wanted in torflow in sbws instead (1 week) - bwauth code needs to be smarter about failed circuits https,//trac.torproject.org/projects/tor/ticket/16559 - assign the 10-second client timeout as the time for failed circuits? - at least publish failure rates https,//trac.torproject.org/projects/tor/ticket/7281 - Understand how accurate the bandwidth authority estimates are https,//trac.torproject.org/projects/tor/ticket/7177 - integration tests (1 week) - a practice transition in the test network (1 week, but split up) - set up torflow - set up sbws - compare and switch - work little-t tor needs (3 weeks) - report version of bwscanners in votes https,//trac.torproject.org/projects/tor/ticket/3723 - stop relays reporting zero observed bandwidths https,//trac.torproject.org/projects/tor/ticket/24250 - relays should regularly do a larger bandwidth self-test https,//trac.torproject.org/projects/tor/ticket/22453 - bandwidth testing circuits should use guards sometimes https,//trac.torproject.org/projects/tor/ticket/19009 - report bandwidth more often on test networks https,//trac.torproject.org/projects/tor/ticket/17036 - report bandwidth less often on large, public relays https,//trac.torproject.org/projects/tor/ticket/24104 - non-essential tickets like https,//github.com/pastly/simple-bw-scanner/77 and https,//github.com/pastly/simple-bw-scanner/48 (2 weeks) - if time and motivation - Have sbws client launch tor itself https,//github.com/pastly/simple-bw-scanner/77 - Evaluate using [not threads, but twisted instead (probably)] https,//github.com/pastly/simple-bw-scanner/48 teor: last week: * made my test dirauth a trust pastly's and juga's dirauths * monitored the average bandwidth on the testnet using this script: https://pad.riseup.net/p/tor-vote-bw-average * opened a ticket for getting similar features into consensus-health: https://github.com/pastly/simple-bw-scanner/issues/120 this week: * busy with deadlines * eventually try to set up sbws on the testnet Discussion ========================================= ### Topic: Switch to HTTP(S) ### (Proposed) Topic: SoP - specific topics on this? pastly: where can I find your proposed schedule again? juga: mail pastly: finally found it - personally i'm not sure on how to organize the tasks we've said i'll do for SoP, since that's in a month. Either i do now other stuff not included there (doing so so far), or i start with those tasks and then update to others during SoP

1 0

Standardizing the 'onion service' name
by Damian Johnson 26 Apr '18

26 Apr '18

Hi all, teor suggested engaging the list with #25918 so here we go! Ticket #25918 has a couple goals... 1. Provide a tracking ticket for the rename effort. 2. Come to a consensus on if we should move forward with "onion service" or revert back to "hidden service". The limbo we've been in for months is confusing for our users and we should standardize on a name. Here's the ticket... ======================================== A recent post on tor-dev@ just got me thinking about the roadblocks we have for v2 deprecation. There's a couple I don't believe we're following in trac so lets fix that. For me the biggest is its name. Renaming takes work, and we attempted to rename hidden services in v3 without investing the time to make it happen. We should either fix that or revert to to the old name. To move forward we need to... Have OnionService aliases for controller commands, events, descriptor fields, and anything else referencing 'HS' or 'HiddenService'. Speaking of which, how do we plan to replace abbreviations? Having an 'OSFETCH' or 'OS_CREATED' event doesn't exactly have the same ring as their HS counterparts. ;P Adjust all our docs to be consistent about the name. Renaming takes work. Lesson I learned from Nyx is that it works best if you draw a line in the sand and stand by it. With Nyx, version 2.0 is called Nyx (you won't find any docs saying otherwise) and version 1.x is the legacy 'arm' project. If I was in your shoes I'd opt for the same. Either prioritize the aliases and be firm that v3 are 'Onion Services' or abort the rename. Otherwise this will live in a confusing dual-named limbo land indefinitely. ;P Cheers! -Damian PS. Stem and Nyx have stuck with the old name ("hidden services") and will continue to do so until tor's standardized this.

3 2

[TSoP] stem.client - introductions
by Dave Rolek 25 Apr '18

25 Apr '18

Hi everyone! I'm really excited to announce I'll be working on stem for the 2018 Tor Summer of Privacy! The news is still sinking in. This is going to be my first major chunk of contributing code back to the Tor community. :) In particular, I'll be working on expanding stem's support to act as a Tor client, aka stem.client, speaking with Tor's ORPort. You can see more about stem in general [1], and read concepts for stem.client [2], but at the end of this email I list my high-level plan for the summer project. There's definitely more to a Tor client than one can write in a summer, so my focus is on getting basic client functionality and on fetching v2/v3 Onion Service descriptors. The official project period is May 14th through August 21. My existing work on stem is available by querying through Trac [3] (easier for context) and browsing patches on GitHub [4]. I've been pretty active on IRC recently, but I wanted to broaden this. Anyone is welcome to reach out to me with questions/comments! On OFTC and Trac, I'm dmr. I plan to attend the network team meetings [5] (at least lurk) to get a sense of Tor's protocol development and to be around for any littlet-tor/stem syncing discussions. I'll also be sending updates biweekly to the tor-dev mailing list and writing "/me status: [...]" posts in #tor-dev. Lastly, my mentors for the project are atagar and teor. Really looking forward to this! Cheers! Dave -- My high-level plan is the following in this rough order: * continue prework with stem and the tor specs * get a stronger feel for / strengthen the CI testing for stem * better understand the ecosystem of projects that use stem * begin official coding by tackling minor stem.client improvements and implementing classes for a few more Cell types * continue developing stem.client.Relay and adding further testing using more of the Cell classes already defined * work on relay authentication * move onto fetching relay descriptors from the ORPort * work on the RELAY_EXTEN[sion] cells, to build multi-hop circuits * build upon this to allow actual exit traffic, including implementing RELAY_RESOLVE[D] * work on fetching v2 Onion Service descriptors, since I already have familiarity [6] with that spec, and v2 Onion Services probably will be around for a while yet * work on fetching v3 Onion Service descriptors * work on other stem.client improvements, if time allows * culminate the project with tutorials and a blog post introducing the new user-facing features of stem [1] https://stem.torproject.org/ [2] https://www.torproject.org/getinvolved/volunteer.html.en#pythonTorClient [3] https://trac.torproject.org/projects/tor/query?cc=~dmr&or&reporter=dmr&or&o… ^ a bit noisy, sorry; look at "Core Tor" [4] https://github.com/dmr-x/stem [5] https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam#Meeting… [6] https://github.com/dmr-x/talks

2 1

ahmia ~ summer of privacy
by Stelios Barberakis 24 Apr '18

24 Apr '18

Hello all, My name is Stelios and I am a CS student at Technical University of Crete. This summer I will be working on ahmia <https://ahmia.fi> project, during the "summer of privacy". This will be the first time to be engaged with the tor-ecosystem community as a developer, and I am really glad about it. The current plan includes some updates, enhancements of the codebase and the building/deployment process (including docker), as well as some new features like search filters, but we are looking to come up with new ideas for additional features, etc. Everyone is welcomed to contribute to the brainstorming, so feel free to ask anything you would like, either here or by joining #ahmia channel in OFTC if you feel so. I will do my best to contribute this summer, and hopefully beyond that. I will be sending bi-weekly reports to @tor-dev, so that everyone could be informed about the progress. Best Regards, Stelios -- PGP key: http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xBF6EA91B7CBE3998

2 2

HS desc replay protection and ed25519 malleability
by George Kadianakis 24 Apr '18

24 Apr '18

Hello Ian, isis, and other crypto people around here! Here is an intro: In HSv3 we've been using revision counters (integers++) to do HS desc replay protection, so that bad HSDirs cannot replay old descs to other HSDirs. We recently learned that this is a bad idea from a scalability prespective (multiple sites need to track rev counter...), and also it's needless complexity in the code (tor needs to cache the counters etc.). See ticket #25552 for more details: https://trac.torproject.org/projects/tor/ticket/25552 https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt#n1078 In #25552 we've been making plans to ditch the rev counters and replace them with a casual replay cache. (These replay caches also don't need to be big, since descriptors are only replayable for a day before the ephemeral blinded key changes, and the cache can be reset). Anyhow, now we've been playing the game of "which part of the desc should we use in the replay cache"? The latest plan here has been to use the ed25519 descriptor signature since it's something small, simple and necessarily changes with every fresh descriptor. And this is how we entered the ed25519 malleability scene. The basic question here is, can we use the ed25519 signature in our replay cache and consider it immutable by attackers without the private key? And should we use R, or S, or both? According to RFC8032: Ed25519 and Ed448 signatures are not malleable due to the check that decoded S is smaller than l. Without this check, one can add a multiple of l into a scalar part and still pass signature verification, resulting in malleable signatures. However, neither donna or ref10 include such a check explicitly IIUC. Instead they check whether (RS[63] & 224), which basically ensures that the high 3 bits of S are zeroed, which ensures S < 2^253. Is that equivalent to the RFC check? Because if I'm counting right, for most legit S values you can still add a single l as the attacker and get an S' = S + l < 2^253 equivalent signature (you can't add 2*l tho). So what's the state of ed25519 malleability? I know that after the Bitcoin incident, people have thought about this a lot, so I doubt we are in a broken state, but I just wanted to make sure that we will not mess something up. :) Thanks for the help! :)

5 7

Tor Messenger
by Milan Damjanovic 24 Apr '18

24 Apr '18

Hello, I would like to ask is there any online documentation about how is possible to transmit text messages via tor nodes? I would like to [try to] rewrite the TorChat Messenger which is written in Python 2.7 and wxPython to Python 3x and Qt. I cant find any documents how to transmit messages between two tor nodes so anything would be helpful... Cheers, Milan

2 2

Simple Bandwidth Scanner is now open source
by Matt Traudt 23 Apr '18

23 Apr '18

https://github.com/pastly/simple-bw-scanner Simple Bandwidth Scanner (sbws) aims to replace torflow[0], the bandwidth measuring software bwauths use today. For the most up to date documentation about how it works and why it exists, see the docs at readthedocs, especially the spec[2][3]. sbws's semantic version (currently 0.1.0) should not be considered meaningful until 1.0.0. There are major planned changes to how it works (the HTTPS milestone[1]), and likely a few unplanned changes too. I haven't put much thought into whether sbws should canonically live on gitweb.tpo or on GitHub other than I like GitHub's issue tracking, milestones, Travis CI integration, and readthedocs integration. So it might move ... or it might not. Cheers Matt [0]: https://gitweb.torproject.org/torflow.git [1]: https://github.com/pastly/simple-bw-scanner/milestone/3 [2]: https://sbws.readthedocs.io/en/latest/specification.html [3]: http://d7pxflytfsmz6uh3x7i2jxzzwea6nbpmtsz5tmfkcin5edapaig5vpyd.onion/speci…

1 0

Bandwidth scanner - Summer of Privacy
by juga 23 Apr '18

23 Apr '18

Hi, I will be working on the bandwidth scanner project during the Tor Summer of Privacy. This would include implementing features to the scanner code, writing tests and documentation, deployment in a test network, comparing results, modify little-tor code. The goal is to have bandwidth scanner code that is easier to maintain, understand, run and gives better measurements. There will be more details online and with reports. I've being already contributing on this project, for instance, writing a draft specification for the bandwidth measurements format [0] or reporting/fixing bugs on bwscanner code [1]. I'm MS in computer engineering, senior developer and have worked as sysadmin and researcher. Best regards, juga. GnuPG key fp: 2DA81D01455C3A0032198850F305447AF806D46B [0] https://lists.torproject.org/pipermail/tor-dev/2018-April/013071.html [1] https://github.com/TheTorProject/bwscanner

1 0

Notes from 19 April 2018 Simple Bandwidth Scanner Meeting
by Matt Traudt 19 Apr '18

19 Apr '18

Next meeting is scheduled 26 April 2018 at 2200 UTC in #tor-meeting, **BUT** we are probably going to look into finding a time that works better across three halves of the globe. Pad is here, and is copy/pasted below. https://pad.riseup.net/p/BenwGdgpz0uS Meeting IRC log http://meetbot.debian.net/tor-meeting/2018/tor-meeting.2018-04-19-21.59.log… Simple Bandwidth Scanner meeting 19 April 2018 ============================================== Previous weeks notes: https://lists.torproject.org/pipermail/tor-dev/2018-April/013059.html Updates -------------------------------------------------------- pastly: - Started running an authority in the test net. Don't think it's working yet and/or people have started trusting it - Switched to standard python logging module - Added periodic reachability tests for helper relays - (Helped) add FAQ and glosasry to docs - Learned: no part of sbws is CPU bound. Weird results were due to a helper that is not capable of much bandwidth - Time is still passing. In less than 2w the open source paperwork should be back next week: - Put docs on an onion service - Ask dirauths what they want to do about sbws servers In the coming weeks, I think I need to be transitioning away from "big" feature type stuff that isn't absolutely necessary, and instead focus my time on docs, tests, and making sbws easy to use. I think literal next steps need to include a practice deployment and transition in the testnet. It's one thing to run an sbws scanner and server. I've done that. I want to see other people try it so we can learn what isn't obvious to anyone but me. I want to exercise getting scan results to the dirauth so it can put them in the consensus. i've tried scanner and server and reported problems :) * list tasks for next months: https://pad.riseup.net/p/bfikl6PM0YV9 * plan proposed by teor with testnet deployment: * ... * it'll be very useful to have sbws doc public! juga: last week: * 1st draft spec * setup dirauth, sbws server, sbws scanner * dirauth not yet in consensus * start doc about what we have deployed so far (started to feel a bit lost about what we have/need) * added draft faq and glossary to doc next week: * finish spec * check the status of config files vs cli args, with debug (sorry pastly, didn't follow close the changes) and see whether #91 is really needed * check the status of tests/chutney * implement actually some unit test? * think how to make easier deployment at some point and add something on it? (cron/Makefile/system services?) questions: * pastly: once sbws repo is public, can we work on new releases publicly? [pastly: says yes] teor: last week: * occasional suggestions this week: * make my test dirauth a trust pastly's and juga's dirauths * eventually try to set up sbws on the testnet

1 0