- tor-dev - lists.torproject.org

Re: [tor-dev] First draft of Requirements and Software Design for a Better Tor Performance Measurement Tool
by Karsten Loesing 23 Oct '13

23 Oct '13

[Adding tor-dev@ back to the Cc.] On 10/8/13 7:25 PM, Robert Annessi wrote: > On Friday 06 September 2013 15:58:53 Karsten Loesing wrote: >> I just finished a first draft of a tech report sketching out the >> requirements and a software design for a new Torperf implementation. > >> Feedback in the form of patches to the LaTeX sources is most preferred, >> though email replies are of course fine, too. > > I would add a "Circuit build success/failure rate" to the tests. Recent events > (botnet) clearly showed that such data is needed but missing. Absolutely. The plan is to log stream and circuit information on the tor processes used by Torperf. See section 2.2.1 "Results format" for details: """ Results may include data which appear not immediately relevant to measuring Tor performance, but which may be useful for related purposes. For example, Torperf should include data about circuit failures in its results, even though these circuits may not have been used in actual requests.% \footnote{\url{https://trac.torproject.org/projects/tor/ticket/8662}} """ Also see the appendix that specifies data formats for stream and circuit information. If you have suggestions for making this clearer, can you send me a patch to the LaTeX sources? https://gitweb.torproject.org/user/karsten/tech-reports.git/blob/refs/heads… All the best, Karsten

1 0

Torperf implementation considerations (was: Torperf)
by Kevin Butler 23 Oct '13

23 Oct '13

[cc tor-dev] On 16 September 2013 09:47, Karsten Loesing <karsten(a)torproject.org> wrote: > Hmm, I don't think the HTTP client/server part is the right interface to > write another client and server and call it Torperf compatible. The > Torperf data API would be a better interface for that: people could > write their own experiments and provide data that is Torperf compatible, > or they could use the data that Torperf provides and analyze or > visualize it in a better way. But writing only half of an experiment, > client or server, wouldn't be of much use. > > I thought bit more about this and I don't fully agree (but you've nailed it that the data is the core api). I've come to the conclusion that I think it makes sense for TorPerf to do much of the heavy lifting and the core server aspects, but I think the experiments should be decoupled in such a way as to allow for flexible client implementations. Below are a bunch of semi related musings on how to get to this: Of course as many experiments will just be doing simple http requests, the experiments will really just be wrapper scripts around a bundled default client implementation which would be similar to the one in your perfd branch. Specifically, to aid in this, I'd propose something like the folder structure below: https://etherpad.mozilla.org/iqrgueVFd6 [Why a set of directories? There's not a solid reason other than it forces unique names and gives good opportunity to isolate experiment specific data. If an experiment has no specific data files it should be alright to just have the config file instead of a directory. A directory structure also mimics the idea of a restful web service as mentioned in the pdf, i.e. the user could easily know to go to http:/.../results/myexperiment/ to see results filtered for that single experiment. Either way, it's a minor detail, I just feel it's easier for a user.] A good use of the specific experiment data could be for static files, where anything in an experiments '/public' folder would be served by the webserver while that experiment is running. If wanting to ensure nothing gets proxy cached, the experiment could be responsible for generating random data files for each run. (This is a clear separation of server vs experiment implementation concern.) Another idea could be custom experiment views(js probably) that would transform the experiment results when viewed on the web dashboard. The experiments should not be responsible for determining details such as socks port or control port, the TorPerf service should deal with load balancing a bunch of tor instances on different ports and just tell an experiment 'Do your stuff using this sock port, this public ip:port, etc...' via environment variables. (The config could ask that certain aspects of the torrc file are setup specifically but managing ports and stuff is just asking for user error unless there's some experiment that requires it.) The config should minimally have an execution rate and a command. The command could be to just execute the bundled TorPerf client implementation with boring parameters e.g. just fetch facebook.com and record the normal timings that the default implementation tracks. A more interesting config's command could be the alexa_top_100 where it's just a basic script/makefile that fetches a new alexa list if the one in the local folder is older than X days and then for each site in the list it runs the TorPerf client instance. The TorPerf service instance should be able to run experiments by just executing the commands and recording whatever is written to stdout and stderr. After the command exits, if it's non zero then the stderr output is treated as error messages while if it's zero then it's treated as info messages. The stdout data should be treated as TorPerf results and it's an error if it's not well formed. If it's not well formed it should be captured in the results file for debug purposes. An example of informational output might be that the alexa_top_100 experiment updated it's list before this result set. [On the web interface it should be clear which results sets contained errors or information] Once the experiment finishes, the service should postprocess the results from the experiment and replace any fingerprinted entries (This needs to be well defined) with the server side timing information for that specific fingerprint. Then the server should store the results file in a timestamped file (timestamped probably by experiment start time) and update it's database(if there is one). The experiments should be able to specify their required tor version in their config, but it should accept placeholder values such as the default 'latest', 'latest-stable' or even 'latest(3)' which would run the experiment for all 3 of the latest tor versions. I think the ability to check the performance of the same experiment over multiple Tor versions could be interesting, especially to determine if any one build has caused anomalies in performance. I would expect very few experiments to run across multiple versions though. Additionally, something that could be neat, but it's not clearly in the requirements, should TorPerf be responsible for notifying the user when there are new clients available to run as latest? Would it be useful to be able to specify that some experiments should be run on 'master' or a gitref and that it would be pulled between runs? That's probably not practical. Apologies for the length and lack of order! Kevin

4 11

Help wanted: write user manuals on digital signatures and open proxies
by Runa A. Sandvik 21 Oct '13

21 Oct '13

Hi everyone, I am currently working on a rewrite of the short user manual [1] (relevant tickets are: #5571, #5811, #8003, #8779). The next version of the manual will (hopefully) be easier to read and translate, and it will also contain an updated list of frequently asked questions. I hope to have a version out for review next week. The old version of the manual contains pieces about verifying digital signatures and using open proxies. A number of people have suggested that we either move these pieces into an advanced section of the manual, or that we take them out completely. I intend to take these pieces out completely and suggest that we write separate manuals for them. The new manuals should be easy to read and translate, small enough to send via email, and be in a format which does not get blocked by most email providers (e.g. xhtml). If you would like to help with these two manuals, please take a look at Trac tickets #9883 and #9884. [1]: https://www.torproject.org/dist/manual/short-user-manual_en.xhtml Thanks, -- Runa A. Sandvik

2 2

[Otter/Buoyant] Tor documentation list
by Colin C. 21 Oct '13

21 Oct '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 At the last Buoyant Otter meeting, it was determined that we required a list of the documentation that we currently provide. Runa and I have put together a list, and it is now available on the wiki[1]. If anyone notices something missing, please feel free to edit the wiki. [1]: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Otter/Boisterous… - -- - -Phoul -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBCAAGBQJSZZuQAAoJEJ01Zvx7KO7YTy8QAJTfQnUVqSkqWxh40jmh9I3I qgdx8x8tqugS0kLOD3tDRTU6YvyHjdDYpbcKw87AAVh4cT8vhy0pzEmq2WpTSebA Z19eyytf511TsPC/Ke/yL7Ub0Xqs66j+niIcmcFY3cZIykcpH1lLa5OQ3BR51FYn yo5fznbra9fwv9/TbE4r2sZdBu8dH2Vs9pJtzmsMcgZcPVbZlUTE102qRh31Fsib pU9fepc6SaKQOCDXkFbaIoydVqlr8sovlfmUNqZ3qiups2KBxJHrYGZBLGqkVqAi 12ux8C6WTpuCRIlMbcAyjJjuSiEesQDS7dekXsc35YC5Q8TPWFTXuDDY4XIsZ5wR LGWOgNENl//DOb1KR6S7EX9YI/WcqG9o/mgKKuDapNIQZesP9rfGudZsl8IdhZer QRO7GtfwMqRJEflhm5W3BMVZbQStIbpSyzmbBK+kou6b968oi9xDhGD5yT1Co07W HwV2lwTdGhDCBgY3peymBKul8v4yCKex4IP1tgF//dZrD6CsahtMreyzq8G5Syf3 S4ojrv5AXXqR6X1F8qRYCPkHgAYeKPQ8RZx+ynb+T8P65bZ7yUz51s4pO4Vvhh7H FjdgJk4esBg+BBic8ju3ATfPizaaaf1xHeCdts/qLdYjCNeEamQ6CYPnbw+Onqcs lg3qrQ/S2M88Ba0jr5PO =CfQj -----END PGP SIGNATURE-----

1 0

Sponsor F: update; next meeting [in *two weeks*]
by Tom Lowenthal 21 Oct '13

21 Oct '13

Today, at 1100 Pacific, we spent more than 90 minutes discussing [Sponsor F][]. Here's the summary. **READ THIS**: The next Sponsor F meeting will be held in a mere two weeks on **2013-10-14, at 1100h Pacific in #tor-dev**. This is a schedule change: from now on, the meetings will be every two weeks. It's possible that we may have to increase this to every week, depending on how fast we work, and how long meetings are taking. If you should be at these meetings but cannot make Mondays at 1100h Pacific, please contact me, and I'll start the process of finding a better time or times. If you are individually in the `cc` field of this message, it's because I think that there's something which I think you need to do for Sponsor F before Halloween. You should also come to the next Sponsor F meeting. If you can't make the meeting, or don't think this work applies to you, you should get back to me ASAP so we can fix it. Is something else missing, wrong, or messed up? I'd like to know. [Sponsor F]: https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorF/Year3 * * * * * Core Phase 2 Deliverables ========================= UDP Transport [#10] ------------- **[On Track: Minimal]** Karsten will work with Rob to complete the Shadow simulation of this work, then write up a full report on this, probably with Stephen's help. We expect both tasks to be complete by Halloween. This likely represents a minimal outcome for this deliverable. Combine obfuscation (obfsproxy) with address-diversity (flashproxy) [#11] ------------------------------------------------------------------- **[On Track: Minimal]** The work of integrating obfsproxy with flashproxy is done. George will include this in the next released version of the pluggable transports browser bundle. George will also write a report on this work. Ximin and David will help. By Halloween, the report will be complete and the bundles will either be released or well on their way through testing. This likely represents some combination of minimal or intended outcomes for this deliverable. Bridge Metrics [#12] -------------- **[Done: Intended]** Our reporting code has been merged into master. It will ride the trains or flow downstream or whatever your favorite code development cycle imagery is, and show up in future releases. As it goes through alpha, beta, and release, gets picked up and adopted by more operators, we'll get more comprehensive sample coverage, and better data. This likely represents an ideal outcome for this deliverable. N23 Performance Work [#13] -------------------- **[On Track: Alternate]** Roger doesn't think that N23 is as good as we thought it was, so he's going to write a report on the various performance improvements we've implemented lately; the performance work which we though about, but decided not to implement, and why; and a wishlist of future performance work and research. He'll have this done by Halloween. This likely represents an alternate outcome for this deliverable. Improved Scheduling [#14] ------------------- **[On Track: Intended]** Nick will work with Roger and Andrea to implement an improved scheduler (possibly based on picking randomly, weighted by bandwidth), and -- if possible -- also to refactor `cmux`. If time permits, Nick will also attempt to simulate this using Shadow , probably with Karsten's help. Finally, Nick will produce a full report before Halloween. This likely represents an intended outcome for this deliverable. Alternate `Fast` Flag Allocation [#15] -------------------------------- **[At Risk]** The person who we initially expected to do this work does not seem to be available to do it. We need to find an alternate plan to execute this deliverable. If you think you can do it, please read [ticket #1854][#1854] and get in touch. [#1854]: https://trac.torproject.org/projects/tor/ticket/1854 VoIP Support [#16] ------------ **[On Track: alternate]** Our implementation strategy for this was high-latency send-an-mp3-over-XMPP using Gibberbot/Chatsecure, or a similar system. The internal milestone was to have a release candidate available today. Sadly, Nathan (who is on point for this) wasn't on the call. Fortunately, Nathan [blogged][guardian-1] about Chatsecure's `12.4.0-beta4` ten days ago, and a tantalizingly-named [`ChatSecure-v12.4.2-release.apk`][chatsecure-release] ([sig][chatsecure-release-sig]) is available in the Guardian Project [release directory][guardian-releases]. The outlook seems good, but Tom will follow up with Nathan as soon as possible to verify these outrageous claims. Nathan, if you're reading this, could you get in touch (email, IRC, XMPP, whatever). Thanks! [guardian-1]: https://guardianproject.info/2013/09/20/gibberbots-chatsecure-makeover-almo… [chatsecure-release]: https://guardianproject.info/releases/ChatSecure-v12.4.0-beta4-release.apk [chatsecure-release-sig]: https://guardianproject.info/releases/ChatSecure-v12.4.0-beta4-release.apk.… [guardian-releases]: https://guardianproject.info/releases/ Extended Phase 2 Deliverables ============================= VoIP Support [#17] ------------ **[Limbo: Intended]** Here we're talking about getting a general VoIP client (Mumble) to work over Tor. This discussed in [ticket #5699][#5699], and [instructions][torify-mumble] for using Mumble over Tor are on the wiki. We didn't get an update during the meeting, so if you're working on this, get in touch, eh? [#5699]: https://trac.torproject.org/projects/tor/ticket/5699 [torify-mumble]: https://trac.torproject.org/projects/tor/wiki/doc/TorifyHOWTO/Mumble Simulating Tor [#18] -------------- **[On Track: Intended]** Linus will finish Shadow improvements as needed, and write a full report on his Shadow work. That'll all be done by Halloween. This probably represents the intended outcome of this deliverable. Relays as Bridges [#19] ----------------- **[Done: Minimal]** Using a public relay as a bridge works. Defiance [#20, #21] -------- **[At Risk]** We cannot work on these without open Defiance code. Throttling Classifiers [#22] ---------------------- **[On Track]** Adaptive throttling seems to come with unpleasant anonymity tradeoffs, so we're not doing that right now. It looks like static throttling will work really quite well, without these tradeoffs. Torbrowser Optimistic Data [#23] -------------------------- **[Done: Ideal]** It's in your TBB *right now*. FAQ === **Q**: Why do you keep talking about Halloween? **A**: We're currently in Phase Two of Year 3 of the Sponsor F project. That phase ends on October 31st 2013, so we should try to finish everything as best we can before then. Also: if we don't finish, at midnight (local time) each person who has an outstanding deliverable will be besieged by a seemingly-endless horde of zombies. Others may be besieged too: the zombie report is hazy.

4 6

Re: [tor-dev] Sponsor F: update; next meeting [in *two weeks*]
by matt＠pagan.io 21 Oct '13

21 Oct '13

I wanted to give another update as to the status of patching Mumble to support native Tor proxification. Mumble uses its own connection protocol to make external connections, a protocol which is defined in a 30-page pdf file which comprises most of Mumble's documentation (the comments in the code are quite sparse). Most of the methods laid out in mumble-protocol.pdf are defined in the file mumble.pro (or mumble.proto in the github version). I suspect the method named 'connect' of being a culprit in the DNS leaks. Unfortunately 'connect' is not defined in mumble.pro like the rest of the connection protocol. Right now I am searching for where this method is defined. I have visited #mumble on freenode, but no one there was able to offer any insight. I even wrote a simple tool to help me search for this method, but 'connect' is used in over 30 of mumble's *.cpp files (and none of the *.h files).

1 0

I2P's Syndie (wrt HS publishing)
by George Kadianakis 21 Oct '13

21 Oct '13

Hey people, there seems to be some movement on improving HS publishing (making it easier and more user friendly) lately. I just wanted to point you to a project by I2P called Syndie which looks relevant to your interests: http://syndie.i2p2.de/ (FWIW, I haven't checked its code or its architecture yet.)

2 1

Attentive Otter: Usability issues with existing OTR clients
by Mike Perry 18 Oct '13

18 Oct '13

Here's the major usability pain points I've run in to with various OTR clients over the years. Feel free to chime in with any additional issues you've noticed. 1. The multiple client problem: If you have two devices online simultaneously under the same account, they will fight and endlessly renegotiate. This issue may be specific to OTRv2 only. Arlo pointed out that this change in OTRv3 may solve a lot of these problems: "Both fragmented and unfragmented messages contain sender and recipient instance tags. This avoids an issue on IM networks that always relay all messages to all sessions of a client who is logged in multiple times. In this situation, OTR clients can attempt to establish an OTR session indefinitely if there are interleaving messages from each of the sessions." -- https://otr.cypherpunks.ca/Protocol-v3-4.0.0.html We should be sure we use these features properly, and make sure the js-ctypes wrappers are updated to support the new libotr v3 APIs with these instance tags. The current js-ctypes wrapper may still be v2 only. If for some reason these tags in v3 can only be used if both parties support v3, perhaps we can add some additional code to track OTR state using XMPP resource/device info and tell the OTR lib underneath that different XMPP resources are actually different users, so that at least XMPP accounts won't have this problem. 2. Better TOFU support: We should track only one OTR key for a given XMPP resource/device ID. Some people I communicate with are constantly rotating OTR keys for security and other reasons. However, this doesn't help me any: I have accumulated about 50 keys of unknown age and usage pattern in my OTR fingerprint list for them, and I have no idea which ones have been changed due to compromise, device loss, or simply due to reinstall. Worse, I have no idea which keys should correspond to which XMPP resources/devices. If a user changes the key for a given XMPP resource ID, we should expire the previous key and loudly alert the user of the fact that the resource's key has changed, and ask them if they want to replace the old key with the new key. We should not simply accumulate more keys by default, especially for the same XMPP resource/device, and especially not for unverified keys. At the same time, we should allow a different OTR key for each unique XMPP resource ID, and perhaps allow non-XMPP keys to accumulate at user option. 3. "Unreadable encrypted message" errors should be visible only to advanced users/in the error console, and the client should transparently renegotiate and resend. It looks like OTR has an ERROR_START_AKE flag for this, but for some reason, some states of OTR in Pidgin still cause me to get and send these errors and miss messages though, usually if one client has crashed or restarted without properly ending the conversation. We should try extra hard to force key re-negotiation in these cases. We probably also want an auto-responder after the renegotiation that informs the other party that N messages were missed (for clients that don't support OTR's built-in notifications). 4. OTR shouldn't get stuck in the "Finished" state when you end a conversation. I've noticed in the past that OTR seemed to refuse to automatically negotiate/re-negotiate if it is still "Finished" regardless of your "Require/Force OTR" setting. This is just bad, and often causes plaintext to leak. 5. It should be impossible for our client to send a message in plaintext by default, and the client should discourage counterparties from sending more than one plaintext message. Perhaps related to issue 3, I've not had very good results with the "Require OTR"/"Force OTR" options on many clients. It also seems like many people's clients will still send the first message unencrypted (to get the handshake hint), and on all clients OTR doesn't seem to try very hard to negotiate if an unencrypted message is received. On our side, we should hold those messages and send only the handshake initiation (ie our default OTR mode should be REQUIRE_ENCRYPTION). We may also want to consider a plaintext auto-reply for when we receive unencrypted messages. Something like "Please install and enable OTR before sending further messages" as a separate message in *addition* to attempting an OTR handshake. 6. The UI should have per-message encryption indicators (send and receive) For many clients, the UI is not clear which messages were sent encrypted, especially if the OTR state has changed due to autonegotiation or "Finishing". We should have per-message icons for both sent and received messages that indicate encryption status for that message in the chat scrollback window. 7. Logging should be off by default when OTR is on. -- Mike Perry

3 2

REMINDER: Attentive Otter IM meeting in 1h [1430 Pacific]
by Tom Lowenthal 18 Oct '13

18 Oct '13

Today at **1430h Pacific** (an hour from now), we'll be in #tor-dev talking about how to turn Instantbird into TorMessenger, or MessengeTor, or OnionTalk, or something. Some of you noticed my cunning test of putting different times in the subject and the body. The body was right, it's 1430 Pacific! See y'all soon! -Tom On 15 October 2013 21:57, Tom Lowenthal <me(a)tomlowenthal.com> wrote: > Sorry about this late notice too. I thought I'd sent these times out > already, but apparently I was rather mistaken. > > We'll be in #tor-dev tomorrow Wednesday 2013-10-16 at 1430 Pacific to > talk about the Tor IM project. We may take up to 90 minutes, and plan > to meet again next week, probably at the same time unless there are > conflicts. > > Arlo came up with [a cunning plan][1] m'lord. You should totes read it > and come tomorrow with questions and suggestions about how best to > implement Instantbird as the Tor IM app. > > See y'all tomorrow. Play nice, now. > -Tom > > [1]: https://gist.github.com/arlolra/6969273

2 1

REMINDER: 1 hour until Cute Otter hidden services & publishing
by Tom Lowenthal 16 Oct '13

16 Oct '13

Just under an hour until 1130 Pacific on Wednesday 2013-10-16. We'll be in #tor-dev, talking hidden services and publishing. -Tom On 15 October 2013 21:40, Tom Lowenthal <me(a)tomlowenthal.com> wrote: > Sorry about the late notice, folks. I thought I'd sent these times out > already, but apparently I was rather mistaken. > > We'll be in #tor-dev tomorrow Wednesday 2013-10-16 at 1130 Pacific to > talk about hidden services, scaling thereof, and safe publishing > therewith. We may take up to 90 minutes, and plan to meet again next > week, probably at the same time unless there are conflicts. > > It'd be super neat if you could familiarize yourself with [Sina's > proposal][1] before the meeting. Come with constructive comments and > suggestions. > > See y'all tomorrow. > -Tom > > > [1]: https://lists.torproject.org/pipermail/tor-dev/2013-October/005558.html

1 0