- tor-dev - lists.torproject.org

Help us build Tails 3.2~alpha1 build reproducibly
by anonym 08 Sep '17

08 Sep '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Dear Tails and Tor contributors, dear Reproducible Builds community, We have sent out a first call [1] for testing to build Tails 3.1 reproducibly and we have received some build reports. Thank you very much for your help! We have since then tried to fix most of the identified issues [2] in Tails 3.2~alpha1, and thus we'd kindly like to ask you to try to build the new ISO image again, or even for the first time. Please don't hesitate to contact … [View More]us if you get stuck at some point in the process, for example by connecting to our chatroom [3]! You can also send us email to tails-dev at boum.org (public) or tails at boum.org (private). Note that Tails 3.2~alpha1 is *not* recommended for real usage, since it has not gone through *any* QA. Please use Tails 3.1 instead until Tails 3.2 is released! # How? For your convenience all instructions needed to attempt to reproduce Tails 3.2~alpha1 are included hereafter. However all commands are adapted for Debian Stretch (and Buster/Sid), so your results may vary if you run another Linux distribution. Our full build instructions [4] might help if you are having problems. ## Setup the build environment Building Tails requires the KVM virtual machine hypervisor to be available, a minimum of 1 GiB of free RAM and a maximum of 20 GB of free storage. ### Install dependencies sudo apt-get install \ git \ rake \ libvirt-daemon-system \ dnsmasq-base \ ebtables \ qemu-system-x86 \ qemu-utils \ vagrant \ vagrant-libvirt \ vmdebootstrap && \ sudo systemctl restart libvirtd ### If building as a non-root user (Skip this section if you intend to build Tails as the root user!) Make sure that the user that is supposed to initiate the build is part of the relevant groups: for group in kvm libvirt libvirt-qemu; do sudo adduser $user $group; done Then run `newgrp` (or just reboot) to apply the new group memberships to the session. ## Build Tails 3.2~alpha1 git clone https://git-tails.immerda.ch/tails cd tails git checkout 3.2~alpha1 git submodule update --init rake build # Send us feedback! No matter how your build attempt turned out we are interested in you sending us feedback. For that we'll first need some information of the system you used -- please run these commands in the exact same terminal session that you ran `rake build` in (e.g. run them right after `rake build`)! sudo apt install apt-show-versions || : ( for f in /etc/issue /proc/cpuinfo do echo "--- File: ${f} ---" cat "${f}" echo done for c in free locale env 'uname -a' '/usr/sbin/libvirtd --version' \ 'qemu-system-x86_64 --version' 'vagrant --version' do echo "--- Command: ${c} ---" eval "${c}" echo done if which apt-show-versions >/dev/null then echo '--- APT package versions ---' apt-show-versions qemu:amd64 linux-image-amd64:amd64 vagrant \ libvirt0:amd64 fi ) | bzip2 > system-info.txt.bz2 Please have a look at the generated file with bzless system-info.txt.bz2 to make sure it doesn't contain any sensitive information you do not want to leak in case you send this file to us or make it public! Next, please follow the instructions below that match your situation! ## If the build failed. Please open a ticket on our bug tracker [5] with "Category" set to "Build system" and `system-info.txt.bz2` attached (note that this makes this file public). ## If the build succeeded ... Please compute the SHA-512 checksum of the resulting ISO image: sha512sum tails-amd64-3.2~alpha1.iso and compare it to: 1c928336264fc44821562f2fffbda4da97dcdc38072fce58f55b749fde04ac60055273cfc021b6c57120c5d276980859ffa3a5b0bd0f9c98851f34b682a09b02 tails-amd64-3.2~alpha1.iso Bonus points if you verify the signed (with: [8]) message containing the checksum below (note that manually inserted line-wraps marked with "`\`"). If you run Tails, the verification is very easy! :) [9] - -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 $ sha512sum tails-amd64-3.2~alpha1.iso 1c928336264fc44821562f2fffbda4da97dcdc38072fce58f55b749f \ de04ac60055273cfc021b6c57120c5d276980859ffa3a5b0bd0f9c98 \ 851f34b682a09b02 tails-amd64-3.2~alpha1.iso - -----BEGIN PGP SIGNATURE----- iQJDBAEBCgAtFiEEuiwiL0SsAO2YmTiTmP7GvHUqPbYFAlmxqwAPHHRhaWxzQGJv dW0ub3JnAAoJEJj+xrx1Kj22RgAP/0auINj6Y5svR7DfeRF8HxPdnd2Rw/8VIiaM isN3eQoAmNGUtEe50b9VXY+UidCdWtApbZbyZPKFz9ITJOxp0XeSGS8K2+Y0PZIx NSIEYCx2LEWlzY96ivH2B4pboeq2TIzj/VkPLISYGc80CYRT32OzMRkDDcQn+3+Y 2dkGVf1HPvreZ7c7cUfozay4TNPhKrn2p5IZp1jHgpiq8aAYIv5jcubR//lm1W3S Ol/IpTQrxzCShJHzsCh1l6/7zLSx5Dv8ITTEIHTj2OTCsZdAcFnznB4byaHVfVQ0 jSogb+b2J6skNhlsHtX2Jo9xK6Ni9NKsCzYYQ2KgWufC93Cvpmh5J164CqkI/DEd ixe/KbFURP9sTzEL38ExS2DVbMvFvYTmmBmWzvU3USMo0nWfaErye2RIs4yB2pM6 wbsgy1g93tQ49YUU0eKFcoUQLnQpKJmZVsLacdkGpLTn7pL9uHR3AcUnRT27CSzb anzpyLVTllyApU+Suu3f5rJibx9ysmLu8gHp5/Osd54qzSBR9JVrhMTkgJmZnit4 67wuSWJDQgvhMKxRz+zB/z4biORIfyblviR4rv1ZbAfX/ChP25CWUxG9Tgk1Esp0 LMzroJyQNfmy++6GrGuHJ3miRsKwO66ot84GSc5/pN1cJHKYiNa4HQhfb+VZBXbk HoXLV3eR =Hhyr - -----END PGP SIGNATURE----- ### ... and the checksums match (i.e. reproduction succeeded). Congrats for successfully reproducing Tails 3.2~alpha1! Please send an email to tails-dev at boum.org (public) or tails at boum.org (private) with the subject "Reproduction of Tails 3.2~alpha1 successful" and attach `system-info.txt.bz2` to it. ### ... and the checksums differ (i.e. reproduction failed). Now you are in a great position to help Tails improve its reproducibility! Please install `diffoscope` [7] version 83 or higher. If you run Debian Stretch, that is: echo 'deb http://ftp.debian.org/debian stretch-backports main' \ | sudo tee /etc/apt/sources.list.d/stretch-backports.list && \ sudo apt update && \ sudo apt -o APT::Install-Suggests="true" \ -o APT::Install-Recommends="true" \ install diffoscope -t stretch-backports Then download the official Tails 3.2~alpha1 ISO image [6] and compare it to yours: diffoscope \ --text diffoscope.txt \ --html diffoscope.html \ --max-report-size 262144000 \ --max-diff-block-lines 10000 \ --max-diff-input-lines 10000000 \ path/to/official/tails-amd64-3.2~alpha1.iso \ path/to/your/tails-amd64-3.2~alpha1.iso && \ bzip2 diffoscope.* Please send an email to tails-dev at boum.org (public) or tails at boum.org (private) with the subject "Reproduction of Tails 3.2~alpha1 failed" and attach `system-info.txt.bz2` to it. We also want you attach one (the smallest!) of `diffoscope.txt.bz2` and `diffoscope.html.bz2` to the email, but if they are "big" (say >100 KiB) then please don't bomb our mail inboxes! Instead upload the file to some web-based file-sharing service (we recommend RiseUp [10]) and include the link(s) in the email. Thank you very much for your interest and help! Cheers! The Tails project [1] https://mailman.boum.org/pipermail/tails-dev/2017-August/011591.html [2] https://labs.riseup.net/code/issues/13624 https://mailman.boum.org/pipermail/tails-dev/2017-March/011297.html [3] https://tails.boum.org/support/#talk [4] https://tails.boum.org/contribute/build [5] https://labs.riseup.net/code/projects/tails/issues/new [6] http://dl.amnesia.boum.org/tails/alpha/tails-amd64-3.2~alpha1/ https://mirrors.wikimedia.org/tails/alpha/tails-amd64-3.2~alpha1/ https://mirrors.kernel.org/tails/alpha/tails-amd64-3.2~alpha1/ [7] https://diffoscope.org/ [8] https://tails.boum.org/tails-signing.key [9] https://tails.boum.org/doc/encryption_and_privacy/gpgapplet/decrypt_verify/ [10] https://share.riseup.net/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEUrafEKOweFrQWvtHHYTM8BDMW8cFAlmxuZIACgkQHYTM8BDM W8fBag/+PlnYIFIfxf4KbPh97Tm9hrNlC6bJp5uTC+6WebBamVsEDekQXnXRzEtF wN4btdFG7MJuLHocSqeiV7gX33A07+KaugqCjQ4NJcVkZ6L9HPK6a1bjn9909EWw l/Cfn0NxaSkfGLhOsk2wFOm2BHU890But+j3bOPOU61JNMVUpwiHUbN2DS1xDK2X ZKE0dlOP3O4TC7kee7rRy6PFUVQx0eQkVW+1qrcusEeHEkc/Z6YzuLmNBdcHJ96m /OfkEi4ygY6l9j0B8z4O/vqPkyeU5VWQWwhhnETvJiSDqs5WNLDuttHOsumP0r29 sluAmicjNoM48mWJHwiyLYnEt5SfFHXOqHXPP1k5YzTOMLRfv/F2eH4Zoq/eI8TM sJy2MO+MgShH8SAOe1vJMBlV9Jqs6t85eToQt+39RFCCn4geyEYlynrk2Ka2hE3n ycn2Qa79nbNK8NJve+1rPZTiJq3lgYyXTYyb8chefKyWm1IsAdY8grS0UIYP44Po KAApudKD75DuCf6qczkOChVeAOcABu63RmAJLnJTYnEwGf7JLJ+KHrCdq0FQQS8s /PulDSSDZo9bhLAHx9tO2T6p2snJfZSf9/dmvM5H+kOlMwzY8GciCgN6wlluwlnU 5WMfAMYT5mhuKIH/WIU5BXlKuUj1MuOi0c64q2MQM2GByJ6SOa8= =6m+4 -----END PGP SIGNATURE----- [View Less]

1 3

[release] [protocol] Onionoo 4.1-1.4.1
by iwakeh 31 Aug '17

31 Aug '17

Hi there! Onionoo's protocol was extended and has a minor version jump to 4.1, therefore a new release is available: https://dist.torproject.org/onionoo/4.1-1.4.1/ Protocol changes (also summarized in [0]): * removal of (unused) beta fields in bridge client objects * addition of a new search filter parameter by Tor version, see [1] and [2] for example calls. There were also some internal improvements, which can be found in the changelog [3a]. You might have noticed an increment in … [View More]

1 0

Meeting on IRC to discuss prop280 privcount stuff (Sep 12, 8pm EDT)
by Nick Mathewson 28 Aug '17

28 Aug '17

Hello! On #tor-dev at 8pm EDT on Tuesday Sep 12 (== 00:00 Sep 13 in UTC, I think), we'll have an IRC meeting to talk about proposal 280, and integrating privcount into Tor. If you're interested, please feel free to drop by! The current draft of the proposal is at https://gitweb.torproject.org/torspec.git/tree/proposals/280-privcount-in-t… best wisehes, -- Nick

1 0

Proposal 281: downloading microdescriptors in bulk
by Nick Mathewson 28 Aug '17

28 Aug '17

Filename: 281-bulk-md-download.txt Title: Downloading microdescriptors in bulk Author: Nick Mathewson Created: 11-Aug-2017 Status: Draft 1. Introduction This proposal describes a ways to download more microdescriptors at a time, using fewer bytes. Right now, to download N microdescriptors, the client must send about 44*N bytes in its HTTP request. Because clients can request microdescriptors in any combination, the directory caches cannot pre-compress responses to these requests,… [View More] and need to use less space-efficient on-the-fly compression algorithms. Under this proposal, clients simply say "Send me the microdescriptors I need", given what I know. 2. Combined microdescriptor downloads 2.1. By diff If a client has a consensus with base64 sha3-256 digest X, and it previously had a consensus with base64 sha3-256 digests Y then it may request all the microdescriptors listed in X but not Y, by asking for the resource: /tor/micro/diff/X/Y Clients SHOULD only ask for this resource compressed. Caches MUST NOT answer this request unless they recognize the consensus with digest X, and digest Y. digest Y. If answering, caches MUST reply with all of the microdescriptors that the cache holds that were listed by consensus X, and MUST omit all the microdescriptors that were omitted listed in consensus Y. 2.2. By consensus: If a client has fewer than NMNM% of the microdescriptors listed in a consensus X, it should fetch the resource /tor/micro/full/X Clients SHOULD only ask for this resource compressed. Caches MUST NOT answer this request unless they recognize the consensus with digest X. They should send all the microdescriptors they have that are listed in that consensus. 2.3. When to make these requests Clients should decide to use this format in preference to the old download-by-digest format if the consensus X lists their preferred directory cache as using a new DirCache subprotocol version. (See 5 below.) 3. Performance analysis This is a back-of-the-envelope analysis using a month's worth of consensus documents, and a randomly chosen sample of microdescriptors. On average, about 0.5% of the microdescriptors change between any two consensuses. Call it 50. That means 50*43 bytes == 2150 bytes to request the microdescriptors. It means ~24530 bytes of microdescriptors downloaded, compressed to ~13687 bytes by zstd. With this proposal, we're down to 86 bytes for the request, and we can precompute the compressed output, making it save to use lzma2, getting a compressed result more like 13362. It appears that this change would save about 15% for incremental microdescriptor downloads, most of that coming from the reduction in request size. For complete downloads, a complete set of microdescriptors is about 7700 microdesciptors long. That makes the total number of bytes for the requests 7700*43 == 331100 bytes. The response, if compressed with lzma instead of zstd, would fall from 1659682 to 1587804 bytes, for a total savings of 20%. 5. Compatibility Caches supporting this download protocol need to advertise support of a new DirCache subprotocol version. [View Less]

3 2

Some Tor Devs could get higher pay as PR than as programmers
by Ryan Carboni 24 Aug '17

24 Aug '17

Whenever I read something, with an open mind, but say, in a sandboxed environment (don't run your memes on root), I get impressed by arguments which convince me of things which are not true. Take for instance the following article: https://blog.torproject.org/blog/tor-security-advisory-relay-early-traffic-… It is a self-contradictory argument, we detected these bad relays when they arrived, allowed them for 6 months. Now, what was the available prophylactic measures at the time? Alleged … [View More]

2 1

Draft patchset for PT2.0 support (ticket #21816)
by Robin Tarsiger 24 Aug '17

24 Aug '17

Good day, tor-dev; As some of you know, I've been working on a patchset for Tor to allow it to participate in Pluggable Transports 2.0 configuration, primarily the new JSON Parameter Block SOCKS method (at least that's what I've been calling it in the absence of a more official name), almost but not quite as described in section 3.3.4 of PT2 draft 2 [1]---i.e., basically #21816 [2]. [1] https://www.pluggabletransports.info/assets/PTSpecV2Draft2.pdf [2] https://trac.torproject.org/projects/… [View More]tor/ticket/21816 I'm attaching a draft patchset which adds this functionality, with the intent of getting feedback and making remaining cleanups or other modifications necessary to get it merged into Tor. I have successfully completed circuits through an obfs4 bridge using both obfs4proxy (PT1) and a version of shapeshifter-dispatcher (PT2) using a patched Tor. I've tried to follow the local style, but the preferred implementation strategies aren't always clear, and of course I'd appreciate any reports of other problems. A forked Git repository is also available on Bitbucket [5][6], which will be updated as I make remaining changes. [5] https://bitbucket.org/DasyatidPrime/tor-rtt2017-21816.git (Git) [6] https://bitbucket.org/DasyatidPrime/tor-rtt2017-21816/src (Web) More implementation details are below if you're interested in this; thanks for your attention. I'll try to be around on IRC more during the week, so feel free to ping me there as well. -RTT ... Details: Not visible above, related to the target functionality: - I'm assuming the SOCKS method includes a response with an analogous structure to RFC 1929; PT2 draft 2 doesn't specify one. I've cleared this with blanu, and that's intended for the next draft of the PT2 specification. - Similarly, the length prefix is in fact big-endian; the example in PT2 draft 2 is wrong, though the text is correct. - We're looking into getting an IANA assignment for the method number. Technically this probably meets the requirements for the private use block, but I feel like interop might be easier later on, and it could simplify code paths in places to have a registered number (mainly if it's possible to decouple the method negotiation from the configuration-version plumbing). I believe this is still in limbo. - For the PT2 side, I've been testing against my branch of shapeshifter-dispatcher [3] compiled with my branch of shapeshifter-ipc [4] since there were some breaking changes to Shapeshifter upstream which were otherwise preventing it from interoperating with Tor. I'm planning to help merge fixes into Shapeshifter upstream as I can; it's my current understanding that there aren't any other PT2 managed transport implementations to test against. [3] https://github.com/OperatorFoundation/shapeshifter-dispatcher/tree/rtt2017 [4] https://github.com/OperatorFoundation/shapeshifter-ipc/tree/rtt2017 Issues on my radar currently (comments appreciated): - We probably want unit tests for the (limited) JSON encoding functions, and for the factored-out RFC 1929 encoding functions. Anything else that looks feasibly testable here? - It's not clear to me whether negotiating a PT2 configuration version still allows PT1-style RFC 1929 parameter encoding so that managed transports can support the new configuration version and the new SOCKS method separately. I've assumed it might be possible, so far, but that's not being tested against anything. - I currently restrict parameters to ASCII to avoid either writing a JSON encoder that can spit out invalid JSON or writing a JSON encoder that has to validate incoming UTF-8. The impression I've gotten is that this is probably okay, but if there are counter- examples, I can put in UTF-8 passthrough. - The commit sequence isn't the cleanest. How high a priority is it to reorder/combine patch hunks to make a cleaner one? - We still need a 'changes' file. (What would be an appropriate heading for this? Is this a minor feature, for instance?) A few other questions: - Is there an effective way of doing automated testing of the SOCKS state machine currently in Tor? I didn't see anything obvious in the test directory. This seems like the most fragile part, especially since both the original and modified versions are not very explicit in their state machine nature and are split between multiple files. - Can there ever be more than one managed_proxy_t to a transport name? More generally, is there a relational diagram of the main Tor data structures somewhere? A lot of the way the plumbing for state and configuration information is set up feels kind of fragile. [View Less]

3 3

Tor and IP2Location LITE
by KL Liew 24 Aug '17

24 Aug '17

All, My name is Kim, the founder of IP2Location, a geolocation service provider since 2002. It looks like Tor is looking to review other providers for GeoIP service while I was reading one of a meeting minute for a meeting back in March 2017. https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam/No… We are very interested in contributing to Tor and work on this matter. Tor can host and integrate IP2Location LITE (http://lite.ip2location.com) into their application. … [View More]

6 9

Re: [tor-dev] tor-dev Digest, Vol 79, Issue 4
by John kongtcheu 21 Aug '17

21 Aug '17

Hello Tor developers, I am interested in becoming an open source contributor for Tor, but I don't know where to start some guidance would be appreciated. Thank you, On Sat, Aug 12, 2017 at 8:00 AM, <tor-dev-request(a)lists.torproject.org> wrote: > Send tor-dev mailing list submissions to > tor-dev(a)lists.torproject.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > or, … [View More]via email, send a message with subject or body 'help' to > tor-dev-request(a)lists.torproject.org > > You can reach the person managing the list at > tor-dev-owner(a)lists.torproject.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of tor-dev digest..." > > > Today's Topics: > > 1. Proposal 281: downloading microdescriptors in bulk > (Nick Mathewson) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 11 Aug 2017 13:36:00 -0400 > From: Nick Mathewson <nickm(a)torproject.org> > To: tor-dev(a)lists.torproject.org > Subject: [tor-dev] Proposal 281: downloading microdescriptors in bulk > Message-ID: > <CAKDKvux=eJBh_JsEeiqnhbEbcgRVeMRrbm2G7itZ8kX > y4+M26A(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Filename: 281-bulk-md-download.txt > Title: Downloading microdescriptors in bulk > Author: Nick Mathewson > Created: 11-Aug-2017 > Status: Draft > > 1. Introduction > > This proposal describes a ways to download more microdescriptors > at a time, using fewer bytes. > > Right now, to download N microdescriptors, the client must send > about 44*N bytes in its HTTP request. Because clients can request > microdescriptors in any combination, the directory caches cannot > pre-compress responses to these requests, and need to use less > space-efficient on-the-fly compression algorithms. > > Under this proposal, clients simply say "Send me the > microdescriptors I need", given what I know. > > 2. Combined microdescriptor downloads > > 2.1. By diff > > If a client has a consensus with base64 sha3-256 digest X, and it > previously had a consensus with base64 sha3-256 digests Y then > it may request all the microdescriptors listed in X but not Y, > by asking for the resource: > /tor/micro/diff/X/Y > > Clients SHOULD only ask for this resource compressed. > > Caches MUST NOT answer this request unless they recognize the > consensus with digest X, and digest Y. > digest Y. If answering, caches MUST reply with all of the > microdescriptors that the cache holds that were listed by > consensus X, and MUST omit all the microdescriptors that were > omitted listed in consensus Y. > > 2.2. By consensus: > > If a client has fewer than NMNM% of the microdescriptors listed in a > consensus X, it should fetch the resource > /tor/micro/full/X > > Clients SHOULD only ask for this resource compressed. > > Caches MUST NOT answer this request unless they recognize the > consensus with digest X. They should send all the microdescriptors > they have that are listed in that consensus. > > 2.3. When to make these requests > > Clients should decide to use this format in preference to the > old download-by-digest format if the consensus X lists their > preferred directory cache as using a new DirCache subprotocol > version. (See 5 below.) > > 3. Performance analysis > > This is a back-of-the-envelope analysis using a month's worth of > consensus documents, and a randomly chosen sample of > microdescriptors. > > > On average, about 0.5% of the microdescriptors change between any > two consensuses. Call it 50. That means 50*43 bytes == 2150 > bytes to request the microdescriptors. It means ~24530 bytes of > microdescriptors downloaded, compressed to ~13687 bytes by zstd. > > With this proposal, we're down to 86 bytes for the request, and we > can precompute the compressed output, making it save to use lzma2, > getting a compressed result more like 13362. > > It appears that this change would save about 15% for incremental > microdescriptor downloads, most of that coming from the reduction > in request size. > > For complete downloads, a complete set of microdescriptors is about > 7700 microdesciptors long. That makes the total number of bytes > for the requests 7700*43 == 331100 bytes. The response, if > compressed with lzma instead of zstd, would fall from 1659682 to > 1587804 bytes, for a total savings of 20%. > > > 5. Compatibility > > Caches supporting this download protocol need to advertise > support of a new DirCache subprotocol version. > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > tor-dev mailing list > tor-dev(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > > > ------------------------------ > > End of tor-dev Digest, Vol 79, Issue 4 > ************************************** > [View Less]

2 1

Re: [tor-dev] Feature Request: please consider ship default, Tor bridges
by iry 19 Aug '17

19 Aug '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Roger: > On Thu, Aug 17, 2017 at 07:57:53PM +0000, iry wrote: >> Btw, Collateral Freedom seems to be one of the most effective >> ways to circumvent Internet censorship in China. Circumvention >> tools that depend on Collateral Freedom usually works fine, >> including meek, lantern, psiphon3 etc. Therefore, I see a lot of >> potential work which may benefit the Internet freedom in China. >> For example:… [View More] 1. package meek into Debian 2. host (part of the) >> BridgeDB mirror on Github or AWS 3. #22402: >> https://trac.torproject.org/projects/tor/ticket/22402 > Some hopefully useful thoughts along these lines: > Hi Roger! Thank you for sharing your insights! > A) Most places around the world that need bridges these days need > pluggable transport bridges, not just vanilla bridges. So if we > want to bundle bridge addresses, we should bundle PT bridge > addresses. > I agree! This also makes me think about a small potential improvement on the design of BridgeDB web. When users click the big "Just give me some bridge" on https://bridges.torproject.org/options , they will be provided with vanilla bridges instead of obfs4 bridges. However, those users who choose not to use "Advanced Options" are most likely to be inexperienced and have no idea on what obfd4, PT etc are. Therefore, is it a good idea to provide obfs4 bridges, rather than vanilla ones, in "Just give me some bridge" for better usability and higher success rate? > B) ...and that means we need to make sure that those PTs are well > packaged in Debian too, since the Tor deb by itself would not be > able to use them. > Agreed. I can help to report a bug against obfs4proxy on Debian BTS when the idea is mature. > C) I wonder if we could use the new %include torrc directive in > this situation: https://bugs.torproject.org/1922 I don't have a say on the final decision, but this is also what I am thinking about:> 3. "Bridge" + plain text which is ready to be appended to a torrc file > or to be one of the torrc files in /etc/torrc.d/ (or whatever > torrc.d path Debain decides to use) > That is, when you apt-get install obfs4proxy, is that the right > time to populate /etc/torrc.d/obfs4-bridges with some (probably > commented out to start) lines that let you use those bridges if > you want? How about not commenting out the bridge info as the switch? Instead, user who would like to use bridge can comment out or add the following two lines in /etc/tor/torrc: #UseBridges 1 #ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy I have being considering /etc/torrc.d/ as a place to store all the available settings and considering /etc/tor/torrc as a panel that let user decide what they would like to enable and disable. But I am not sure if this is a good way to think about the relationship between /etc/tor/torrc and /etc/torrc.d/ One thing I am concerned about is, when applying the method above, how can user choose different PTs in the future? For example, when meek in packaged into Debian, the meek package will probably also have a /etc/torrc.d/meek-bridges file. Can we just choose to use obfs4 or meek in /etc/tor/torrc by switching between the following two lines? #ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy #ClientTransportPlugin meek exec /usr/bin/meek-client (I can test with that, but people who is familiar with the mechanism behind it can probably answer that.) Best, iry -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZmHZUAAoJEKFLTbxtzdU8Y2gP+wfDoSOTprVqe4rByzHVhUup LXvy4WnfHeUq03t+H8W5QwJCPdsFssFZae/EsfUU5Q490GqhAFryi8rHmOHluSGl FmX9SIxQXT3RV7iEFN7w4qKqYHLEKrRTklWOLrDauHZe3eJEUyn49VCROtBat88Y bdnrav4D443fFD/ugk8C3K5u4oNEWgtaq9natsLkPFbXpNcB06gd/S23Vj4VsOP0 +FUmIHmtzzMk0iTAVjGrW8X/z6/lyqk6Nj/lvwfNhdIJ5gbk5F848sl71VBeK3of Q7rdCjglZ3/tPRfrE+d40cfQWmOpw7doozC7LKdDbxCtx/LJ0WFq2PvwmO+uUFOG 3QMAV0w58JWsUsLW80ifcx7T+0O7QeAMASqWqKva7d1Y2PgqNCJfDJDHOX/KuEy3 TUi8o5WkIUUtyPaMwnYXW9VkaEKIHBXxfjYBwg0llfGU9HKJmTj+yCrMMHB2t0m0 OlZUFx9E7U11mhWzGM64sICHuob8VjwsLgLPMJc1+ocDobo0HrJr1ZtWmMZScp60 SpK35EwU8jKZJTSXtI/SKtDdHGhGmxm4NvLy26TWSqHbEzlcelK+yqkifnLHr3tb we61NdV2ZA7XCrp+o3Q7NprMjRGtaP1aze2bQTjJfkLDdrShkZFw3/hYk75cRpvk tkf7Tl4oLtoa5lnNhB/S =sP5G -----END PGP SIGNATURE----- [View Less]

1 0

Feature Request: please consider ship default Tor bridges
by iry 19 Aug '17

19 Aug '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello Tor people! A set of Tor bridges are shipped with Tor browser bundle[0], helping users in Tor-censored area to connection to the Tor network. Since system Tor users may also face the censorship problem, shall we ship some Tor bridges along with the tor package? The request is firstly reported[0] to Debian BTS and I got the following reply by Peter: > If upstream starts shipping bridges with their Tor releases, that > would … [View More]naturally result in the Tor package shipping bridges as > well. > > I do not know whether that's a good idea or not, but I don't think > deviating from upstream would be particularly worthwhile. The following is some related information which may help the future discussion: The possible formats to hold those bridges can be: 1. JSON which is also the way tor-connection-wizard used so far[1]; 2. plain text which is the same formatt given by the BridgeDB[2] by Tor Project; 3. "Bridge" + plain text which is ready to be appended to a torrc file or to be one of the torrc files in /etc/torrc.d/ (or whatever torrc.d path Debain decides to use) The default bridge shipped with tor package should be exactly the same bridges contained in bridge_prefs.js[0] shipped with the latest stable TBB. This is because: 1. The servers hosting default bridges are set up for huge amount of traffic; 2. The servers hosting default bridges are probably audited by TPO for better security; 3. Using a different set of bridges will distinguish the anon-connection-wizard bridge users from the TBB bridge users, which compromises their anonymity. What do you think? Looking forward to hearing your insights! Best, iry [0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=872456 [1]: https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/Bundl e-Data/PTConfigs/bridge_prefs.js?h=7.5a3-linux [2]: https://github.com/irykoon/anon-connection-wizard/blob/master/usr/share/ anon-connection-wizard/bridges_default [3]: https://bridges.torproject.org/options Best, iry -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZldAaAAoJEKFLTbxtzdU8QHEP/0DyAtD95JOKEkDulsyuuVfx uYT+5WGaRSFntqRq91RcNkHLHDy61JtLXhr5VIcz/tYIIe3TVZhI9idqbPgJMZc+ xxS/4r5qhkkcJ5X99xo7Jerz/0Y/4CKboREAkSstz15RL3FNLF6mwFZgWsiZ4rMa SkruE8qchz1KIUuWKZyx3HioloZgIHQkvqQ6fE4asGIs8gnaKeofpSwRGq85/Vcq T8D0WTqCFweLFaYzCWMtO7bVKXrfqC8rGLesLPJhxZbl0MJ3H/5TdvbPHn6VwRH0 AZCT5q7A+0fC/+HHwsn9SFhMo0TaIOtZBonOH58X3OEamKrmJOwqESvCPqILtMC3 pU2PtoOSDQEd684b2hxoR+0uRMOew+CJ+U7lzyh7yYU9x3jv//9CsFGcKcD+FoFG zrkDPJ75uzYGJjHZUFpnk8opwVy49TghYiVvjwm9/PXXQVvEiCGNEt7W1wTHX3Ja DygMsGN8GXg6AqWRESg4NcI8N/4U2EUEt+Li45u0qsTJ/uYmfamsC/WoacfjznaD JQVKjJQlhuk7F3qUPuxtxaGCLopLJd/uAUyYaI/HPrFeR3uIawSzCW9prTkk/E7n wAop3mErdp6JYnTacUb4pcYINIQf1c7FymbngrAmJzljxH4apZpBPugitAYvtJ2X 2OBJpUV0CtEJH3poicdq =oct9 -----END PGP SIGNATURE----- [View Less]

3 3

[release] CollecTor 1.2.1
by iwakeh 18 Aug '17

18 Aug '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! A bugfix release for CollecTor is available: https://dist.torproject.org/collector/1.2.1/ The bug fixed by this release affects CollecTors sanitizing bridge descriptors and was discovered due to the recent bridge authority outage. Please direct comments and questions to the metrics-team mailing list [2]. And, of course, bug reports [3] or feature requests [4] can be filed in trac. Cheers, iwakeh [1] https://gitweb.… [View More]

1 0

Re: [tor-dev] Doesn't hidden services break RFC 3986?
by Ryan Carboni 14 Aug '17

14 Aug '17

Also: just because it's HTTP/S running over a different network stack, doesn't make it a new scheme. Just because your dinner arrives on a different plate doesn't mean the recipe has changed. :-) --- https://bugzilla.mozilla.org/show_bug.cgi?id=1228457 Tor is a strange sort of sacred cow highly revered leaks much methane But RFC 7686 literally bent over backwards to rewrite multiple standards because Tor was long existing. ".onion" shouldn't be resolved or sent over the internet. Sure, … [View More]

1 0

Getting started coding on Tor [Re: tor-dev Digest, Vol 79, Issue 4]
by Nick Mathewson 14 Aug '17

14 Aug '17

On Sat, Aug 12, 2017 at 3:02 PM, John kongtcheu <johnkongtcheu(a)gmail.com> wrote: > Hello Tor developers, > > I am interested in becoming an open source contributor for Tor, but I > don't know where to start some guidance would be appreciated. > > Thank you, > Hi! We have some introductory material over here, including a list of subprojects and what they're implemented in: https://www.torproject.org/getinvolved/volunteer If you're interested in Tor itself, … [View More]

1 0

Doesn't hidden services break RFC 3986?
by Ryan Carboni 14 Aug '17

14 Aug '17

https://tools.ietf.org/html/rfc3986#section-3 By placing the scheme within the authority as a tld while using the same authority as the HTTP specification, this probably breaks RFC 3986 and maybe others. I might be a bit late in saying this.

3 2

Help us build Tails reproducibly
by u 09 Aug '17

09 Aug '17

Dear Tails and Tor contributors, dear Reproducible Builds community, As you might know, Tails [1] has received the Mozilla Open Source Software award (MOSS) to make Tails ISO images build reproducibly. Since this project has started, less than a year ago, we've made huge progress and we've finally seen some ISO images build reproducibly on the build environments of our core developers as well as on our isobuilder machines. (See our previous reports [2]). However, there are still some … [View More]remaining issues which we'd like to know more about in order to fix them. That's why we are asking for your help: Please try and build the Tails 3.1 ISO image and report your findings back to us. You will find all instructions for doing so hereafter. Please don't hesitate to contact us if you get stuck at some point in the process, for example by connecting to our chatroom [3]. You can also send us email to tails-dev(a)boum.org (public) or tails(a)boum.org (private). # How? For your convenience all instructions needed to attempt to reproduce Tails 3.1 are included hereafter. However all commands are adapted for Debian Stretch (and Buster/Sid), so your results may vary if you run another Linux distribution. Our full build instructions [4] might help if you are having problems. ## Setup the build environment Building Tails requires the KVM virtual machine hypervisor to be available, a minimum of 1 GiB of free RAM and a maximum of 20 GB of free storage. ### Install dependencies sudo apt-get install \ git \ rake \ libvirt-daemon-system \ dnsmasq-base \ ebtables \ qemu-system-x86 \ qemu-utils \ vagrant \ vagrant-libvirt \ vmdebootstrap && \ sudo systemctl restart libvirtd ### If building as a non-root user (Skip this section if you intend to build Tails as the root user!) Make sure that the user that is supposed to initiate the build is part of the relevant groups: for group in kvm libvirt libvirt-qemu; do sudo adduser $user $group; done Then run `newgrp` (or just reboot) to apply the new group memberships to the session. ## Build Tails 3.1 git clone https://git-tails.immerda.ch/tails cd tails git checkout 3.1 git submodule update --init rake build # Send us feedback! No matter how your build attempt turned out we are interested in you sending us feedback. For that we'll first need some information of the system you used -- please run these commands in the exact same terminal session that you ran `rake build` in (e.g. run them right after `rake build`)! sudo apt install apt-show-versions || : ( for f in /etc/issue /proc/cpuinfo do echo "--- File: ${f} ---" cat "${f}" echo done for c in free locale env 'uname -a' '/usr/sbin/libvirtd --version' \ 'qemu-system-x86_64 --version' 'vagrant --version' do echo "--- Command: ${c} ---" eval "${c}" echo done if which apt-show-versions >/dev/null then echo '--- APT package versions ---' apt-show-versions qemu:amd64 linux-image-amd64:amd64 vagrant \ libvirt0:amd64 fi ) | bzip2 > system-info.txt.bz2 Please have a look at the generated file with bzless system-info.txt.bz2 to make sure it doesn't contain any sensitive information you do not want to leak in case you send this file to us or make it public! Next, please follow the instructions below that match your situation! ## If the build failed. Please open a ticket on our bug tracker [5] with "Category" set to "Build system" and `system-info.txt.bz2` attached (note that this makes this file public). ## If the build succeeded ... Please compute the SHA-512 checksum of the resulting ISO image: sha512sum tails-amd64-3.1.iso and compare it to: 843427fa13446c4b7134a10d3269b693317bbb898759e9d4e5dd8a25583372bed767e575974f5ca0229f1b44a99d4c7b64872c3dc433c0caf8965961cac9fb30 ### Use the SHA256sum from our signed upgrade files instead This is optional, but if you want to use an authenticated checksum, you can find the sha256 checksum in our upgrade files: https://tails.boum.org/upgrade/v1/Tails/3.0.1/amd64/stable/upgrades.yml .. which are signed by the Tails signing key [7]: https://tails.boum.org/upgrade/v1/Tails/3.0.1/amd64/stable/upgrades.yml.pgp The SHA256 checksum should be: 0ef1c7d880308ee9f98c255b2658b75445cc84622eae2944a342dcc50cea71c7 ### ... and the checksums match (i.e. reproduction succeeded). Congrats for successfully reproducing Tails 3.1! Please send an email to tails-dev(a)boum.org (public) or tails(a)boum.org (private) with the subject "Reproduction of Tails 3.1 successful" and attach `system-info.txt.bz2` to it. ### ... and the checksums differ (i.e. reproduction failed). Now you are in a great position to help Tails improve its reproducibility! Please install `diffoscope` [8] version 83 or higher. If you run Debian Stretch, that is: echo 'deb http://ftp.debian.org/debian stretch-backports main' \ | sudo tee /etc/apt/sources.list.d/stretch-backports.list && \ sudo apt update && \ sudo apt -o APT::Install-Suggests="true" \ -o APT::Install-Recommends="true" \ install diffoscope Then download the official Tails 3.1 ISO image [6] and compare it to yours: diffoscope \ --text diffoscope.txt \ --html diffoscope.html \ --max-report-size 262144000 \ --max-diff-block-lines 10000 \ --max-diff-input-lines 10000000 \ path/to/official/tails-amd64-3.1.iso \ path/to/your/tails-amd64-3.1.iso && \ bzip2 diffoscope.* Please send an email to tails-dev(a)boum.org (public) or tails(a)boum.org (private) with the subject "Reproduction of Tails 3.1 failed" and attach `system-info.txt.bz2` to it. We also want you attach one (the smallest!) of diffoscope.txt.bz2 and diffoscope.html.bz2 to the email, but if they are "big" (say >100 KiB) then please don't bomb our mail inboxes! Instead upload the file to some web-based file-sharing service (we'd recommend RiseUp [9]) and include the link(s) in the email. Thank you very much for your interest and help! Cheers! The Tails project [1] http://tails.boum.org [2] https://tails.boum.org/news/report_2017_06/, https://tails.boum.org/news/report_2017_05/, https://mailman.boum.org/pipermail/tails-dev/2017-March/011297.html [3] https://tails.boum.org/support/#talk [4] https://tails.boum.org/contribute/build [5] https://labs.riseup.net/code/projects/tails/issues/new [6] http://dl.amnesia.boum.org/tails/stable/tails-amd64-3.1/tails-amd64-3.1.iso [7] https://tails.boum.org/news/signing_key_transition/ [8] https://diffoscope.org/ [9] https://share.riseup.net/ [View Less]

1 0

[release] Onionoo 4.0-1.3.0
by iwakeh 08 Aug '17

08 Aug '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! Another release of Onionoo is available: https://dist.torproject.org/onionoo/4.0-1.3.0/ This release is a maintenance release: * It adapts to use the improvements of metrics-lib 2.0.0 and * Debian Stretch dependencies and * steps up to Jetty9. (See the changelog [0] and git repo for details.) Please direct comments and questions to the metrics-team mailing list [1]. Cheers, iwakeh [0] https://gitweb.torproject.org/onionoo.… [View More]

1 0

Various Tor keys, their formats, and usages [was: Bridge Bandwidth Scanner Internship]
by isis agora lovecruft 08 Aug '17

08 Aug '17

Hello! The following was my attempt last week to summarise all of our various keys in server descriptors, how they are formatted, and what they are used for, in the hopes that this is useful in alleviating possible future confusions for others. Let me know (and, more importantly! let Jake know since he's writing a Rust parser for bridge descriptors as part of the Bridge Bandwidth Scanner project) if I got anything wrong. :) ----- Forwarded message from isis agora lovecruft <isis(a)… [View More]torproject.org> ----- > From: isis agora lovecruft <isis(a)torproject.org> > Subject: Re: Bridge Bandwidth Scanner Internship > Date: Thu, 27 Jul 2017 19:52:47 +0000 > Message-ID: <20170727195247.GI24882(a)patternsinthevoid.net> > To: jnz(a)riseup.net > Reply-To: isis(a)torproject.org > Delivered-To: <isis+torproject(a)patternsinthevoid.net> > > jnz(a)riseup.net transcribed 1.4K bytes: > > Hey Isis, > > > > I have started the sanitization spec and the parser. A lot of the stuff > > I had to learn for the two overlaps, so I am working on them > > simultaneously. Conceptually, I understand the sanitization process, but > > I am a little bit confused about what some of the fields are in the > > descriptors. > > > > Q1. What exactly is the difference between "master-key", "identity-key", > > and "signing-key"? > > > > After reading through the descriptor spec, cert spec, and other docs, I > > am pretty confused on what each of the above means.I'm using the Nom > > crate for the parser, I've never used it before but so far I'm really > > liking it. Have you ever used it? > > This is specified (not all that well, imho) in dir-spec.txt… it's really > confusing. > > For versions of tor new enough to have prop#220 and prop#228 (>0.2.7.2-alpha): > > - "identity-ed25519" is a certificate containing a base64-encoded ed25519 > key. The bridge uses this to authenticate to clients. > > - "master-key-ed25519" is the ed25519 key which is in the > "identity-ed25519" certificate. > > - "onion-key" is an RSA-1024 key, encoded as a PKCS#1 RSAPublicKey > structure, encoded in base64, and wrapped in "-----BEGIN RSA PUBLIC > KEY-----" and "-----END RSA PUBLIC KEY-----". This key is used to > encrypt CREATE cells to in the older TAP handshake protocol. (TAP is > almost entirely unused on the network, last I checked. We'll likely be > removing "onion-key"s in the next year or so, after we EOL some old tor > versions.) > > - "ntor-onion-key" is a public curve25519 key, base64-encoded. This is used > in the newer NTor handshake protocol, which replaced TAP. It is still > quite common to see routers which have both an "onion-key" and an > "ntor-onion-key". > > - "signing-key" is the same type and format as "onion-key" above, but it's > used as a long-term identity key, for authenticating the bridge to > clients. (Similar to "identity-ed25519" just with RSA signatures.) > > - "router-sig-ed255519" is a only present iff "identity-ed25519" is > present, and it's an ed25519 signature of a SHA-256 digest of the > "entire" descriptor (Minus the "@purpose bridge" header since the Bridge > Authority is the one who adds that. So from "router " on the first line > up to and including the first space after "router-sig-ed25519".) This > "entire" descriptor is first prefixed with the string "Tor router > descriptor signature v1" before hashing and signing. After the signature > is produced, it is encoded in base64 and the trailing "="s are removed. > > - "router-signature" contains a signature of the PKCS#1-padded SHA-1 digest > of the "entire" server descriptor, taken from the beginning of the > "router" line (skipping "@purpose bridge" again), through (and > including) the newline after the "router-signature" line. The signature > is made with the router's "signing-key" (a.k.a. identity key). > > For older tor versions (without prop220 and/or prop228): > > - They will have "onion-key", "ntor-onion-key", "signing-key", and > "router-signature", as above. > > I've attached a tarball of today's descriptors for this bridge from the > public/default bridges in Tor Browser: > > https://atlas.torproject.org/#details/FEC8FB380DABA9D3C80790B634E4540BF5D09… > > Let me know if you want some more examples! I only grabbed one because they > are pretty annoying to pull out of the giant, unordered, undeduplicated > files the BridgeAuth produces (at least without writing a full parser). > > Also lmk if you have more questions, the entangled ball of crypto junk we > shove into descriptors is pretty confusing. […snip…] ----- End forwarded message ----- Best regards, -- ♥Ⓐ isis agora lovecruft _________________________________________________________ OpenPGP: 4096R/0A6A58A14B5946ABDE18E207A3ADB67A2CDB8B35 Current Keys: https://fyb.patternsinthevoid.net/isis.txt [View Less]

1 0

X25519 ←→ Ed25519 key conversions and specs [was: Bridge Bandwidth Scanner Internship]
by isis agora lovecruft 08 Aug '17

08 Aug '17

jnz(a)riseup.net transcribed 2.0K bytes: > Hey Isis, > > I have a few crypto questions for you. The bonus question is purely out > of interest (I searched for the answer, but didn't get a > good/understandable answer, if you are busy don't worry about it). Hey Jake! I'm CCing the conversation to tor-dev, since these are all really good questions. (Also you found another problem in our specs!) > Q1. What does the bit argument in the the ntor-onion-key-crosscert field > … [View More]do? > > The following is an example ntor-onion-key-crosscert field. The bit > argument is the "1" following ntor-onion-key-crosscert keyword. > > > ntor-onion-key-crosscert 1 > > -----BEGIN ED25519 CERT----- > > AQoABl2EAdjttTuvK9jQzdyi+Mh46ky1Jk9+Hw+FJ6eos5zUxcBDAJF+TGU6oZij > > DOToeg+/lIaEkv0RwIrl5aJta/jf2EdTgHRfCsMOLEWMfqjOA6pmkwc3jnWvWNms > > 7NVBzUcMWgA= > > -----END ED25519 CERT----- > > Q2. The following sentence is the bit argument's documentation from the > dir-spec. Could you possibly explain to me what this means? What is a > public key's "sign"? > > > Bit must be "0" or "1". It indicates the sign of the ed25519 public key corresponding to the ntor onion key. > > The following paragraph is also in the dir-spec, I know that it is > relevant, but I don't understand what it means. > > > C. Converting a curve25519 public key to an ed25519 public key > > > > Given a curve25519 x-coordinate (u), we can get the y coordinate > > of the ed25519 key using > > > > y = (u-1)/(u+1) > > > > and then we can apply the usual ed25519 point decompression > > algorithm to find the x coordinate of the ed25519 point to check > > signatures with. > > > > Note that we need the sign of the X coordinate to do this > > operation; otherwise, we'll have two possible X coordinates that > > might have correspond to the key. Therefore, we need the 'sign' > > of the X coordinate, as used by the ed25519 key expansion > > algorithm. > > > > To get the sign, the easiest way is to take the same private key, > > feed it to the ed25519 public key generation algorithm, and see > > what the sign is. > > BONUS. What is the difference between a curve25519 public key and an > ed25519 public key? Why would you want to convert back and forth between > the two? Okay, this is going to be a long answer. Let me know if parts don't make sense. So, as you noted, dir-spec.txt says [0]: > Bit must be "0" or "1". It indicates the sign of the ed25519 > public key corresponding to the ntor onion key. > > To compute the ed25519 public key corresponding to a > curve25519 key, see appendix C. There are a few different representations of the curve over the Galois field GF(2²⁵⁵ - 19) called curve25519. The (arguably) most useful is the twisted Edwards form of the curve (see our Rust documentation [1] for further details on point forms and more literature) defined for affine points P = (x,y) by the equation -x²+y² = 1 + dx²y² (1) where d = - 121665/121666 This is due to the fact that efficient, complete formulae (meaning that the same formulae, e.g. for point addition, can be used for all points, so you don't have messy if/then statements to deal with exceptional points) exist for Edwards curves. (Recently, complete formulae for Weierstrass curves and other prime-order curves have also been described. [2] However, they are still slightly less-efficient for most tasks.) There's a useful technique called a Montgomery ladder, which is an approach to constant-time (if implemented in a sane manner) point multiplication (i.e. computing nP by taking the point P and adding it to itself n times). This is particularly useful for elliptic curve Diffie-Hellman (ECDH) implementations, but it requires first converting a point from a certain representation (a.k.a. within a specific spatial coordinate system), usually "extended homogeneous twisted Edwards coordinates" (see Hişil's thesis [3], which is the explanation of all the different forms and group laws), into the affine point Q = (u,v) as it would lie on the Montgomery form of the curve: bv² = u³ + au² + u (2) where a = 486662 b = 1 The conversion is rather straighforward, as you can see from mine and Henry de Valence's code [4], but there's a problem in that the compressed form (i.e. "give me the serialised version that is just an array of 32 bytes") of points on the Montgomery curve are just the u-coordinate. You can use this u-coordinate to recover the v-coordinate, [5] and then use both u and v to recover the original Edwards x-coordinate, [6] but there's two possible x-coordinates, one positive and one negative. So, if you want to be able to use a key for both purposes, then, when using Montgomery form (e.g. an X25519 key a.k.a. a "curve25519 key") then you have to also pass around the sign of x. It's kind of icky. In this case, the "ntor-onion-key-crosscert" is an Ed25519 signature (so the public key is a point in twisted Edwards form) of the relay's public master identity key, computed using the "ntor-onion-key" (which is an X25519 key, so a point in Montgomery form). The verifier knows the "ntor-onion-key", but needs to know the sign of x in order to be able to convert it into the Edwards-form public key to check this signature. Following through the code tor is using, it appears that if "Bit" is "0", then x will be positive. If "Bit" is "1" then x will be negative. Tor's behaviour w.r.t. the sign bit (using either curve25519-donna [7] or ref10 [8]) appears to be identical to other implementations, namely Open Whispersystems' code [9] and curve25519-dalek. [10] It appears it isn't specified anywhere what the "Bit" is supposed to mean in terms of parsing the ntor-onion-cert into an Ed25519 key though. :( I made an attempt at specifying it better. [11] As for the question of why one would want to convert back and forth between the two… normally, you wouldn't want to do this. There's nothing concretely horribly wrong or insecure about it, but there are plenty of more theoretical concerns regarding any type of key reuse for different protocols, primarily because it removes any sense of domain separation, such that an attack on one protocol might be assisted by or contribute to attacking the other. With separate keys for separate purposes, you can breathe a lot easier knowing that if, e.g. your ECDH keys were compromised, that your signing key (being a totally different key) is (probably) unaffected. The only time where is does make sense to share a key between two protocols is what Nick's design for the cross-certifications, i.e.: a) Prove that you control the private part of an encryption key by converting it to a signing key, then signing the public master identity key, and b) Authenticate the signature by using the master identity key to sign the entire document (including the other signature). > Thanks for all the help, I hope that you are having a nice day and... > Happy Hacking. > > - Jake No worries! Thanks for the excellent questions and catching more bugs in our specs! [0]: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt?id=ce09d266#n536 [1]: https://docs.rs/curve25519-dalek/0.10.0/curve25519_dalek/curve/index.html [2]: https://ellipticnews.wordpress.com/2016/05/30/complete-group-laws-for-prime… [3]: https://github.com/isislovecruft/library--/blob/master/cryptography%20%26%2… [4]: https://github.com/isislovecruft/curve25519-dalek/blob/f2883028d/src/curve.… [5]: https://github.com/isislovecruft/curve25519-dalek/blob/f2883028d/src/curve.… [6]: https://github.com/isislovecruft/curve25519-dalek/blob/f2883028d/src/curve.… [7]: https://gitweb.torproject.org/tor.git/tree/src/ext/ed25519/donna/ed25519_to… [8]: https://gitweb.torproject.org/tor.git/tree/src/ext/ed25519/ref10/keyconv.c?… [9]: https://github.com/WhisperSystems/curve25519-java/blob/master/android/jni/e… [10]: https://github.com/isislovecruft/curve25519-dalek/blob/f2883028d/src/curve.… [11]: https://gitweb.torproject.org/torspec.git/commit/?id=2395f34affbe97c19d7bb9… Best, -- ♥Ⓐ isis agora lovecruft _________________________________________________________ OpenPGP: 4096R/0A6A58A14B5946ABDE18E207A3ADB67A2CDB8B35 Current Keys: https://fyb.patternsinthevoid.net/isis.txt [View Less]

1 0

Need help on my new work related to TOR Project
by Nima Azizzadeh 25 Jul '17

25 Jul '17

Hello My name is Nima Azizzadeh and I'm a second year of mastering Cyber Security major at University of Tehran, Iran. I am extremely interested in pursuing independent research. An extensive research experience will greatly help me consolidate my future career choice. Recently I'm interested in tor project vulnerabilities and I looked at some papers about it. This paper "The Effect of DNS on Tor’s Anonymity, NDSS, February 2017" attracted a great deal of my attention. A lot of research has … [View More]

2 1

prop224: Deprecating SHA1 circuit digests
by teor 24 Jul '17

24 Jul '17

Hi all, At the moment, Tor uses SHA1 for the running digests of circuit cell payloads. Some of the prop224 code seems to use SHA256 for the digests for client to service rendezvous circuits. But that's not in the spec yet (see #22995 at [0]). How and when do we plan to move away from using SHA1 in Tor circuits? For non-onion service circuits, this would mean: * implementing support for SHA256 [1] circuit digests * adding a Relay protocol version 3 * teaching clients to use SHA256 digests … [View More]with relays with Relay protocol >= 3 For onion service circuits, it's more complicated, because the following circuit types can't use relay versions from the consensus: * client to intro * service to rend * client to service (Using relay versions from the consensus leaks which consensus clients and services have, which reduces the anonymity set.) Here are the upgrade mechanisms in prop224 at the moment, for both circuit protocol versions and any necessary handshake material: client to intro: * the protocol version could be in a proto line to each intro point, but this isn't implemented yet * the handshake data can be in the link-specifiers (I think?) service to rend * the protocol version could be in the EXT_FIELD in the INTRODUCE cell, but this isn't implemented yet * the handshake data can be in the link-specifiers (I think?) client to service: * the protocol version is in the create2-formats in the descriptor * the handshake data is in HANDSHAKE_INFO in the RENDEZVOUS cells * SHA256 digests are implemented, but not documented in prop224 [0] I suggest we make the following changes to prop224 to make this happen: Protocol version information: * add the relevant relay protocol versions to the intro point section of the descriptor * put the relevant relay protocol versions in an EXT_FIELD in the INTRODUCE cell * check create2-formats contains all the version info we will need to change the client to service circuit protocol version Downgrade resistance: * teach clients and services to use the highest common protocol between client/service and relay, excluding protocols that are below the minimum required protocol versions * work out how we will tell clients to no longer accept an old create2-formats line from a service [0]: https://trac.torproject.org/projects/tor/ticket/22995 [1]: By SHA256, I mean "a good secure hash at the time" T -- Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------ [View Less]

3 5

Thoughts on the Counter-RAPTOR paper
by Roger Dingledine 20 Jul '17

20 Jul '17

Hi Prateek, Yixin, (and please involve your other authors as you like), (I'm including tor-dev here too so other Tor people can follow along, and maybe even get involved in the research or the discussion.) I looked through "Counter-RAPTOR: Safeguarding Tor Against Active Routing Attacks": https://arxiv.org/abs/1704.00843 For the tl;dr for others here, the paper: a) comes up with metrics for how to measure resilience of Tor relays to BGP hijacking attacks, and then does the measurements; b) … [View More]describes a way that clients can choose their guards to be less vulnerable to BGP hijacks, while also considering performance and anonymity loss when guard choice is influenced by client location; and c) builds a monitoring system that takes live BGP feeds and looks for routing table anomalies that could be hijack attempts. Here are some hopefully useful thoughts: ----------------------------------------------------------------------- 0) Since I opted to write these thoughts in public, I should put a little note here in case any journalists run across it and wonder. Yay research! We love research on Tor -- in fact, research like this is the reason Tor is so strong. For many more details about our perspective on Tor research papers, see https://blog.torproject.org/blog/tor-heart-pets-and-privacy-research-commun… ----------------------------------------------------------------------- 1a) The "live BGP feed anomaly detection" part sounds really interesting, since in theory we could start using it really soon now. Have you continued to run it since you wrote the paper? Have you done any more recent analysis on its false positive rate since then? I guess one of the real challenges here is that since most of the alerts are false positives, we really need a routing expert to be able to look at each alert and assess whether we should be worried about it. How hard is it to locate such an expert? Is there even such a thing as an expert in all routing tables, or do we need expertise in "what that part of the network is supposed to look like", which doesn't easily scale to the whole Internet? Or maybe said another way, how much headway can we make on automating the analysis, to make the frequency of alerts manageable? I ask because it's really easy to write a tool that sends a bunch of warnings, and if some of them are false positives, or heck even if they're not but we don't know how to assess how bad they really are, then all we've done is make yet another automated emailer. (We've made a set of these already, to e.g. notice when relays change their identity key a lot: https://gitweb.torproject.org/doctor.git/tree/ but often nobody can figure out whether such an anomaly is really an attack or what, so it's a constant struggle to keep the volume low enough that people don't just ignore the mails.) The big picture question is: what steps remain from what you have now to something that we can actually use? 1b) How does your live-BGP-feed-anomaly-detector compare (either in design, or in closeness to actually being usable ;) to the one Micah Sherr was working on from their PETS 2016 paper? https://security.cs.georgetown.edu/~msherr/reviewed_abstracts.html#tor-data… 1c) Your paper suggests that an alert from a potential hijack attempt could make clients abandon the guard for a while, to keep clients safe from hijack attempts. What about second-order effects of such a design, where the attacker's *goal* is to get clients to abandon a guard, so they add some sketchy routes somewhere to trigger an alert? Specifically, how much easier is it to add sketchy routes that make it look like somebody is attempting an attack, compared to actually succeeding at hijacking traffic? I guess a related question (sorry for my BGP naivete) is: if we're worried about false positives in the alerts, how much authentication and/or attribution is there for sketchy routing table entries in general? Can some jerk drive up our false positive rate, by adding scary entries here and there, in a way that's sustainable? Or heck, can some jerk DDoS parts of the Internet in a way that induces routing table changes that we think look sketchy? These are not reasons to not take the first steps in the arms race, but it's good to know what the later steps might be. ----------------------------------------------------------------------- 2a) Re changing guard selection, you should check out proposal 271, which resulted in the new guard-spec.txt as of Tor 0.3.0.x: https://gitweb.torproject.org/torspec.git/tree/guard-spec.txt I don't fully understand it yet (so many things!), but I bet any future guard selection change proposal should be relative to this design. 2b) Your guard selection algorithm makes the assumption that relays with the Guard flag are the only ones worth choosing from, and then describes a way to choose from among them with different weightings. But you could take a step back, and decide that resilience to BGP hijack should be one of the factors for whether a relay gets the Guard flag in the first place. It sounded from your analysis like some ASes, like OVH, are simply bad news for (nearly) all Tor clients. Your proposed guard selection strategy reduced, but did not eliminate, the chances that clients would get screwed by picking one of these OVH relays. And the tradeoff was that by only reducing the chances, you left the performance changes not as extreme as you might have otherwise. How much of the scariness of a relay is a function of the location of the particular client who is considering using it, and how much is a function of the average (expected) locations of clients? That is, can we identify relays that are likely to be bad news for many different clients, and downplay their weights (or withhold the Guard flag) for everybody? The advantage of making the same decision for all clients is that you can get rid of the "what does guard choice tell you about the client" anonymity question, which is a big win if the rest of the effects aren't too bad. Which leads me to the next topic: ----------------------------------------------------------------------- 3) I think you're right that when analyzing a new path selection strategy, there are three big things to investigate: a) Does the new behavior adequately accomplish the goal that made you want a new path selection strategy (in this case resilience to BGP attacks)? b) What does the new behavior do to anonymity, both in terms of the global effect (e.g. by flattening the selection weights or by concentrating traffic in fewer relays or on fewer networks) and on the individual epistemic side (e.g. by leaking information about the user because of behavior that is a function of sensitive user details)? c) What are the expected changes to performance, and are there particular scenarios (like high load or low load) that have higher or lower impact? I confess that I don't really buy your analysis for 'b' or 'c' in this paper. Average change in entropy doesn't tell me whether particular user populations are especially impacted, and a tiny Shadow simulation with one particular network load and client behavior doesn't tell me whether things will or won't get much worse under other network loads or other client behavior. I can't really fault this paper though, because the structure of an academic research paper means you can only do so much in one paper, and you did a bunch of other interesting things instead. We, the Tor research community, really need better tools for reasoning about the interaction between anonymity and performance. In fact, there sure have been a lot of Tor path selection papers over the past decade which each invent their own ad hoc analysis approach for showing that their proposed change doesn't impact anonymity or performance "too much". Is it time for a Systemization of Knowledge paper on this area -- with the goal of coming up with best practices that future papers can use to provide more convincing analysis? --Roger [View Less]

2 1

[release] CollecTor 1.2.0
by iwakeh 13 Jul '17

13 Jul '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! a new CollecTor version is available: https://dist.torproject.org/collector/1.2.0/ The main new features are the switch from Torperf to Onionperf (see [0] for background) and the upgrade to metrics-lib-2.0.0. For more see the change log [1]. In addition, CollecTor's web-pages are now part of Tor Metrics web-site: https://metrics.torproject.org/collector.html. Please update any bookmarks and report any wrong links to … [View More]

1 0

Fuzzing tor_inet_pton(): A fuzzing example
by Nick Mathewson 05 Jul '17

05 Jul '17

So, there was a bug report of an assertion failure in tor_inet_pton(), and I wanted to track it down. Here's my reconstruction of my process, in case it helps anybody else write fuzzing code. First, I wrote a new fuzzing target in src/test/fuzz/fuzz_pton.c: ====== #include "orconfig.h" #include "or.h" #include "fuzzing.h" int fuzz_init(void) { return 0; } int fuzz_cleanup(void) { return 0; } int fuzz_main(const uint8_t *stdin_buf, size_t data_size) { struct in_addr *in = tor_malloc(… [View More]sizeof(*in)); struct in6_addr *in6 = tor_malloc(sizeof(*in6)); char *cp = tor_memdup_nulterm(stdin_buf, data_size); int in_ok = tor_inet_pton(AF_INET, cp, in); int in6_ok = tor_inet_pton(AF_INET6, cp, in6); tor_free(cp); tor_free(in); tor_free(in6); log_info(LD_GENERAL, "%d %d", in_ok, in6_ok); return 0; } ====== Notes: * The init and cleanup functions don't have anything to do above, but the test harness in fuzzing_common.c expects to find them. * This fuzzer tries to parse each address twice: first as an IPv4 address, then as an IPv6 address. * The fuzzer puts its outputs into heap-allocated RAM to improve the odds that any bugs will get caught by asan, though this shouldn't really be necessary. * Because tor_inet_pton() takes a nul-terminated string, this fuzzer nul-term inates its input. You shouldn't do that unless you're fuzing a function that requires nul-terminated inputs. Next, I edited the script in scripts/codegen/fuzzing_include_am.py to add "pton" to the "FUZZERS" list at the top of the file, and I re-ran the script and used its output to replace src/test/fuzz/include.am, so that our build system can know about the new fuzzer. I knew I would need a corpus of pton examples, so I started one in fuzzing-corpora (fuzzing-corpora is a separate repository). I made a new "pton" directory there, and filled it with 4 or 5 examples of things that looked like they might be addresses. I chose: 127.0.0.1 1.2.3.4 ::1 80f0:9999:ffff::3 Each example went into its own file, without a terminating newline. I tried the fuzzing corpora examples out with the fuzzing program, to make sure that I got the expected results $ ./src/test/fuzz/fuzz-pton --info < ../tor-fuzz-corpora/pton/b Jul 05 15:00:56.371 [info] fuzz_main(): 1 0 ===== AFL Then I re-built Tor for use with AFL: ./configure CC=afl-gcc --enable-fragile-hardening make clean AFL_HARDEN=1 make fuzzers To fuzz it, I started by telling afl-fuzz to run with no memory limit, writing results to pton-output, taking inputs from the corpus: afl-fuzz -i ../tor-fuzz-corpora/pton -o ./pton-output/ -m none \ -- ./src/test/fuzz/fuzz-pton AFL wound up complaining about some kernel settings, so I had to do this as root. YMMV. AFL told me to do this: echo core >/proc/sys/kernel/core_pattern cd /sys/devices/system/cpu echo performance | tee cpu*/cpufreq/scaling_governor Then AFL started running! I watched the "total paths" value climb. That's how many different distinct paths through the code that the fuzzer was able to find. I didn't see any "uniq crashes" or "uniq hangs" values, so I knew the fuzzer hadn't found anything. ===== Libfuzzer After that had run for a day, I stopped it and tried again with libfuzzer: ./configure --enable-libfuzzer CC=/home/nickm/build/llvm-build/bin/clang make clean make fuzzers Obviously, you'll need to point CC to your own clang here. I wasn't able to use the system clang because libfuzzer needs features I didn't have. Then I made a new file for the libfuzzer results, and ran libfuzzer using the original corpus and the output from AFL: mkdir pton-output-libfuzzer ./src/test/fuzz/lf-fuzz-pton pton-output-libfuzzer/ \ ./pton-output/queue/ ../tor-fuzz-corpora/pton/ And libfuzzer began giving results. Every "NEW" line was a new path; every "pulse" line meant that time had passed but no new paths were found. [View Less]

1 0

Open topics of prop247: Defending Against Guard Discovery Attacks using Vanguards
by George Kadianakis 04 Jul '17

04 Jul '17

Hello, here is some background information and summarizing of proposal 247 "Defending Against Guard Discovery Attacks using Vanguards" for people who plan to work on this in the short-term future. I include a list of open design topics (probably not exhaustive) and a list of engineering topics. Some engineering stuff can be done parallel to the design stuff. ==================== Background info ==================== * Proposal: https://gitweb.torproject.org/torspec.git/tree/proposals/247-hs-… [View More]guard-disco… * Discussion: ** Initial prop247 thread: https://lists.torproject.org/pipermail/tor-dev/2015-July/009066.html ** Recent prop247 thread: https://lists.torproject.org/pipermail/tor-dev/2015-September/009497.html ** Reading group notes of prop247: https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html ==================== Design topics ==================== * Optimize proposal parameters ** Optimize guardset sizes ** Optimize guardset lifetimes and prob distributions (minXX/maxXX/uniform?) ** To take informed decision, we might need a prop247 simulator, or an actual PoC with txtorcon * HOW to choose second-layer and third-layer guards? ** Should they be Guards? middles? Vanguards? Serious security / load balancing implications! ** Can guardsets share guards between them or are they disjoint? Particularly third-layer sets ** background: https://lists.torproject.org/pipermail/tor-dev/2016-January/010265.html * HOW to avoid side-channel guard discovery threats? ** Can IP/RP be the same as first-layer guard? ** Can first-layer guard be the same as third-layer guard? ** background: https://gitweb.torproject.org/user/mikeperry/torspec.git/commit/?h=guard_di… * Change path selection for IP circs to avoid third-layer guard linkability threats. ** Switch from [HS->G1->M->IP] to [HS->G1->G2->G3->IP] or even to [HS->G1->G2->G3->M->IP]. ** Consider the latter option for HSDir circs as well? ** background: https://gitweb.torproject.org/user/mikeperry/torspec.git/commit/?h=guard_di… * Should prop247 be optional or default? ** Consider making it optional for a testing period? * How does prop247 affects network performance and load balancing? ** especially if it's enabled by default? ** Update load balancing proposal? * Correct behavior for multiple HSes on single host? * Does prop247 influence guard fingerprinting (#10969) and should we care enough? ==================== Engineering topics ==================== * What's a good entrynodes API to implement prop247? * What's a good state file API to implement prop247? * Write prop247 simulator to verify security goals and optimize proposal parameters (see above). * Write PoC with txtorcon! * Write PoC with little-t-tor! ============================================================ [View Less]

5 7

Re: [tor-dev] Default parsing order of config files in Debain tor package
by iry 03 Jul '17

03 Jul '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi Daniel! Thank you so much for your immediate, detailed and informative answer. I really like the example you offered as it clearly illustrates the parsing order. Your answer is very helpful for me to decide where I should place the %include line and Tor config files. I really appreciate your help. Best, iry Daniel: > The precedence for tor options is the following (1 overrides 2, > etc...): > > 1. Command line options. 2. … [View More]Configuration file options (your > /etc/torrc). 3. Defaults file options (your > /usr/share/tor/tor-service-defaults-torrc). > > In the same file, options that appear later override earlier > options. > > Currently, there is no torrc.d directory created when you install > the tor package. However, you can use a %include in the > configuration file or in the defaults file. When you insert a > %include in a file, it works as if all the options for the included > file or folder were written on the line of the %include. If you're > including a folder, the files will be processed in lexicographic > order and files starting with a dot will be ignored. > > Here is an example: > > tor-service-defaults-torrc: SomeOption 0 %include /etc/tor/torrc.d/ > # SomeOption is now 2 SomeOption 3 # SomeOption is now 3 > > /etc/tor/torrc.d/01_one: SomeOption 1 > > /etc/tor/torrc.d/02_two: SomeOption 2 > > > With this configuration, the value for some option is 3. But we can > have a torrc with %include too: > > /etc/torrc: SomeOption 4 # SomeOption is now 4 %include > /etc/tor/foo.torrc # SomeOption is now 5 SomeOption 6 # SomeOption > is now 6 > > /etc/tor/foo.torrc: SomeOption 5 > > With both these files, the value for SomeOption is 6. > > There are also different types of options and some can take > multiple values. For more information see the section "Mid-level > semantics" on this file: > https://gitweb.torproject.org/tor.git/tree/doc/torrc_format.txt > > Best regards, Daniel -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJZWZ1gAAoJEKFLTbxtzdU84gAQAKVu8UcbYHXIaZ+sUkiENUeW lSBnvOHeRcIBAOe1LfClgkAFiVWhHwfG2ll7cGDTiMxn7IuDlv+sUFtwpqRVFgYH kOxJiVS1kORQJ8t8DuXh3rQZTUTKOkhCCepb8maHeCSED0yUGSXs+wzpFMAyKJk+ 2B73YR/hQd2XgpZx9LcpfkdznTXF/jpOPZthZWFAkwm1yNAlvwAmfXgADV4lQy5K RRCwYmAgdSS9OLrDMj0G1lVnvsr7qUgfePzLpp1FBkMf/E8nhL6NBCK+jJc+BA+R 3YKn5pKtJ5/KaAwutQ5nlI3+mdVoDnJI0wWGHxL6ZJPwfCMANqkrbRa8/Wq0i7ZB u1sG9aWJzTXPaI/OZQ8VE5bvh9mLSfwo0RpzYv8CVsFgu2VnMPk5EW13d33+Dzjz 5GVcA0s0V1AA7uZ/NGUrcUZ/jO+z5Argre7RxkCnAgjItKb2vJK5hGnj7gVQO0il bcx+e47MyLVNoi7tOaoJmOusEEMQB1wMG6S1Pc+o/apqW6p2VEUKrTh+DSp72+LX zDPOxAleJjbntXZkGrDO0nay21y1LWYL2bOHpWomjd03f9v6br/n9oofNX6pN7Xn B9FIOa0PxzCT2BwcROnEDEB9mCHDbIRSEvuwpbbdz3StRW3pLzU1Q2e6WO+eF7dH OpDeWlO5/Z6IJKV4eesz =9riT -----END PGP SIGNATURE----- [View Less]

1 0