- tor-dev - lists.torproject.org

txtorcon 'all_routers' issue
by meejah 23 Feb '17

23 Feb '17

As per https://github.com/meejah/txtorcon/issues/203 all released versions of txtorcon that have TorState.all_routers will, upon a NEWCONSENSUS event, make this set empty. There is already a fix on master for the real issue and it will be in the next release. For users of released versions there is a workaround, pointed out by evilaliv3: simply use TorState.routers.values() instead. Cheers, meejah

1 0

List of Tor Exit IP address: Is this available from Tor Control Port
by Fabio Pietrosanti (naif) - lists 22 Feb '17

22 Feb '17

Hi all, is the list of Tor Exit IP addresses available from Tor Control Port or only from https://check.torproject.org/exit-addresses via TorDNSEL ? -naif

2 1

announcing: txmix, a twisted python mixnet library
by dawuud 19 Feb '17

19 Feb '17

Hi. I'd like to announce txmix, a library for writing mixnets in python. https://github.com/applied-mixnetworks/txmix txmix is GPL3 licensed and currently depends on: sphinxmixcrypto, Twisted, txtorcon, attrs, eliot and Tor Project's chutney (for integration tests). txmix allows you to write mixnets with reduced code complexity and transport agnosticism... however I've choosen to focus my development efforts on the Tor onion transport for several reasons, the onion transport has some properties which the UDP transport does not: - reliability - forward secrecy - NAT penetration Additionally the Tor anonymity properties are a welcome bonus since they aren't really overlapping with those of the mixnet, they are vulnerable to different types of attacks and accomodate for totally different threat models. Currently txmix has some integration tests that use chutney to spin up a local 25 relay tor network; builds a mixnet using onion services as the transport and then tests functionality by sending and receiving messages on the mixnet. future development ------------------ - txmix could certainly be used to write various kinds of mixnets, however, we are missing a PKI. Without some kind of public key infrastructure how will mix clients learn about mix public keys? - sphixmixcrypto library needs improvements that would allow us to specify more message types such as dummy/cover traffic message and heartbeat messages. These of course are essential for the defense against certain known active attacks against some mix types. questions, code review and pull requests welcome. David Stainton

1 0

Netflow padding
by grarpamp 19 Feb '17

19 Feb '17

Are these the current / recommended paper refs for eyeballing things on this to date? torspec/proposals/251-netflow-padding.txt torspec/proposals/254-padding-negotiation.txt

2 1

Re: [tor-dev] GAEuploader (Adam Fisk)
by Katherine Li 19 Feb '17

19 Feb '17

Hi Adam, my experience is also that GAE is blocked in China. It is possible to find unblocked IPs. XXnet, for example, has a built-in scanne <https://github.com/XX-net/XX-Net/wiki/%E2%80%9C%E6%89%AB%E6%8F%8FIP%E2%80%9…>r for Google's unblocked IPs. GAEuploader currently does not have built-in scanning for unblocked IP addresses. We definitely want to incorporate this feature in the future, and any contribution on this would be much appreciated. On Wed, Feb 1, 2017 at 9:40 AM, Adam Fisk <afisk(a)getlantern.org> wrote: > In our experience GAE is basically blocked in China at least. Is that > your experience as well? I believe it's possible to find IPs that are > unblocked, but you'd have to embed those IPs somehow into the client > or integrate the scanning. Katherine Li https://github.com/katherinelitor/GAEuploader

1 0

Tor Browser and Mozilla addon verification
by bancfc＠openmailbox.org 18 Feb '17

18 Feb '17

Hi, does Tor Browser check addon code for tampering for addons from the Mozilla server?

2 2

[release] metrics-lib/DescripTor 1.6.0
by iwakeh 17 Feb '17

17 Feb '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! Another release of metrics-lib/DescripTor is available: https://dist.torproject.org/descriptor/1.6.0/ - From the change-log: * Major changes - Deprecate DescriptorDownloader in favor of the much more widely used DescriptorCollector. * Medium changes - Parse "proto" lines in server descriptors, "pr" lines in status entries, and "(recommended|required)-(client|relay)-protocols" lines in consensuses and votes. - Parse "shared-rand-.*" lines in consensuses and votes. ... and quite a bit more see [0]. Please direct comments and questions to the metrics-team mailing list [1]. Cheers, iwakeh [0] https://gitweb.torproject.org/metrics-lib.git/plain/CHANGELOG.md?id=10042d3… [1] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYpyhdAAoJEPeEx9Sa/gvikq0QAKKhd2v04WGwokOGD+j7HNJC ylEamW7BSxMHCecgkMULy7RvTkmUmw/OJ66Ob2chdqVd6di8/fej3Sra+mpxcx9f mibg7UTLtMPgmoKMOp2qL7++HYuhWOowU+6e5rF1Ps0vVBZHWAD2iRCMX+jU2J+p XzYcVJ4YyV7ovfITDi1hDMvOXqDRrvhbH5bFDgXM0b5mWyKfjb/p35Y5wivEFPxp DYfVQ231IhDaN8ews3jXpuU8MILUBxzybOK+xtnAfYI4SXxMrJ16LmEpuP5lL2j8 9FfdG2U9MTdxnfEEYVPV/NrBRxZ9r+6d6pTtySmJ8D2s1hnwF3higTYm1DPCfU5U /m1AtLJsFvGVd/qJ4TNmWww8r1WgZlPzggVHgsHL28lpGK3L8ryQrbA0Nr6AVuFO sHDG4Fq5iegoWPsgGIcD8ThJlgvi5a6nS0l79OL+DUv4is2zQeI+zOgiaVeu/CmE JlRkMaA3U2AxayqBSJCwHKdFOu75cHCErTYEQt8Oy+h3rhrw2ucfUe/fKyMOVWps g156i/PsroggHMZWackXBv9tPJyK57ipTEr84ljg2jsZYTUsrYzN13+v/bkrOWXe jTOiicdFJ8Kfyj9Ng5afbLmt24W7BYQSqp4BM4MdEmZ2sY4Zq3iTPMqzBGai3OV4 8EJ5gsOCP7WEmSJcMNyg =EBek -----END PGP SIGNATURE-----

1 0

Prop224 oppurtunity: keygen, crypt, sign, encoding tools
by grarpamp 17 Feb '17

17 Feb '17

Tor could ship with a tool to offline generate all the various keys, encrypt and sign with them, for debug, test, and use with other apps that tie to tor. And a tool to translate strings between different encodings in use. Or at least provide howto and links in the docs to third party tools that users could use for key ops and translation. Since those howto topics appear on the lists now and then. We here might code up openssl, python functions, etc on the fly. However beginning users are typically looking for simple purpose dedicated tools, or example docs using prebuilt tools off the net. This tends to apply to budding application development in onionland, developing things that look at and use tor, etc.

5 4

GSoC 2017 - my name is Marcin
by Marcin Czarnecki 13 Feb '17

13 Feb '17

Hello all, tl;dr can go on about Tor and anonymity and privacy related things for hours, give me a chance I am studying Computer Science at the University of Brighton (in the UK). I am interested in participating in Google Summer of Code 2017 with the Tor project. I am a web security and anonymity enthusiast, tried to work with the Tor Project for GSoC 2016 but it did not work out. I have not contributed to the Tor's code base but have been an active, semi advanced user for past 3 years. Have been experimented with setting up exit-nodes (ended up with my box being considered the #1 most evil machine in the world :)), creating Tor-clearnet gateways, hosting own HS, trying to exploit other HSes, playing with encryption etc. I think that gave a me a lot of insight of how Tor works (still amazed and finding new things when reading docs). I am planning to base my proposal on the idea on a easier way of hosting HS so it would become more accessible to non tech savvy Tor users plus enforcing good security practices when hosting a HS (got some experience and cool ideas [citation needed]). My main skills are in Linux and back-end web development but I am also interested in scripting and automation as well as encryption. Can talk about Bitcoin for hours (security, anonymity, etc.), thinking about using that as my other project proposal as this should be the currency of Tor and more integrated into Tor (Tor Browser extension?). When it comes to programming I feel confident with PHP, Python and HTML, CSS, JS stuff, and know a bit of other languages but I am a fast learner! When it comes to tor related projects, two years ago I did this small crawler with web interface for searching (#2 most evil machine) in Python: https://github.com/marncz/torcrawler Cool feature on which I stopped working was HTTP headers fingerprinting which could led to grouping Hidden Services by hosts providers. Feel free to contact me: email: marnczarnecki[at]gmail.com IRC at OFTC: marncz Many thanks, Marcin

2 1

Issue regarding IRC channel
by Jaskaran Singh 12 Feb '17

12 Feb '17

Hi, As of writing this mail, I can't access the Tor-Dev channel on OFTC neither using my registered nick on desktop client nor through the web interface. Looks like the channel has turned invite only. Would be great if someone could look into this. Regards, Jaskaran Veer Singh

3 2

Re: [tor-dev] [RFC] Proposal for the encoding of prop224 onion addresses
by Vi Grey 08 Feb '17

08 Feb '17

> Here is the suggested scheme: > > onion_address = base32(version + pubkey + checksum) > checksum = SHA3(".onion checksum" + version + pubkey) Wouldn't SHA3 be computational overkill if we're just worrying about the checksum making sure the .onion address wasn't mistyped? My suggestion would be to use something like this for the checksum: checksum = HMAC-CRC32(".onion checksum", version + pubkey)[2:] (with the HMAC key as its first argument) HMAC-CRC16 could be used as well instead of a truncated HMAC-CRC32, but CRC16 is a lot less standard than CRC32. Also, should we consider including a Version option eventually in the ADD_ONION control port command when using a KeyBlob? It wouldn't matter much for this new version and probably wouldn't matter much for a while, but it might be good to keep this in mind for the future.

4 3

archive.is and archive.fo are using CloudFlare. Is the TorBrowser add-on cfc useless now?
by Christian Pietsch 07 Feb '17

07 Feb '17

Hi Yawning, hi all, sorry if this is old news, but I just noticed that archive.is not only redirects me to archive.fo (which I noticed before), but archive.fo now greets me with a CloudFlare captcha. Considering that cfc was created in order to evade the CloudFlare captcha, this is quite a disappointment. Is cfc useless now? Can it be fixed? David F suggested using Memento which I think is an interesting idea, having participated in one of their hackathons. One caveat: Memento would point to to various archives not all of which might be suitable. What is the reason why archive.org is not used? I hear they are almost done setting up an onion service for the Internet Archive. Cheers, C: On Mon, Apr 18, 2016 at 10:16:31PM +0000, Yawning Angel wrote: > Source: https://git.schwanenlied.me/yawning/cfc -- Christian Pietsch, volunteering for Digitalcourage e.V., Bielefeld, Germany Nominations sought for the German Big Brother Awards: https://bigbrotherawards.de Verfassungsbeschwerde gegen Vorratsdatenspeicherung: https://digitalcourage.de/weg-mit-vds Appell gegen den geplanten Datenschutz-Abbau: https://digitalcourage.de/rettet-den-datenschutz Umfrage zum Datenschutztag 2017: https://datenschutztag.org/survey/

2 2

generate relay fingerprint without tor given the datadir/keys folder?
by nusenu 04 Feb '17

04 Feb '17

Hi, given the files within the datadir/keys folder (without the datadir/fingerprint file), is there an easy way to generate the relay fingerprint? (using openssl?) According to the spec [1] the fingerprint is the SHA1 hash of the public key. (I assume RSA pubkey) According to the tor man page [2] the RSA public key should be in keys/secret_id_key. openssl rsa -in secret_id_key -pubout| ..? |sha1sum thanks, nusenu [1] > "fingerprint" fingerprint NL > > [At most once] > > A fingerprint (a HASH_LEN-byte of asn1 encoded public key, encoded in > hex, with a single space after every 4 characters) for this router's > identity key. A descriptor is considered invalid (and MUST be > rejected) if the fingerprint line does not match the public key. [2] > DataDirectory/keys/secret_id_key > A relay’s RSA1024 permanent identity key, including private and > public components. Used to sign router descriptors, and to sign > other keys.

2 4

onionoo: understanding 'exit_policy_v6_summary'
by nusenu 01 Feb '17

01 Feb '17

Hi Karsten, given this torrc configuration and exit policy: https://lists.torproject.org/pipermail/tor-relays/2017-January/011806.html would you expect onionoo's 'exit_policy_v6_summary' to be not set? [1] https://onionoo.torproject.org/protocol.html#details writes: > Missing if the relay rejects all connections to IPv6 addresses. thanks, nusenu [1] https://atlas.torproject.org/#details/5E762A58B1F7FF92E791A1EA4F18695CAC667… {"nickname":"sorrentini","fingerprint":"5E762A58B1F7FF92E791A1EA4F18695CAC6677CE","or_addresses":["128.199.76.145:443","[2400:6180:0:d0::18a7:d001]:443"],"last_seen":"2017-01-27 13:00:00","last_changed_address_or_port":"2017-01-11 09:00:00","first_seen":"2016-11-30 03:00:00","running":true,"flags":["Exit","Fast","Running","Stable","Valid"],"country":"sg","country_name":"Singapore","region_name":"Central Singapore Community Development Council","city_name":"Singapore","latitude":1.2855,"longitude":103.8565,"as_number":"AS133165","as_name":"Digital Ocean, Inc.","consensus_weight":420,"host_name":"128.199.76.145","last_restarted":"2017-01-21 03:29:31","bandwidth_rate":1073741824,"bandwidth_burst":1073741824,"observed_bandwidth":1225379,"advertised_bandwidth":1225379,"exit_policy":["reject 0.0.0.0/8:*","reject 169.254.0.0/16:*","reject 127.0.0.0/8:*","reject 192.168.0.0/16:*","reject 10.0.0.0/8:*","reject 172.16.0.0/12:*","reject 128.199.76.145:*","accept *:53","accept *:80","accept *:110","accept *:143","accept *:220","accept *:443","accept *:873","accept *:989-990","accept *:991","accept *:992","accept *:993","accept *:995","accept *:1194","accept *:1293","accept *:3690","accept *:4321","accept *:5222-5223","accept *:5228","accept *:9418","accept *:11371","accept *:64738","reject *:*"],"exit_policy_summary":{"accept":["53","80","110","143","220","443","873","989-993","995","1194","1293","3690","4321","5222-5223","5228","9418","11371","64738"]},"contact":"0x44BB1BA79F6C6333 <tor-admin AT zumbi dot com dot ar>","platform":"Tor 0.2.9.8 on Linux","effective_family":["$82C92FBAF2196EC346670D12BB9650FE9FF55741","$EFD2EEB91E5C5D8CB999B1EC68D89E51F8776AC7"],"consensus_weight_fraction":8.568019E-6,"guard_probability":0.0,"middle_probability":0.0,"exit_probability":3.799685E-5,"recommended_version":true,"measured":true}

3 5

[RFC] Directory structure of prop224 onion services
by George Kadianakis 01 Feb '17

01 Feb '17

Hey list, with service-side prop224 implementation moving forward, we need to pin down the directory structure of prop224 onion services. This will be very similar to the current directory structure, but with some mods to facilitate assymetric client authorization keys and offline keys. As people have pointed out, the HS directory structure matters less after the introduction of ephemeral ADD_ONION onion services, but still it's an important part of onion service sysadmin UX. So the HiddenServiceDir directory will contain the following items: - "./hostname" [FILE] This is a file containing the onion address of the onion service. As you can see it's the same filename as in v2. Should we suffix it with v3 to make it clear that it's v3 onion? Would we ever have v2 and v3 onions living in the same directory? - "./private_key_ed25519" [FILE] This is the file containing the private master ed25519 key of the onion service. If offline keys are _enabled_, then this file doesn't exist and instead a directory is made containing blinded keys for every day [TODO: The directory format here will be specified in the future]. - "./client_authorized_pubkeys" [FILE] If client authorization is _enabled_, this is a newline-separated file of "<client name> <pubkey>" entries for authorized clients. You can think of it as the ~/.ssh/authorized_keys of onion services. - "./client_authorized_privkeys/" [DIRECTORY] "./client_authorized_privkeys/alice" [FILE] "./client_authorized_privkeys/bob" [FILE] "./client_authorized_privkeys/charlie" [FILE] If client authorization is _enabled_ _AND_ if the hidden service is responsible for generating and distributing private keys for its clients, then this directory contains files with client's private keys. The idea is that these files can be shredded and deleted after the private key has been passed to the client. For more context here, please read the client authorization thread in [tor-dev] and see 'Appendix F' of prop224 for more details on how this works. So this is it. The above should handle most uses of onion services + client authorization. The directory format of offline keys will be specified as we move forward with implementation. Hope things here are not too controversial. Looking forward to your feedback. In a few days, I will add a small Appendix section to prop224 with the above, and also fix the parts of 'Appendix F' that got outdated since then. Cheers!

6 12

Re: [tor-dev] GAEuploader
by Katherine Li 01 Feb '17

01 Feb '17

Hi Tim, Thank you for your insightful questions! My responses are below: *> Is this the cheapest zone/region for computation or bandwidth? If I am not in the US, would it be faster or cheaper for me to use a local google data center?* Yes, it would be faster for you to use a local google data center <https://cloud.google.com/appengine/docs/locations>. I have added a variable "REGION" in the script of the uploader. You can now change the region by replacing "us-central" with another region. I have also added an explanation of how to do this in the README. Below is a list of all the regions: REGION SUPPORTS STANDARD SUPPORTS FLEXIBLE asia-northeast1 YES YES europe-west YES NO us-central YES YES us-east1 YES YES You can also obtain this list via the following terminal command <https://cloud.google.com/sdk/gcloud/reference/app/regions/list>(with GAEuploader as your current directory): ./google-cloud-sdk/bin/gcloud app regions list As you pointed out, I have set the zone to zone3 in my original code. However, it turned out that zone is relevant only when working with Google Compute Engine <https://cloud.google.com/compute/>, which is not the case for GAEuploader. So I have removed the line "subprocess.check_call([" ./google-cloud-sdk/bin/gcloud", "beta", "config", "set", "compute/zone", " zone3"]) *> Which tor bridge does the traffic for these instances go through? (Or does this set up a tor bridge as well?)* The bridge uses https://meek.bamsoftware.com/, which is on the same host as the meek-azure bridge, but not rate-limited like meek-azure is. You can find more information on Tor relays at its Atlas page: https://atlas.torproject.org/#details/C20658946DD706A7A2181159A1A04C D838570D04 If you want to use a different bridge, you can edit the file appengine/reflect.go before running uploader: forwardURL = "https://meek.bamsoftware.com/" Thanks again for your input! Please let me know if you have any more questions/feedback. Best, Katherine Li

2 1

git-update: transparently torified git pulls
by carlo von lynX 31 Jan '17

31 Jan '17

Hi, here it is. Style may be crude as I usually write perl. #!/bin/sh # # A variant of git pull which operates over Tor and # - figures out when 'torify' needs to be used # - shows the changes that were made to the repository # - before attempting to merge # --lynX & heldensaga, 2017 # # I recommend piping the output into # vim -R "+set syntax=diff" # but since that isn't everybody's choice of editor # I won't work it into this script... # would be more elegant to tell you how to # configure this in your .git/config? gitdiff="git diff -b -M77% --minimal" # 'resolve' has served me well as default strategy gitmerge="git merge -s resolve" branch="$*" if test x"$*" = x; then branch="origin" fi # echo $branch url=`git remote get-url $branch` if test x"$url" = x; then exit fi # echo $url case $url in http*) torify="" echo "*** Fetching from $url via web proxy" ;; *) torify="torsocks" echo "*** Fetching from $url via torsocks" esac # echo $torify $torify git fetch $branch \ && $gitdiff master..$branch/master \ && exec $gitmerge $branch/master

2 2

ExitPortStatistics interpretation
by nusenu 30 Jan '17

30 Jan '17

Hi, I'm looking at ExitPortStatistics. Since the spec [1] is not very specific, I wanted to confirm that my assumption is correct: The current tor implementation includes the 10 most relevant ports, correct? (highest number of bytes or stream) example: exit-streams-opened 80=5178868,182=1092,443=3499276,5000=29600,5753=8920,6881=43496,8080=31184,8333=472,8999=16572,51413=51496,other=1925104 so the ports 80 182 443 5000 5753 6881 8080 8333 8999 51413 are the most used exit ports on that given relay (not by that order). thanks, nusenu [1] https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1082: > "exit-kibibytes-written" port=N,port=N,... NL > [At most once.] > "exit-kibibytes-read" port=N,port=N,... NL > [At most once.] > > List of mappings from ports to the number of kibibytes that the > relay has written to or read from exit connections to that port, > rounded up to the next full kibibyte. Relays may limit the > number of listed ports and subsume any remaining kibibytes under > port "other". > > "exit-streams-opened" port=N,port=N,... NL > [At most once.] > > List of mappings from ports to the number of opened exit streams > to that port, rounded up to the nearest multiple of 4. Relays may > limit the number of listed ports and subsume any remaining opened > streams under port "other".

2 5

[release] Onionoo 3.2-1.1.0
by iwakeh 30 Jan '17

30 Jan '17

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! Another release of Onionoo is now available here: https://dist.torproject.org/onionoo/3.2-1.1.0/ Besides some technical improvements that are listed in the changelog [0] this release contains a protocol extensions [1]: * Extended order parameter to "first_seen" and added response meta data fields that facilitate paging access to Oninoo data (see [2] for details). * A bug was fixed, that prevents noting bridges as first seen on January 1, 1970 when in fact they were never mentioned in a bridge network status and only learned about from their self-published bridge server descriptors (see ticket [3]). * The data on the main Onionoo instance [4] was cleaned to not report the wrong first_seen dates anymore. Please direct comments and questions to the metrics-team mailing list [5]. Cheers, iwakeh [0] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md?id=5b219203b87… [1] https://onionoo.torproject.org/protocol.org [2] https://onionoo.torproject.org/protocol.html#responses [3] https://trac.torproject.org/projects/tor/ticket/20994 [4] https://onionoo.torproject.org [5] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYjv3nAAoJEPeEx9Sa/gviBU0P/j/ABxgGW7w46SHALGqJWazC /0bF3OipaNQc8UXwejwh7GGH9IZJxk1W5IUMrf9Fu7USTrcmCmhZowBPNbh6vc64 QnFtOPgDbm/S7lURhLr56UIb68zpbbtCQu3++mj2r4oep3g3avKOcIFyNYsLnM8z qGhLRIhSyJtHC0ZLojzDgRzT3gD4qKiU79rofJysaevoHRzusSDlHB7m/LBLcGz3 NFBemfB3NtAWJcl8G50hileMp2uLnepZ5eyfWV1PmNvT3j96x4L7sUwVywdKHzjs P31R7ysgT3oYf2lcwsyCAZLrBF+ZklVOGivHcxTByVJf7At4N/j86V5iSr0I7H4n 3KKdIjzFjwYLN8E9LUeqiD+v4+a4Z+tJO56XV2O0P4r3szMaSxVK13XNriHlEp6A 3rKvyJDpGS40Kijna0H5E7NdHS6za1S2Pj/DJr62WFft/6lz48cm6HdzS7IQTRGP zHYuYcT5tMn1jS72PQmIJd8x7hP8C0G18lp9vs6au/S4rfw+/XXgmpzYJM5JQjKu jgFCNuw9V8egk8v9f25K3viv+MT19zZjrCyKTNFig28Fda0EQqLvDYiWBbNBqmyt dtd0JOYx78ynVXUBue9dPGg2YvLXhVuftdESpkxMzjGxakFRrLD2klokn9sXT9Rf i2r5Ugu0gYqkVb93DcEy =+jNS -----END PGP SIGNATURE-----

1 0

tor 0.2.9.9 gcc 4.2.1
by grarpamp 30 Jan '17

30 Jan '17

Ancient gcc says src/or/connection.c:1843: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/or/connection.c:1843: warning: passing argument 1 of 'TO_OR_CONN' discards qualifiers from pointer target type src/test/test_dir.c:3700: warning: assuming signed overflow does not occur when assuming that (X - c) >= X is always true src/test/test_dir.c:3828: warning: assuming signed overflow does not occur when assuming that (X - c) >= X is always true src/test/vote_descriptors.inc:93: warning: string length '4135' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_microdesc.c:654: warning: string length '5844' is greater than the length '4095' ISO C99 compilers are required to support src/test/test_descriptors.inc:305: warning: string length '10571' is greater than the length '4095' ISO C99 compilers are required to support

2 2

onionoo.tpo stuck at 2016-05-13 12:00
by nusenu 29 Jan '17

29 Jan '17

Hi Karsten, I was surprised that ornetradar did not send a single email for yesterday's new relays. After looking into it, it turned out it is an onionoo problem. "relays_published":"2016-05-13 12:00:00" https://onionoo.torproject.org/details?limit=4 regards, nusenu

2 11

./configure in 0.2.9.9 is ~broken?
by loki der quaeler 26 Jan '17

26 Jan '17

Hi - Trying to configure 0.2.9.9’s build with this command: ./configure --with-libevent-dir=/usr/local/lib --with-openssl-dir=/usr/local/openssl/ fails with: checking for libevent directory... configure: WARNING: We found the libraries for libevent, but we could not find the C header files. You may need to install a devel package. configure: WARNING: On most Redhat-based systems, you can get headers for libevent by installing the libevent-devel RPM package configure: error: Missing headers; unable to proceed. whereas this worked (and still works) when being issued against 0.2.8.12’s source code I have libevent-devel (and libevent-headers) installed on my system - and have been building tor on the system for many years now. Does this ring a bell with anyone?

2 4

Proposal 274: A Name System API for Tor Onion Services
by George Kadianakis 26 Jan '17

26 Jan '17

Hello list, we've been busy discussing secure name systems for the past few months, and how we could integrate them with Tor. We had a few people ask us for precise interfaces, so here is an attempt for a modular name system API. It's largely based on the PT API, which has been pretty successful. The proposal was brainstormed on a Sunday morning in Seattle between Yawning, David and me. I wrote the actual proposal contents so any innacuracies, impossibilities and errors are mine :) If people like it and are interested in implementing it, we can draft out an implementation plan for any person who wants to go for it. But for now, please consider this proposal text as a draft that is meant to be improved from this list's feedback. Have fun and looking forward to your comments (and code). --- Filename: 274-naming-layer-api.txt Title: A Name System API for Tor Onion Services Author: George Kadianakis, Yawning Angel, David Goulet Created: 04-Oct-2016 Status: Draft Table Of Contents: 1. Introduction 1.1. Motivation 1.2. Design overview and rationale 2. System Specification 2.1. System overview [SYSTEMOVERVIEW] 2.2. System Illustration 2.3. System configuration [TORRC] 2.3.1. Tor name resolution logic 2.4. Name system initialization [INITPROTOCOL] 2.5. Name resolution using NS API 2.5.1. Message format 2.5.2. RESOLVED status codes 2.5.3. Further name resolution behavior 2.6. Cancelling a name resolution request 2.7. Launching name plugins [INITENVVARS] 2.8. Name plugin workflow [NSBEHAVIOR] 2.8.1. Name plugin shutdown [NSSHUTDOWN] 2.9. Further details of stdin/stdout communication 2.9.1. Message Format 3. Discussion 3.1. Using second-level domains instead of tld 3.2. Name plugins handling all tlds '*' 3.3. Deployment strategy 3.4. Miscellaneous discussion topics 4. Acknowledgements A.1: Example communication Tor <-> name plugin [PROTOEXAMPLE] A.2: Example plugins [PLUGINEXAMPLES] 1. Introduction This proposal specifies a modular API for integrating name systems with Tor. 1.1. Motivation Tor onion service addresses are decentralized and self-authenticated but they are not human-memorable (e.g. 3g2upl4pq6kufc4m.onion). This is a source of poor usability, since Internet users are familiar with the convenient naming of DNS and are not used to addresses being random text. In particular, onion addresses are currently composed of 16 random base32 characters, and they look like this: 3g2upl4pq6kufc4m.onion vwakviie2ienjx7t.onion idnxcnkne4qt76tg.onion vwakviie2ienjx6t.onion When Proposal 224 get deployed, onion addresses will become even bigger: 53 base32 characters. That's: llamanymityx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymle.onion lfels7g3rbceenuuqmpsz45z3lswakqf56n5i3bvqhc22d5rrsza.onion odmmeotgcfx65l5hn6ejkaruvai222vs7o7tmtllszqk5xbysola.onion qw3yvgovok3dsarhqephpu2pkiwzjdk2fgdfwwf3tb3vgzxr5kba.onion Over the years Tor users have come up with various ad-hoc ways of handling onion addresses. For example people memorize them, or use third-party centralized directories, or just google them everytime. We believe that the UX problem of non-human-memorable problems is not actually solved with the above ad-hoc solutions and remains a critical usability barrier that prevents onion services from being used by a wider audience. 1.2. Design overview and rationale During the past years there has been lots of research on secure naming and various such systems have been proposed (e.g. GNS, Namecoin, etc.). Turns out securely naming things is a very hard research problem, and hence none of the proposed systems is a clear winner: all of them carry various trade-offs. Furthermore, none of the proposed systems has seen widespread use so far, which makes it even harder to pick a single project. Given the plenitude of options, one approach to decide which system is best is to make various decent name systems available and let the Tor community and the sands of time pick the winner. Also, it might be that there is no single winner, and perhaps different specialized name system should be used in different situations. We believe that by getting secure name systems actually get utilized by real users, the whole field will mature and existing systems will get battle-hardened. Hence, the contribution of this proposal is a modular Name System API (NSA) that allows developers to integrate their own name systems in Tor. The interface design is name-system-agnostic, and it's heavily based on the pluggable transports API (proposal 180). It should be flexible enough to accommodate all sorts of name systems (see [PLUGINEXAMPLES]). 2. System Specification A developer that wants to integrate a name system with Tor needs to first write a wrapper that understands the Tor Name System API (NS API). Using the Name System API, Tor asks the name system to perform name queries, and receives the query results. The NS API works using environment variables and stdin/stdout communication. It aims to be portable and easy to implement. 2.1. System overview [SYSTEMOVERVIEW] Here is an overview of the Tor name system: Alice, a Tor user, can activate various name systems by editing her torrc file and specifying which tld each name system is responsible for. For this section, let's consider a simple fictional name system, unicorn, which magically maps domains with the .corn tld to the correct onion address. Here it is: OnionNamePlugin 0 .corn /usr/local/bin/unicorn After Alice enables the unicorn plugin, she attempts connecting to elephantforum.corn. Tor will intercept the SOCKS request, and use the executable at /usr/local/bin/unicorn to query the unicorn name system for elephantforum.corn. Tor communicates with the unicorn plugin using the Tor NS API through which name queries and their results can be transported using stdin/stdout. If elephantforum.corn corresponds to an onion address in the unicorn name system, unicorn should return the onion address to Tor using the Tor NS API. Tor must then internally rewrite the elephantforum.corn address to the actual onion address, and initiate a connection to it. 2.2. System Illustration Here is a diagram illustrating how the Tor Name System API works. The name system used in this example is GNS, but there is nothing GNS-specific here and GNS could be swapped for any other name system (like hosts files, or Namecoin). The example below illustrates how a user who types debian.zkey in their Tor browser gets redirected to sejnfjrq6szgca7v.onion after Tor consults the GNS network. Please revisit this illustration after reading the rest of the proposal. | $~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$ | 1. $ 4. GNS magic!! $ | User: SOCKS CONNECT to $ debian.zkey -> sejnfjrq6szgca7v.onion$ | http://debian.zkey/ $~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$~~~~~~~$ | $ +-----|-----------------------------------------+ $ |+----v-----+ 2. +---------+| 3. $ ||Tor | debian.zkey |Tor || debian.zkey +-$-------+ ||Networking------------------------->Naming -------------------------> | ||Submodule | |Submodule|| Tor Name System API | GNS | || <------------------------- <------------------------- wrapper| || | 6. | ||5. | | |+----|-----+ sejnfjrq6szgca7v.onion +---------+|sejnfjrq6szgca7v.onion +---------+ +-----|-----------------------------------------+ | 7. | Tor: Connect to | http://sejnfjrq6szgca7v.onion/ v 2.3. System configuration [TORRC] As demonstrated in [SYSTEMOVERVIEW], a Tor user who wants to use a name system has to edit their configuration file appropriately. Here is the torrc line format: OnionNamePlugin <priority> <tld> <path> where <priority> is a positive integer denoting the priority with which this name plugin should be consulted. <tld> is a string which restricts the scope of this plugin to a particular tld. Finally, <path> is a filesystem path to an executable that speaks the Tor Name System API and can act as an intermediary between Tor and the name system. For example here is a snippet from a torrc file: OnionNamePlugin 0 .hosts /usr/local/bin/local-hosts-file OnionNamePlugin 1 .zkey /usr/local/bin/gns-tor-wrapper OnionNamePlugin 2 .bit /usr/local/bin/namecoin-tor-wrapper OnionNamePlugin 3 .scallion /usr/local/bin/community-hosts-file 2.3.1. Tor name resolution logic When Tor receives a SOCKS request to an address that has a name plugin assigned to it, it needs to perform a query for that address using that name plugin. If there are multiple name plugins that correspond to the requested address, Tor queries all relevant plugins sorted by their priority value, until one of them returns a successful result. If two plugins have the same priority value, Tor MUST abort. If all plugins fail to successfuly perform the name resolution, Tor SHOULD default to using the exit node for name resolution. XXX or not? because of leaks? 2.4. Name system initialization [INITPROTOCOL] When Tor finds OnionNamePlugin lines in torrc, it launches and initializes their respective executables. When launching name plugins, Tor sets various environment variables to pass data to the name plugin (e.g. NS API version, state directory, etc.). More information on the environment variables at [INITENVVARS]. After a name plugin initializes and parses all needed environment variables, it communicates with Tor using its stdin/stdout. The first line that a name plugin sends to stdout signifies that it's ready to receive name queries. This line looks like this: INIT <VERSION> <STATUS_CODE> [<STATUS_MSG>] where VERSION is the Tor NS API protocol version that the plugin supports, STATUS_CODE is an integer status code, and STATUS_MSG is an optional string error message. STATUS_CODE value 0 is reserved for "success", and all other integers are error codes. See [PROTOEXAMPLE] for an example of this protocol. 2.5. Name resolution using NS API Here is how actual name resolution requests are performed in NS API. 2.5.1. Message format When Tor receives a SOCKS request to an address with a tld that has a name plugin assigned to it, Tor performs an NS API name query for that address. Tor does this by printing lines on the name plugin stdout as follows: RESOLVE <QUERY_ID> <NAME_STRING> where QUERY_ID is a unique integer corresponding to this query, and NAME_STRING is the name to be queried. When the name plugin completes the name resolution, it prints the following line in its stdout: RESOLVED <QUERY_ID> <STATUS_CODE> <RESULT> where QUERY_ID is the corresponding query ID and STATUS_CODE is an integer status code. RESULT is the resolution result (an onion address) or an error message if the resolution was not succesful. See [PROTOEXAMPLE] for an example of this protocol. XXX Should <RESULT> be optional in the case of failure? 2.5.2. RESOLVED status codes Name plugins can deliver the following status codes: 0 -- The name resolution was successful. 1 -- Name resolution generic failure. 2 -- Name tld not recognized. 3 -- Name not registered. 4 -- Name resolution timeout exceeded. XXX add more status codes here as needed 2.5.3. Further name resolution behavior Tor and name plugins MAY cache name resolution results in memory as needed. Caching results on disk should be avoided. Tor SHOULD abort (or cancel) an ongoing name resolution request, if it takes more than NAME_RESOLUTION_TIMEOUT seconds. XXX NAME_RESOLUTION_TIMEOUT = ??? Tor MUST validate that the resolution result is a valid .onion name. XXX should we also accept IPs and regular domain results??? XXX perhaps we should make sure that results are not names that need additional name resolution to avoid endless loops. e.g. imagine some sort of loop like this: debian.zkey -> debian-bla.zkey -> debian.zkey -> etc. 2.6. Cancelling a name resolution request Tor might need to cancel an ongoing name resolution request (e.g. because a timeout passed, or the client is not interested in that address anymore). In this case, Tor sends the following line to the plugin stdout as follows: CANCEL <QUERY_ID> to which the name plugin, after performing the cancellation, SHOULD answer with: CANCELED <QUERY_ID> 2.7. Launching name plugins [INITENVVARS] As described in [INITPROTOCOL], when Tor launches a name plugin, it sets certain environment variables. At a minimum, it sets (in addition to the normal environment variables inherited from Tor): "TOR_NS_STATE_LOCATION" -- A filesystem directory path where the plugin should store state if it wants to. This directory is not required to exist, but the plugin SHOULD be able to create it if it doesn't. The plugin MUST NOT store state elsewhere. Example: TOR_NS_STATE_LOCATION=/var/lib/tor/ns_state/ "TOR_NS_PROTO_VERSION" -- To tell the plugin which versions of this configuration protocol Tor supports. Future versions will give a comma-separated list. Plugins MUST accept comma-separated lists containing any version that they recognize, and MUST work correctly even if some of the versions they don't recognize are non-numeric. Valid version characters are non-space, non-comma printing ASCII characters. Example: TOR_NS_PROTO_VERSION=1,1a,2,4B "TOR_NS_PLUGIN_OPTIONS" -- Specifies configuration options for this name plugin as a semicolon-separated list of k=v strings with options that are to be passed to the plugin. Colons, semicolons, equal signs and backslashes MUST be escaped with a backslash. If there are no arguments that need to be passed to any of the plugins, "TOR_NS_PLUGIN_OPTIONS" MAY be omitted. For example consider the following options for the "banana" name plugin: TOR_NS_PLUGIN_OPTIONS=timeout=5;url=https://bananacake.com Will pass to banana the parameters 'timeout=5' and 'url=https://bananacake.com'. XXX Do we like this option-passing interface? Do we have any lessons from our PT experiences? XXX Add ControlPort/SocksPort environment variables. See [PROTOEXAMPLE] for an example of this environment 2.8. Name plugin workflow [NSBEHAVIOR] Name plugins follow the following workflow: 1) Tor sets the required environment values and launches the name plugin as a sub-process (fork()/exec()). See [INITENVVARS]. 2) The name plugin checks its environment, and determines the supported NS API versions using the env variable TOR_NS_PROTO_VERSION. 2.1) If there are no compatible versions, the name plugin writes an INIT message with a failure status code as in [INITPROTOCOL], and then shuts down. 3) The name plugin parses and handles the rest of the environment values. 3.1) If the environment variables are malformed, or otherwise invalid, the name plugin writes an INIT message with a failure status code as in [INITPROTOCOL], and then shuts down. 4) After the name plugin completely initializes, it sends a successful INIT message to stdout as in [INITPROTOCOL]. Then it continues monitoring its stdin for incoming RESOLVE messages. 6) When the name plugin receives a RESOLVE message, it performs the name resolution and replies with the appropriate RESOLVED message. 7) Upon being signaled to terminate by the parent process [NSSHUTDOWN], the name plugin gracefully shuts down. 2.8.1. Name plugin shutdown [NSSHUTDOWN] To ensure clean shutdown of all name plugins when Tor shuts down, the following rules apply for name plugins: Name plugins MUST handle OS specific mechanisms to gracefully terminate (e.g. SIGTERM). Name plugins SHOULD monitor their stdin and exit gracefully when it is closed. 2.9. Further details of stdin/stdout communication 2.9.1. Message Format Tor communicates with its name plugins by writing NL-terminated lines to stdout. The line metaformat is <Line> ::= <Keyword> <OptArgs> <NL> <Keyword> ::= <KeywordChar> | <Keyword> <KeywordChar> <KeyWordChar> ::= <any US-ASCII alphanumeric, dash, and underscore> <OptArgs> ::= <Args>* <Args> ::= <SP> <ArgChar> | <Args> <ArgChar> <ArgChar> ::= <any US-ASCII character but NUL or NL> <SP> ::= <US-ASCII whitespace symbol (32)> <NL> ::= <US-ASCII newline (line feed) character (10)> Tor MUST ignore lines with keywords that it doesn't recognize. 3. Discussion 3.1. Using second-level domains instead of tld People have suggested that users should try to connect to reddit.zkey.onion instead of reddit.zkey. That is, we should always preserve .onion as the tld, and only utilize second-level domains for naming. The argument for this is that this way users cannot accidentally leak addresses to DNS, as the .onion domain is reserved by RFC 7686. The counter-argument here is that this might be confusing to users since they are not used to the second-level domain being special (e.g. co.uk). Also, what happens when someone registers a 16-character name, that matches the length of a vanilla onion address? We should consider the concerns here and take the right decision. 3.2. Name plugins handling all tlds '*' In [TORRC], we assigned a single tld to each name plugin. Should we also accept catch-all tlds using '*'? I'm afraid that this way a name system could try to resolve even normal domains like reddit.com . Perhaps we trust the name plugin itself, but maybe the name system network could exploit this? Also, the catch-all tld will probably cause some engineering complications in this proposal (as it did for PTs). 3.3. Deployment strategy We need to devise a deployment strategy that will allow us to improve the UX of our users as soon as possible, but without taking hasty, sloppy or uneducated decisions. For starters, we should make it easy for developers to write wrappers around their secure name systems. We should develop libraries that speak the NS API protocol and can be used to quickly write wrappers. Similar libraries were quite successful during pluggable transport deployment; see pyptlib and goptlib. In the beginning, name plugins should be third-party applications that can be installed by interested users manually or through package managers. Users will also have to add the appropriate OnionNamePlugin line to their torrc. This will be a testing phase, and also a community-growing phase. After some time, and when we get a better idea of how name plugins work for the community, we can start considering how to make them more easily usable. For example, we can start by including some name plugins into TBB in an optional opt-in fashion. We should be careful here, as people have real incentives for attacking name systems and we should not put our users unwillingly in danger. 3.4. Miscellaneous discussion topics 1. The PT spec tries hard so that a single executable can expose multiple PTs. In this spec, it's clear that each executable is a single name plugin. Is this OK or a bad idea? Should we change interfaces so that each name plugin has an identifier, and then use that identifier for things? 2. Should we make our initialization protocol _identical_ to the PT API initialization protocol? That is, use ENV-ERROR etc. instead of INT? 3. Does it make sense to support reverse queries, from .onion to names? So that people auto-learn the names of the onions they use? 4. Acknowledgements Proposal details discussed during Tor hackfest in Seattle between Yawning, David and me. Thanks to Lunar and indolering for more discussion and feedback. Appendix A.1: Example communication Tor <-> name plugin [PROTOEXAMPLE] Environemnt variables: TOR_NS_STATE_LOCATION=/var/lib/tor/ns_state TOR_NS_PROTO_VERSION=1 TOR_NS_PLUGIN_OPTIONS=timeout=5;cache=/home/user/name_cache Messages between Tor and the banana name plugin: Name plugin (banana) -> Tor: INIT 1 0 Tor -> Name plugin (banana): RESOLVE 1 daewonskate.banana Name plugin (banana) -> Tor: RESOLVED 1 0 jqkscnkne4qt91iq.onion Tor -> Name plugin (banana): RESOLVE 1 architecturedirect.zkey Name plugin (banana) -> Tor (banana): RESOLVE 1 2 "zkey not recognized tld" Tor -> Name plugin (banana): RESOLVE 1 origamihub.banana Name plugin (banana) -> Tor (banana): RESOLVE 1 2 wdxfpaxar4dg12vd.onion Appendix A.2: Example plugins [PLUGINEXAMPLES] Here are a few examples of name plugins for brainstorming: a) Simplest plugin: A local hosts file. Basically a local petname system that maps names to onion addresses. b) A remote hosts file. A centralized community hosts file that people trust. c) Multiple remote hosts files. People can add their own favorite community hosts file. d) Multiple remote hosts files with notaries and reputation trust. Like moxie's convergence tool but for names. e) GNS f) OnioNS g) Namecoin/Blockstart

10 14

log: ORPort/DirPort address does not match descriptor address
by nusenu 24 Jan '17

24 Jan '17

Hi, how bad is it to simply ignore ORPort/DirPort address mismatch log entry on a relay behind (1:1) NAT? I assume tor figures out the public IPv4 address anyway, no? The IPv4 ORPort address 192.168.1.1 does not match the descriptor address 1.2.3.4. If you have a static public IPv4 address, use 'Address <IPv4>' and 'OutboundBindAddress <IPv4>'. If you are behind a NAT, use two ORPort lines: 'ORPort <PublicPort> NoListen' and 'ORPort <InternalPort> NoAdvertise'. I'm explicitly specifying (private) IP addresses in ORPort/DirPort (and OutboundBindAddress) lines to avoid binding to the same ports when running >2 instances (with >1 public IP). https://github.com/nusenu/ansible-relayor/issues/101 thanks, nusenu

3 5

GAEuploader
by Katherine Li 23 Jan '17

23 Jan '17

Hello Tor-Dev Community, GAE(Google App Engine) uploader has just released v1.0-beta. GAEuploader enables clients to set up their own private meek-google in one easy step: just run it and paste the bridge line into Tor Browser. Background information: Meek <https://trac.torproject.org/projects/tor/wiki/doc/meek#Overview> is pluggable transport that uses domain fronting. Meek-google got suspended <https://lists.torproject.org/pipermail/tor-talk/2016-June/041699.html>, but clients can set up their own Google App Engine app and configure their Tor Browser accordingly. However, the process of creating and deploying the Google App Engine can be confusing. GAEuploader significantly simplifies this process for clients. I would really appreciate user testing on GAEuploader. You can download it at: https://github.com/katherinelitor/GAEuploader/releases README: https://github.com/katherinelitor/GAEuploader Tor wiki page, containing step-by-step screenshots of how to use: https://trac.torproject.org/projects/tor/wiki/doc/GAEuploader *GAEuploader runs on Mac or Linux, but currently does not support Windows. *GAEuploader uses the Google Cloud SDK <https://cloud.google.com/sdk/docs/#linux>. If you are interested in how GAEuploader works, the README contains the list of commands that were run. If you have any questions, feedback or bug reports, please email me at katherineli.tor(a)gmail.com. I will respond promptly. Best, Katherine Li

3 2