February 2018 - tor-dev - lists.torproject.org

Stem ORPort protocol support
by Damian Johnson 07 Feb '18

07 Feb '18

Hi all. Over the last few months Tim and I have been collaborating on Python support for the ORPort protocol. With it you can download descriptors without a DirPort, and possibly fancier things in the future like full circuit construction. Tim put together a wonderful proof of concept called Endosome... https://github.com/teor2345/endosome ... and I just finished integrating it into Stem... https://gitweb.torproject.org/stem.git/tree/stem/client/__init__.py With stem.client you can now download descriptors... import stem.client with stem.client.Relay.connect('127.0.0.1', 12345, [3]) as relay: circ = relay.create_circuit() circ.send('RELAY_BEGIN_DIR', stream_id = 1) desc = circ.send('RELAY_DATA', 'GET /tor/server/authority HTTP/1.0\r\n\r\n', stream_id = 1).data circ.close() print(desc) When run this looks like... % python demo.py HTTP/1.0 200 OK Date: Wed, 07 Feb 2018 18:42:41 GMT Content-Type: text/plain Content-Encoding: identity Expires: Fri, 09 Feb 2018 18:42:41 GMT router Unnamed 97.113.177.53 12345 0 23456 identity-ed25519 -----BEGIN ED25519 CERT----- AQQABm/qAazUltT1iUUbIMw8VNNhGb50FDHKJz6S94FLQNxL0LObAQAgBAAapbO9 iLFD0l9SEiEMFQWIT2VnbLyCZKvbrxTs5ULC1l1hQPoui6Y/lEd3yjrQhIs/vl6R 1S6FbwSFDmiXOzq47mFrse4C71ht3TpLOD0F3wiyjWtsqU1k7iPmmpejUgs= -----END ED25519 CERT----- master-key-ed25519 GqWzvYixQ9JfUhIhDBUFiE I'd like to emphasize this is still very alpha. The API isn't set in stone and there's no doubt quite a few rough edges. However, I wanted the list to be aware just in case anyone would care to build on it. I plan to draw a line at 'download descriptors through ORPorts' but I'd be delighted to help others if there's more ambitious directions they'd care to go (potentially all the way up to a Python Tor client, similar to Orchid). Now that we've reached this milestone I'm taking a break to focus on Stem support for v3 Onion Services for a bit. However, when I come back the next things on my dance card are... a. Support ORPort downloads in the stem.descriptor.remote module. b. More integ tests so Stem can be used as a tool for testing tor's ORPort. c. Give more thought to the API we'd like to vend. d. Brainstorm a GSoC project idea that expands these capabilities. Cheers! -Damian

3 4

[release] Onionoo 5.0-1.10.1
by iwakeh 07 Feb '18

07 Feb '18

Hi there! A fix for the most recent Oninonoo version is available at: https://dist.torproject.org/onionoo/5.0-1.10.1/ The changes [1] concern also the resolution of graph data: three months weights and bandwidth data are now provided in 24 hours detail. Deployment of the new version 5.0-1.10.1 finished successfully for all onionoo.torproject.org instances. The changes for bandwidth data might take another day to be visible. So, we need to have a little patience. Please direct comments and questions to the metrics-team mailing list [2]. Cheers, iwakeh [1] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md?id=a7418b7c4d1… [2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

[release] Onionoo 5.0-1.10.0
by iwakeh 07 Feb '18

07 Feb '18

Hi there! the new Oninonoo version is available at: https://dist.torproject.org/onionoo/5.0-1.10.0/ All changes [1] this time are supposed to make writing of bandwidth, clients, uptime, and weights documents independent of system time, which is a step toward making Onionoo instances serve the same data where possible. Other steps remain and will be addressed in future [2]. Deployment of version 5.0-1.10.0 will start at 13:00 UTC for all onionoo.torproject.org instances. It might cause a a few hours without updates as we need to make additional backups. For those interested, I will post a deployment finished message to metrics-team mailing list [3] approximately during UTC evening. Please direct comments and questions to the metrics-team mailing list [3]. Cheers, iwakeh [1] https://gitweb.torproject.org/onionoo.git/plain/CHANGELOG.md [2] https://trac.torproject.org/projects/tor/ticket/25002 [3] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

Re: [tor-dev] Proposal: only parse .torrc files in torrc.d directory
by iry 05 Feb '18

05 Feb '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Thank you so much for the immediate feedback, teor! Could you please help me to review the requirements (TODO list) below: * %include /etc/torrc.d means parsing all the files in the directory * implement %include /etc/torrc.d/*.extension syntax * let Tor log each config file it loads * warn on potentially unexpected files * document *.conf as the recommend usage * document how to manually migrate from * to *.conf * recommend /etc/torrc.d/*.conf as "a default extension for packagers" > In my experience, mailing lists are good for discussion, trac > tickets are good for tasks, and gitlab/oniongit are good for code > review. Got it! Thank you very much! Best Regards, iry -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEzKSpZKlpRovTotu+oUtNvG3N1TwFAlp4wsIACgkQoUtNvG3N 1Tx3JA/5AXuMfqo7jww4LeRvIUfs69+Fu98MV8X1GLKaTnqDH7MnDhOW5KxUY8HU FbVK040GhV7UGMCP83fCbSnFVw4uNFMENToYVFB649Lg5/BAOzbvQ0Lc6i1B+fUt W5rq3rzJxvMSG3q8EXPgxxHWUGl2AKzGkKBeIxc4WYt1aZX9FXpbVCZn3Rnu5Gj1 OeesVXdkO0TGb0r99qEBg7CqGUUT28L8VveiJlZHyRUFpwrKdBlX8hVhpIaQYEDV 9NXMTtRd4eHUFrRs7ZINpg6LiUKcicVVOHj1p0qx1DV5T9tdXVpBemHTJQtmQAU+ 3iJgyoNHDez0WBlIwhXm0zr3QFot2/MV2D+yKAIJGwVlWOtEFnqUGyX+LwxD6XQ8 Qzj4XcOLbs+hIB6h/QPXGgQnP7seLZhJMQAReMu64uvGx+zt15Avm3DAbz5rgwnR pYQatcG/2/6sESgNz4NXjPaxz6nokylsKSMaynd1aHHzsOc3vLVWRbHbMvSoC8Mu iKeUhhg01pO6eSzAzM9tnFGb0TaSrjR5NSnq3BoFeSnPNqZdsmehBOMMFXtj68QC Dn78/gC0++W0DZONXSf2HdARwP+asNTzMnyobQHw3ikgLtCxCNjCLOTycyrCMkNS Ar8cbOEVaDG802JWUMdWnFbbE6CfNWh5egx3jSzw19NDM+qGM8E= =mJnZ -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] Proposal: only parse .torrc files in torrc.d directory
by iry 05 Feb '18

05 Feb '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi teor! > Could you please look at existing .d folders of any other projects > tell me what you think? Perhaps discuss this with Tor Project. > [...] From my quick search, it appears that the Debian feature > request is still open, and no other distro is using torrc.d yet. > But you should check, too. I went through all Tor packages listed here: https://trac.torproject.org/projects/tor/wiki/doc/packages and no distros shipped a torrc with %include line enabled. I know Whonix will not use torrc.d before next stable version. I also did a grep -r -i "%include" on Tails source code and I do not think Tails has enabled it by default. nickm suggested proposed to create a new syntax to take care of the compatibility: > %include /etc/torrc.d/*.conf Here is my thoughts on this: 1. I agree that "[a]nybody who currently has a working setup will have it fail if we start requiring a suffix that they didn't know to provide", which is not good for compatibility. But, letting people still use or will be able to use a setting that is not recommended anymore seems also not to be a very good idea? Considering the potential danger of parsing all the files, shall we go a little bit aggressive? I would rather break people's current potentially dangerous settings. What do you think? 2. Since no distros I know has enabled this feature by default, I guess there are only a very small number of users has enabled this feature. Will an info in the release note saying "%include /etc/torrc.d/ will only pase files suffixed with .torrc files" be enough to inform them? Maybe we can even document the manual migration somewhere. 3. %include /etc/torrc.d/*.conf syntax is very flexible so that Tor does not have to decide which extension names should be parsed. 4. %include /etc/torrc.d/*.conf syntax explicitly says which extension name will be used rather than the implicit document. 5. But is it a good idea to make the syntax that flexible? For example, anon-connection-wizard will generate a torrc files in torrc.d directory, I (and maybe many other developers) prefer writing to a file that I can guarantee it will be parsed in most case. If I write to 40_anon-connection-wizard.conf but some people set to pase .torrc or anything else only, it will be not be very good? (I do not want anon-connection-wizard to touch /etc/tor/torrc) Finally, do you think it is a good idea to switch to the ticket for further discussion to avoid cross posting and high volume on @tor-dev? Thank you very much! Best Regards, iry -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEzKSpZKlpRovTotu+oUtNvG3N1TwFAlp4pFoACgkQoUtNvG3N 1TwSkRAAnBRytWCeP8ocCOJ0FmJZVq4ANznDWWBoYg+iaRPUMYbU/Tro0j7ljJMe snf0ji+Eu8DqTjg/HgmQ9gVuiJNWE7jfvnaAE7nDxeqd23J68ek5yHeIwonnaVPq IrWeCQDAAUUz42rAcoHBsy/tGS/kiq0OMf3yf6Pzq02UsQ4Ob46kD4ySNnirRXce a/mJG1zMXoAYnM84bKnm6bD4Gx5qiyK+O61e70z4b4ArPqRpxmfQoU5RAELMHAFU hq9JtdxN/FBvNq7NUOboAYcX1FdkWLLEaQXWB/sgtS94OvCIX0hfrtis0/UFPTe5 wQN1FQCFBpPkHOLuvszNln9WvPf0XSWqCDVdZh7Fd9GrELfrVwEJYC2+1rukPWZq U0ZJ0smDkdijHBc53i6+23175GyaQ+uMoSebouQ2vh9hbf2qx1EMnSFc5pSz8y5b eJgUA6WFZvcTgUFmwYpe3X2E3QcoYHD8UNoBZiD0ehXrWTcxScT0kWNcF06v3dhr 4Doo9wxDmF/ZusRrpyTRCZ5LsPCCL1spphxNqlCIm9MsfUCHBUbULBKn+uV2fNb6 4Xk2gu6eA258ZEfYDhottLpsk8V15Jx7F+Jz/W0zlL5OCwDzhPcG44C7OfEzeW5u h42Jw6YsQkAgGXnKzRG9lW7emVkLLhF0wlCSeWY8QxHGnxShdLY= =EnSw -----END PGP SIGNATURE-----

2 1

onionoo: varnish 503
by nusenu 05 Feb '18

05 Feb '18

https://onionoo.torproject.org/details?search=123 > Error 503 Backend fetch failed > > Backend fetch failed > Guru Meditation: > Varnish cache server thanks for having a look -- https://mastodon.social/@nusenu twitter: @nusenu_

2 1

Re: [tor-dev] Proposal: only parse .torrc files in torrc.d directory
by iry 04 Feb '18

04 Feb '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 > To be more precise, most tools accept files ending in ".conf". We > might want tor to accept ".conf" for consistency. > > I suggest we also accept files called "torrc", or ending in > ".torrc". This should probably also include files called literally > ".torrc". > Thank you for your suggestions, teor! I have updated it in the ticket. > Since this is a breaking change, we might want to check if any > distribution has adopted another naming scheme, and avoid breaking > that naming schem I agree. I will wait for a little bit before implementing it so that we may get more input on the issue. :) > If you'd like your patch to be reviewed quickly: * put the ticket > in 0.3.3, because it's a bugfix * change the status to "needs > review" once you have a patch Thank you so much for your instructions! Best Regards, iry -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEzKSpZKlpRovTotu+oUtNvG3N1TwFAlp3dnwACgkQoUtNvG3N 1TycXQ//X5m4c9gucDY2idIHULxzIDdfgAFnPOyjzyqDLjM8y4/Uu/3xV4SX3gX8 M2qguNmqMX2+OuoWTHPo+4S+L4qS12XibUha1lwcWxjXGyPkZ7iwIvj7a0RoBsb1 Ln6DcpBGcPt61trQg4eBLI2JbAOQRXh3J/jl9djNzs64uj/0WtxXcUhBDPiJyPz1 yunB8EuO3NJIZ9n1SoT6d9HpF8oLPu4E9PkhFi0uflPKdZhwAUqCLW4J4zqlK2eB 8cQezMHvqNmTYS9pmQgw6lz0xAL+6NQZWUeFLFE+TvIqB8RwFQkAcb3mfTfDFvpf K/tOiSF+9OvMEVonB1efWUhaxSBHc8MwQFfWaWTNNndHkO6lVGYtwPC7UfrteTY5 cr0oB6xE5U+La/2GnwP5i2200v0IgNi1S82okXIbGbMyEgEXzkC4+6xIsq40ksGy 8hRHEusLz8FUefKioxdiyPmHZgrq1jzQGSgFQKafKLR0YkfFgnbIX1alMgVIatS0 xbr53GXtQVY658T/lAdXSIqwb5regJ9NB/+LtB7HvzFbFMCnT4g1yDYLSsiwWX/z 2KJH8+OOKy6WMRmoTbMXb1J+WnBTVfazUc+fOs4IK2tlnMDs7TukcS9FtNHS1G1d AlgML+AfnuO3m8SKR1Vsbl5PRypA7Vz0a4kkzy1dLgpaqVxP6Rw= =vdE1 -----END PGP SIGNATURE-----

2 1

onionoo.tpo stuck at 2018-01-21 22:00
by nusenu 04 Feb '18

04 Feb '18

Hi Karsten, just wanted to let you know that the delta between relays_published and current time is unusually high. https://onionoo.torproject.org/details?limit=0 {"version":"5.0", "build_revision":"0bce98a", "relays_published":"2018-01-21 22:00:00", This is currently blocking ornetradar reports. thanks for having a look, nusenu -- https://mastodon.social/@nusenu twitter: @nusenu_

2 11

Re: [tor-dev] Proposal: only parse .torrc files in torrc.d directory
by iry 03 Feb '18

03 Feb '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 >> Therefore, may I propose to let Tor parse only the files whose >> name ends with .torrc ? > > Yes, this is standard behaviour among many tools. > >> Or maybe even only parse number_filename.torrc for better >> consistency and for more clear priority order? > > No, this is counter-intuitive. It will confuse many users. It is > not how most other tools work. Thank you so much for the feedback, teor! I have opened the ticket #25140 for this. Shall I try to make a patch on this? Or is it preferred to leave it to developers from TPO? Thank you very much! Best Regards, iry -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEzKSpZKlpRovTotu+oUtNvG3N1TwFAlp2hnAACgkQoUtNvG3N 1Tz3GBAAghKBKm7JVd5PyO7HGBClshO/fUOG2ippVm5hY9sgmzVPxVauJC+CWJN9 RH6YfJwyLJeAHu0o3YD1+LBHvccxTKKQYL48q8aopfWkh8ALk4h9+O5hwpPPIsGC qV7iJ+v4QfpxYkhAUFh8FOy+m5WqsOurScBzcjEnmu2IpIkpnwv1NKLaVrArpMv9 lrGtQds/7dCYC57boSUNgTTJtbuqgNhRBM1oqkGisHsKBiLYExvztS22aLj9uVEX km5c+JJuwEKuuordP4YiFMdES1UYx9Bdm/LrUNWG4dB/ZfjwZYSu/KPsqQcDwmNy 25iovL5KUkVg8S05QwdcCnXm2B5oofLMJ4hW0bkjozjLUt3na131uJumOR7Q2W9f 9aKAGwRzQ624rDHn3SWCC1HKMhP0Y6EhCDyzX5914kRakikas6hmtUWmrSqFvZLO p0WzWuNB05BdRXImTXxPdLyOxasB/eMt31mQXKBSSiwdzJxe3rfnDb//P2vcv9ex ofQNx672qtAgmHp9WWeKY6qas36sZeckEigIlcUN27KFBwDyCGfCebk5tPIGDzC9 aAqmiTqqYCqphHb3i/xX3WwGwk6+Q1ixezOfYFxbAOa96NUDV2ldYIXgBcsAWh/+ RbZ8hI/oOwAug/C9ig5xFpZAy86bbqDApG6wrGNg5gp3qVKDlnA= =zlMh -----END PGP SIGNATURE-----

2 1

Proposal: only parse .torrc files in torrc.d directory
by iry 03 Feb '18

03 Feb '18

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Dear Tor Developers, I have been testing and using the torrc.d feature for a while, and here is a potential improvement we may make. Currently, when using a torrc.d directory, for example: > %include /etc/torrc.d/ Every file in the directory will be treated and parsed as a valid Tor configuration file. However, sometime, this may not be what users and developers want. For example, users may use /etc/torrc.d/50_user.torrc as the place to put their own torrc configurations. But sometimes, when they use a text editor to edit it, the text editor will leave a /etc/torrc.d/50_user.torrc~ file which will also be treated as a valid torrc file. Another example that also happens very frequently is, when dpkg does an update on /etc/torrc.d/30_distribution.torrc, users' previous configuration can be saved as /etc/torrc.d/30_distribution.torrc.dpkg-old which will also be parsed by Tor. In best case users will just be frustrated because Tor does not work as expected and in worst case this could be dangerous. This could be a severe problem especially because of the following reasons: 1. filename.torrc~ filename.torrc.dpkg-old has higher priority than filename.torrc when Tor does the parsing. 2. In most cases, this will happen without being noticed by the normal suer. Therefore, may I propose to let Tor parse only the files whose name ends with .torrc ? Or maybe even only parse number_filename.torrc for better consistency and for more clear priority order? Thank you very much! Looking forward to hearing your insights! Best Regards, iry -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEzKSpZKlpRovTotu+oUtNvG3N1TwFAlp2Oa0ACgkQoUtNvG3N 1TxfBA/9FfFhBfI08ocI6Wluodg58lppbG+N1LOGxoTSH2iq53P7GaWvZd9OULP1 ezTDbnlPJ3jreBYRnze8/MaUFA/x6wJTAcyM71xktwuR1Np47StY8ishZnbLpI5z D7r65kKZyxrcCht99oSOFuqakVb0bFJJJRF1rvgt9gDHZT9+0V1/GVAv0w1SEb7J LYazQyjtIgCJQaDdZvRsulnlBuFF6KvVNZp4EEKfggihM480SJV1gRExDBrEYQEN BqtYixEe2P2MHPZSV7Szp28D87I391GwrHuLGsrlhL92OPPygVlc+Amyw9rEMij4 y/sXlSN+62Q2UV2CvFGzxSJBNhstJ4Meh91yKDf+Lm0CER5ydEUWtLJLYaIGmjS/ iQNQUd/6Y2dIyzeZuXFm03cfIXtQNdQFAhahNvNEeUOcJ4Qk4IsHXUi7MSzHcQTb lv0E/2IzleXWp57L9rDPayA7eUpNMHlZkVyH42WunGcK+uz6PIT/bvBc5L1b/z3/ zkaPleDivBV5czlBtQIwRURibbUuveDtGfacM/3pmQ5XuYYaA6UvRExHePMeQvM+ Y01YTMlEcqL4LcjHSJiGamdWRPtGazbXVT1bl4VRJ1RqGqrvFE65I+xdg01UnSWy tgsOUhEwrMUxUdsumNpb6sEgpUmZ/NzKbi7zgGywEmxVkUN67lc= =zue4 -----END PGP SIGNATURE-----

2 1