Patches to improve mobile hidden service performance
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi all, I've been experimenting with small changes to Tor to improve the performance of mobile hidden services. The following patches for Tor and jtorctl make two performance improvements: 1. Each time the network's enabled, don't try to build introduction circuits until we've successfully built a circuit. This avoids a problem where we'd try to build introduction circuits immediately, all the circuits would fail, and we'd wait for 5 minutes before trying again. 2. Added a command to the control protocol to purge any cached state relating to a specified hidden service. This command can be used before trying to connect to the service to force Tor to download a fresh descriptor. It's a more fine-grained alternative to SIGNAL NEWNYM, which purges all cached descriptors and also discards circuits. https://code.briarproject.org/akwizgran/briar/blob/9e5e2e2df24d84135f14adaa4... https://code.briarproject.org/akwizgran/briar/blob/9e5e2e2df24d84135f14adaa4... The Tor patch is based on the tor-0.2.24 tag, and the jtorctl patch is based on the Guardian Project's repo, which is ahead of the upstream repo (n8fr8 needs commit privileges for upstream, I think). https://github.com/guardianproject/jtorctl I've only done small-scale testing of these patches so far. If they seems like they might be useful I'll create a trac ticket to merge them. Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJUM6W7AAoJEBEET9GfxSfM5FkIAL+gPk2ZmGvSnt/gVQGZwqH5 mpqGoPIfZPycfhm/dgOjE4MSANcGo7kTqnh85ir9TQvjtNAEkfkTL7GB9G+DOd2X ghje+HCWNRU8WlFD27rrxSUzT+8IvfVaJaj+sCYtA44ib3qpz4XXKXee8xTW2epV dnCLtW7TBUjQEHqbL0aGtsKEHvgVQcLWPxusEioefBPXo6a+8cRKLY+EJvqxaMze mdOQBh+wa7SIyMQi4JqgzjtKAoAqpVzXYsIrmxT1hlCOOaNDjbFUN35QbrZMT6B+ zNCuWIIC6JfBWccfxDsAew2MC8WNf05nvgM67cTFU2nCI6MwYYf4x2xtEOcA5qs= =LweI -----END PGP SIGNATURE-----
On Oct 7, 2014, at 2:35 AM, Michael Rogers <michael@briarproject.org> wrote:
Hi all,
I've been experimenting with small changes to Tor to improve the performance of mobile hidden services. The following patches for Tor and jtorctl make two performance improvements:
1. Each time the network's enabled, don't try to build introduction circuits until we've successfully built a circuit. This avoids a problem where we'd try to build introduction circuits immediately, all the circuits would fail, and we'd wait for 5 minutes before trying again.
This sounds like a useful patch. Do you mean that status/circuit-established currently returns true after DisableNetwork is set? I suggest submitting that separately from the control changes.
2. Added a command to the control protocol to purge any cached state relating to a specified hidden service. This command can be used before trying to connect to the service to force Tor to download a fresh descriptor. It's a more fine-grained alternative to SIGNAL NEWNYM, which purges all cached descriptors and also discards circuits.
You’ll need to update control-spec. I’m curious what would happen if you use this command on a HSDir or for a local service - those cases need to be defined. I wonder what problem you’re trying to solve, and if this is the best solution. From a quick reading of the code tor’s behavior is: 1. If an intro point remains, try it 2. If we run out of intro points, re-fetch the descriptor 3. If we’ve tried fetching the descriptor from all HSDirs within the past 15 minutes, fail 4. If the descriptor is re-fetched, try all intro points again The list of HSDirs used in the past 15 minutes seems to be cleared when a connection succeeds, when we give up connecting to a service, or on rendezvous failures. Even if you hit that 15-minute-wait case, it looks like another connection attempt will re-fetch the descriptor and start over. See rendclient.c:1065, leading to purge_hid_serv_from_last_hid_serv_requests. Can you point out where your client gets stalled, or where I’ve read the logic wrong?
https://code.briarproject.org/akwizgran/briar/blob/9e5e2e2df24d84135f14adaa4...
https://code.briarproject.org/akwizgran/briar/blob/9e5e2e2df24d84135f14adaa4...
The Tor patch is based on the tor-0.2.24 tag, and the jtorctl patch is based on the Guardian Project's repo, which is ahead of the upstream repo (n8fr8 needs commit privileges for upstream, I think).
https://github.com/guardianproject/jtorctl
I've only done small-scale testing of these patches so far. If they seems like they might be useful I'll create a trac ticket to merge them.
Cheers, Michael
_______________________________________________ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
- John
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 08/10/14 02:55, John Brooks wrote:
1. Each time the network's enabled, don't try to build introduction circuits until we've successfully built a circuit. This avoids a problem where we'd try to build introduction circuits immediately, all the circuits would fail, and we'd wait for 5 minutes before trying again.
This sounds like a useful patch. Do you mean that status/circuit-established currently returns true after DisableNetwork is set?
Yup, that's right. So when the network is re-enabled we try building intro circuits straight away, in contrast to startup when we wait for the first circuit to be built.
I suggest submitting that separately from the control changes.
Thanks, I'll do that.
2. Added a command to the control protocol to purge any cached state relating to a specified hidden service. This command can be used before trying to connect to the service to force Tor to download a fresh descriptor. It's a more fine-grained alternative to SIGNAL NEWNYM, which purges all cached descriptors and also discards circuits.
You?ll need to update control-spec. I?m curious what would happen if you use this command on a HSDir or for a local service - those cases need to be defined.
Hmm, good point. The behaviour will be similar to SIGNAL NEWNYM but only affecting a single service. I'll look into the details.
I wonder what problem you?re trying to solve, and if this is the best solution. From a quick reading of the code tor?s behavior is:
1. If an intro point remains, try it 2. If we run out of intro points, re-fetch the descriptor 3. If we?ve tried fetching the descriptor from all HSDirs within the past 15 minutes, fail 4. If the descriptor is re-fetched, try all intro points again
The list of HSDirs used in the past 15 minutes seems to be cleared when a connection succeeds, when we give up connecting to a service, or on rendezvous failures.
Even if you hit that 15-minute-wait case, it looks like another connection attempt will re-fetch the descriptor and start over. See rendclient.c:1065, leading to purge_hid_serv_from_last_hid_serv_requests. Can you point out where your client gets stalled, or where I?ve read the logic wrong?
I agree with your reading - this change is meant to avoid the first failed connection attempt in cases where we know it's likely to fail. Another way to approach the problem would be to make the retention time of the HS descriptor cache configurable. Then clients connecting to mobile hidden services could set a shorter retention time than the default. But that would affect all hidden services used by the client. What I'm trying to achieve with this patch is a way of saying "I know this specific descriptor is likely to be out-of-date, don't bother trying it". Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJUNQCRAAoJEBEET9GfxSfM49sIAKA8/m7PkC+GwgAe9zVYkcaP 540HFG73NFvtXK/z3Ww/prJsSwAJFSwXnKmYHLOp4upZ9E4+dtt82gOWorDxh9jR ubv3nGIFZ4D+90LS5O28OHAGwz/9zdx7533xRI58pXMQQpuWcqsfk9KECKWolRcY cEcLYXeUvnadbfR1HjEUp4RjXaw0KzXQwhXushs6bLTBUV29phhShDggHTYx/k4u 9FMHjLMn7neGnwvLyRUsaT9SCejB1HQJkpXGMwLKBB3B8VM8Gc1ZClISLz54qQQq x5hWiV11NI5sTNVLuDYuA7xj2SIyTXtJGEj9y2N1STV1smVW1GOKkSW5hyzVoX4= =LWzn -----END PGP SIGNATURE-----
On Tue, Oct 7, 2014 at 4:35 AM, Michael Rogers <michael@briarproject.org> wrote:
2. Added a command to the control protocol to purge any cached state relating to a specified hidden service. This command can be used before trying to connect to the service to force Tor to download a fresh descriptor. It's a more fine-grained alternative to SIGNAL NEWNYM, which purges all cached descriptors and also discards circuits.
I've only done small-scale testing of these patches so far. If they seems like they might be useful I'll create a trac ticket to merge them.
I've added a link to this thread to the below ticket. Somewhere therein I and Qingping were describing need and syntax to flush/fetch hidden service descriptors via control. For which your HS descriptor purge may be a useful component. If it is able to flush both single specified, or all, HS desc's that could be good semantic so as not to disturb other state. Thanks. # Allow controllers to retrieve HS descriptors from Tor https://trac.torproject.org/projects/tor/ticket/3521
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 08/10/14 08:34, grarpamp wrote:
I've added a link to this thread to the below ticket. Somewhere therein I and Qingping were describing need and syntax to flush/fetch hidden service descriptors via control. For which your HS descriptor purge may be a useful component. If it is able to flush both single specified, or all, HS desc's that could be good semantic so as not to disturb other state. Thanks.
# Allow controllers to retrieve HS descriptors from Tor https://trac.torproject.org/projects/tor/ticket/3521
Thanks! The patch only supports purging a single descriptor at present, but it would be easy enough to make the hostname optional and purge all descriptors if it's absent. Cheers, Michael -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBCAAGBQJUNQENAAoJEBEET9GfxSfMzNAIAJVrOc3qinqDu04niBFFcNI+ robNA9DHi7g7EF5sGBdoIuE1aO3JVx2mApNT+IVAQgH1iZSzRz/2jWCeeC12nDKQ dhkHD7fej02Zk+f3mCzf534/2/LFxyTCAPkPyE+GGhljrmExSBINo1u0KzedTwsu 0pDjld2rVTF/h0ktgxuzpVIhXlruQon8As1Tz3vOzCTIaIQxb8nwc6u6ShxhQmHT LnxRCcdChsIyFsafsn7rJDDMbnnNn5h751d0YwiELJNetb1l1W9lh8ekSuUCpqMp L4Vr8TZrwKuFHaRLroMXN94FoosuA9nRIt21iW5eQ6pfhV8wkqscA2zOha3w5Kg= =67Md -----END PGP SIGNATURE-----
On Wed, Oct 8, 2014 at 5:17 AM, Michael Rogers <michael@briarproject.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 08/10/14 08:34, grarpamp wrote:
I've added a link to this thread to the below ticket. Somewhere therein I and Qingping were describing need and syntax to flush/fetch hidden service descriptors via control. For which your HS descriptor purge may be a useful component. If it is able to flush both single specified, or all, HS desc's that could be good semantic so as not to disturb other state. Thanks.
# Allow controllers to retrieve HS descriptors from Tor https://trac.torproject.org/projects/tor/ticket/3521
Thanks! The patch only supports purging a single descriptor at present, but it would be easy enough to make the hostname optional and purge all descriptors if it's absent.
BTW, if you'd like submit this patch for inclusion in 0.2.6, you might want to open a new ticket for it. It doesn't seem completely subsumed by #3521 to me. At first glance, the basic idea seems plausible, though I haven't looked at the code too closely. (My workflow ensures that eventually we should get around to every patch with an open ticket against a release, but it's not very good at capturing things from tor-dev.) many thanks for the patch, and best wishes, -- Nick
participants (4)
-
grarpamp -
John Brooks -
Michael Rogers -
Nick Mathewson