I searched for the Snowflake bridge in Atlas, and couldn't find it. Its fingerprint is 2B280B23E1107BB62ABFC40DDCC8824814F80A72. Its torrc is stock "Last updated 9 October 2013 for Tor 0.2.5.2-alpha" except for these settings: ContactInfo David Fifield dcf@torproject.org SOCKSPort 0 ORPort 9001 BridgeRelay 1 ExtORPort auto ServerTransportPlugin snowflake exec /usr/local/bin/snowflake-server --acme-hostnames snowflake.bamsoftware.com --acme-email dcf@torproject.org --log /var/log/tor/snowflake-server.log ServerTransportListenAddr snowflake 0.0.0.0:443
Its ORPort 9001 is blocked by the local firewall, because it is meant to be only a Snowflake bridge, and not a vanilla bridge. (Most of the default Tor Browser obfs4 bridges are configured the same way, with their ORPort blocked.) There are these messages in the log (which I exxpected): [warn] Your server (37.218.242.151:9001) has not managed to confirm that its ORPort is reachable. Please check your firewalls, ports, address, /etc/hosts file, etc.
Why is the bridge not appearing in Atlas? I initially suspected https://bugs.torproject.org/18050, whose changelog entry is: - Check that both the ORPort and DirPort (if present) are reachable before publishing a relay descriptor. Otherwise, relays publish a descriptor with DirPort 0 when the DirPort reachability test takes longer than the ORPort reachability test. Closes bug #18050. Reported by "starlight", patch by "teor". Bugfix on 0.1.0.1-rc, commit a1f1fa6ab on 27 Feb 2005. But if it's the case that an unreachable ORPort causes descriptors not to be uploaded, then why do the default obfs4 bridges appear in Atlas? For example: https://atlas.torproject.org/#details/D9C805C955CB124D188C0D44F271E9BE57DE21... https://atlas.torproject.org/#details/D3D4A456FCB5F301F092F6A49ED671B84B432F... https://atlas.torproject.org/#details/11A3982C417AF37230F576006405BE5338F41C... Actually, now that I look at it, I notice some other default bridges are not present in Atlas, for example the two from https://bugs.torproject.org/21917, which went out in Tor Browser 6.5.2: C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4 0BAC39417268B96B9F514E7F63FA6FBA1A788955
What's going on and how can we fix it? You can find a list of default bridge fingerprints here: https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/Bundle-Da...
On Fri, May 05, 2017 at 04:30:52PM -0700, David Fifield wrote:
But if it's the case that an unreachable ORPort causes descriptors not to be uploaded, then why do the default obfs4 bridges appear in Atlas?
Tor relays (and bridges) test their reachability by making circuits that loop back to themselves, and they consider themselves reachable when an incoming connection sends a create cell (see the end of onionskin_answer()).
You might think that these two actions are more connected, i.e. that it needs to be one of the loop circuits that sends the create cell, but no, they're completely disconnected. So the relay (or bridge) can launch all the loop circuits it wants, and they can all fail, but if something causes an incoming connection that sends a create cell, it will happily conclude that it's reachable.
So it's likely that the reason the default bridges are publishing to the bridge authority is because somebody used them via obfs4, at which point they decided they were reachable, at which point they decided it was cool to publish.
You're right that this is a fragile situation. Maybe we should recommend that if you firewall your ORPort, you also set "AssumeReachable 1" in your torrc?
--Roger
On Sat, May 06, 2017 at 03:41:28AM -0400, Roger Dingledine wrote:
On Fri, May 05, 2017 at 04:30:52PM -0700, David Fifield wrote:
But if it's the case that an unreachable ORPort causes descriptors not to be uploaded, then why do the default obfs4 bridges appear in Atlas?
Tor relays (and bridges) test their reachability by making circuits that loop back to themselves, and they consider themselves reachable when an incoming connection sends a create cell (see the end of onionskin_answer()).
You might think that these two actions are more connected, i.e. that it needs to be one of the loop circuits that sends the create cell, but no, they're completely disconnected. So the relay (or bridge) can launch all the loop circuits it wants, and they can all fail, but if something causes an incoming connection that sends a create cell, it will happily conclude that it's reachable.
So it's likely that the reason the default bridges are publishing to the bridge authority is because somebody used them via obfs4, at which point they decided they were reachable, at which point they decided it was cool to publish.
Okay, thanks. It still doesn't fully make sense to me, because while some of the default bridges are in Atlas, not all of them are (for example the two from https://bugs.torproject.org/18050). I don't think it's possible that they haven't gotten *any* client traffic.
I wonder if it has something to do with the tor version number?
You're right that this is a fragile situation. Maybe we should recommend that if you firewall your ORPort, you also set "AssumeReachable 1" in your torrc?
I've just set "AssumeReachable 1"; let's see if that helps anything.
On Sat, May 06, 2017 at 09:25:11AM -0700, David Fifield wrote:
You're right that this is a fragile situation. Maybe we should recommend that if you firewall your ORPort, you also set "AssumeReachable 1" in your torrc?
I've just set "AssumeReachable 1"; let's see if that helps anything.
Setting "AssumeReachable 1" seems to have worked.
https://atlas.torproject.org/#details/5481936581E23D2D178105D44DB6915AB06BFB...
On Sat, May 06, 2017 at 09:25:11AM -0700, David Fifield wrote:
Okay, thanks. It still doesn't fully make sense to me, because while some of the default bridges are in Atlas, not all of them are (for example the two from https://bugs.torproject.org/21917). I don't think it's possible that they haven't gotten *any* client traffic.
I wonder if it has something to do with the tor version number?
I checked all the other default bridges and all of them but 2 are publishing statistics: C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4 cymrubridge31 0BAC39417268B96B9F514E7F63FA6FBA1A788955 cymrubridge33 I will contact the operators and ask them to try setting "AssumeReachable 1".
I suspect it has something to do with the Tor version number. Here are the other default bridges and their version numbers. The only old one is the last one, which is the one I had to set AssumeReachable on to get it to show up.
A09D536DD1752D542E1FBB3C9CE4449D51298239 LeifEricson obfs3,obfs4,fte Tor 0.3.1.0-alpha-dev on Linux 1E05F577A0EC0213F971D81BF4D86A9E4E8229ED ndnop0 obfs3 Tor 0.2.9.10 on Linux 4C331FA9B3D1D6D8FB0D8FBBF0C259C360D97E6A ndnop2 obfs3 Tor 0.2.9.10 on Linux AF9F66B7B04F8FF6F32D455F05135250A16543C9 Unnamed Tor 0.2.8.7 on Linux 0E858AC201BF0F3FA3C462F64844CBFFC7297A42 pdxtorbridge01 Tor 0.2.9.10 on Linux 1E326AAFB3FCB515015250D8FCCC8E37F91A153B wisctorbridge01 Tor 0.2.8.8 on Linux FC562097E1951DCC41B7D7F324D88157119BB56D wisctorbridge02 Tor 0.2.9.10 on Linux A17A40775FBD2CA1184BF80BFC330A77ECF9D0E9 wisctorbridge03 Tor 0.2.9.10 on Linux 8DFCD8FB3285E855F5A55EDDA35696C743ABFC4E ndnop3 obfs4 Tor 0.2.9.10 on Linux BBB28DF0F201E706BE564EFE690FE9577DD8386D ndnop5 obfs4 Tor 0.2.9.10 on Linux 752CF7825B3B9EA6A98C83AC41F7099D67007EA5 riemann obfs4 Tor 0.2.8.9 on Linux 7B126FAB960E5AC6A629C729434FF84FB5074EC2 noether obfs4 Tor 0.2.8.7 on Linux A832D176ECD5C7C6B58825AE22FC4C90FA249637 MaBishomarim obfs4 Tor 0.2.9.10 on Linux 8FB9F4319E89E5C6223052AA525A192AFBC85D55 Mosaddegh obfs4 Tor 0.2.9.10 on Linux 00DC6C4FA49A65BD1472993CF6730D54F11E0DBB JonbesheSabz obfs4 Tor 0.2.9.10 on Linux C73ADBAC8ADFDBF0FC0F3F4E8091C0107D093716 GreenBelt obfs4 Tor 0.2.9.8 on Linux FE7840FE1E21FE0A0639ED176EDA00A3ECA1E34D Azadi obfs4 Tor 0.2.9.10 on Linux CDF2E852BF539B82BD10E27E9115A31734E378C2 Lisbeth obfs4 Tor 0.2.9.10 on Linux FC259A04A328A07FED1413E9FC6526530D9FD87A NX01 obfs4 Tor 0.3.0.6 on Linux B9E7141C594AF25699E0079C1F0146F409495296 TorLandMeek meek Tor 0.2.9.10 on Linux 97700DFE9F483596DDA6264C4D7DF7641E1E39CE cymrubridge02 meek Tor 0.2.9.9 on Linux 2B280B23E1107BB62ABFC40DDCC8824814F80A72 Unnamed snowflake Tor 0.2.5.12 on Linux
appendix: commands to generate this data: grep -o -E '"\w+ [0-9.:]+ [0-9A-F]+' bridge_prefs.js | sed -e 's/.//' | while read transport addr fpr; do wget -c -O $fpr.json https://onionoo.torproject.org/details?fingerprint=$(echo $fpr | xxd -r -p | sha1sum | awk '{print $1}'); done for a in *.json; do echo -n "$a "; jq -j '.bridges[]|(.hashed_fingerprint,"\t",.nickname,"\t",(.transports//[]|join(",")),"\t",.platform)' $a; echo; done
On Wed, May 24, 2017 at 09:31:45PM -0700, David Fifield wrote:
On Sat, May 06, 2017 at 09:25:11AM -0700, David Fifield wrote:
Okay, thanks. It still doesn't fully make sense to me, because while some of the default bridges are in Atlas, not all of them are (for example the two from https://bugs.torproject.org/21917). I don't think it's possible that they haven't gotten *any* client traffic.
I wonder if it has something to do with the tor version number?
I checked all the other default bridges and all of them but 2 are publishing statistics: C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4 cymrubridge31 0BAC39417268B96B9F514E7F63FA6FBA1A788955 cymrubridge33 I will contact the operators and ask them to try setting "AssumeReachable 1".
The operator of these bridges set "AssumeReachable 1" and the bridges still are not publishing statistics ("No Tor relays or bridges matched your query :("): https://atlas.torproject.org/#details/C8CBDB2464FC9804A69531437BCF2BE31FDD2E... https://atlas.torproject.org/#details/0BAC39417268B96B9F514E7F63FA6FBA1A7889...
According to the operator, the bridges are running version 0.2.9.10, so my hypothesis about it having something to do with old version numbers is wrong.
Roger suspects that the reason the bridges haven't reported any statistics is because they haven't received any connections. That possibility seems unlikely to me, because they are default bridges shipping in Tor Browser, and additionally I have an hourly test running that makes an obfs4 connection and builds a circuit.
I tried downloading one of the bridge's bridge-extra-info descriptors (which would contain information about how many connections it has had), but I can't quite make it work. Using this torrc file: UseMicroDescriptors 0 DownloadExtraInfo 1 FetchUselessDescriptors 1 ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy ControlPort 9051 CookieAuthentication 1 UseBridges 1 Bridge obfs4 38.229.1.78:80 C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4 cert=Hmyfd2ev46gGY7NoVxA9ngrPF2zCZtzskRTzoWXbxNkzeVnGFPWmrTtILRyqCTjHR+s9dg iat-mode=1 And this Stem script: from stem.control import Controller with Controller.from_port(port = 9051) as controller: controller.authenticate() print controller.get_server_descriptor("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4") I can get the bridge-server-descriptor, but not the bridge-extra-info. Here are some lines from bridge-server-descriptor: platform Tor 0.2.9.10 on Linux proto Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4 LinkAuth=1 Microdesc=1-2 Relay=1-2 published 2017-06-01 21:19:10 fingerprint C8CB DB24 64FC 9804 A695 3143 7BCF 2BE3 1FDD 2EE4 uptime 613949 bandwidth 1073741824 1073741824 5628404 extra-info-digest 8B5F0BD647B3C4AF2C57F148FF6A1FB8B695B0AE 398ZHFBxUpTRfLxv+pSMY3BGMUYlzogXMG40dhjPgnA
(Furthermore, with the torrc shown above, tor doesn't save *any* extra-info descriptors, despite the presence of "DownloadExtraInfo 1" and "FetchUselessDescriptors 1". The datadir contains cached-certs, cached-consensus, and cached-descriptors, but no cached-extrainfo. Only after I change "UseBridges 1" to "UseBridges 0" does the cached-extrainfo file appear--but then of course it doesn't contain any information on bridges.)
On 2 Jun 2017, at 08:20, David Fifield david@bamsoftware.com wrote:
On Wed, May 24, 2017 at 09:31:45PM -0700, David Fifield wrote:
On Sat, May 06, 2017 at 09:25:11AM -0700, David Fifield wrote:
Okay, thanks. It still doesn't fully make sense to me, because while some of the default bridges are in Atlas, not all of them are (for example the two from https://bugs.torproject.org/21917). I don't think it's possible that they haven't gotten *any* client traffic.
I wonder if it has something to do with the tor version number?
I checked all the other default bridges and all of them but 2 are publishing statistics: C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4 cymrubridge31 0BAC39417268B96B9F514E7F63FA6FBA1A788955 cymrubridge33 I will contact the operators and ask them to try setting "AssumeReachable 1".
The operator of these bridges set "AssumeReachable 1" and the bridges still are not publishing statistics ("No Tor relays or bridges matched your query :("): https://atlas.torproject.org/#details/C8CBDB2464FC9804A69531437BCF2BE31FDD2E... https://atlas.torproject.org/#details/0BAC39417268B96B9F514E7F63FA6FBA1A7889...
According to the operator, the bridges are running version 0.2.9.10, so my hypothesis about it having something to do with old version numbers is wrong.
...
I tried downloading one of the bridge's bridge-extra-info descriptors (which would contain information about how many connections it has had), but I can't quite make it work. Using this torrc file: ... DownloadExtraInfo 1 FetchUselessDescriptors 1
These options only make tor clients fetch relay extra info descriptors.
But that's not really clear in the man page: https://www.torproject.org/docs/tor-manual.html.en
I logged this bug so we clarify that: https://trac.torproject.org/projects/tor/ticket/22491
... And this Stem script: from stem.control import Controller with Controller.from_port(port = 9051) as controller: controller.authenticate() print controller.get_server_descriptor("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4")
...
I can get the bridge-server-descriptor, but not the bridge-extra-info. Here are some lines from bridge-server-descriptor: platform Tor 0.2.9.10 on Linux proto Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4 LinkAuth=1 Microdesc=1-2 Relay=1-2 published 2017-06-01 21:19:10 fingerprint C8CB DB24 64FC 9804 A695 3143 7BCF 2BE3 1FDD 2EE4 uptime 613949 bandwidth 1073741824 1073741824 5628404 extra-info-digest 8B5F0BD647B3C4AF2C57F148FF6A1FB8B695B0AE 398ZHFBxUpTRfLxv+pSMY3BGMUYlzogXMG40dhjPgnA
You asked your bridge for its server descriptor and it gave it to you.
Did you try: print controller.get_extrainfo_descriptors("8B5F0BD647B3C4AF2C57F148FF6A1FB8B695B0AE")
https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot...
Since your bridge knows its own extra info descriptor, it should serve it to you.
I don't know how to find a server that caches *all* bridge extra infos. Maybe you should try running these queries against the bridge authority?
(How does OnionOO do it?)
(Furthermore, with the torrc shown above, tor doesn't save *any* extra-info descriptors, despite the presence of "DownloadExtraInfo 1" and "FetchUselessDescriptors 1".
This is because your bridge is your directory guard.
If you want to fetch relay extra infos through your bridge, you need to configure it to download and serve relay extra infos. I'm not sure how to do that (maybe DirCache 1?), so I opened this ticket: https://trac.torproject.org/projects/tor/ticket/22492
The datadir contains cached-certs, cached-consensus, and cached-descriptors, but no cached-extrainfo. Only after I change "UseBridges 1" to "UseBridges 0" does the cached-extrainfo file appear--but then of course it doesn't contain any information on bridges.)
That's because your client selects a directory mirror that caches extra infos for relays.
T -- Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
Thanks for your informative reply.
On Mon, Jun 05, 2017 at 02:37:00PM +1000, teor wrote:
On 2 Jun 2017, at 08:20, David Fifield david@bamsoftware.com wrote:
... And this Stem script: from stem.control import Controller with Controller.from_port(port = 9051) as controller: controller.authenticate() print controller.get_server_descriptor("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4")
...
I can get the bridge-server-descriptor, but not the bridge-extra-info. Here are some lines from bridge-server-descriptor: platform Tor 0.2.9.10 on Linux proto Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4 LinkAuth=1 Microdesc=1-2 Relay=1-2 published 2017-06-01 21:19:10 fingerprint C8CB DB24 64FC 9804 A695 3143 7BCF 2BE3 1FDD 2EE4 uptime 613949 bandwidth 1073741824 1073741824 5628404 extra-info-digest 8B5F0BD647B3C4AF2C57F148FF6A1FB8B695B0AE 398ZHFBxUpTRfLxv+pSMY3BGMUYlzogXMG40dhjPgnA
You asked your bridge for its server descriptor and it gave it to you.
Did you try: print controller.get_extrainfo_descriptors("8B5F0BD647B3C4AF2C57F148FF6A1FB8B695B0AE")
https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot...
Since your bridge knows its own extra info descriptor, it should serve it to you.
I don't know how to find a server that caches *all* bridge extra infos. Maybe you should try running these queries against the bridge authority?
get_extrainfo_descriptors doesn't work as a method of stem.control.Controller: AttributeError: 'Controller' object has no attribute 'get_extrainfo_descriptors'
Calling get_extrainfo_descriptors from stem.descriptor.remote returns an empty list. (499D92E08769BFC0B7941C74031335B9EC9E9BAE is the new extra-info-digest I got from running just now.) import stem.descriptor.remote print list(stem.descriptor.remote.get_authorities()) print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4")) print list(stem.descriptor.remote.get_extrainfo_descriptors("499D92E08769BFC0B7941C74031335B9EC9E9BAE")) This is the output. Bifroest is in the get_authorities list, so I assume it's querying the bridge authority somewhere in there. ['maatuska', 'tor26', 'Bifroest', 'longclaw', 'dizum', 'gabelmoo', 'moria1', 'dannenberg', 'Faravahar'] [] []
(How does OnionOO do it?)
I thought that Onionoo was getting extrainfos from Collector. I assumed that was the reason why Onionoo doesn't work for the two bridges in question, because they aren't in Collector.
(Furthermore, with the torrc shown above, tor doesn't save *any* extra-info descriptors, despite the presence of "DownloadExtraInfo 1" and "FetchUselessDescriptors 1".
This is because your bridge is your directory guard.
If you want to fetch relay extra infos through your bridge, you need to configure it to download and serve relay extra infos. I'm not sure how to do that (maybe DirCache 1?), so I opened this ticket: https://trac.torproject.org/projects/tor/ticket/22492
I'm not really interested in fetching extrainfos through the bridge. I was only trying to do that for the bridge itself, in order to see how much traffic the bridge is getting. I thought if Stem couldn't do it, I might be able to pull the answer out of tor's state directory. Actually I suspected that the bridge would not cache extrainfos for other bridges, but I thought it might at least return its own extrainfo.
What I'm trying to understand is why these two bridges are behaving differently than other default bridges, even though as far as I know they are configured the same way (obfs4 is the only open port, ORport is firewalled shut). I'm concerned that we are undercounting the number of obfs4 users while these bridges are not reporting statistics.
On 5 Jun 2017, at 15:06, David Fifield david@bamsoftware.com wrote:
You asked your bridge for its server descriptor and it gave it to you.
Did you try: print controller.get_extrainfo_descriptors("8B5F0BD647B3C4AF2C57F148FF6A1FB8B695B0AE")
https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot...
Since your bridge knows its own extra info descriptor, it should serve it to you.
I don't know how to find a server that caches *all* bridge extra infos. Maybe you should try running these queries against the bridge authority?
get_extrainfo_descriptors doesn't work as a method of stem.control.Controller: AttributeError: 'Controller' object has no attribute 'get_extrainfo_descriptors'
Calling get_extrainfo_descriptors from stem.descriptor.remote returns an empty list. (499D92E08769BFC0B7941C74031335B9EC9E9BAE is the new extra-info-digest I got from running just now.) import stem.descriptor.remote print list(stem.descriptor.remote.get_authorities()) print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4")) print list(stem.descriptor.remote.get_extrainfo_descriptors("499D92E08769BFC0B7941C74031335B9EC9E9BAE")) This is the output. Bifroest is in the get_authorities list, so I assume it's querying the bridge authority somewhere in there. ['maatuska', 'tor26', 'Bifroest', 'longclaw', 'dizum', 'gabelmoo', 'moria1', 'dannenberg', 'Faravahar'] [] []
I think the documentation at: https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot... is out of date: as far as I know, newer stem versions try fallback directories rather than authorities.
Even if that's not the case, stem will choose a random authority for you, which will (8/9 times) tell you that it knows nothing about any bridge extra infos. (I don't think authorities cache bridge extra infos.)
Try passing Bifroest's address and DirPort to the endpoints= argument to get_extrainfo_descriptors(): https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot...
T -- Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
On Mon, Jun 05, 2017 at 03:15:08PM +1000, teor wrote:
On 5 Jun 2017, at 15:06, David Fifield david@bamsoftware.com wrote:
Calling get_extrainfo_descriptors from stem.descriptor.remote returns an empty list. (499D92E08769BFC0B7941C74031335B9EC9E9BAE is the new extra-info-digest I got from running just now.) import stem.descriptor.remote print list(stem.descriptor.remote.get_authorities()) print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4")) print list(stem.descriptor.remote.get_extrainfo_descriptors("499D92E08769BFC0B7941C74031335B9EC9E9BAE")) This is the output. Bifroest is in the get_authorities list, so I assume it's querying the bridge authority somewhere in there. ['maatuska', 'tor26', 'Bifroest', 'longclaw', 'dizum', 'gabelmoo', 'moria1', 'dannenberg', 'Faravahar'] [] []
I think the documentation at: https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot... is out of date: as far as I know, newer stem versions try fallback directories rather than authorities.
Even if that's not the case, stem will choose a random authority for you, which will (8/9 times) tell you that it knows nothing about any bridge extra infos. (I don't think authorities cache bridge extra infos.)
Try passing Bifroest's address and DirPort to the endpoints= argument to get_extrainfo_descriptors(): https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot...
Thanks again for your suggestions. Passing endpoints= doesn't seem to do anything. >>> import stem.descriptor.remote >>> BIFROEST = ("37.218.247.217", 80) >>> print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4", endpoints=(BIFROEST,))) [] It also doesn't work when trying the fingerprint of another default bridge, or of one I just got from bridge.torproject.org. >>> print list(stem.descriptor.remote.get_server_descriptors("A09D536DD1752D542E1FBB3C9CE4449D51298239", endpoints=(BIFROEST,))) [] >>> print list(stem.descriptor.remote.get_server_descriptors("924E408BAE68C6C47A06BCDF6A44A53930708092", endpoints=(BIFROEST,))) [] I get the same output with and without the endpoints= argument, and whether or not I provide the SHA-1 hash of the fingerprint.
Strangely, this call seems to just hang: >>> print list(stem.descriptor.remote.get_extrainfo_descriptors("499D92E08769BFC0B7941C74031335B9EC9E9BAE", endpoints=(BIFROEST,)))
I tried some other things in Stem but none of them worked. >>> from stem.control import Controller >>> with Controller.from_port(port = 9051) as controller: >>> controller.authenticate() >>> print controller.get_server_descriptor("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4") router cymrubridge31 38.229.1.78 443 0 0... >>> print controller.get_info("desc/id/C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4") router cymrubridge31 38.229.1.78 443 0 0... >>> print controller.get_info("extra-info/digest/499D92E08769BFC0B7941C74031335B9EC9E9BAE") stem.InvalidArguments: GETINFO request contained unrecognized keywords: extra-info/digest/499D92E08769BFC0B7941C74031335B9EC9E9BAE
>>> import stem.connection >>> import stem.socket >>> control_socket = stem.socket.ControlPort(port = 9051) >>> stem.connection.authenticate(control_socket) >>> control_socket.send("GETINFO desc/id/C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4") >>> print control_socket.recv() desc/id/C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4= router cymrubridge31 38.229.1.78 443 0 0... OK >>> control_socket.send("GETINFO extra-info/digest/499D92E08769BFC0B7941C74031335B9EC9E9BAE") >>> print control_socket.recv() Unrecognized key "extra-info/digest/499D92E08769BFC0B7941C74031335B9EC9E9BAE"
On 5 Jun 2017, at 15:53, David Fifield david@bamsoftware.com wrote:
On Mon, Jun 05, 2017 at 03:15:08PM +1000, teor wrote:
On 5 Jun 2017, at 15:06, David Fifield david@bamsoftware.com wrote:
Calling get_extrainfo_descriptors from stem.descriptor.remote returns an empty list. (499D92E08769BFC0B7941C74031335B9EC9E9BAE is the new extra-info-digest I got from running just now.) import stem.descriptor.remote print list(stem.descriptor.remote.get_authorities()) print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4")) print list(stem.descriptor.remote.get_extrainfo_descriptors("499D92E08769BFC0B7941C74031335B9EC9E9BAE")) This is the output. Bifroest is in the get_authorities list, so I assume it's querying the bridge authority somewhere in there. ['maatuska', 'tor26', 'Bifroest', 'longclaw', 'dizum', 'gabelmoo', 'moria1', 'dannenberg', 'Faravahar'] [] []
I think the documentation at: https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot... is out of date: as far as I know, newer stem versions try fallback directories rather than authorities.
Even if that's not the case, stem will choose a random authority for you, which will (8/9 times) tell you that it knows nothing about any bridge extra infos. (I don't think authorities cache bridge extra infos.)
Try passing Bifroest's address and DirPort to the endpoints= argument to get_extrainfo_descriptors(): https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot...
Thanks again for your suggestions. Passing endpoints= doesn't seem to do anything.
import stem.descriptor.remote BIFROEST = ("37.218.247.217", 80) print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4", endpoints=(BIFROEST,)))
[] It also doesn't work when trying the fingerprint of another default bridge, or of one I just got from bridge.torproject.org. ...
At this point, you are deep in the internals of stem, the control spec, the dir spec, and their various implementations in Tor.
Atagar might be able to help with the stem side of things. I'd encourage you to open a ticket for the bridges that aren't reporting, and we'll try to reproduce and diagnose.
Can you get logs (and torrcs) from those bridges to confirm whether they think they are producing extra info descriptors?
T
-- Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
On Mon, Jun 05, 2017 at 11:51:04PM +1000, teor wrote:
Can you get logs (and torrcs) from those bridges to confirm whether they think they are producing extra info descriptors?
I've asked the operator but not gotten a reply yet.
You asked your bridge for its server descriptor and it gave it to you.
Did you try: print controller.get_extrainfo_descriptors("8B5F0BD647B3C4AF2C57F148FF6A1FB8B695B0AE")
https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot...
Hi teor, minor correction: you link to the DescriptorDownloader's method but cite it as being part of Controller. This is why David didn't find it later. The Controller can fetch some descriptor types. For instance...
https://stem.torproject.org/api/control.html#stem.control.Controller.get_ser... https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#can-i-g...
However, the Controller doesn't provide extrainfo descriptors. This is for a couple reasons...
1. The control protocol only allows it to be retrieved by the descriptor digest rather than relay fingerprint, which is a pita...
https://gitweb.torproject.org/torspec.git/tree/control-spec.txt#n562
2. This isn't what you want anyway. Fetching decriptors from the tor process only gets you the cached descriptors which isn't what folks generally want. So you're right that the DescriptorDownloader is the way to go.
I don't know how to find a server that caches *all* bridge extra infos. Maybe you should try running these queries against the bridge authority?
Bridge descriptors are not public like normal descriptors. To work of course the tor client can retrieve it if you know the bridge address but beyond that I think the only thing available is the sanitized descriptors from CollecTor...
https://collector.torproject.org/recent/bridge-descriptors/
I think the documentation at: https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remot... is out of date: as far as I know, newer stem versions try fallback directories rather than authorities.
Unfortunately it doesn't. I had to revert that because they lack extrainfo docs...
https://gitweb.torproject.org/stem.git/commit/?id=758f632
Thanks again for your suggestions. Passing endpoints= doesn't seem to do anything. >>> import stem.descriptor.remote >>> BIFROEST = ("37.218.247.217", 80) >>> print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4", endpoints=(BIFROEST,))) []
For what it's worth stem has authority information so this can also be done with...
import stem.descriptor.remote bifroest = stem.descriptor.remote.get_authorities()['Bifroest'] print list(stem.descriptor.remote.get_server_descriptors("C8CBDB2464FC9804A69531437BCF2BE31FDD2EE4", endpoints=((bifroest.address, bifroest.dir_port),)))
It also doesn't work when trying the fingerprint of another default bridge, or of one I just got from bridge.torproject.org.
Alas, this is getting into areas I'm not too familiar with. To prevent enumaration I'm sure bridges are handled specially.
Atagar might be able to help with the stem side of things.
Happy to help if there's any other stem questions.
Cheers! -Damian
-
Damian Johnson -
David Fifield -
Roger Dingledine -
teor