[tor-bugs] #19420 [Metrics/Onionoo]: No AS data for some relays

[Tor Bug Tracker & Wiki]

#19420: No AS data for some relays
-----------------------------+------------------------------
 Reporter:  twim             |          Owner:  karsten
     Type:  defect           |         Status:  needs_review
 Priority:  Medium           |      Milestone:
Component:  Metrics/Onionoo  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:  as, asn, geoip   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------

Comment (by karsten):

 Replying to [comment:3 twim]:
 > Thanks for looking, karsten!
 >
 > > Would that be better?
 >
 > For me it looks like we definelety should include all that we have, i.e.
 ASNs even without better-to-have AS names. Downgrade just will bring
 another errors.

 Agreed.  If I don't hear any other concerns, I'll merge the patch and
 deploy it tomorrow (Friday).

 > IMHO, in the long-term we should avoid using GeoIP databases (like
 MaxMind's) and do vanilla IP to ASN mapping. And thus avoid mysteriuos
 latitude/longtitude/radius/city fields. Proably something like this [1].
 We have to have more precise and recent AS mapping (especially for
 little-t-tor).

 We're currently unable to produce our own database, so we'll have to rely
 on some third party to do that for us.  Researching alternatives is
 certainly an option, but it's not a priority right now.  I might change my
 mind about that if MaxMind continues to screw up with the ASN database or
 screws up with the country/city databases in the near future.

 There's also #19118 if you're curious.  Also not a priority right now,
 sadly.

 > Anyway tracking MaxMind's layout change is a way to nowhere.

 In this case I don't think they changed their data format but rather that
 they broke something that still produces the same data format as before.

 > [1] https://quaxio.com/bgp/
 >
 > ---
 > FYI have a look at your battle with GeoIP:
 > `$ git shortlog --grep "MaxMind\|GeoIP"`
 > {{{
 > Karsten Loesing (14):
 >       Use recent GeoIP database without A1 entries.
 >       Simplify GeoIP cleanup code, update to May files.
 >       Extract GeoIP lookup code and test it.
 >       MaxMind's GeoIP files use ISO-8859-1, not UTF-8.
 >       Merge two writer classes to speed up rDNS lookups.
 >       Switch to using MaxMind's GeoLite2 city database.
 >       Add unit tests for new GeoIP2 code, and fix a bug.
 >       Move front-end parts of NodeStatus to SummaryDocument.
 >       Handle UTF-8 characters in GeoIP lookup results.
 >       Adapt to MaxMind's new GeoLite2 City format.
 >       Fix character encoding when reading GeoIP files.
 >       Add more tests for UTF-8 characters in GeoIP files.
 >       Fix character encoding of ASN database file.
 >       Support additional columns in GeoLite2 files.
 > }}}

 Haha, nice.  Now you know why I'm not too keen to repeat that battle with
 another GeoIP/ASN data provider.  And honestly, I expect that to be a
 similar battle.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/19420#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online