Please help us find a working alternative to using MaxMind's GeoLite2 databases
Hi relay operators, as you might have heard, MaxMind has changed access and use of their GeoLite2 databases: https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi... This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket: https://trac.torproject.org/projects/tor/ticket/32978 Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is. I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database. The result is that 7669 relays (93%) had the same country code and ASN. I put the remaining 7% on the following wiki page: https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison I'd like to hear from you which data source is right and which is wrong (or if both are wrong). If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020. Thanks for helping! All the best, Karsten
Sorry for the reply here. I am in an endless cycle of: Trac thinks your submission might be Spam. To prove otherwise please provide a response to the following. and capture solving does not really work. Checking my relays its hard to tell what is correct. As a German IXC I do own the IPs but the servers are in the Netherlands. If you prefer the ownership of the IPs Maxmind is correct with Germany, if you prefer where the actual servers are the new one is correct. Pick your poison :) At least the new one has my new AS, so I am a happy bunny after all. niftybunny
On 12. Feb 2020, at 16:05, Karsten Loesing <karsten@torproject.org> wrote:
Signed PGP part Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping!
All the best, Karsten
Hi,
On 13 Feb 2020, at 03:14, niftybunny <abuse-contact@to-surf-and-protect.net> wrote:
Checking my relays its hard to tell what is correct. As a German IXC I do own the IPs but the servers are in the Netherlands. If you prefer the ownership of the IPs Maxmind is correct with Germany, if you prefer where the actual servers are the new one is correct. Pick your poison :) At least the new one has my new AS, so I am a happy bunny after all.
There are often at least 2 correct answers for GeoIP, sometimes more: * the physical location of the server, * the business address of the data centre operator, * the business address of the owner or reseller of the server. Some servers and businesses have layers of owner/operator/reseller/ corporate subsidiaries, so it can get very complicated. I once operated relays that were obviously in Australia (because of the traceroute and latency), but were geolocated to the US, because their reseller was in the US. T
Teor, yeah, I have seen my /25 located in Germany, Netherlands and UK. Really depends how you see it. Karsten, all my 78 relays are perfectly located where the physical servers are. Hope this helps. Cheers, niftybunny
On 12. Feb 2020, at 23:46, teor <teor@riseup.net> wrote:
Signed PGP part Hi,
On 13 Feb 2020, at 03:14, niftybunny <abuse-contact@to-surf-and-protect.net> wrote:
Checking my relays its hard to tell what is correct. As a German IXC I do own the IPs but the servers are in the Netherlands. If you prefer the ownership of the IPs Maxmind is correct with Germany, if you prefer where the actual servers are the new one is correct. Pick your poison :) At least the new one has my new AS, so I am a happy bunny after all.
There are often at least 2 correct answers for GeoIP, sometimes more: * the physical location of the server, * the business address of the data centre operator, * the business address of the owner or reseller of the server.
Some servers and businesses have layers of owner/operator/reseller/ corporate subsidiaries, so it can get very complicated.
I once operated relays that were obviously in Australia (because of the traceroute and latency), but were geolocated to the US, because their reseller was in the US.
T
since https://trac.torproject.org/projects/tor/ticket/26585 didn't make any progress since a long time I'm glad to see planed improvements in this area.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
some of the relays in that table actually changed their ASN location between the last maxmind update (Oct 2019) and now (all ASMK relays). -- https://mastodon.social/@nusenu
Hi Karsten For the RIPE Region (the resources from RIPE) the correct source is the RIPE Database. There is an "country" attribute on the inetnum and inetnum6 objects. Last time I checked maxmind, it was completely wrong about 185.220.100.0/22. I reported to them, but didn't get any response. And it looks as they didn't update since then. I will check your list for the 185.220.100.0/22 prefixes and report in the ticket, later. Tim Am 12. Februar 2020 16:05:04 MEZ schrieb Karsten Loesing <karsten@torproject.org>:
Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping!
All the best, Karsten
On 2020-02-12 16:05, Karsten Loesing wrote:
Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping!
Hello again, I'm bumping this thread, because there's (finally) news on finding an alternative to MaxMind: https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978 (Yes, you'll have to scroll down quite a bit to find the news, it's a long ticket.) Can relay operators please take a look at that ticket and see if they find their relay in the table with different country results? It would be neat to identify patterns there and possibly improve the accuracy of the new database. Thanks for your help! All the best, Karsten
On Tue, 7 Jul 2020 at 10:20, Karsten Loesing <karsten@torproject.org> wrote:
On 2020-02-12 16:05, Karsten Loesing wrote:
Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping!
Hello again,
I'm bumping this thread, because there's (finally) news on finding an alternative to MaxMind:
https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978
(Yes, you'll have to scroll down quite a bit to find the news, it's a long ticket.)
Can relay operators please take a look at that ticket and see if they find their relay in the table with different country results?
It would be neat to identify patterns there and possibly improve the accuracy of the new database.
Thanks for your help!
I run 3 relays with Scaleway, one in Paris and 2 in Amsterdam, the new alternative places them all in France. whois correctly describes the range as being in Amsterdam: % Information related to '51.15.0.0/17AS12876' route: 51.15.0.0/17 descr: SCALEWAY descr: Amsterdam, Netherlands origin: AS12876 mnt-by: MNT-TISCALIFR created: 2019-10-03T15:11:06Z last-modified: 2019-10-03T15:11:06Z source: RIPE % This query was served by the RIPE Database Query Service version 1.97.2 (WAGYU)
Hi Karsten I'm the operator of bauruine51 (185.204.1.239). Finland is the correct location and not Russia. Other relays with the same hoster which are also wrong: - SingularET - AutoNoMe - whatevr8 - BlackHall The nodes that are FI in MaxMind and DE in your alternative are mostly (all?) Hetzner Finland servers. e.g. Ranlvor and HORUS1. Best regards, Stefan On 07.07.20 11:19, Karsten Loesing wrote:
On 2020-02-12 16:05, Karsten Loesing wrote:
Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping! Hello again,
I'm bumping this thread, because there's (finally) news on finding an alternative to MaxMind:
https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978
(Yes, you'll have to scroll down quite a bit to find the news, it's a long ticket.)
Can relay operators please take a look at that ticket and see if they find their relay in the table with different country results?
It would be neat to identify patterns there and possibly improve the accuracy of the new database.
Thanks for your help!
All the best, Karsten
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi Karsten, I am the operator of relays "torturing" and "BurningMan". The right country for torturing is Netherlands and for BurningMan Finland. Cheers Am 07.07.2020 um 11:19 schrieb Karsten Loesing:
On 2020-02-12 16:05, Karsten Loesing wrote:
Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping! Hello again,
I'm bumping this thread, because there's (finally) news on finding an alternative to MaxMind:
https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978
(Yes, you'll have to scroll down quite a bit to find the news, it's a long ticket.)
Can relay operators please take a look at that ticket and see if they find their relay in the table with different country results?
It would be neat to identify patterns there and possibly improve the accuracy of the new database.
Thanks for your help!
All the best, Karsten
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
My relay (912709C7F560CC3E9FE9604B58D598B1BFAED8C7) is not on the list. The Metrics pages says it’s in the UK, but actually it’s located in Germany. I guess the new alternative keeps showing it wrongly in the UK. -- fl4co
Il giorno 7 lug 2020, alle ore 11:19, Karsten Loesing <karsten@torproject.org> ha scritto:
On 2020-02-12 16:05, Karsten Loesing wrote:
Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping!
Hello again,
I'm bumping this thread, because there's (finally) news on finding an alternative to MaxMind:
https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978 <https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978>
(Yes, you'll have to scroll down quite a bit to find the news, it's a long ticket.)
Can relay operators please take a look at that ticket and see if they find their relay in the table with different country results?
It would be neat to identify patterns there and possibly improve the accuracy of the new database.
Thanks for your help!
All the best, Karsten
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays>
My relays: * themerrythoughts is not on the list * thesistersofmercy is neither on the list * joydivision is located in RU, but is in UA * projectpitchfork is not on the list * thecure is not on the list * bauhaus is not on the list * clanofxymox is not on the list lg Sebastian On 07/07/2020 11:19, Karsten Loesing wrote:
On 2020-02-12 16:05, Karsten Loesing wrote:
Hi relay operators,
as you might have heard, MaxMind has changed access and use of their GeoLite2 databases:
https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-usi...
This affects Onionoo and tor, and I'm trying to find a working alternative on the following ticket:
https://trac.torproject.org/projects/tor/ticket/32978
Today I think I found a possible alternative by using data from another provider. But before I name it here, I'd first want to find out how accurate it is.
I tried resolving relay IP addresses of relays that have been running in the past week and compared that to our existing lookups using MaxMind's October database.
The result is that 7669 relays (93%) had the same country code and ASN.
I put the remaining 7% on the following wiki page:
https://trac.torproject.org/projects/tor/wiki/doc/MetricsGeoipComparison
I'd like to hear from you which data source is right and which is wrong (or if both are wrong).
If you'd like to help, please leave comments on the ticket or in the "Comment" column on that wiki page by February 19, 2020.
Thanks for helping!
Hello again,
I'm bumping this thread, because there's (finally) news on finding an alternative to MaxMind:
https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978
(Yes, you'll have to scroll down quite a bit to find the news, it's a long ticket.)
Can relay operators please take a look at that ticket and see if they find their relay in the table with different country results?
It would be neat to identify patterns there and possibly improve the accuracy of the new database.
Thanks for your help!
All the best, Karsten
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- Sebastian Elisa Pfeifer Wehlistrasse 40/2/1 1200 Wien PGP Key ID: 0x923719A753677B95 https://sebastian-elisa-pfeifer.eu Phone (AT): +43 1 22 600 45 Phone (DE): +49 221 596 19 4570 Mobile: +43 699 195 79 087 Fax: +43 1 22 600 45 - 10
On Wed, 2020-07-08 at 20:38 +0200, Sebastian Elisa Pfeifer wrote:
* joydivision is located in RU, but is in UA
Not sure if it's me not being a native English speaker or you being a bit ambiguous, but I hope this makes it a bit more clear: joydivision is incorrectly listed as being in RU. It is actually in UA. (please correct me if this is wrong) Imre
On 08/07/2020 22:53, Imre Jonk wrote:
On Wed, 2020-07-08 at 20:38 +0200, Sebastian Elisa Pfeifer wrote:
* joydivision is located in RU, but is in UA
Not sure if it's me not being a native English speaker or you being a bit ambiguous, but I hope this makes it a bit more clear:
joydivision is incorrectly listed as being in RU. It is actually in UA.
I think the problem is that I was really tired, haha. I wanted to say: "joydivision is listed as being in RU, but is actually in UA" as you said! I have made a list: https://sebastian-elisa-pfeifer.eu/tor.html Lg
On 07.07.2020 11:19, Karsten Loesing wrote:
https://gitlab.torproject.org/tpo/metrics/trac/-/issues/32978
MaxMind is wrong (gb) Alternative 2 is correct (de) A751D8BE TorOrDie4privacyNET gb de Not listet on issues/32978 list. But they are wrong on metrics. Both servers are in (lu) Roost Luxembourg. Provider FranTech/BuyVM is in United States (LA & NY) 5D84900DBE6D6365684A9675B81A68ACE9577A68 376DC7CAD597D3A4CBB651999CFAD0E77DC9AE8C Where does https://torstatus.rueckgr.at get the country codes from? My are all right there. -- ╰_╯ Ciao Marco! Debian GNU/Linux It's free software and it gives you freedom!
participants (12)
-
fl4co -
Imre Jonk -
Karsten Loesing -
lists＠for-privacy.net -
niftybunny -
nusenu -
Pascal Terjan -
Sebastian Elisa Pfeifer -
Stefan Spühler -
teor -
Tim Niemeyer -
Tor-abuse