Hey folks! My relay which is currently hibernating shows something strange. Hibernation is set to 19 TB’s of outgoing Traffic. Hetzner Cloud shows ~16TB outgoing traffic and the relays log itself 38TB. vnstat reports exactly 20.38 TiB tx so in conclusion Hetzner can not count there Bits and Bytes or the same as protection is not correct. Because if Hetzner’s stats are correct i would have send 3 TB’s of Traffic to other Tor servers and clients internally. Because internal traffic is not counted. I guess that a Juniper routers net flow tool is correct so how can i look at this? Because my second relay in Helsinki does have this too with a difference of 2 TB’s. "Heartbeat: Tor's uptime is 27 days 1:39 hours, with 0 circuits open. I've sent 38486.76 GB and received 38489.42 GB. We are currently hibernating.“ When i divide that by two this gives me 19 TB’s of outgoing Traffic. So i guess that this is a bug? Thanks! Tobias
On Sat, Jul 28, 2018 at 07:59:08PM +0200, Tobias Sachs wrote:
Hibernation is set to 19 TB???s of outgoing Traffic. Hetzner Cloud shows ~16TB outgoing traffic and the relays log itself 38TB.
Can you give us your actual torrc lines? Also, how about the log lines, particularly the ones that talk about when it woke up, when it went into hibernation, when it plans to wake up again, etc?
vnstat reports exactly 20.38 TiB tx so in conclusion Hetzner can not count there Bits and Bytes or the same as protection is not correct.
Maybe some of these transmitted bytes went to other Hetzner servers, which they don't count as "really" sending bytes to the Internet.
Because if Hetzner???s stats are correct i would have send 3 TB???s of Traffic to other Tor servers and clients internally. Because internal traffic is not counted.
Ah, yes, this is what you suggested. Maybe? That's a lot (a large fraction) given that Hetzner doesn't have that large a fraction of the Tor network these days.
I guess that a Juniper routers net flow tool is correct so how can i look at this? Because my second relay in Helsinki does have this too with a difference of 2 TB???s.
"Heartbeat: Tor's uptime is 27 days 1:39 hours, with 0 circuits open. I've sent 38486.76 GB and received 38489.42 GB. We are currently hibernating.???
When i divide that by two this gives me 19 TB???s of outgoing Traffic. So i guess that this is a bug?
Tor said you sent 38TB, so that's 38TB of outgoing traffic. Maybe your 27 days contained two hibernation periods? That would be one possible explanation for 19+19=38. --Roger
Roger Dingledine:
On Sat, Jul 28, 2018 at 07:59:08PM +0200, Tobias Sachs wrote:
Hibernation is set to 19 TB???s of outgoing Traffic. Hetzner Cloud shows ~16TB outgoing traffic and the relays log itself 38TB.
Can you give us your actual torrc lines?
Also, how about the log lines, particularly the ones that talk about when it woke up, when it went into hibernation, when it plans to wake up again, etc?
vnstat reports exactly 20.38 TiB tx so in conclusion Hetzner can not count there Bits and Bytes or the same as protection is not correct.
Maybe some of these transmitted bytes went to other Hetzner servers, which they don't count as "really" sending bytes to the Internet.
Because if Hetzner???s stats are correct i would have send 3 TB???s of Traffic to other Tor servers and clients internally.
seems reasonable to me
Because
internal traffic is not counted.
Ah, yes, this is what you suggested. Maybe? That's a lot (a large fraction) given that Hetzner doesn't have that large a fraction of the Tor network these days.
Hetzner hosts > 7% of the tor network capacity (#3 on the biggest ASes on the tor network). If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock) -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
On 07/29/2018 12:45 AM, nusenu wrote:
Roger Dingledine:
On Sat, Jul 28, 2018 at 07:59:08PM +0200, Tobias Sachs wrote:
<SNIP>
Because internal traffic is not counted.
Ah, yes, this is what you suggested. Maybe? That's a lot (a large fraction) given that Hetzner doesn't have that large a fraction of the Tor network these days.
Hetzner hosts > 7% of the tor network capacity (#3 on the biggest ASes on the tor network).
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
It'd be better for relays to avoid connecting within an AS, right? Rather than just within /16. Or at least, if there were enough AS diversity. Which I'm guessing there isn't. So that would limit the number of possible circuits too much. Is that accurate?
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
It'd be better for relays to avoid connecting within an AS, right?
better according to what metric?
Rather than just within /16. Or at least, if there were enough AS diversity. Which I'm guessing there isn't. So that would limit the number of possible circuits too much. Is that accurate?
it will certainly limit the possible circuits a lot since the biggest 10 autonomous systems make up about 50% of the entire tor network. -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
On 07/29/2018 02:26 PM, nusenu wrote:
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
It'd be better for relays to avoid connecting within an AS, right?
better according to what metric?
Risk of coordinated compromise.
Rather than just within /16. Or at least, if there were enough AS diversity. Which I'm guessing there isn't. So that would limit the number of possible circuits too much. Is that accurate?
it will certainly limit the possible circuits a lot since the biggest 10 autonomous systems make up about 50% of the entire tor network.
Yeah, that was my vague recollection.
Mirimir:
On 07/29/2018 02:26 PM, nusenu wrote:
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
It'd be better for relays to avoid connecting within an AS, right?
better according to what metric?
Risk of coordinated compromise.
that is a very generic and short description -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
I did want to note one thing about these big ASes... sure, they may be big ASes, but they are still lacking in one major area - Exits. OVH has almost 4.5 Gbit/s of relay bandwidth available within the AS. However, if you search for exits, that rapidly drops to just under 750 mbit/s. I'm more than positive all of the other big ASes are the same way. A little off topic, but it just amazes me how much exit capacity these sites actually have, but people aren't willing to sign up for services whose TOS permits running an exit (or can't afford it), so they run a relay at an overly saturated site. Thanks, Conrad -- Conrad Rockenhaus Fingerprint: 8049 CDBA C385 C451 3348 776D 0F72 F2B5 26DA E93F Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x0F72F2B526DAE93F https://www.rockenhaus.com ------ Get started with GreyPony Anonymization Today! https://www.greyponyit.com -----Original Message----- From: tor-relays <tor-relays-bounces@lists.torproject.org> On Behalf Of nusenu Sent: Sunday, July 29, 2018 4:43 PM To: tor-relays@lists.torproject.org Subject: Re: [tor-relays] AS awareness Mirimir:
On 07/29/2018 02:26 PM, nusenu wrote:
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
It'd be better for relays to avoid connecting within an AS, right?
better according to what metric?
Risk of coordinated compromise.
that is a very generic and short description -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
...it just amazes me how much exit capacity these sites actually have, but people aren't willing to sign up for services whose TOS permits running an exit ..., so they run a relay at an overly saturated site.
Conrad
A couple of big European ones didn't notice exits were running in there data centres despite the ban on them so if you pay monthly it is worth a try. Rob
Hey folks! How about trying it with /8 and if that fails falling back to /16. Is it possible to set the limit to /14 /12 or maybe /10? And Conrad, the main concern is mostly like in my case to get kicked out while you still have production systems running or the worst case a police raid. Best Regards Tobias
Am 02.08.2018 um 08:05 schrieb <conrad@rockenhaus.com> <conrad@rockenhaus.com>:
I did want to note one thing about these big ASes... sure, they may be big ASes, but they are still lacking in one major area - Exits.
OVH has almost 4.5 Gbit/s of relay bandwidth available within the AS. However, if you search for exits, that rapidly drops to just under 750 mbit/s.
I'm more than positive all of the other big ASes are the same way.
A little off topic, but it just amazes me how much exit capacity these sites actually have, but people aren't willing to sign up for services whose TOS permits running an exit (or can't afford it), so they run a relay at an overly saturated site.
Thanks,
Conrad
-- Conrad Rockenhaus Fingerprint: 8049 CDBA C385 C451 3348 776D 0F72 F2B5 26DA E93F Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x0F72F2B526DAE93F https://www.rockenhaus.com ------ Get started with GreyPony Anonymization Today! https://www.greyponyit.com
-----Original Message----- From: tor-relays <tor-relays-bounces@lists.torproject.org> On Behalf Of nusenu Sent: Sunday, July 29, 2018 4:43 PM To: tor-relays@lists.torproject.org Subject: Re: [tor-relays] AS awareness
Mirimir:
On 07/29/2018 02:26 PM, nusenu wrote:
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
It'd be better for relays to avoid connecting within an AS, right?
better according to what metric?
Risk of coordinated compromise.
that is a very generic and short description
-- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Tobias Sachs:
How about trying it with /8 and if that fails falling back to /16.
Is it possible to set the limit to /14 /12 or maybe /10?
can you elaborate what you mean? and what you are trying to solve? thanks, nusenu -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
conrad@rockenhaus.com:
I did want to note one thing about these big ASes... sure, they may be big ASes, but they are still lacking in one major area - Exits.
we where talking about AS awareness in the context of traffic accounting for relay operators, feels like you jumped to another topic - namely overall AS diversity or AS diversity specifically for exit traffic? (which is fine, I just want to make sure I understood you correctly)
OVH has almost 4.5 Gbit/s of relay bandwidth available within the AS. However, if you search for exits, that rapidly drops to just under 750 mbit/s.
When talking about AS diversity, absolute numbers don't tell much unless you put them in context to other absolute numbers or you switch to overall fraction.
A little off topic, but it just amazes me how much exit capacity these sites actually have, but people aren't willing to sign up for services whose TOS permits running an exit (or can't afford it), so they run a relay at an overly saturated site.
there aren't many ASes that hold a bigger exit probability fraction than OVH SAS (AS16276) - currently just 2, so I wouldn't agree to "OVH is lacking exits when compared to other ASes" -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
This is my public keyfile. I know this doesn't relate to this message but I couldn't find an actual email address to send it to. I want to run a bridge relay but am having trouble setting it up. On 8/2/2018 at 6:05 AM, conrad@rockenhaus.com wrote:I did want to note one thing about these big ASes... sure, they may be big ASes, but they are still lacking in one major area - Exits. OVH has almost 4.5 Gbit/s of relay bandwidth available within the AS. However, if you search for exits, that rapidly drops to just under 750 mbit/s. I'm more than positive all of the other big ASes are the same way. A little off topic, but it just amazes me how much exit capacity these sites actually have, but people aren't willing to sign up for services whose TOS permits running an exit (or can't afford it), so they run a relay at an overly saturated site. Thanks, Conrad -- Conrad Rockenhaus Fingerprint: 8049 CDBA C385 C451 3348 776D 0F72 F2B5 26DA E93F Public Key: https://pgp.key-server.io/pks/lookup?op=get&search=0x0F72F2B526DAE93F https://www.rockenhaus.com ------ Get started with GreyPony Anonymization Today! https://www.greyponyit.com -----Original Message----- From: tor-relays On Behalf Of nusenu Sent: Sunday, July 29, 2018 4:43 PM To: tor-relays@lists.torproject.org Subject: Re: [tor-relays] AS awareness Mirimir:
On 07/29/2018 02:26 PM, nusenu wrote:
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
It'd be better for relays to avoid connecting within an AS, right?
better according to what metric?
Risk of coordinated compromise.
that is a very generic and short description -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 3 Aug 2018, at 06:26, Tony Peck <tonypeck@hushmail.com> wrote:
This is my public keyfile. I know this doesn't relate to this message but I couldn't find an actual email address to send it to. I want to run a bridge relay but am having trouble setting it up.
Try: https://trac.torproject.org/projects/tor/wiki/TorRelayGuide#Bridge T
Try:
https://trac.torproject.org/projects/tor/wiki/TorRelayGuide#Bridge
T
On that page they have "Fast exit relays (>=100 MBit/s)" and "MBit/s (Mbps)". MB means megabytes I would think. Rob
On Sun, Aug 05, 2018 at 07:39:49PM -0800, I wrote:
https://trac.torproject.org/projects/tor/wiki/TorRelayGuide#Bridge
On that page they have "Fast exit relays (>=100 MBit/s)" and "MBit/s (Mbps)". MB means megabytes I would think.
Well, you're in holy war territory there. The only answer I've found is to always say MBit or MByte, and never say MB because people will get excited and/or confused. And as far as I can tell the wiki page above already does it this way. --Roger
https://trac.torproject.org/projects/tor/wiki/TorRelayGuide#Bridge
On that page they have "Fast exit relays (>=100 MBit/s)" and "MBit/s (Mbps)". MB means megabytes I would think.
bits are not capitalized. It is Mbit/s or more simply Mbps. And tor is primarily a network tool interfacing with network providers and net hardware at network speeds, not a disk storage or memory testing or webhosting (ugh) app, so it's not Bytes or powers of two prefixes. People screw this stuff up all the time. The international standards docs with how to express things correctly have been posted to this list many times. Feel free to read them and then fix that page.
-----Original Message----- From: nusenu- changed https://trac.torproject.org/projects/tor/wiki/TorRelayGuide?action=diff&vers...
impressive but very rare
Hi!
Am 29.07.2018 um 00:51 schrieb Roger Dingledine <arma@mit.edu>:
On Sat, Jul 28, 2018 at 07:59:08PM +0200, Tobias Sachs wrote:
Hibernation is set to 19 TB???s of outgoing Traffic. Hetzner Cloud shows ~16TB outgoing traffic and the relays log itself 38TB.
Can you give us your actual torrc lines?
Log notice file /var/log/tor/notices.log RunAsDaemon 1 ORPort 443 ORPort [2a01:4f8:1c1c:af5::1]:443 Nickname GermanCraft3 RelayBandwidthRate 100 MBits RelayBandwidthBurst 200 MBits AccountingMax 19TB AccountingStart month 1 00:00 AccountingRule out ContactInfo 4096R/0x4cf76925833e2e24 Knight <knight AT germancraft dot net> - 1MTXtuSCCTf6J3TiUnk1ePwgaHt9h6uQaU DirPort 80 ExitPolicy reject *:* ExitPolicy reject6 *:*
Also, how about the log lines, particularly the ones that talk about
when it woke up, when it went into hibernation, when it plans to wake up again, etc?
Jul 29 06:25:01.000 [notice] Configured hibernation. This interval began at 2018-07-01 00:00:00; the scheduled wake-up time was 2018-07-01 07:27:40; we expect to exhaust our quota for this interval around 2018-07-31 23:32:40; the next interval begins at 2018-08-01 00:00:00 (all times local)
vnstat reports exactly 20.38 TiB tx so in conclusion Hetzner can not count there Bits and Bytes or the same as protection is not correct.
Maybe some of these transmitted bytes went to other Hetzner servers, which they don't count as "really" sending bytes to the Internet.
Because if Hetzner???s stats are correct i would have send 3 TB???s of Traffic to other Tor servers and clients internally. Because internal traffic is not counted.
Ah, yes, this is what you suggested. Maybe? That's a lot (a large fraction) given that Hetzner doesn't have that large a fraction of the Tor network these days.
I guess that a Juniper routers net flow tool is correct so how can i look at this? Because my second relay in Helsinki does have this too with a difference of 2 TB???s.
"Heartbeat: Tor's uptime is 27 days 1:39 hours, with 0 circuits open. I've sent 38486.76 GB and received 38489.42 GB. We are currently hibernating.???
When i divide that by two this gives me 19 TB???s of outgoing Traffic. So i guess that this is a bug?
Tor said you sent 38TB, so that's 38TB of outgoing traffic.
Maybe your 27 days contained two hibernation periods? That would be one possible explanation for 19+19=38.
—Roger
Hetzner hosts > 7% of the tor network capacity (#3 on the biggest ASes on the tor network).
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock)
So that means the same-as protection is a check that it is not the same /16 Block? That would make sense. https://bgp.he.net/AS24940#_prefixes <https://bgp.he.net/AS24940#_prefixes> Hetzner is holding 17 /16 and smaller netblocks. So it is possible that the complete route is within one AS? o.O The IP is: 159.69.2.239 Thanks for your help! Tobias
Tobias Sachs:
Hetzner hosts > 7% of the tor network capacity (#3 on the biggest ASes on the tor network).
their actual fraction might be even bigger because I used onionoo data to attribute relays to specific ASes and onionoo didn't switch to a better IP->AS number source yet (so some relays have missing AS level data like yours as you can see on Relay Search) https://trac.torproject.org/projects/tor/ticket/26585
If I know the relays IP I could give you the probabilities of your relay relaying traffic to others in the same AS (since a relay will usually not be used with others in the same /16 netblock) So that means the same-as protection is a check that it is not the same /16 Block? That would make sense.
I'm not sure what you mean by "same-as protection", do you want to elaborate?
Hetzner is holding 17 /16 and smaller netblocks. So it is possible that the complete route is within one AS? o.O
The (BGP) routing itself does not have any affect on the /16 netblock rule in tor.
The IP is: 159.69.2.239
In your /16 netblock (159.69.0.0/16) there are only 7 relays with a total of 0.169% consensus weight fraction (so the fraction of this should not really matter compared to the overall fraction of Hetzner) btw: please set a proper MyFamily on your relays https://metrics.torproject.org/rs.html#search/contact:0x4cf76925833e2e24 -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
On 07/28/2018 07:59 PM, Tobias Sachs wrote:
Hibernation is set to 19 TB’s of outgoing Traffic. Hetzner Cloud shows ~16TB outgoing traffic Hetzner doesn't bill outgoing traffic to other Hetzner servers, so relay-to-relay communication might be counted by Tor, but not by Hetzner IMO.
-- Toralf PGP C4EACDDE 0076E94E
participants (10)
-
conrad＠rockenhaus.com -
grarpamp -
I -
Mirimir -
nusenu -
Roger Dingledine -
teor -
Tobias Sachs -
Tony Peck -
Toralf Förster