stem library: huge difference between resolvers "proc" and others, eg. "netstat"
I do wonder about the differences of "proc" versus the other 3 ("netstat, "lsof" and "ss") related to the Inbound/Outbound values at my Tor relay. As an example I copied below the output of "proc" and "netstat". Does anybody have a clue about those differences? mr-fox ~ # python ~/stem/docs/_static/example/relay_connections.py --ctrlport 9051 --resolver proc 0.3.4.0-alpha-dev uptime: 06:48:29 flags: Exit, Fast, Guard, Running, Stable, V2Dir, Valid +------------------------------+------+------+ | Type | IPv4 | IPv6 | +------------------------------+------+------+ | Inbound to our ORPort | 2107 | 6 | | Inbound to our DirPort | 15 | 0 | | Outbound to a relay | 1753 | 0 | | Outbound exit traffic | 155 | 302 | | Outbound uncategorized | 119 | 0 | +------------------------------+------+------+ | Total | 4149 | 308 | +------------------------------+------+------+ +------------------------------+------+------+ | Exit Port | IPv4 | IPv6 | +------------------------------+------+------+ | 80 (HTTP) | 111 | 61 | | 143 (IMAP) | 1 | 0 | | 443 (HTTPS) | 0 | 229 | | 465 (SMTP) | 1 | 0 | | 993 (IMAPS) | 13 | 0 | | 5222 (Jabber) | 8 | 2 | | 5223 (Jabber) | 6 | 0 | | 7777 | 1 | 0 | | 8080 (HTTP Proxy) | 4 | 4 | | 8333 (Bitcoin) | 2 | 2 | | 8888 (NewsEDGE) | 2 | 0 | | 50002 (Electrum Bitcoin SSL) | 6 | 4 | +------------------------------+------+------+ | Total | 155 | 302 | +------------------------------+------+------+ mr-fox ~ # python ~/stem/docs/_static/example/relay_connections.py --ctrlport 9051 --resolver netstat 0.3.4.0-alpha-dev uptime: 06:48:41 flags: Exit, Fast, Guard, Running, Stable, V2Dir, Valid +------------------------------+------+------+ | Type | IPv4 | IPv6 | +------------------------------+------+------+ | Inbound to our ORPort | 4883 | 6 | | Inbound to our DirPort | 35 | 0 | | Inbound to our ControlPort | 1 | 0 | | Outbound to a relay | 4174 | 0 | | Outbound exit traffic | 390 | 293 | | Outbound uncategorized | 267 | 0 | +------------------------------+------+------+ | Total | 9750 | 299 | +------------------------------+------+------+ +------------------------------+------+------+ | Exit Port | IPv4 | IPv6 | +------------------------------+------+------+ | 80 (HTTP) | 300 | 68 | | 110 (POP3) | 1 | 0 | | 143 (IMAP) | 4 | 0 | | 443 (HTTPS) | 0 | 214 | | 465 (SMTP) | 1 | 0 | | 1883 | 1 | 0 | | 3128 (SQUID) | 1 | 0 | | 5222 (Jabber) | 21 | 2 | | 5223 (Jabber) | 18 | 0 | | 6667 (IRC) | 4 | 0 | | 7777 | 2 | 0 | | 8080 (HTTP Proxy) | 14 | 3 | | 8333 (Bitcoin) | 8 | 2 | | 8888 (NewsEDGE) | 3 | 0 | | 50002 (Electrum Bitcoin SSL) | 12 | 4 | +------------------------------+------+------+ | Total | 390 | 293 | +------------------------------+------+------+ -- Toralf PGP C4EACDDE 0076E94E
Hi Toralf. Unfortunately I don't know offhand, though I'd certainly be curious if you find the answer. At the end of the day connection resolvers use the proc contents, so this might be a bug in how I'm filtering connections in Stem or maybe a bug with its underlying proc resolver. Unfortunately I can't troubleshoot this without a local repro. If you'd care to dig in I'd suggest adjusting the script a little to print the connections, then see in what way netstat differs from proc. Is it a strict superset? Does it have duplicates? Anything about the connections which differ that seem interesting? Cheers! -Damian On Wed, Mar 7, 2018 at 2:25 PM, Toralf Förster <toralf.foerster@gmx.de> wrote:
I do wonder about the differences of "proc" versus the other 3 ("netstat, "lsof" and "ss") related to the Inbound/Outbound values at my Tor relay. As an example I copied below the output of "proc" and "netstat". Does anybody have a clue about those differences?
mr-fox ~ # python ~/stem/docs/_static/example/relay_connections.py --ctrlport 9051 --resolver proc 0.3.4.0-alpha-dev uptime: 06:48:29 flags: Exit, Fast, Guard, Running, Stable, V2Dir, Valid
+------------------------------+------+------+ | Type | IPv4 | IPv6 | +------------------------------+------+------+ | Inbound to our ORPort | 2107 | 6 | | Inbound to our DirPort | 15 | 0 | | Outbound to a relay | 1753 | 0 | | Outbound exit traffic | 155 | 302 | | Outbound uncategorized | 119 | 0 | +------------------------------+------+------+ | Total | 4149 | 308 | +------------------------------+------+------+
+------------------------------+------+------+ | Exit Port | IPv4 | IPv6 | +------------------------------+------+------+ | 80 (HTTP) | 111 | 61 | | 143 (IMAP) | 1 | 0 | | 443 (HTTPS) | 0 | 229 | | 465 (SMTP) | 1 | 0 | | 993 (IMAPS) | 13 | 0 | | 5222 (Jabber) | 8 | 2 | | 5223 (Jabber) | 6 | 0 | | 7777 | 1 | 0 | | 8080 (HTTP Proxy) | 4 | 4 | | 8333 (Bitcoin) | 2 | 2 | | 8888 (NewsEDGE) | 2 | 0 | | 50002 (Electrum Bitcoin SSL) | 6 | 4 | +------------------------------+------+------+ | Total | 155 | 302 | +------------------------------+------+------+
mr-fox ~ # python ~/stem/docs/_static/example/relay_connections.py --ctrlport 9051 --resolver netstat 0.3.4.0-alpha-dev uptime: 06:48:41 flags: Exit, Fast, Guard, Running, Stable, V2Dir, Valid
+------------------------------+------+------+ | Type | IPv4 | IPv6 | +------------------------------+------+------+ | Inbound to our ORPort | 4883 | 6 | | Inbound to our DirPort | 35 | 0 | | Inbound to our ControlPort | 1 | 0 | | Outbound to a relay | 4174 | 0 | | Outbound exit traffic | 390 | 293 | | Outbound uncategorized | 267 | 0 | +------------------------------+------+------+ | Total | 9750 | 299 | +------------------------------+------+------+
+------------------------------+------+------+ | Exit Port | IPv4 | IPv6 | +------------------------------+------+------+ | 80 (HTTP) | 300 | 68 | | 110 (POP3) | 1 | 0 | | 143 (IMAP) | 4 | 0 | | 443 (HTTPS) | 0 | 214 | | 465 (SMTP) | 1 | 0 | | 1883 | 1 | 0 | | 3128 (SQUID) | 1 | 0 | | 5222 (Jabber) | 21 | 2 | | 5223 (Jabber) | 18 | 0 | | 6667 (IRC) | 4 | 0 | | 7777 | 2 | 0 | | 8080 (HTTP Proxy) | 14 | 3 | | 8333 (Bitcoin) | 8 | 2 | | 8888 (NewsEDGE) | 3 | 0 | | 50002 (Electrum Bitcoin SSL) | 12 | 4 | +------------------------------+------+------+ | Total | 390 | 293 | +------------------------------+------+------+
-- Toralf PGP C4EACDDE 0076E94E
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 03/10/2018 10:39 AM, Toralf Förster wrote:
Will try to find the time - FWIW it seems just to be an IPv4 issue - the IPv6 numbers are roughly the same. And I should mention (again?) that I do run 2 exit relay at the same ip adress, just with different ports.
-- Toralf PGP C4EACDDE 0076E94E
On 03/10/2018 01:05 AM, Damian Johnson wrote:
Unfortunately I can't troubleshoot this without a local repro. If you'd care to dig in I'd suggest adjusting the script a little to print the connections, then see in what way netstat differs from proc. Is it a strict superset? Does it have duplicates? Anything about the connections which differ that seem interesting?
Cheers! -Damian
Well, comparing the results of "proc" and "netstat" via print ("debug: %s:%i\t%s:%i" % (laddr, lport, raddr, rport) )) shows these things : 1. both have few duplicate lines (2-3 for netstat, about 6 for proc) 2. proc expands the ipv6 addresses, eg.: 2a01:4f8:190:514a::2 into 2a01:04f8:0190:514a:0000:0000:0000:0002 3. the resolver netstat have significant lines more w/o any patterns why 4. proc is the only resolver which differs, lsof, netstat and ss have comparable results 5. the output of netstat is nearly identical to "na -n | wc -l" -- Toralf PGP C4EACDDE 0076E94E
participants (2)
-
Damian Johnson -
Toralf Förster