On 9 Nov 2015, at 18:56, Matthew Finkel matthew.finkel@gmail.com wrote:

Which version of Tor were you previously running? Have you seen these messages within the last few days, after you upgraded?

Hi SiNA, Matthew,

I can reproduce some unusual behaviour from Faravahar simply using wget. It seems that a web cache in front of Faravahar's dirport might be misbehaving.

*Exit Notice - Reproducible Incorrect X-Your-Address-Is*

I just retrieved the Faravahar "exit notice" from a US AWS address, then a few seconds later made a query from an Australian home IP address.

Both queries returned the US AWS address in X-Your-Address-Is. (A previous query from the Australian address a few minutes beforehand had the correct address.)

I can reproduce this behaviour: whichever query starts first is the one whose IP address gets recorded.

Subsequent queries get the same IP address for several tens of seconds afterwards.

I'm using: wget --save-headers http://154.35.175.225/

The headers I see look like:

HTTP/1.0 200 OK Age: 976 Date: Mon, 09 Nov 2015 08:35:20 GMT Expires: Mon, 09 Nov 2015 08:55:20 GMT Connection: Keep-Alive ETag: "KXJDGONEHLPTLKVYRY" Content-Type: text/html X-Your-Address-Is: <not my IP address> Content-Encoding: identity Content-Length: 38126

*Directory Documents - Once-Off Header Corruption*

This caching behaviour isn't reproducible with the actual documents, but I did see one instance of corrupted headers: * Content-Length is misspelt Xontent-Length, * Connection and ETag are present, but they're not typically present in the headers of other directory documents from Faravahar. * Expires is in the location it would typically be in for the exit notice (after Date), not the directory documents (last, after Content-Encoding)

wget --save-headers http://154.35.175.225/tor/status-vote/current/consensus-ns

HTTP/1.0 200 OK Age: 975 Date: Mon, 09 Nov 2015 08:43:32 GMT Expires: Mon, 09 Nov 2015 09:00:00 GMT Xontent-Length: Connection: Close ETag: "KXJDGONEHLSKRWTYRY" Content-Type: text/plain X-Your-Address-Is: 180.200.153.214 Content-Encoding: identity

The second download with exactly the same URL did not contain [C|X]ontent-Length, Connection, or ETag.

It's probably worth mentioning that similar queries to other directory authorities do not show the same behaviour.

Tim

Tim Wilson-Brown (teor)

teor2345 at gmail dot com PGP 968F094B

teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

On 9 Nov 2015, at 20:31, Matthew Finkel matthew.finkel@gmail.com wrote:

On Mon, Nov 09, 2015 at 08:04:55PM +1100, Tim Wilson-Brown - teor wrote:

...

...

On 9 Nov 2015, at 18:56, Matthew Finkel matthew.finkel@gmail.com wrote:

Which version of Tor were you previously running? Have you seen these messages within the last few days, after you upgraded?

Hi SiNA, Matthew,

I can reproduce some unusual behaviour from Faravahar simply using wget. It seems that a web cache in front of Faravahar's dirport might be misbehaving.

For some reason I can't reproduce your results, but I agree that what you saw and what the various tor log snippets show does seem to point towards a kind of frontend caching proxy. This is unfortunate, but I don't see a way for the dirauth code to make a mistake like this, and we'd expect other authories would show the same symptoms if that was the case, as well.

I am happy to provide files exhibiting these issues if needed.

Tim

Tim Wilson-Brown (teor)

teor2345 at gmail dot com PGP 968F094B

teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

On 9 Nov 2015, at 20:45, Roger Dingledine <arma@mit.edu mailto:arma@mit.edu> wrote:

On Mon, Nov 09, 2015 at 08:04:55PM +1100, Tim Wilson-Brown - teor wrote:

...

Subsequent queries get the same IP address for several tens of seconds afterwards.

Woah. Are we setting the Expires: http header in our Tor answer based on how long we think the *payload* will remain valid, and the proxy in the middle is caching both the payload and the headers for that time?

That's what most caches do with unknown headers: cache them for the same period as the payload.

HTTP 1.0 is silent on whether headers should be cached - it just says content should be cached. http://www.w3.org/Protocols/HTTP/1.0/spec.html#Expires

HTTP 1.1 specifies that headers are cached along with content, except for a few headers that don't get cached. (It also specifies that "not-modified" and similar responses replace headers from the original response. But I doubt that's relevant here.) https://tools.ietf.org/html/rfc7234

For HTTP 1.1 compliant caches, Tor can specifically indicate that X-Your-IP-Address-Is is a hop-by-hop header by using a the "Connection" header like this: Connection: close X-Your-IP-Address-Is ("close" indicates that Tor doesn't support persistent connections.) This would cause the X-Your-IP-Address-Is header to not ever be sent out by the cache, not even on the first connection. If Tor sees the IP address of the cache, this is what we want. But if it sees the IP address of the relay, we want the first response to have the X-Your-IP-Address-Is header, and any cached responses to not have it (see below). https://tools.ietf.org/html/rfc7230#section-6.1

Alternately, we could use: Cache-Control: no-cache="X-Your-IP-Address-Is" This is interpreted as a request not to cache the header at all, or, if the no-cache="<header-name>" feature is not supported, it's generally interpreted as a request not to cache the entire document. (This also causes the cache to attempt to revalidate the header, which might not be what we want, as Tor might not support cache revalidation.) https://tools.ietf.org/html/rfc7234#section-5.2.2

These are HTTP 1.1 features, but Tor provides a HTTP 1.0 response. HTTP 1.1 caches are supposed to handle headers by applying the HTTP 1.1 rules, regardless of the version of the response.

For HTTP 1.0 compliant caches, Tor can disable caching: Pragma: no-cache (This will also disable caching for HTTP 1.1 caches unless we provide a more generous Cache-Control header, like the one above.) https://tools.ietf.org/html/rfc7234#section-5.4

If we disable caching, this will put more load on Faravahar. But at least we'll get correct behaviour.

If we remove the header, some relays might not be able to find their IP address as quickly. (Do we believe IP addresses from other relays, or just the authorities?)

Tim

Tim Wilson-Brown (teor)

teor2345 at gmail dot com PGP 968F094B

teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F

On 2015-11-09 08:56, Matthew Finkel wrote:

On Wed, Nov 04, 2015 at 08:00:52AM +0100, Logforme wrote:

...

After seeing this I upgraded to the latest version 0.2.7.4-rc and on the restart it again used Faravahar to (correctly) guess the IP:

Nov 04 07:47:49.000 [notice] Guessed our IP address as 84.219.173.60 (source: 154.35.175.225).

Which version of Tor were you previously running? Have you seen these messages within the last few days, after you upgraded?

Sorry about not replying, I have been on vacation. It seems the issue has an explanation already (caching) but here's my answer for completeness:

Previously I was running version 0.2.6.10 from what was then the latest Debian wheezy tor-experimental package. As I wrote above I upgraded to the (at the time) latest wheezy tor-experimental, 0.2.7.4-rc. During my vacation the relay have not restarted, 22 days uptime at the moment, so I assume the IP address has not fluctuated.

These messages only happened rarely, weeks apart. I have only seen Faravahar giving the wrong IP, not any other Auth server.