A response to George’s comment: "The timeline here is that we are hoping the proposal _and_ the implementation to be ready by mid-December… I'm currently OK with the two statistics in: <https://people.torproject.org/~karsten/volatile/238-hs-relay-stats.txt>… I feel that any other statistics will need to be carefully analyzed.” I believe Roger created a branch implementing these two statistics as well as the number of HS descriptors requests at an HSDir, and I believe that he run those on some relays (at least Moritz’s). Are you just recreating this work? Did those relays stop collecting those statistics? What happened to that data? It won’t be terribly interesting if all we do is report *fewer* statistics collected at a later date than at the kickoff meeting. I also think that we should identify some question we hope to investigate for January, such as: 1. How much HS traffic is there? Already semi-answered by Roger this summer, as I just mentioned. 2. How many descriptors are there? Ditto. 3. How many descriptors are never requested? Now we’re getting somewhere. 4. How What is the median or maximum number of requests? This would be incredibly informative about the skew of HS popularity. 5. How many faliures/anomalies do we observe? This would help us figure out how well HSes are working and how broken/abusive client behavior is. And is there a reason this process has to be so slow? Is it the security review? Roger managed to pump out a branch for stats collection and get it generating data within a week. It’s pretty pathetic if we can’t do better ;-) Cheers, Aaron On Nov 20, 2014, at 9:49 PM, Karsten Loesing <karsten@torproject.org> wrote:

On 20/11/14 13:42, George Kadianakis wrote:

...

"A. Johnson" <aaron.m.johnson@nrl.navy.mil> writes:

...

...

...

...

George and I have been working on a small proposal to add two hidden-service related statistics: number of hidden services and total hidden-service traffic.

Great, I’m starting to focus more on this project now. Well, actually I’m going on a trip for a week today, but *then* I’m focusing more on this project :-)

Sounds great! We're meeting every Tuesday at 16:00 UTC in #tor-dev. Feel free to drop by.

Excellent. I won’t be there this coming Tuesday, but I’ll be there the next Tuesday.

...

Replicas mean that each descriptor is stored under two identifiers, so that's two places. Further, descriptor identifiers change once per day, so during a 24-hour period, there are up to four descriptor identifiers for a hidden service.

That makes sense. It would be nice if the statistics would allow you to identify how long (i.e. how many hour periods) each descriptor was observed being published. That would allow us to figure out if there are lots of short-lived services or fewer long-lived services. Publishing statistics every hour would pretty much take care of this. If you are really set on 24 hours, then perhaps you could add the total number of published descriptors in addition to the number of *unique* published descriptors.

Also, my suggestion about using additive noise applies equally well to the descriptor statistics. And multiplicative noise is a *bad idea* if you don’t have some adjustment for small values (e.g. 10% noise of a 0 value is 0, and 10% of 1 is only 0.1).

...

We have been thinking about many more hidden-service related statistics in a separate document. We're currently discussing whether we should turn it into a tech report, because we'll probably not want to implement most of those statistics. If you have remarks or more ideas, please feel free to edit the document. We're going to have a public review round for this, too, but that might not happen in the next week or two.

https://etherpad.wikimedia.org/p/hs_stats_78281091

Great! I think we should go for at least a little more data in the current proposal (what is the timeline for this, btw?). I think we should come up with a list of statistics we might imagine gathering and identify the subset of those that we’re comfortable gathering at this point. For example, I think failure statistics is much more innocuous than other data, and those would be very useful. For example, they would help us understand how to improve the protocol is failing, and it might help us identify misuse of hidden services (e.g. by botnets clients stupidly looking for non-existent descriptors or by malicious crawlers attempting to brute force descriptors). So here are some ideas: 1. Number of fetch requests for descriptors that don’t exist (number of fetch requests that do succeed would of course be very useful as well) 2. Number of descriptor publishes to the wrong HSDir (actually I suspect that the HSDir doesn’t check this and wants to be accepting of any publish) 3. Number of rendezvous circuits that never connect (from the RP perspective) 4. Number of rendezvous circuits on which no data cells are ever sent

(CC'ed [tor-dev])

Thanks, George, for moving the discussion here.

Here's the latest proposal draft where I incorporated Aaron's suggestions:

https://gitweb.torproject.org/user/karsten/torspec.git/blob/refs/heads/hs_st...

If people on this list have more feedback, please reply here. Thanks!

All the best, Karsten

...

Thanks for the input Aaron!

The timeline here is that we are hoping the proposal _and_ the implementation to be ready by mid-December. Then we are hoping that we can deploy the code to a few relays so that we have some data by January.

So, time is tight.

I'm currently OK with the two statistics in: https://people.torproject.org/~karsten/volatile/238-hs-relay-stats.txt

I feel that any other statistics will need to be carefully analyzed. We should add the ideas you mentioned in the etherpad, and get them included in the tech report (which we are also hoping to have ready in some form by mid-January).

The tech report is supposed to contain and analyze most of the HS statistics we can think of. It will likely contain many stats that we will never do, but also some stats that might be a good idea. The good ones we should eventually integrate to the Tor proposal and write code for.

...

...

Thanks for the very valuable input! Let me know if the following draft looks okay, and I'll start another thread on tor-dev@.

https://people.torproject.org/~karsten/volatile/238-hs-relay-stats-2014-11-2...

"Lab(\epsilon/C)” -> "Lap(\epsilon/C)” (that was my mistake. I think having the added noise both parameterized and included in the reported statistics is an idea worth thinking about. Making it a parameter allows you to easily change it without upgrading. Including it in the statistics would allow us to correct better for noise if different relays might be adding different amounts of noise due to inconsistent opinions of the noise parameter (if this should never happen, then I guess this wouldn’t be necessary).

So again, sorry that I’m not going to be very responsive on this for the next week. I’m really happy that you’re working on it!

Best, Aaron

On Nov 21, 2014, at 9:39 AM, George Kadianakis <desnacked@riseup.net> wrote:

I also believe that some of these extra stats (e.g. "How many failures/anomalies do we observe?") should first be done on a privnet instead of the real network. That can give us some preliminary results, and then we can consider doing them on the real network. Maybe we can also have some privnet stats by January.

I think this is a great idea. While we might not learn (at first) the absolute number of failures that occur in the *real* network, we will at least be able to say things about the *fraction* of requests that fail. That can be collected in a large ShadowTor network. In fact, I would advocate implementing the collection of all of the stats that Aaron has requested; for those stats that still need some security analysis before people are convinced, we can enable those in TestingTorNetwork mode for now. If they are blessed, collecting them in “normal” mode becomes easier. In fact, might we also consider doing this for even more of the statistics from the etherpad (the ones that make sense for privnets)? I suppose at some point there will exist an implementation bottleneck, but being able to say as much as possible in January - even if it is only from privnet stats - is a win. We can then hope to be able to say more about the real network by the following deadline. -Rob

"A. Johnson" <aaron.m.johnson@nrl.navy.mil> writes:

...

Roger's branch was a PoC that wrote stats on the log file. I don't think we have newer data than what is in #13192. It's unclear whether the relays stopped collecting statistics, or they just haven't updated the trac ticket.

If we could check on that and get that data, that would be really helpful. Then we could do analysis in parallel with the better extra-info implementation.

I asked Moritz but he told me that he stopped collecting those stats...

...

Also, Roger's stats were counting cells from both RP and IP circuits. It's unclear whether we will take the same approach; atm I find it more reasonable to only count RP cells/circuits.

IP stats are also interesting, but I agree less so than RP stats alone.

Yes. Also, IP stats can be linked to specific HSes, RP stats shouldn't be linkable to specific HSes.

...

BTW, did Roger do the "How many HSes are there?" HSDir stats? Is there a ticket for that?

I am fairly sue he at least counted descriptor updates at an HSDir. I have a slide bullet saying "We estimate about 30 to 50K hidden services are updating their descriptors each day” from the kickoff meeting, and I recall Roger talking about that. Because the question is what the “dark matter” of Hidden Services consists of, that is, the 30-50K HSes less the ~1500 that are publicly available and were responding at that time.

Yep, found it at #13195.

...

In any case, Roger told us that answering the questions "Approx. how many HSes are there?" and "How much bw is HS bw?" are the important parts of what we need to have by January. Our plan was to have that, plus a document with various future statistics we might or might not do. Do you think that's not sufficient?

I’m not sure about “not sufficient”, but as I said, Roger already reported estimates for those last time. But I’d go with his opinion on this - it is Tor’s part of the project.

...

Security review is indeed a big part. I'm not persuaded that just collecting all kinds of statistics from the Tor network is always good or helpful [0]. I personally prefer to do this methodically and with sufficient time for thinking and feedback, instead of starting to collect various statistics in a short time. I feel that getting pressured about *moar statistics* is a slippery slope that leads to badness.

OK, makes sense. So let’s start tackling the hard question: what exactly do hidden services want to protect? Some questions:

To all the questions below, my answer would probably be: "Yes, to the extend possible". Even though those properties are not really related to hiding the location of the HS, I believe that the fewer info the adversary knows about a specific HS the harder it is to deanonymize it. This philosophy can be seen in various places of the HS spec, for example by the fact that ephemeral keys are used for introduction points so that they don't know which HS they are serving, or by the fact that HS descriptors will soon become encrypted so that the HSDirs cannot read them.

1. Should HSes be able to hide that they even exist at all in the system? If so, counting the number of hidden services reduces this somewhat (up to the added noise/inaccuracy). And by the way, random noise doesn’t necessarily hide this, because over time, if you choose new noise every measurement period and the number of HSes is constant, then the average will eventually reveal the exact number. Ideas to handle this: reuse randomness (except now that reveals exactly when HSes are added or removed), round to the nearest multiple of some bucket size (although what about the one HS that puts you into the next bucket..) Doing against an active adversary (not one who just looks at your reported stats) is much harder, of course, because you need to prevent HSDirs from knowing how many real descriptors they have.

The fact that noise is not very effective here is true, but I also acknowledge that this could be a useful stat, so we need to find the right balance. I'm hoping that the noise and the fact that the number of HSes is not really constant, will be able to obfuscate the exact number of HSes. So that if an adversary wanted to enumerate all of them in the current network, he would be off by a hundred or so.

2. Should HSes be able to hide their (pseudonymous) popularity (i.e. number of users, connections per user)? If so, collecting RP cell counts already leaks averages and puts a lower bound on the max.

That's also true. Hopefully RP cell counts won't be able to reveal the popularity of a specific HS though, which is the important part IMO.

3. Should client HS lookups be hidden so that nobody knows what’s being queried or how often? If so, collecting descriptors requests could reveal a very active set of clients.

Personally, I don't think we should count HSDir descriptor requests. HSDir descriptor requests can be linked back to specific HSes, which I think is bad.

These are hard questions because HSes are only designed to hide location, but there also appears to be a strong desire to make it hard to learn anything else about them. But there are good reasons (e.g. designing protocols improvements, troubleshooting problems, watching for malicious behavior) to learn *something* about HSes.

<snip>

...

[0]: Did you know that relays (and bridges) report bandwidth statistics every *15* minutes? I have no idea if this is a good idea to do, especially for relays that see very few clients.

I did know this. It does seem potentially revealing of, say, the guard used by a hidden service because you can easily modulate the HSes traffic in 15 minute intervals. Somebody should think about how what statistics gathering might reveal and if that’s cool.

Hm. I made #13838 for this.