Hi all,
For all my sins I wrote parts of the algorithm that is at fault here.
I also echo, and confirm all the problems mentioned. One thing that would greatly help tune such systems is a database of known censored periods from different jurisdictions. The issue is that "anomalies" occur all the time -- and tor is presumably only interested in "intersting anomalies" that related to attacks.
Now I know more about this field, and happy to work with others to improve the state of the detector if there is interest.
George
On Thu, Aug 20, 2015 at 7:36 PM, l.m ter.one.leeboi@hush.com wrote:
Hi Joss,
Thank you for the fine paper. I look forward to reading it. Karsten would be keen on it too (and maybe also your offer) if you haven't already forwarded it to them. My interest in fixing it is (mostly) recreational. I have some thoughts on how to proceed, but I'm not a representative of tor project.
Regards --leeroy
Hi,
These are well identified issues. We've been working here on a way to improve the current filtering detection approach, and several of the points above are things that we're actively hoping to work into our approach. Differentiating 'filtering' from 'other events that affect Tor usage' is tricky, and will most likely have to rely on other measurements from outside Tor. We're currently looking at ways to construct models of 'normal' behaviour in a way that incorporates multiple sources of data.
We have a paper up on arXiv that might be of interest. I'd be interested to be in touch with anyone who's actively working on this. (We have code, and would be very happy to work on getting it into production.) I've shared the paper with a few people directly, but not here on the list.
arXiv link: http://arxiv.org/abs/1507.05819
We were looking at any anomalies, not only pure Tor-based filtering events. For the broader analysis, significant shifts in Tor usage are very interesting. It's therefore useful to detect a range of unusual behaviours occurring around Tor, and have a set of criteria within that to allow differentiating 'hard' filtering events from softer anomalies occurring due to other factors.
Joss
Dr. Joss Wright | Research Fellow Oxford Internet Institute, University of Oxford http://www.oii.ox.ac.uk/people/?id=176
tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev