On Mar 6, 2013, at 3:58 PM, Sam Smith s@msmith.net wrote:
Hey all,
Hello :)
I'm looking at detection and identification of some transparent proxies (of different kinds). Has anyone already got tests that effectively but generically merge manipulation/http_host and experimental/squid.py ?
I am not sure what you mean exactly by "effectively but generically merge manipulation/http_host and experimental/squid.py".
There is a currently unmerged branch of mine that add some features to the HTTP Host test. Specifically it tests for some HTTP Proxy bypassing techniques (prepending a \n to the GET method and postfixing the Host header field with a \t), it checks if filtering is happening also on subdomains, if it's done based on fuzzy matching. You can find it here: https://github.com/TheTorProject/ooni-probe/pull/51 (this is the branch: https://github.com/hellais/ooni-probe/tree/experimental-tests).
The tests that are most relevant to your endeavor are:
* nettests/manipulation/http_invalid_request_line.py (for example bluecoat transparent HTTP proxies are know to crash with some, but not all of these requests)
* nettests/manipulation/http_host.py, with the added patches in my experimental-tests branch it will also attempt some filter bypassing techniques that can be useful to understand what the device in question is.
* nettests/manipulation/http_header_field_manipulation.py, when pointing this towards a oonib test helper you will understand if the device in question is adding extra HTTP Headers or performing some other HTTP header mangling.
Also, I'm testing on a debian (6.0) VM running in a parallels 8 on OS X. Ooni successfully runs tests once , and then requires a debian reboot in order to run again (especially http_host, it was less repeatable with others). I have no idea what's going on, and it'll be a pain to debug, but it's the sort of thing someone's probably interested in for other purposes ;)
That is quite peculiar behavior. What exactly is it that leads you to need to reboot? Are you seeing some errors after the first run?
What version of ooniprobe are you running?
Feel free to drop by on IRC: irc.oftc.net #ooni.
~ Art.