Re: [tor-dev] bananaphone obfsproxy module
David Stainton <dstainton415@gmail.com> writes:
Hi,
Yeah... I should add a doc string to the BaseTransport __init__ explaining that it runs upon connect.
OK yesterday I implemented transport class method called setup()... The BananaphoneTransport overrides setup()... storing the markov model in a class attribute. I had to modify pyobfsproxy.py and run the class setup() method after the external cli args are parsed.
I made it work for external cli mode... by making pyobfsproxy.py run the setup method for each transport after args.validation_function(args) runs... because the setup method needs some args to read the corpus file and build the markov model.
What should I do to get arguments passed to obfsproxy when running in managed mode?
Hm. Good question. You probably want to go to obfsproxy/managed/server.py and obfsproxy/managed/client.py and place a strategic run_transport_setup() hook. I would place it before the launch_transport_listener() call. You might also need to use obfsproxy/transports/get_transport_class() somewhere in there. Another nice thing would be if you could pass the pt_config struct in setup(), so that transports can use pt_config elements during initialization. As an example, see how obfs2 configures its shared-secret in its __init__ using getServerTransportOptions(). That could be done in its setup(), I think. Does this make sense? If you don't want to do this coding, I can do it at some point next week. Cheers! ( And while I'm on it, let me tell you about the different ways that you can pass parameters to obfsproxy. I wanted to document this somewhere, and this thread seems like a good excuse :) - External mode: You can pass global parameters to external mode transports by using the register_external_mode_cli() and validate_external_mode_cli() functions. - Managed mode: - Global parameters: These are parameters that are passed to obfsproxy on startup. - Server: On the server-side we can pass global parameters to obfsproxy by using the TOR_PT_SERVER_TRANSPORT_OPTIONS environment variable. Obfsproxy exposes those parameters using the transport_config struct and obfsproxy/common/transport_config.py:getServerTransportOptions . - Client: On the client-side we don't have a way to pass global parameters to obfsproxy yet. If we ever need to, we can do it with environment variables here too. - Per-connection parameters: These are parameters which are supposed to be different for each connection. An example of such a parameter can be a shared-secret used for a specific bridge connection. - Server: On the server-side we don't have a way to pass per-connection parameters at the moment. - Client: On the client-side we pass per-connection parameters using the SOCKS handshake as a covert channel. Transports can retrieve these parameters using the handle_socks_args() method. )
George, Thanks for the info! This should approximate your suggestion: https://github.com/david415/obfsproxy/tree/david-bananaphone-managed Let me know your suggestions for improvement. Either way I'll work on this more soon. Onward! David On Sun, Nov 10, 2013 at 3:21 AM, George Kadianakis <desnacked@riseup.net> wrote:
David Stainton <dstainton415@gmail.com> writes:
Hi,
Yeah... I should add a doc string to the BaseTransport __init__ explaining that it runs upon connect.
OK yesterday I implemented transport class method called setup()... The BananaphoneTransport overrides setup()... storing the markov model in a class attribute. I had to modify pyobfsproxy.py and run the class setup() method after the external cli args are parsed.
I made it work for external cli mode... by making pyobfsproxy.py run the setup method for each transport after args.validation_function(args) runs... because the setup method needs some args to read the corpus file and build the markov model.
What should I do to get arguments passed to obfsproxy when running in managed mode?
Hm. Good question.
You probably want to go to obfsproxy/managed/server.py and obfsproxy/managed/client.py and place a strategic run_transport_setup() hook. I would place it before the launch_transport_listener() call. You might also need to use obfsproxy/transports/get_transport_class() somewhere in there.
Another nice thing would be if you could pass the pt_config struct in setup(), so that transports can use pt_config elements during initialization. As an example, see how obfs2 configures its shared-secret in its __init__ using getServerTransportOptions(). That could be done in its setup(), I think.
Does this make sense? If you don't want to do this coding, I can do it at some point next week.
Cheers!
( And while I'm on it, let me tell you about the different ways that you can pass parameters to obfsproxy. I wanted to document this somewhere, and this thread seems like a good excuse :)
- External mode: You can pass global parameters to external mode transports by using the register_external_mode_cli() and validate_external_mode_cli() functions.
- Managed mode: - Global parameters: These are parameters that are passed to obfsproxy on startup.
- Server: On the server-side we can pass global parameters to obfsproxy by using the TOR_PT_SERVER_TRANSPORT_OPTIONS environment variable. Obfsproxy exposes those parameters using the transport_config struct and obfsproxy/common/transport_config.py:getServerTransportOptions . - Client: On the client-side we don't have a way to pass global parameters to obfsproxy yet. If we ever need to, we can do it with environment variables here too.
- Per-connection parameters: These are parameters which are supposed to be different for each connection. An example of such a parameter can be a shared-secret used for a specific bridge connection.
- Server: On the server-side we don't have a way to pass per-connection parameters at the moment. - Client: On the client-side we pass per-connection parameters using the SOCKS handshake as a covert channel. Transports can retrieve these parameters using the handle_socks_args() method. )
David Stainton <dstainton415@gmail.com> writes:
George,
Thanks for the info!
This should approximate your suggestion: https://github.com/david415/obfsproxy/tree/david-bananaphone-managed
Let me know your suggestions for improvement. Either way I'll work on this more soon.
Looks fine! BTW, does it work for you? Did you try the server-side using the ServerTransportOptions torrc option (tor >= 0.2.5.1-alpha)? Some nitpicky comments: - It doesn't really matter in this case but: transport_class = transports.get_transport_class(transport, 'socks') might be better as: transport_class = transports.get_transport_class(transport, 'server') in obfsproxy/managed/server.py. - I see you are passing 'None' to setup() when in external mode. Maybe you can move the run_transport_setup() call to do_external_mode() (after pt_config is instantiated), and then pass the pt_config to setup(). This way external-mode transports can learn the state location (it's in TransportConfig) during setup() (Scramblesuit might like this). If you do so, there is no reason to do 'if transport_config is not None' in obfsproxy/transports/bananaphone_transport.py. - It would be nice if obfs2 also used the setup() method to set its shared_secret. I'm referring to the block of code below: # Check for shared-secret in the server transport options. Cheers!
Looks fine!
BTW, does it work for you? Did you try the server-side using the ServerTransportOptions torrc option (tor >= 0.2.5.1-alpha)?
I have only tested it using the external mode. I've gotten obfsproxy + bananaphone to run with tor in managed mode; using the ServerTransportOptions... But I didn't test that it actually works. OK I'll try testing with tor >= 0.2.5.1-alpha using the ServerTransportOptions.
Some nitpicky comments: - It doesn't really matter in this case but: transport_class = transports.get_transport_class(transport, 'socks') might be better as: transport_class = transports.get_transport_class(transport, 'server') in obfsproxy/managed/server.py.
OK I will fix.
- I see you are passing 'None' to setup() when in external mode. Maybe you can move the run_transport_setup() call to do_external_mode() (after pt_config is instantiated), and then pass the pt_config to setup(). This way external-mode transports can learn the state location (it's in TransportConfig) during setup() (Scramblesuit might like this).
If you do so, there is no reason to do 'if transport_config is not None' in obfsproxy/transports/bananaphone_transport.py.
- It would be nice if obfs2 also used the setup() method to set its shared_secret. I'm referring to the block of code below: # Check for shared-secret in the server transport options.
OK I will do this... but can I do it all in my bananaphone branch or do you want the obfs2 change in another branch?
David Stainton <dstainton415@gmail.com> writes:
Looks fine!
BTW, does it work for you? Did you try the server-side using the ServerTransportOptions torrc option (tor >= 0.2.5.1-alpha)?
I have only tested it using the external mode.
I've gotten obfsproxy + bananaphone to run with tor in managed mode; using the ServerTransportOptions... But I didn't test that it actually works.
OK I'll try testing with tor >= 0.2.5.1-alpha using the ServerTransportOptions.
Some nitpicky comments: - It doesn't really matter in this case but: transport_class = transports.get_transport_class(transport, 'socks') might be better as: transport_class = transports.get_transport_class(transport, 'server') in obfsproxy/managed/server.py.
OK I will fix.
- I see you are passing 'None' to setup() when in external mode. Maybe you can move the run_transport_setup() call to do_external_mode() (after pt_config is instantiated), and then pass the pt_config to setup(). This way external-mode transports can learn the state location (it's in TransportConfig) during setup() (Scramblesuit might like this).
If you do so, there is no reason to do 'if transport_config is not None' in obfsproxy/transports/bananaphone_transport.py.
- It would be nice if obfs2 also used the setup() method to set its shared_secret. I'm referring to the block of code below: # Check for shared-secret in the server transport options.
OK I will do this... but can I do it all in my bananaphone branch or do you want the obfs2 change in another branch?
Please do it in your bananaphone branch if that's more convenient to you. (If you want to be more adventurous, you can even make a new branch on top of the current upstream master with your setup() function and the obfs2 changes. I will merge that faster than the bananaphone changes.)
Please do it in your bananaphone branch if that's more convenient to you.
(If you want to be more adventurous, you can even make a new branch on top of the current upstream master with your setup() function and the obfs2 changes. I will merge that faster than the bananaphone changes.)
OK I gave BaseTransport the setup class method and used it to save obfs2's shared secret : https://github.com/david415/obfsproxy/commits/david-transport-setup I have not yet tested it. I will do that soon.
David Stainton <dstainton415@gmail.com> writes:
Please do it in your bananaphone branch if that's more convenient to you.
(If you want to be more adventurous, you can even make a new branch on top of the current upstream master with your setup() function and the obfs2 changes. I will merge that faster than the bananaphone changes.)
OK I gave BaseTransport the setup class method and used it to save obfs2's shared secret :
https://github.com/david415/obfsproxy/commits/david-transport-setup
I have not yet tested it. I will do that soon.
Merged. Thanks! I tested it and it seems to work. Please test it too when you get the chance and let me know :)
OK I tested obfsproxy obfs2 in managed mode with tor and it works... But I guess that doesn't really test my changes since I'd have to pass it a shared_secret """ - Client: On the client-side we don't have a way to pass global parameters to obfsproxy yet. If we ever need to, we can do it with environment variables here too. """ Are you saying that we cannot use a shared secret with obfs2 in managed mode with Tor? I've tested in external cli mode and it works when passing a shared secret on the commandline. David On Wed, Nov 13, 2013 at 7:39 AM, George Kadianakis <desnacked@riseup.net> wrote:
David Stainton <dstainton415@gmail.com> writes:
Please do it in your bananaphone branch if that's more convenient to you.
(If you want to be more adventurous, you can even make a new branch on top of the current upstream master with your setup() function and the obfs2 changes. I will merge that faster than the bananaphone changes.)
OK I gave BaseTransport the setup class method and used it to save obfs2's shared secret :
https://github.com/david415/obfsproxy/commits/david-transport-setup
I have not yet tested it. I will do that soon.
Merged. Thanks! I tested it and it seems to work.
Please test it too when you get the chance and let me know :)
David Stainton <dstainton415@gmail.com> writes:
OK I tested obfsproxy obfs2 in managed mode with tor and it works... But I guess that doesn't really test my changes since I'd have to pass it a shared_secret
""" - Client: On the client-side we don't have a way to pass global parameters to obfsproxy yet. If we ever need to, we can do it with environment variables here too. """
Are you saying that we cannot use a shared secret with obfs2 in managed mode with Tor?
No, it is possible. You just need to use the k=v parameters of the Bridge line in your torrc. These will be passed as per-connection parameters during the SOCKS handshake from Tor to obfsproxy. In obfsproxy, the parameters will be passed to your transport using handle_socks_args().
Yeah obfs2 works perfectly... in managed mode passing the shared secret. I'd love to contribute some documentation or demonstrate example usage of obfsproxy... Shouldn't we setup a wiki for this purpose? And finally I tested obfsproxy in managed mode with the bananaphone transport... and it works! It's laggy... but it works ;-) It's interesting to note that I got a couple of these in my client side tor log: Nov 14 20:17:16.000 [warn] Your Guard xxx ($xxx) is failing a very large amount of circuits. Most likely this means the Tor network is overloaded, but it could also mean an attack against you or potentially the guard itself. Success counts are 56/184. Use counts are 8/8. 176 circuits completed, 0 were unusable, 120 collapsed, and 14 timed out. For reference, your timeout cutoff is 60 seconds. Also I tested and was able to pass transport options to obfsproxy bananaphone and that works now that I fixed the BananaphoneTransport setup method. Onward! David On Thu, Nov 14, 2013 at 1:12 AM, George Kadianakis <desnacked@riseup.net> wrote:
David Stainton <dstainton415@gmail.com> writes:
OK I tested obfsproxy obfs2 in managed mode with tor and it works... But I guess that doesn't really test my changes since I'd have to pass it a shared_secret
""" - Client: On the client-side we don't have a way to pass global parameters to obfsproxy yet. If we ever need to, we can do it with environment variables here too. """
Are you saying that we cannot use a shared secret with obfs2 in managed mode with Tor?
No, it is possible.
You just need to use the k=v parameters of the Bridge line in your torrc. These will be passed as per-connection parameters during the SOCKS handshake from Tor to obfsproxy. In obfsproxy, the parameters will be passed to your transport using handle_socks_args().
David Stainton <dstainton415@gmail.com> writes:
Yeah obfs2 works perfectly... in managed mode passing the shared secret. I'd love to contribute some documentation or demonstrate example usage of obfsproxy... Shouldn't we setup a wiki for this purpose?
Then I should introduce you to: https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports Only a few days ago I added the bananaphone transport to that wiki page. Feel free to contribute! BTW, it's definitely the case that obfsproxy (and the whole pluggable transport space) would benefit from more/better documentation. (As an example, I've been meaning to make a wiki page for the pluggable transport combiner that is currently in the works (#7167, #9744, #10061), but I haven't got to it yet.)
And finally I tested obfsproxy in managed mode with the bananaphone transport... and it works! It's laggy... but it works ;-)
Excellent news!
It's interesting to note that I got a couple of these in my client side tor log:
Nov 14 20:17:16.000 [warn] Your Guard xxx ($xxx) is failing a very large amount of circuits. Most likely this means the Tor network is overloaded, but it could also mean an attack against you or potentially the guard itself. Success counts are 56/184. Use counts are 8/8. 176 circuits completed, 0 were unusable, 120 collapsed, and 14 timed out. For reference, your timeout cutoff is 60 seconds.
Also I tested and was able to pass transport options to obfsproxy bananaphone and that works now that I fixed the BananaphoneTransport setup method.
Onward!
David
On Thu, Nov 14, 2013 at 1:12 AM, George Kadianakis <desnacked@riseup.net> wrote:
David Stainton <dstainton415@gmail.com> writes:
OK I tested obfsproxy obfs2 in managed mode with tor and it works... But I guess that doesn't really test my changes since I'd have to pass it a shared_secret
""" - Client: On the client-side we don't have a way to pass global parameters to obfsproxy yet. If we ever need to, we can do it with environment variables here too. """
Are you saying that we cannot use a shared secret with obfs2 in managed mode with Tor?
No, it is possible.
You just need to use the k=v parameters of the Bridge line in your torrc. These will be passed as per-connection parameters during the SOCKS handshake from Tor to obfsproxy. In obfsproxy, the parameters will be passed to your transport using handle_socks_args().
Hi Isis, George, I think the Bananaphone transport might need another modification to the obfsproxy api. Isis pointed out that we really don't want to pass the corpus filename via ServerTransportOptions... but instead only read it via optparse'ed commandline. Actually I think the encodingSpec is the only Bananaphone options that needs to get passed to the bridge database. I cannot use register_external_mode_cli for this purpose because we want to run obfsproxy in managed mode with tor... So should I create another stub method for BaseTransport that can be overridden... it could be used to register managed-mode cli arg parser... which populates class attributes of the transport. What do you think? David On Thu, Nov 14, 2013 at 1:23 PM, George Kadianakis <desnacked@riseup.net> wrote:
David Stainton <dstainton415@gmail.com> writes:
Yeah obfs2 works perfectly... in managed mode passing the shared secret. I'd love to contribute some documentation or demonstrate example usage of obfsproxy... Shouldn't we setup a wiki for this purpose?
Then I should introduce you to: https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports Only a few days ago I added the bananaphone transport to that wiki page. Feel free to contribute!
BTW, it's definitely the case that obfsproxy (and the whole pluggable transport space) would benefit from more/better documentation. (As an example, I've been meaning to make a wiki page for the pluggable transport combiner that is currently in the works (#7167, #9744, #10061), but I haven't got to it yet.)
And finally I tested obfsproxy in managed mode with the bananaphone transport... and it works! It's laggy... but it works ;-)
Excellent news!
It's interesting to note that I got a couple of these in my client side tor log:
Nov 14 20:17:16.000 [warn] Your Guard xxx ($xxx) is failing a very large amount of circuits. Most likely this means the Tor network is overloaded, but it could also mean an attack against you or potentially the guard itself. Success counts are 56/184. Use counts are 8/8. 176 circuits completed, 0 were unusable, 120 collapsed, and 14 timed out. For reference, your timeout cutoff is 60 seconds.
Also I tested and was able to pass transport options to obfsproxy bananaphone and that works now that I fixed the BananaphoneTransport setup method.
Onward!
David
On Thu, Nov 14, 2013 at 1:12 AM, George Kadianakis <desnacked@riseup.net> wrote:
David Stainton <dstainton415@gmail.com> writes:
OK I tested obfsproxy obfs2 in managed mode with tor and it works... But I guess that doesn't really test my changes since I'd have to pass it a shared_secret
""" - Client: On the client-side we don't have a way to pass global parameters to obfsproxy yet. If we ever need to, we can do it with environment variables here too. """
Are you saying that we cannot use a shared secret with obfs2 in managed mode with Tor?
No, it is possible.
You just need to use the k=v parameters of the Bridge line in your torrc. These will be passed as per-connection parameters during the SOCKS handshake from Tor to obfsproxy. In obfsproxy, the parameters will be passed to your transport using handle_socks_args().
participants (2)
-
David Stainton -
George Kadianakis