[tor-dev] Small FTE question

[Kevin P Dyer]

Hi George,

On Sat, Jan 4, 2014 at 7:12 AM, George Kadianakis <desnacked at riseup.net> wrote:
>> On Fri, Jan 3, 2014 at 6:20 AM, George Kadianakis <desnacked at riseup.net>
>> wrote:
>>> Hello Kevin,
>>>
>>> I saw your recent changes to the FTE codebase. The code looks nice!
>>>
>>> I then tried to test it, but I got a bit confused by the CLI. I wanted
>>> to
>>> try the good ol' ncat test, where I put FTE in the middle, an ncat
>>> listener on the server side, and an ncat client on the client side, and
>>> throw bytes around. How would I do that with the FTE CLI? I looked at
>>> https://fteproxy.org/docs/0.2/fteproxy.html but I'm not sure how to use
>>> the server_port and client_port switches properly. Can I configure the
>>> client-side to push data to a specific destination, or is it always a
>>> SOCKS listener?
>>
>> As a first place to start, it is probably best to have a look at
>> "systemtests" in the root of the fteproxy source. This might be
>> sufficient for your purposes.
>>
>
>
> Hm `systemtests` seems indeed relevant to my interests. What is it? Is it
> integration tests?

Exactly. It's an integration test script that tests all the FTE
formats registered with fteproxy. I've improved the level of
documentation of this script and increased the verbosity of it's
output. Please give the latest version a try.

> I tried to run `systemtests` and I got stuff like "+ manual-http: 22.78
> Mbps (5.62s)" in stdout, but when I used wireshark during the tests I saw
> a flood of 'SYN' and then 'RST, ACK' packets on the wire. Seems like it
> can't establish a connection to the listener at 8080. Why is that? Did the
> tests run?

The output "+ manual-http: 22.78 Mbps (5.62s)" indicates successful
completion of a test using the manual-http format. Hopefully the new
version of the script should much clearer.

The SYNs and RSTs you were seeing was a result of an attempt to detect
when the fteproxy client/server successfully initialized. I was using
socket.connect aggressively in a tight loop, which is overkill. I've
resolved this issue, such that the majority of traffic generated from
this script will be actual fteproxy traffic. (Just to be clear: this
was a problem with the "systemtests" script and not fteproxy.)

> BTW, you might want to check out `obfsproxy/test/tester.py'. It's the
> integration tester of obfsproxy. It's quite simple: it opens a client and
> a listener, pushes some traffic on one end, and checks if it's received
> intact on the other end. The code is a bit aged (it's there since
> C-obfsproxy), but it might be useful.

I'll take a look at that. Overall, the systemtests script is
relatively simple, and the majority of the logic is very
fteproxy-specific.

>> The longer answer: When not run in managed mode (--managed), fteproxy
>> runs as a simple TCP proxy. Included in the fteproxy code is
>> "bin/socksproxy" which I spin up and is destination for all data
>> received by the fteproxy server. So, the typical (non Tor) testing
>> scenario is:
>>
>> [SOCKS client] < - > [fteproxy client] < - > [fteproxy server] < - >
>> [SOCKS server]
>>
>> * Socks client connects to the port specified as client_port on the
>> fteproxy client.
>> * The specified server port should be the same on the fteproxy client
>> and server.
>> * The fteproxy server proxy_port should be the port that the SOCKS
>> server is listening on.
>>
>
> Hm, do you have a bash recipe to test this? When I want to test obfsrpoxy
> I do something like this:
>
> """
> # Set up obfsproxy client
> ./bin/obfsproxy obfs2 client 127.0.0.1:6666 --dest=127.0.0.1:7777
> # Set up obfsproxy server
> ./bin/obfsproxy obfs2 server 127.0.0.1:7777 --dest=127.0.0.1:8888
> # Set up server-side data listener
> ncat -k -l -p 8888
> # Start up client-side data pusher
> ncat localhost 6666
> """
>
> How would you test that for FTE? I want to push some data around and see
> how packets look like on the wire.

I've checked in two scripts [1,2] which demonstrate how to test
fteproxy with netcat. The concise script [1] takes advantage of the
default configuration parameters of fteproxy. The verbose script [2]
shows, explicitly, how to configure an fteproxy client and server.

You may also want to take a look at the fteproxy help (./bin/fteproxy
--help) which lists the command-line configuration parameters and
their defaults.

-Kevin

[1] https://raw.github.com/kpdyer/fteproxy/master/scripts/netcat_simple.sh
[2] https://raw.github.com/kpdyer/fteproxy/master/scripts/netcat_verbose.sh