On Mon, Apr 29, 2013 at 17:44 PM, Nick Mathewson <nickm at alum.mit.edu> wrote:
Right now, all chutney does is configure and launch a small Tor network, but the small network requires a little babysitting to make it bootstrap. (For example, you sometimes need to send the clients a SIGHUP once the authorities have made their first consensus.
So, the first bits for improving chutney would be to automate the network generation as much as possible (leaving no loose ends)?
What Chutney doesn't do (but should!) is actually test the network it launches. Right now, to make sure the network has bootstrapped, you have to read through its log files. If you want to test the network with any traffic, you need to use an application of your choice to send traffic through one of the Tor clients, and verify that the output as is expected. And if you want to make sure that Tor is running without weird warnings, you need to manually inspect the log files again. It would be good if Chutney grew into, or were replaced by, a tool that could actually do testing right that.
Now, we want chutney to be robust enough to encompass all the manual that is mentioned above & automate it as much as possible or atleast print the tasks to be performed by the tester sequentially. We would need a set of static reliable web pages if we want to automate the traffic testing? Also, we would parse the log files for errors or warnings so that we dont have to manually check for them. * Construct a circuit to an exit which allows port 80 and request a site through it. * Do the same for an exit which rejects port 80. * Construct a one-hop circuit through an exit. * Try to use a circuit that runs through a bridge. * Let tor pick the path of a circuit and assert that the first relay's a guard and the last is an exit. * Attempt to request a page from a hidden service. * Host a hidden service and request something from ourselves. * Act as a relay and try to construct a circuit that includes ourselves. * ... etc... As Damian listed down all the tests one needs to carry out for actual network testing (keeping in mind unreliability of the network at times), what all additional tests would we like chutney to have?
Hello! As part of the Stem Integ tests, nickm suggested integrating gcov into Stem. I did not get that part. Gcov is for code coverage of C/C++ source code. We could integrate it to Stem, but then it would involve executing command line statements on a gcov installed machine. Also, we could add similar code coverage tool for Stem itself. Did I get it right? How about adding coverage.py to Stem?
Hello! As part of the Stem Integ tests, nickm suggested integrating gcov into Stem. I did not get that part. Gcov is for code coverage of C/C++ source code. We could integrate it to Stem, but then it would involve executing command line statements on a gcov installed machine.
Hi Ashish. If the process of integrating gcov is to compile it with tor then that's certainly an option. The steps would be... 1. Do it on your local system so you're familiar with the steps involved. 2. Open a ticket and work with weasel (the person administering our jenkins instance) with specific instructions and work with him to get it set up for our automated test runs.
Also, we could add similar code coverage tool for Stem itself.
Certainly. Coverage is a nice tool that can produce html output. We ran it over stem a while back to get some notion of our code coverage... https://trac.torproject.org/7821 But it would be nice if runTests.py had a '--coverage' argument that used the coverage tool (if available) to include this report. Filed a ticket for this... https://trac.torproject.org/8806 Cheers! -Damian
On Tue, Apr 30, 2013 at 10:29 AM, Ashish Patil <cool_asis_is@yahoo.in> wrote:
On Mon, Apr 29, 2013 at 17:44 PM, Nick Mathewson <nickm at alum.mit.edu> wrote:
Right now, all chutney does is configure and launch a small Tor network, but the small network requires a little babysitting to make it bootstrap. (For example, you sometimes need to send the clients a SIGHUP once the authorities have made their first consensus.
So, the first bits for improving chutney would be to automate the network generation as much as possible (leaving no loose ends)?
The network configuration works fine. The bootstrapping is a bit tricky. Personally, though, I would focus on the actual testing first: having a test for "did the bootstrapping finish" would be invaluable if you're trying to actually make sure the bootstrapping finishes.
What Chutney doesn't do (but should!) is actually test the network it launches. Right now, to make sure the network has bootstrapped, you have to read through its log files. If you want to test the network with any traffic, you need to use an application of your choice to send traffic through one of the Tor clients, and verify that the output as is expected. And if you want to make sure that Tor is running without weird warnings, you need to manually inspect the log files again. It would be good if Chutney grew into, or were replaced by, a tool that could actually do testing right that.
Now, we want chutney to be robust enough to encompass all the manual that is mentioned above & automate it as much as possible or atleast print the tasks to be performed by the tester sequentially. We would need a set of static reliable web pages if we want to automate the traffic testing?
I think that instead, you'd want a little program that ran a server locally. Having an external webserver would mean that network problems, or problems in the external webserver, would make Tor untestable while they were getting fixed.
Also, we would parse the log files for errors or warnings so that we dont have to manually check for them.
* Construct a circuit to an exit which allows port 80 and request a site through it. * Do the same for an exit which rejects port 80. * Construct a one-hop circuit through an exit. * Try to use a circuit that runs through a bridge. * Let tor pick the path of a circuit and assert that the first relay's a guard and the last is an exit. * Attempt to request a page from a hidden service. * Host a hidden service and request something from ourselves. * Act as a relay and try to construct a circuit that includes ourselves. * ... etc...
As Damian listed down all the tests one needs to carry out for actual network testing (keeping in mind unreliability of the network at times), what all additional tests would we like chutney to have?
I think for a while at least, we'd want to be coverage-driven: looking for an area of the code that isn't getting called during testing, and adding a test for it, then iterating. best wishes, -- Nick
participants (3)
-
Ashish Patil -
Damian Johnson -
Nick Mathewson