On 9/18/13 3:49 AM, Kevin Butler wrote:
Executing scripts and reading stdout/stderr is probably too low-level. I think we need a Python/Twisted (or whatever language Torperf will be written in) interface for running an experiment and retrieving results.
You're probably right on stdout/err being too low level, but most experiments would be reusing the provided implementation, just wrapping them in a simple manner. Anyway I think there's a language agnostic way to get this working, without forcing any extensibility on matching Torperfs language of choice. I've tried to be pretty generic in my attached changes. :)
I don't see how we could make new experiments language agnostic. These new experiments will want to re-use Torperf classes/components, which they can't do if they're just "shell scripts". They need to implement an interface provided by Torperf, and Torperf needs to provide functionality via an interface that the experiments understand. If an experiment is in fact just a shell script, it should be trivial to write a wrapper class to call that. But I think that's the exception, not the rule.
Or maybe we have a different understanding of an "experiment". Can you give an example for an experiment that is not listed in Section 3 of the design document and state how you'd want to integrate that into Torperf without touching its sources?
Well, thanks for your input! As I said above, it would help a lot if you added these ideas to the appropriate sections of the design document.
Please see attached.
Awesome! I applied your patch, though I tweaked some parts and commented out other parts, explaining my reasons in the LaTeX sources. Happy to discuss these things further if you want!
I also added an Appendix A with suggested data formats. Maybe these data formats make it clearer what I'd expect as output from an experiment.
https://people.torproject.org/~karsten/volatile/torperf2.pdf
If you have further suggestions how to improve the requirements or design, please send me a new patch, and I'll apply it and comment on it. Thanks!
All the best, Karsten