Hi everyone,
I'm an undergraduate student at University of Campinas, in Brazil, starting now I'll be working all year, >=20h/week on research about and improvements to tor hidden services under the supervision of Prof. Diego F. Aranha.
I'm generally very inexperienced both with Tor (I've been reading papers and specs, but don't know my way around the source yet) and with programming C in the real world (contrived class projects are basically all I've done), so there will be a lot to ramp up to.
Given the above time and experience constraints, I want to 1. do something that is not urgent/blocking; 2. stay as far away as possible from security-sensitive code, and 3. bite off something that is doable, including ramp-up, within a semester.
Going through the "Hidden Services need some love" blog post, I found the item "Analyze Hidden Service Circuit Establishment Timing With Torperf" fitting. From reading the tickets, it seems like I would need to add some instrumentation (#3459) and then do the measurements.
Some questions:
- Is there anything else that would fit the above constraints that you need done more? I'm starting to write the grant proposal (submission is in 2 weeks) and I can still change it to anything else. (I'll dedicate the same amount of time to the project this year regardless of having the grant approved.)
- Who should I report to?
- Assuming I stick to the timing analysis above, are there any specific questions that you would like the analysis to answer?
Thanks a lot!
Cheers, Helder
If you decided to work on profiling hidden service, I would suggest you take a look at chutney[1] and shadow[2]. Torperf is not under active maintenance anymore and it can be easily replaced by chutney.
A fully automated hidden service profiling tool will be very handy. As the community is currently designing next generation hidden service protocol, such tool will help developers evaluate different designs and implementations.
[1] https://gitweb.torproject.org/chutney.git [2] http://shadow.github.io/
On 03/21/2014 03:05 PM, Helder Ribeiro wrote:
Hi everyone,
I'm an undergraduate student at University of Campinas, in Brazil, starting now I'll be working all year, >=20h/week on research about and improvements to tor hidden services under the supervision of Prof. Diego F. Aranha.
I'm generally very inexperienced both with Tor (I've been reading papers and specs, but don't know my way around the source yet) and with programming C in the real world (contrived class projects are basically all I've done), so there will be a lot to ramp up to.
Given the above time and experience constraints, I want to 1. do something that is not urgent/blocking; 2. stay as far away as possible from security-sensitive code, and 3. bite off something that is doable, including ramp-up, within a semester.
Going through the "Hidden Services need some love" blog post, I found the item "Analyze Hidden Service Circuit Establishment Timing With Torperf" fitting. From reading the tickets, it seems like I would need to add some instrumentation (#3459) and then do the measurements.
Some questions:
- Is there anything else that would fit the above constraints that you need
done more? I'm starting to write the grant proposal (submission is in 2 weeks) and I can still change it to anything else. (I'll dedicate the same amount of time to the project this year regardless of having the grant approved.)
Who should I report to?
Assuming I stick to the timing analysis above, are there any specific
questions that you would like the analysis to answer?
Thanks a lot!
Cheers, Helder
tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Hi Qingping, thanks for the help! Answer below:
On Tue, Mar 25, 2014 at 2:44 AM, Qingping Hou dave2008713@gmail.com wrote:
If you decided to work on profiling hidden service, I would suggest you take a look at chutney[1] and shadow[2]. Torperf is not under active maintenance anymore and it can be easily replaced by chutney.
Thanks for the pointers. I'm still not very clear on what each of shadow, chutney, experimentor, torperf or even oprofile should be used for. I'm guessing chutney/oprofile (maybe toperf, originally, too?) are more useful for profiling processes, with less control over what happens in the network, and shadow/experimentor are more useful for network-level simulation. Please correct me if I'm wrong.
A fully automated hidden service profiling tool will be very handy. As the community is currently designing next generation hidden service protocol, such tool will help developers evaluate different designs and implementations.
Great! Do you know what kinds of things are most useful to measure first? Is it more useful at this point to: 1. measure time spent on functions within a process, to see if there's anything taking up too much time, for example, at the hidden service's OP during the handshake; or 2. simulate load on a hidden service and see how request response time climbs with number of clients, etc.?
Thanks!
Cheers, Helder
[1] https://gitweb.torproject.org/chutney.git [2] http://shadow.github.io/
On 03/21/2014 03:05 PM, Helder Ribeiro wrote:
Hi everyone,
I'm an undergraduate student at University of Campinas, in Brazil,
starting
now I'll be working all year, >=20h/week on research about and
improvements
to tor hidden services under the supervision of Prof. Diego F. Aranha.
I'm generally very inexperienced both with Tor (I've been reading papers and specs, but don't know my way around the source yet) and with programming C in the real world (contrived class projects are basically
all
I've done), so there will be a lot to ramp up to.
Given the above time and experience constraints, I want to 1. do
something
that is not urgent/blocking; 2. stay as far away as possible from security-sensitive code, and 3. bite off something that is doable, including ramp-up, within a semester.
Going through the "Hidden Services need some love" blog post, I found the item "Analyze Hidden Service Circuit Establishment Timing With Torperf" fitting. From reading the tickets, it seems like I would need to add some instrumentation (#3459) and then do the measurements.
Some questions:
- Is there anything else that would fit the above constraints that you
need
done more? I'm starting to write the grant proposal (submission is in 2 weeks) and I can still change it to anything else. (I'll dedicate the
same
amount of time to the project this year regardless of having the grant approved.)
Who should I report to?
Assuming I stick to the timing analysis above, are there any specific
questions that you would like the analysis to answer?
Thanks a lot!
Cheers, Helder
tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
On 27/03/14 19:25, Helder Ribeiro wrote:
Great! Do you know what kinds of things are most useful to measure first? Is it more useful at this point to:
- measure time spent on functions within a process, to see if there's
anything taking up too much time, for example, at the hidden service's OP during the handshake; or 2. simulate load on a hidden service and see how request response time climbs with number of clients, etc.?
I have locally a modified version of chutney that is a bit more extensible and have some configuration for it to do Hidden Service stuff. If you bug me in a couple of days, I could make it public, and talk you through using it.
Load testing is something I have actually been looking at over the last few weeks, but I have not yet got anywhere (mostly due to some annoying bugs in Twisted).
On 03/27/2014 03:36 PM, Christopher Baines wrote:
On 27/03/14 19:25, Helder Ribeiro wrote:
Great! Do you know what kinds of things are most useful to measure first? Is it more useful at this point to:
- measure time spent on functions within a process, to see if there's
anything taking up too much time, for example, at the hidden service's OP during the handshake; or 2. simulate load on a hidden service and see how request response time climbs with number of clients, etc.?
I have locally a modified version of chutney that is a bit more extensible and have some configuration for it to do Hidden Service stuff. If you bug me in a couple of days, I could make it public, and talk you through using it.
It would be awesome if you push it to upstream so everyone can use it :)
On 27/03/14 22:14, Qingping Hou wrote:
On 03/27/2014 03:36 PM, Christopher Baines wrote:
On 27/03/14 19:25, Helder Ribeiro wrote:
Great! Do you know what kinds of things are most useful to measure first? Is it more useful at this point to:
- measure time spent on functions within a process, to see if there's
anything taking up too much time, for example, at the hidden service's OP during the handshake; or 2. simulate load on a hidden service and see how request response time climbs with number of clients, etc.?
I have locally a modified version of chutney that is a bit more extensible and have some configuration for it to do Hidden Service stuff. If you bug me in a couple of days, I could make it public, and talk you through using it.
It would be awesome if you push it to upstream so everyone can use it :)
A discussion about the aims of chutney needs to take place before this can even be considered, as I am not aware how my changes fit within the wider goals of the project (if any exist). While the modifications fit my use cases, it probably breaks the way in which others use it.
I think that the changes I have made might be beneficial, hence the "make it public", however they are a bit to rushed and rough to be obviously the right direction (at least from my perspective).
On 03/27/2014 03:25 PM, Helder Ribeiro wrote:
Hi Qingping, thanks for the help! Answer below:
On Tue, Mar 25, 2014 at 2:44 AM, Qingping Hou dave2008713@gmail.com wrote:
If you decided to work on profiling hidden service, I would suggest you take a look at chutney[1] and shadow[2]. Torperf is not under active maintenance anymore and it can be easily replaced by chutney.
Thanks for the pointers. I'm still not very clear on what each of shadow, chutney, experimentor, torperf or even oprofile should be used for. I'm guessing chutney/oprofile (maybe toperf, originally, too?) are more useful for profiling processes, with less control over what happens in the network, and shadow/experimentor are more useful for network-level simulation. Please correct me if I'm wrong.
Correct.
A fully automated hidden service profiling tool will be very handy. As the community is currently designing next generation hidden service protocol, such tool will help developers evaluate different designs and implementations.
Great! Do you know what kinds of things are most useful to measure first? Is it more useful at this point to:
- measure time spent on functions within a process, to see if there's
anything taking up too much time, for example, at the hidden service's OP during the handshake; or 2. simulate load on a hidden service and see how request response time climbs with number of clients, etc.?
I would say 2 is better. I have done some initial profiling on low load HS and found that time spent on functions (i.e. computation) is negligible compared the time spent on creating circuits and cell transmitting.
-
Christopher Baines -
Helder Ribeiro -
Qingping Hou