-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey Tim, I see that I may need to set some environment variables as told by the git readme but the error message is on a different env var: cory@Nulix ~/tor/tor =) make test-network make all-am make[1]: Entering directory `/home/cory/tor/tor' make[1]: Leaving directory `/home/cory/tor/tor' ./src/test/test-network.sh test-network.sh: missing 'chutney' in CHUTNEY_PATH (~/tor/chutney/) make: *** [test-network] Error 1 I also tried modifying my .bashrc so that the CHUTNEY_PATH pointed to the executable but no luck. Do I need to set the chutney path? - - Cory -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVluRoAAoJEB7DuCIJauCgUNUQAICDPM+vQ1YPU36B13beWgsX OUN6wQKdjtnRP9aCvusf49vomr+E8g2JcIvR+KiRKTDtnvVUMijsl1TZCmyTptEh 61opZJBweCP78MGuaeMXjJg3Ijw5vFDV6tszJ+Z/5xY/f2PinKO2Db+JWI1/oSP3 kMERE/Va+ehL8u4Nx/meP9VzVQDFZH1nz381FQFAPFYSv9+JR1x4HFGWLElOH0l8 1+hcGhhtRD5mzYoDLIx6Lr/3jIJxmbJiiOmpNQcfW3ISHTrgOB/Wo2n6oO/OcjEk L66YvFpZNYqWQ8ZXf+ZN3cC/+sTpWvYDOcdKv9LJvEmHNSevmU+i2+nMoeanpXzX z5eYNCE9unfitNg6hPoQ8x7wD+LLpBDEA42QoQVzx26WMCk3ziu9JIsI5/VfZ7Is /Ny+gQ4jv1Jlk+SEu6ZmRVuyN8sipXnkseXGcpqVR9HBWe7Hy+RfUpODhR3cOy7o EBClLrP8s4H0PUb2WYU4IQrIp2xy8cqqCIaDhs4XLhdXgaX2667irM9uEPw6ZMhL vceyU0LM0v5O+FfzFtdUC4QwfLAbzbiwRi+w74c6bneWWFTi/w5RRaNUiv/IZVMW XTh22mRIOkZXQ3dTc2TsI4TjhT2U95oMiO8hFjDBTS9RfZk/w+unToyJ2aed4UxK 9WC/r/vqUpgD4mljBr76 =9Cvw -----END PGP SIGNATURE-----
On 4 Jul 2015, at 05:37 , Cory Pruce <corypruce@gmail.com> wrote:
Signed PGP part Hey Tim,
I see that I may need to set some environment variables as told by the git readme but the error message is on a different env var:
cory@Nulix ~/tor/tor =) make test-network make all-am make[1]: Entering directory `/home/cory/tor/tor' make[1]: Leaving directory `/home/cory/tor/tor' ./src/test/test-network.sh test-network.sh: missing 'chutney' in CHUTNEY_PATH (~/tor/chutney/) make: *** [test-network] Error 1
I also tried modifying my .bashrc so that the CHUTNEY_PATH pointed to the executable but no luck. Do I need to set the chutney path?
The CHUTNEY_PATH variable needs to point to a directory containing a chutney executable. So you seem to have it right the first time. What do you get when you run: ls -l ~/tor/chutney/ I would expect to see an executable script called "chutney" listed in that directory, along with the other chutney distribution files. The test-network.sh script is complaining that the "chutney" script is missing. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/04/2015 12:11 AM, teor wrote:
The CHUTNEY_PATH variable needs to point to a directory containing a
chutney executable.
So you seem to have it right the first time. What do you get when you run: ls -l ~/tor/chutney/
I would expect to see an executable script called "chutney" listed in
that directory, along with the other chutney distribution files. The test-network.sh script is complaining that the "chutney" script is missing.
I got it :-) Thanks! -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVmDV0AAoJEB7DuCIJauCgWKcP/RgIpxwwZ8bb4bp6zcC5Mg4t 489IYkSxnmUQb3+s1jlKPlY3Lb2f5jkvLshWcsKvUzCR7JVVYGw1ZJ+criPD+n0C /hEY5gRfiBectMY482BLETeoO+BTv52Wdxy/wWkMmzYzGeknsK2DmWZPEGVCwfcv NYcwkLb8IUFsHWeNfjacqzcXw2kDwrdzoBwpZzNnBLOh3ZdC2ZvtvwCt0GgT/V/2 IDzouczOiQ9IsWQIJQU2gWWH5+69V+RfuP49Y2LJV0k9Cp9X/9Effr390b6j5yy6 TlFJptvRQ5XMBeQ8ds1ivJXKFxm3u3BPfpL2GNWzTEb8XNhw9strlS0fXF/cYlxH lr0/9TDANros/zcxphi6p4fJWeMwVJDGmYUpHOB2lDCapR35hWbd3NnLaznFK5v3 ktHAaycDn0kOep9nEF9aYkKZDHmG1RxwZUqQzKl4tmftaShTBfboL65PiMtPY4C3 jINfm1jA7brgVMTTnCAFSBgmXWt64vRCDl0mb1AqEe8xJwcPdwYNSmSuMy8tLEtO CQF7/LumZeIqlU3CKZhTi/0FGlTxLrrAqeHb8HFoRvU4UkekWvqthpGOnD0TAGlo uW0FmmVB47KFI7Jcq89wkPY7Dbh65QQ2QZoZIHDFnZlDHzyL62SdcNhwWfvV9r6m EK2MM1QyPH15CMBfS6rh =Dm36 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 One more thing for right now: how should I do benchmarks with chutney. Should I measure the averages of how long it takes to complete the make test-network command? - - Cory -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVmDe/AAoJEB7DuCIJauCgpQQQALOaJ2uJsSWpRIKzQZ4E8bR0 XJV6cY5oUpypvGx0JpvPvfmbKVJey98WwLw1tbyMHBKiq+w+oTdkCyUNiHKBuTTd 694iM7GzPeJFgI4NGbVlPQ+fQmysB8ujK/D4pWiywBm1+Y81p6g4mdBrvvYSy/bF E9PEor3HqYgp24R8FNzXCz6J5/yLKFJV7MHCsKKiPM5xqemULR9rMQTyY71OjTTV OOcVcmatNSaqZvXYkPUvNrHqeLPQ4Ys/bSmqAUXii64kTqlOQpwbmlLk+6j6Ysbw x0W5Ke1BiFkoQQF61U7uUnpnKRwHbzzc4M69wnYm0OmlLD9iEnaQAj3fUCg1c+O+ nLwt5MIT7G/X0aeM/MXdxthmjoCoCYtVeNk9hSk3adSm1E7nUKjws2Uf6H0nsLmC HCAUk6tpAvI+Rl/IE+xwatTWr0TeisELtpcudu+R2qEIsgdi5D6xkoAQ3qTP+Zuy wNjdxNOKhB/KrxJLVeeCTtKrIawpOjKxc4jVV0yOt40jj+b952DKjLOvbzG09f7r 2rf0mw/btvcJOcCMbEggtyZ9PMBdKt9i3zE0Ihofz8o9ILI8YuiklyuxK+zejtFB EPDv64c7dPX0rE7y1jJ+tqfZZFjhV7FaRscNDmo/fw9T78IzDc7gCZY/lxpETjZr tHo7BBNDtFgIRncjenRY =zkhF -----END PGP SIGNATURE-----
On 5 Jul 2015, at 05:45 , Cory Pruce <corypruce@gmail.com> wrote:
One more thing for right now: how should I do benchmarks with chutney. Should I measure the averages of how long it takes to complete the make test-network command?
make test-network is dominated by the 25 second delay waiting for the Tor test network to bootstrap. So it's not going to help much. I'm working on a chutney branch to measure bandwidth on "chutney verify", but it doesn't have any command-line arguments yet (it's all constants in the code). I'll see if I can pull it into shape today. https://trac.torproject.org/projects/tor/ticket/14175 Even with these bandwidth measurement changes, there's something else to think about: chutney will measure the combined throughput of 4-5 tor instances, and 4n - 5n cpuworker threads, where n is the number of cores on your machine. But this isn't the performance you're interested in for multithreaded crypto changes - you want to know how a single instance + n cpuworker threads performs. (A chutney test network is far *more* parallel than a typical tor relay.) To get an accurate benchmark, you could run one tor instance per machine, or, at the very least, run the client on a slow machine, and everything else on a fast machine, so that the client's multithreaded crypto is the limiting factor. But this seems like a lot of work, and I'm not sure how much accuracy you'll gain. As a first step, you could minimise the number of tor instances, which might make multithreading improvements easier to measure. You'll find the basic-min network helpful for this: ./src/test/test-network.sh --flavour basic-min Then check if you're using ~100% of all cores when you push large amounts (100MB+) of data through the network using #14175 (when it's done!) If you're not using 100%, then you'll be able to see any multithreaded improvements when you run the test again. If you are seeing 100% usage already, get more cores or more machines, and re-run the tests. Let me know how you go with this. You could also modify tor to use single-hop connections, then measure single-hop bandwidth, by making a 1-hop connection and pushing data through it. There won't be as much client crypto as the 3 or 4-hop scenario; and you'll still have the client and destination on the one machine, unlike the single relay real-world scenario. But it could be closer to real-world multithreaded performance, as you'll only be measuring 2n threads. (Ideally, you want to measure n threads.) You must *never* use a tor binary built like this on the public tor network, as it has no anonymity. To make tor use 1-hop circuits for everything, change DEFAULT_ROUTE_LEN to 1 in or.h End dire warning about loss of anonymity. Of course, 1-hop circuits might hide some subtle multithreading bugs, as there's less crypto happening overall. So please test the correctness of your code with DEFAULT_ROUTE_LEN 3 as well. Give me 8 hours or so to work on #14175, I'll try and get it into a usable state. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/04/2015 06:19 PM, teor wrote:
make test-network is dominated by the 25 second delay waiting for the
Tor test network to bootstrap. So it's not going to help much.
I'm working on a chutney branch to measure bandwidth on "chutney
verify", but it doesn't have any command-line arguments yet (it's all constants in the code). I'll see if I can pull it into shape today.
Awesome I'm excited to try it out :D
Even with these bandwidth measurement changes, there's something else
to think about:
chutney will measure the combined throughput of 4-5 tor instances, and
4n - 5n cpuworker threads, where n is the number of cores on your machine. But this isn't the performance you're interested in for multithreaded crypto changes - you want to know how a single instance + n cpuworker threads performs. (A chutney test network is far *more* parallel than a typical tor relay.)
To get an accurate benchmark, you could run one tor instance per
machine, or, at the very least, run the client on a slow machine, and everything else on a fast machine, so that the client's multithreaded crypto is the limiting factor. But this seems like a lot of work, and I'm not sure how much accuracy you'll gain.
As a first step, you could minimise the number of tor instances, which
might make multithreading improvements easier to measure.
You'll find the basic-min network helpful for this: ./src/test/test-network.sh --flavour basic-min
Then check if you're using ~100% of all cores when you push large amounts (100MB+) of data through the network using #14175 (when it's done!) If you're not using 100%, then you'll be able to see any multithreaded improvements when you run the test again. If you are seeing 100% usage already, get more cores or more machines, and re-run the tests.
Let me know how you go with this. I too was thinking of how I would set up the test environment and this is probably more than I could have come up with in days! I'm all ready to check the cpu usage when you pump out the update.
You could also modify tor to use single-hop connections, then measure single-hop bandwidth, by making a 1-hop connection and pushing data through it. There won't be as much client crypto as the 3 or 4-hop scenario; and you'll still have the client and destination on the one machine, unlike the single relay real-world scenario. But it could be closer to real-world multithreaded performance, as you'll only be measuring 2n threads. (Ideally, you want to measure n threads.)
You must *never* use a tor binary built like this on the public tor network, as it has no anonymity. To make tor use 1-hop circuits for everything, change DEFAULT_ROUTE_LEN to 1 in or.h End dire warning about loss of anonymity.
Of course, 1-hop circuits might hide some subtle multithreading bugs, as there's less crypto happening overall. So please test the correctness of your code with DEFAULT_ROUTE_LEN 3 as well.
Done. I changed it to start with 1 and I will try 3 as well. If I shouldn't test this out on the public tor network, should I set up everything locally?
Give me 8 hours or so to work on #14175, I'll try and get it into a
usable state.
Tim
Dude, thanks a bunch for you help. I'm really excited to start :D I'm going to read through the initial design and the code to see what functions/structures/constants/etc. need to be changed. Let me know when you release #14175 and I'll be happy to be the test guinea pig =-) - - Cory -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVmWdxAAoJEB7DuCIJauCgT28P/R1WliIB8P/3tCCPuSLE1oqX vheyu1ZaPPfYOZqemo5xvScyuoPQjASmlM6EG20IQddSrLdz3MU/eUDD604X/tmY r4/LvrWrHek+ER3kCFYvKvYCLO94pY6Sr5zha01iZHw0Vibvypp8lFQR4HOnjvGt LD0LOHJXwotFcHmHfdElHT3VozFhHPHqrTQlgI8ziFl+i54XPb8yuqRbT0glwX/r WztAjiFzuMybnA18IYFRmruLk2bb9ZgQOBGmonvWEi/FVdxkuiXc6b+UyC1q9IiG 4hdDVuwVvwa4iDJ8fWm1rGd81mMlIU6FuV77UOUZYg8/4EHFfFzzVynm0V1QhXzr /UKhTk/3alqcydDZ3rmeX0wsHpI9RKgrwxSXFTuFm+iydjZRcbeXS9ftv88jykMW RJ7qwmRfib0e1U+Uk3yXHHnb7k1Qt1kHxG5rFlFivsXSE06rVfq2visHlLJopAxJ 0bZ93d37tDN71yi8lBmko4mcmO4iKbTjAiw42h2Nj0iYvaM8FBrAz5i65QrlgBa8 LOTjEcxgFxeglIvInmnlf3muJvKrEYH9ikF2f6Wd1ggIqBzKxPxfyhC/0EPPJ9to PYdqHH2VzwimhwXergEec6EWnTTyYE66iDHszzLV8KZNkFa+7nFq3VNK6KneIwTD uklR2RCxwPjUgNaOwbdF =z2cK -----END PGP SIGNATURE-----
On 6 Jul 2015, at 03:20 , Cory Pruce <corypruce@gmail.com> wrote:
On 07/04/2015 06:19 PM, teor wrote:
You could also modify tor to use single-hop connections, then measure single-hop bandwidth, by making a 1-hop connection and pushing data through it. There won't be as much client crypto as the 3 or 4-hop scenario; and you'll still have the client and destination on the one machine, unlike the single relay real-world scenario. But it could be closer to real-world multithreaded performance, as you'll only be measuring 2n threads. (Ideally, you want to measure n threads.)
You must *never* use a tor binary built like this on the public tor network, as it has no anonymity. To make tor use 1-hop circuits for everything, change DEFAULT_ROUTE_LEN to 1 in or.h End dire warning about loss of anonymity.
Of course, 1-hop circuits might hide some subtle multithreading bugs, as there's less crypto happening overall. So please test the correctness of your code with DEFAULT_ROUTE_LEN 3 as well.
Done. I changed it to start with 1 and I will try 3 as well. If I shouldn't test this out on the public tor network, should I set up everything locally?
Yes, and testing locally gives you much better control of all the factors which affect performance.
Give me 8 hours or so to work on #14175, I'll try and get it into a
usable state.
Dude, thanks a bunch for you help. I'm really excited to start :D I'm going to read through the initial design and the code to see what functions/structures/constants/etc. need to be changed. Let me know when you release #14175 and I'll be happy to be the test guinea pig =-)
Yeah, that was 8 optimistic, uninterrupted hours. Otherwise known as "fantasy" hours. Give me a day or two to fit the work in. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Yes, and testing locally gives you much better control of all the
factors which affect performance.
Can I set everything up using vm's or maybe just a single relay?
Yeah, that was 8 optimistic, uninterrupted hours. Otherwise known as
"fantasy" hours.
Give me a day or two to fit the work in.
Haha I figured :p no worries, just let me know. - - Cory -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVmXkcAAoJEB7DuCIJauCg4xkQAJhL80/YQGVPj5PFvYNFzZ2p S5ctPj+fAqxWLYovqeoPekO+MWVbscB2RRczOMZMdqUyiSEhck+FcI8+Yyf/7g10 aofq389aV2Tn+SNvaY3kNKsfD96TS1b3+Va8+gV5UYNv7ymY/4FZ7zon3rvj4mbX My0d3AzxndCe6V+18D0kew1of60H9GTKwNATfNKxPT4a282ujl6EJcenYH4f407/ SSehXnXtdnSDdmlSSLX8/TTrVxcG0nTMVEOEdpdf383SAjKLzyE5xVUmbzRXZHZx S4+7oJ9gz667vmnTp1lZv4DlSVsbhflsELp5KzYrDuWYHGStyPMJv53J/yftCEdS bZO4LZTILqG9vPW60214TN9esEDLc3q7dD21aIyr1WsYRyw7ord0nKUbwoZXuawm ulGm0yLJdIwfF+GYsNhRYKbZhMBon4GMHbQdEFl+k94nosh21RjhJdeSjcluyLCj CIRhvHjeQV4HV6ehqWMzqDV/dwMPxjrhg1+TppWeLiOAw8ZLLMxSCNgfam8oe/Sd w3Z/JOrpixJX3ceW6Yz4GCan+e3HIuHFwBppdCk8JRBZX95bMxThyLSdnRFZ3GkY o1qBAdnucYpNMvQVux1ioTlYLrHXL82CSaToxWH3aH8WNIE/jUTE5Lt/kiPsbhim Tp6tQ9mpbcCYS1NtRi4/ =tjfB -----END PGP SIGNATURE-----
On 6 Jul 2015, at 04:36 , Cory Pruce <corypruce@gmail.com> wrote:
Yes, and testing locally gives you much better control of all the
factors which affect performance.
Can I set everything up using vm's or maybe just a single relay?
Well, your ideal scenario is a separate box / VM, with at least 2 separate cores, for each of the tor instances you're pushing data through (2-3 with a path length of 1, 4-5 with a path length of 3, as you'll need to account for cannibalization). However, anything you do to approach this ideal scenario is an improvement. And running everything on the same box / VM should still give you some idea, as long as CPU usage on all CPUs isn't ~100%. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Well, your ideal scenario is a separate box / VM, with at least 2
separate cores, for each of the tor instances you're pushing data through (2-3 with a path length of 1, 4-5 with a path length of 3, as you'll need to account for cannibalization).
However, anything you do to approach this ideal scenario is an
improvement. My first thought was this is a job for my r pi but it is only single core. In time I can definitely, find the nodes for the test.
And running everything on the same box / VM should still give you some
idea, as long as CPU usage on all CPUs isn't ~100%.
Haha I guess this is a "we'll wait and see" situation. Let me know if there is anything I can do for chutney as well - - Cory -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVmdbuAAoJEB7DuCIJauCgGhEP/16kpyoOqg0RPJC2GMKTOD+W FerMLlMkD9++VcL31cEB88Tos1A69m13Oi7W81vsRxP9MANJpbB2KLnndIxdvmI0 1lt1kWI8JquHquTb7jcFBz8aWamDyMMaP13qDO1xGBl11w1p+Cdo9HM0LYzwrGDK frlJC8/SeKL0oZJfzT+ZO2Nd7EhQbfkSjimD2+ZDTSFhB4cYv/dkLSNLTUdyvFd4 p3/YPR7QRe5Ebef4GgJjhx0NEDWn6JXasDaTxo6qa5K9xOCZxRv32ZS1e5cILTXK 1pd4NUytpdFmm5YB7gCsSpubC4FWIfaSr/NS1O9F/6LQXXsaLKVITqaA7hvyicQQ LMyWVkmt5aoKAOJ8nAub0OuoRxnUUwz3NR364KQCgKRHWcOAHBtlomsx3N2xb++R PDq/MkBajWBUxxWpQnRfp4860Vhpg3JvzOGLqz/AwZ+ZKTBX4KCFCzcK9NtJaczw VC705A7DK5QcrZfa3j92DVA8osoCOl6cRd+SDCll4pR8zNOG6H9FY56wPQ/GCoi8 76rON+Hkgfu1VDsh/BJ/vMCldGkMpNL3OAaRu40qsAcMt76x+qKr9wj6iQubbfJW o1t4+LP11F0S7zTb2R0CA0e+1OqPC+Mx85iFG0C7/yifKXZrUCRH0vj8/ztekWEM I8yILyPsaHhD23runbwv =xUz2 -----END PGP SIGNATURE-----
On 6 Jul 2015, at 11:16 , Cory Pruce <corypruce@gmail.com> wrote:
And running everything on the same box / VM should still give you some
idea, as long as CPU usage on all CPUs isn't ~100%.
Haha I guess this is a "we'll wait and see" situation. Let me know if there is anything I can do for chutney as well
Well, you could test my latest branches for #14175: https://trac.torproject.org/projects/tor/ticket/14175#comment:8 There's a branch which modifies src/test/test-network.sh in tor, and another with the performance measurement code in chutney. The command-line arguments/environmental variables are in the ticket, and I've modified the chutney README to include performance testing. Let me know if anything isn't clear. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Well, you could test my latest branches for #14175:
Hey Tim, I got the branch of chutney and tor and made sure that the commands you run in the comments of the issue exist. What do you think would be a good way to start testing? Begin with a static analysis of the code? Verify that the bandwidth is correct? Let me know what you think is important/feasible. - - Cory -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVnwgeAAoJEB7DuCIJauCgPCQP+wb1uMbW3eo0ATq0aahxrwoK Nc9H7p1vtYfLNeN1Kai4XDOetxNmPSYhhuYRpDuSeZ1OJjBqAJ/EI9gl0iDRzaP+ byFA8VrLQTrJ1j7lj0Dz/foIOlcsnKyufRtVwy/yWGkbBpG6u+UWvWVrNyU9iy3w sNyMiniqiI4Z9SLhVtO/oV+Jb1UMSGaj2//nRoxhGY1uv1ExRcjXhMIaPeQFgRO7 BJhhjIo9RtV0pN/Z0ZrrU8xbA3XooRTBjXyoZxLLp7fi/YRLsUywn4qJ/jjh9NhG 6OeoYTdD2Na1HOX7oBNsDteoobE7LSnuc8RUNtu2wyvlyzL58RvHyADmuoJELZkf 9AXMIjxELmRLl0aYN2B+Ge/0/QhQTpa/fTKsj4WUG+KYMRbhDD2mfJo/K0noByao bJsbnuucYu0ie/y608aiGwGxmSJIZy3GnCdu9dUzqassAGGkVGdgQ+fXkVVFxUrH HwFUH2eLle3rkGO1a5q0GSOxzCJhD3sZoI2HIjjCPw+oaAgNMC5k7joAl/9DZ8rH IFUshRDgjHLnIlACAOZjmc4hbelVApFtddVzBJHxaGkmKT60p8ZIRljifNRFYP2L C186YksygXjDGfxOSoDVCP6TOlKN7J8HRxcepf+9Gi/+Vp7W6aLOIjc+S+bWOWMP 1YaAhNWDQR8Jiy5v+8eF =wqKb -----END PGP SIGNATURE-----
On 10 Jul 2015, at 09:47 , Cory Pruce <corypruce@gmail.com> wrote:
Signed PGP part
Well, you could test my latest branches for #14175:
Hey Tim, I got the branch of chutney and tor and made sure that the commands you run in the comments of the issue exist. What do you think would be a good way to start testing? Begin with a static analysis of the code?
If you can read Python and shell script, then checking I haven't made any obvious coding errors in my changes would help. But that might require becoming familiar with the codebase - which may take some effort. The diffs are here, or you can use git diff: https://github.com/teor2345/chutney/compare/feature14175-chutney-performance... https://github.com/teor2345/tor/compare/feature14175-chutney-performance-v2 Also, I was only using Python 2, so I might have accidentally introduced some incompatibilities with Python 3.
Verify that the bandwidth is correct?
Since it's the localhost, CPU-limited, massively-parallel bandwidth, there's no "correct" value. I'm not even sure what sane values are, but we'll get an idea once people start competing for the biggest numbers.
Let me know what you think is important/feasible.
Does it run? When you make performance improvements, does the bandwidth increase? (Or, far more easily: when you deliberately slow down the code, does the bandwidth tank?) Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7
On 10 Jul 2015, at 11:35 , teor <teor2345@gmail.com> wrote:
On 10 Jul 2015, at 09:47 , Cory Pruce <corypruce@gmail.com> wrote:
Signed PGP part
Well, you could test my latest branches for #14175:
Hey Tim, I got the branch of chutney and tor and made sure that the commands you run in the comments of the issue exist. What do you think would be a good way to start testing? Begin with a static analysis of the code?
If you can read Python and shell script, then checking I haven't made any obvious coding errors in my changes would help. But that might require becoming familiar with the codebase - which may take some effort.
The diffs are here, or you can use git diff: https://github.com/teor2345/chutney/compare/feature14175-chutney-performance... https://github.com/teor2345/tor/compare/feature14175-chutney-performance-v2
This commit contains all the tor changes: https://github.com/teor2345/tor/commit/128d4a68967fb2986965e9f8443f362a9dc20... (The github comparison picked the wrong base branch, so it's huge and unhelpful.) Tim Wilson-Brown (teor) teor2345 at gmail dot com pgp ABFED1AC https://gist.github.com/teor2345/d033b8ce0a99adbc89c5 teor at blah dot im OTR D5BE4EC2 255D7585 F3874930 DB130265 7C9EBBC7
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
If you can read Python and shell script, then checking I haven't made
any obvious coding errors in my changes would help. But that might require becoming familiar with the codebase - which may take some effort.
The diffs are here, or you can use git diff:
https://github.com/teor2345/chutney/compare/feature14175-chutney-performance...
https://github.com/teor2345/tor/compare/feature14175-chutney-performance-v2 Awesome, yup I can read those languages. I'll hopefully wrap my head around chutney's and tor's code in a reasonable amount of time. Gotta start somewhere =-)
When you make performance improvements, does the bandwidth increase? (Or, far more easily: when you deliberately slow down the code, does
the bandwidth tank?) Sounds like a good idea to me! Thanks again, Cory -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJVn2xXAAoJEB7DuCIJauCg38cQAKWH+Ovs0Kko+dyXHaKyysnH pvxViQqLSWvXGdz09U+hBlzH743PigD7l8ibUFbyWaDz62WcXQRw6oo6OqgTJjN5 x1C3aTcL/766wfTHqXWUUfqgWB+gemrBsW7W8bZHJzvHUcs6y69KX1gUVavyJQuj aGM5JTkXftdJgkWZTGnIPAtcFPWV+nKY/k2bUo/jZQaLz6uUdklM1KYH/8u2ZqE/ f/Q/bPOyO2yG2Y9BJ9ZPAYjDB5ENMuSVUdk2/NJuEubBEEd2LvVLlufu3m5uuYn6 sW44pVbU2zk7NtbTeY+uVFacLjbMDXun39hL+PXM4SSfp+Tvtgbq28Z6L6qT6Bgu +YKUvn4xmo4jScEYKItCP/JMhz2ssvcHvGSSOa7dP1rdFqeluQQUmp7bYDXGcz5d fZszqX/0uRXYe3hf+ln7fxPoCsLZtg3W9EcfmSrutPLyifwXpKGMjRoDbe2BRvm9 OMmvTWCj0UiLzz2pHGblaFNpw1l+3uEvzXUca08eLCVLIZZE2awc1je+OBggoY7w urVQXIc9vAl6kIirrQVOLssSZrjFbsDEpnEpr4IoONeW0M+WkEO92roOBIQp+C/O eHp93NPoGdKlU1U3bFwaDKqXYbBoF0GzC+cI5UO5Jqyq59muQFxBeokW6vX/RNPj 3zblrb+9uqADwl4kRu39 =s11f -----END PGP SIGNATURE-----
On Sun, 05 Jul 2015 10:20:50 -0700 Cory Pruce <corypruce@gmail.com> wrote:
Dude, thanks a bunch for you help. I'm really excited to start :D I'm going to read through the initial design and the code to see what functions/structures/constants/etc. need to be changed. Let me know when you release #14175 and I'll be happy to be the test guinea pig =-)
Maybe look at these for a start. It's something that's on my TODO list, but I wouldn't complain if someone else happens to do it before I do, and it would help HS scalability considerably[0]. https://trac.torproject.org/projects/tor/ticket/13737 https://trac.torproject.org/projects/tor/ticket/13738 (If you happen to be more interested in making non-HS use cases faster, then look elsewhere. :P) Regards, -- Yawning Angel [0]: I do have a branch that makes circuit build crypto substantially faster that I've been poking at so the situation will be better in 0.2.7.x, but it's not via threading...
participants (3)
-
Cory Pruce -
teor -
Yawning Angel