Hello Karsten,
hope you are doing well!
I've been working on the S61 performance experiments [0] and I would appreciate some help with onionperf.
I have done various onionperf measurements using something the following command: $ onionperf measure -i --tgen ~/tgen/build/src/tgen --tor ~/onionperf/tor/src/app/tor --drop-guards 10
I put each of the measurements on a different directory and now I want to analyze them and derive the CDF-TTFB graphs etc. I attempted doing that using the following calls:
$ onionperf analyze --tgen ./tgen-client/onionperf.tgen.log --torctl ./tor-client/onionperf.torctl.log $ onionperf visualize --data onionperf.analysis.json.xz "test"
Unfortunately, the 'visualize' call can fail for the attached 'onionperf-mbps.json.xz':
$ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements" 2020-11-03 15:51:31 1604411491.540736 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz 2020-11-03 15:51:31 1604411491.577864 [onionperf] [INFO] done! 2020-11-03 15:51:31 1604411491.586845 [onionperf] [INFO] NumExpr defaulting to 8 threads. /user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding. /user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding. Traceback (most recent call last): File "/user/.local/bin/onionperf", line 4, in <module> __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf') File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script exec(script_code, namespace, namespace) File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module> File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 48, in plot_all File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 205, in __plot_throughput_ecdf File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 235, in __draw_ecdf File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 5000, in dropna raise KeyError(list(np.compress(check, subset))) KeyError: ['mbps']
Then for another onionperf run (attached as 'onionperf-44028.json.xz') it gave me a different error:
$ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements" 2020-11-03 15:50:03 1604411403.946028 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz 2020-11-03 15:50:03 1604411403.976088 [onionperf] [INFO] done! Traceback (most recent call last): File "/user/.local/bin/onionperf", line 4, in <module> __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf') File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script exec(script_code, namespace, namespace) File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module> File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 38, in plot_all File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 129, in __extract_data_frame KeyError: '44028'
Because of the two errors above I'm unable to proceed with producing any results. Any idea what I might be doing wrong? Is there any other data you might need to help me with this?
Thanks a lot!
[0]: https://trac.torproject.org/projects/tor/wiki/org/roadmaps/CoreTor/Performan... https://gitlab.torproject.org/tpo/core/tor/-/issues/40157
On 2020-11-03 15:01, George Kadianakis wrote:
Hello Karsten,
Hi George!
Indeed, that's a bug in the visualize mode.
However, before it fails it writes a .csv file that tells us why: none of the measurements are successful! I'm seeing lots of TOR/CANT_ATTACH errors in that file. There's something wrong in your measurement setup. If you fix that, you'll be able to visualize the results.
(We should still fix the bug and produce a nicer error message.)
In this case it's failing even before writing a CSV file. Still, when looking at the .xz file I'm seeing only errors, too, so it's a related issue here.
Hope this helps!
All the best, Karsten
On 2020-11-03 17:16, Karsten Loesing wrote:
Hi again!
Did you figure out what went wrong? Do you need help figuring that out?
(We should still fix the bug and produce a nicer error message.)
I'm going to file an issue and start working on a possible fix tomorrow.
All the best, Karsten
Karsten Loesing karsten@torproject.org writes:
Hey hey Karsten,
yes there was an issue with the port forwarding (or the incoming IP addr) and tgen could not do its thing, and I didn't realize because there were not any errors exposed to this effect.
In any case, I fixed this and then onionperf worked just fine. For example see here https://gitlab.torproject.org/tpo/core/tor/-/issues/40157#note_2714605
So no worries about this, it's all good on this front.
Also onionperf has been perfoming just fine in general for the purposes of #40157 so far.
Cheers! (and welcome back (?))
On 2020-11-23 16:49, George Kadianakis wrote:
Hello!
Understood. I'm not sure how to make it easier to spot configuration errors like this in the future. In fact, we had a very similar problem with one of the long-running instances, op-hk5, and it took us two weeks to figure that out. I just opened a issue to discuss possible fixes:
https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/40011
I also opened an issue to avoid the tracebacks you ran into in the future:
https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/40012
Awesome! Please let me know if there's anything else causing trouble. Or just open an issue.
Cheers! (and welcome back (?))
Yes, I'm back!
Cheers! Karsten
Karsten Loesing karsten@torproject.org writes:
Hello again Karsten,
your help was valuable last time and I managed to overcome those issues.
I've been doing more experiments recently (see https://gitlab.torproject.org/tpo/core/tor/-/issues/40157#note_2716591) and while the data collection is still ongoing, I decided to do some visualizations to make sure that I'm not collecting useless data all this time.
So I took collected onionperf data but when I tried to visualize them I got a similar error like the one from my previous email:
""" $ onionperf visualize --data onionperf.analysis.json.xz "bla" 2020-12-02 21:19:04 1606936744.475013 [onionperf] [INFO] loading analysis results from /home/user/tmp/onionperf/onionperf-data-2558933-close/onionperf.analysis.json.xz 2020-12-02 21:19:05 1606936745.245664 [onionperf] [INFO] done! Traceback (most recent call last): File "/home/user/.local/bin/onionperf", line 4, in <module> __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf') File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script self.require(requires)[0].run_script(script_name, ns) File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script exec(script_code, namespace, namespace) File "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module> File "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main File "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize File "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 38, in plot_all File "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 129, in __extract_data_frame KeyError: '36178' """
However this time the onionperf.analysis.json.xz file looks more reasonable. I'm attaching it to this email just in case you have the time to check it out and tell me what might be going wrong.
FWIW, the data was collected in a manner similar to this: $ onionperf measure -i --tgen ./tgen --tor ./tor --torclient-conf-file ./torrc --tgen-connect-ip 1.2.3.4 --tgen-listen-port 443 --tgen-connect-port 443 --tgen-pause-initial 20 --tgen-pause-between 2 --tgen-transfer-size '50 KiB'
Thanks a lot!
On 2020-12-02 20:23, George Kadianakis wrote:
Hi George!
This is related to the issues you saw before. Please try out the patch that I wrote for #40012 last week for the visualization part (no need to re-do the measurements):
$ git remote add karsten https://gitlab.torproject.org/karsten/onionperf.git
$ git fetch karsten
$ git checkout -b task-40012 karsten/task-40012
If you want, please comment on the ticket whether this worked for you!
All the best, Karsten
-
George Kadianakis -
Karsten Loesing