October 2013 - tor-commits - lists.torproject.org

[tech-reports/master] Add feedback from Sathya.
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit d6cba8aa8383555d9e12e0e791c1466b055b8e45 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Sep 24 12:00:45 2013 +0200 Add feedback from Sathya. --- 2013/torperf2/torperf2.tex | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex index 9a0b080..1fe89ec 100644 --- a/2013/torperf2/torperf2.tex +++ b/2013/torperf2/torperf2.tex @@ -148,6 +148,12 @@ it as soon as practical without additional user intervention. % speak Torperf's programming language. If people want to add a simple % experiment, they can write a wrapper for using their tool in Torperf. +This requirement is still subject to discussion, because it's unclear how +this will work when users just \verb+apt-get install torperf+. +Ideally if someone writes a good experiment, they should send the patches +upstream and get it merged, and then we update Torperf to include those +tests and then the users just update Torperf with their package managers. + \subsubsection{User-defined tor version or binary} A key part of measurements is the tor software version or binary used to @@ -169,7 +175,7 @@ across tor releases. Note that the tor version should be contained in the results. It might be beneficial to provide a mechanism to download and verify the -signature of new tor versions as they are released. The user could speficy +signature of new tor versions as they are released. The user could specify if they plan to test stable, beta or alpha versions of tor with their Torperf instance. @@ -439,7 +445,7 @@ set up and tear down hidden services. should take care of the entire experiments lifecycle and reserve an exclusive tor instance for its lifetime. Finally, collect the results and post-process them (if applicable) before saving the data to the -results database. If the experiment fails it should be possible to track +results data store. If the experiment fails it should be possible to track down the reason for failure from looking at the experiment results. \item[request scheduler] Start new requests following a previously configured schedule. @@ -459,8 +465,8 @@ median weg page, accept POST requests, provide measurement results via RESTful API, present measurement results on web page. \item[Alexa top-X web pages updater] Periodically retrieve list of top-X web pages. -\item[results database] Store request details, retrieve results, -periodically delete old results if configured. +\item[results data store] Store request details, retrieve results, +periodically delete old results if configured, possibly using a database. \item[results accumulator] Periodically collect results from other Torperf instances, warn if they're out of date. \item[analysis scripts] Command-line tools to process measurement results

1 0

[tech-reports/master] Add feedback from Sathya and Kevin Butler.
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 779143997c9a4ee8b511a1c9b628942d39e27263 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Mon Sep 16 10:45:17 2013 +0200 Add feedback from Sathya and Kevin Butler. --- 2013/torperf2/torperf2.tex | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex index 6cfbcad..029d8a1 100644 --- a/2013/torperf2/torperf2.tex +++ b/2013/torperf2/torperf2.tex @@ -103,6 +103,23 @@ Fortunately, running client and server on the same physical should not have any effect on measurement results, because all requests and responses traverse the Tor network. +Reasons for \emph{not} separating client and server are: +\begin{itemize} +\item it's easier to deploy client and server together than deploying them +separately and configuring the client to use the server; +\item measurement results can easily contain client and server timestamps +of a measurement; +\item assuming the host has sufficient free bandwidth capacity, measurement +results shouldn't be affected by client and server running on the same +host; and +\item it's not unusual to deploy client and server talking over Tor in a +single tool: think of how a tor relay does bandwidth self-tests to itself. +\end{itemize} + +What's not easily possible with this approach is to configure client and +server to run on different hosts. Is there an experiment where this +might be useful? + \subsubsection{User-defined tor version or binary} A key part of measurements is the tor software version or binary used to @@ -343,7 +360,9 @@ txtorcon for communicating with tor clients, and stem for event parsing. The following list outlines tasks that the new Torperf needs to perform. These could be implemented as Python classes, though this list was not -written with Python or Twisted specifics in mind: +written with Python or Twisted specifics in mind:% +\footnote{An alternative to Python/Twisted might be Go, though it doesn't +have tor controller libs like txtorcon or stem.} \begin{description} \item[configuration handler] Validate and parse configuration file or

1 0

[tech-reports/master] Added some ideas on extensibility
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 0628a193d3b33b37b8fe44a03c0d39d4d5f071a7 Author: Kevin Butler <haqkrs(a)gmail.com> Date: Wed Sep 18 02:40:06 2013 +0100 Added some ideas on extensibility --- 2013/torperf2/torperf2.tex | 51 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 48 insertions(+), 3 deletions(-) diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex index 029d8a1..ceb5677 100644 --- a/2013/torperf2/torperf2.tex +++ b/2013/torperf2/torperf2.tex @@ -120,6 +120,24 @@ What's not easily possible with this approach is to configure client and server to run on different hosts. Is there an experiment where this might be useful? +\subsubsection{Experiment extensibility considerations} + +It should be easy for a user to implement or install an experiment that isn't +bundled with the core distribution. Installing an experiment should be as +simple as unzipping a folder or config file into an experiments folder. +Optimally, the Torperf service would detect the new experiment and schedule +it as soon as practical without additional user intervention. + +For implementing, this will require a clearly defined interface for developers +to work with. +\footnote{Ideally, this interface would allow experiment implementations in +any language, not only the language of choice for the core Torperf service.} +This could be as simple as conforming to a common data format or a tight +integration with the core service components which could allow lighter weight +experiment implementations. If the experiment reuses the services +file server implementation, the service should post-process the result +file(s) to add any additional serverside timing information available. + \subsubsection{User-defined tor version or binary} A key part of measurements is the tor software version or binary used to @@ -128,8 +146,22 @@ It should be easy to specify a tor version that is not the current tor version shipped with the OS. Similarly, it should be easy to point to a custom tor binary to use for measurements. + It should be possible to run different experiments with different tor versions or binaries in the same Torperf service instance. +An experiment should be able to be configured to run for multiple +versions of tor, potentially simultaneously, to spot performance change +across tor releases. Note that the tor version should be contained in the +results. +\footnote{Loosely specified tor versions, e.g. 'latest-stable, git:master, +\textasciitilde>0.2.4', +would be a boon in reducing error prone configuration updates everytime +there are new tor versions released.} + +It might be beneficial to provide a mechanism to download and verify the +signature of new tor versions as they are released. The user could speficy +if they plan to test stable, beta or alpha versions of tor with their +torperf instance. \subsubsection{User-defined third-party software version or binary} @@ -137,7 +169,7 @@ Similar to the previous requirement, it should be easy to specify custom versions or binaries of third-party software other than the version currently shipped with the OS. This applies, for example, to Firefox when attempting to make measurements -more reaslistic. +more realistic. \subsection{Results requirements} @@ -175,10 +207,14 @@ plausible, too. \subsubsection{Results web interface} Torperf should provide all its results via a public web interface. +\footnote{It is a requirement to also provide a machine readable representation +to enable easier development of custom visualisations of the result data.} This includes measured timestamps as well as any meta data collected in an experiment. Results should be available for all measurements as well as for a subset -specified by measurement time or filtered by other criteria. +specified by measurement time or filtered by other criteria. The interface +should highlight experiment failures and provide a convenient mechanism to +run a set of experiments on demand. \subsubsection{Results accumulator} @@ -216,7 +252,8 @@ codebases, which is suboptimal.} \subsubsection{Results parsing library} The new Torperf should come with an easy-to-use library to process its -results. +results. The library should also make it simple to create Torperf compatible +results or clearly highlight areas of non conformance with existing results. Alternatively, this library could be provided and maintained as part of stem or another parsing library. @@ -374,6 +411,14 @@ client processes, using a previously configured tor version or binary. \item[tor controller] Connect to tor's control port, force-set guards if required by the experiment, register for and handle asynchronous events, set up and tear down hidden services. +\item[experiment scheduler] Run experiments following a schedule. Must also +handle immediate scheduling requests by users. +\item[experiment runner] Handle the execution of a scheduled experiment. It +should take care of the entire experiments lifecycle and reserve an +exclusive tor instance for its lifetime. Finally, collect the results +and post-process them (if applicable) before saving the data to the +results database.If the experiment fails it should save as much information +as possible for future inspection. \item[request scheduler] Start new requests following a previously configured schedule. \item[request runner] Handle a single request from creation over various

1 0

[tech-reports/master] Add data formats.
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 268ea07f7abf79e6ae39dc6b34b2fd5c9456ce83 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Sep 18 12:08:48 2013 +0200 Add data formats. Based on discussion in #8662. --- 2013/torperf2/torperf2.tex | 161 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 161 insertions(+) diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex index abc9c39..9a0b080 100644 --- a/2013/torperf2/torperf2.tex +++ b/2013/torperf2/torperf2.tex @@ -3,6 +3,7 @@ \usepackage{graphicx} \usepackage{enumerate} \usepackage{hyperref} +\usepackage{longtable} \begin{document} @@ -238,6 +239,9 @@ It would be useful to have the accumulator warn if a remote service instance becomes stale and doesn't serve recent results anymore. Note how the service instance name or identifier should become part of Torperf results meta data. +The accumulating Torperf instance must verify that all requested data +contains the expected Torperf service identifier and otherwise discard the +received data. \subsubsection{Truncate old results} @@ -298,6 +302,7 @@ that one that uses optimistic data.% \footnote{\url{https://trac.torproject.org/projects/tor/ticket/3875}} \subsubsection{Static file downloads} +\label{sec:static-file-downloads} Another major reason for rewriting Torperf is to supersede the existing, bit-rotting codebase. @@ -465,5 +470,161 @@ Could also become part of stem instead, together with a tutorial. \bibliography{torperf2} +\appendix +\section{Data formats} + +In this section we give examples of the suggested data formats to be +produced by Torperf. + +\subsection{File download} + +A file download document contains all data and meta data from making a +static file download over Tor. +It will only be used by the experiment specified in +\ref{sec:static-file-downloads}. +In the following, we give a file download example document and define its +fields: + +\begin{longtable}{p{8.5cm}p{7cm}} +\verb+{+ \\ +\verb+ "type": "file_download",+ & +Document type string; always \verb+"file_download"+ for this document +type; required.\\ +\verb+ "guid": "ec2_9001_1365473921.75",+ & +Globally unique identifier for a file download document, consisting of +service identifier, tor SOCKS port, and request start time; required.\\ +\verb+ "source": "ec2",+ & +Configured service identifier; required.\\ +\verb+ "socks": 9001,+ & +Configured tor SOCKS port; required.\\ +\verb+ "filesize": 51200,+ & +Configured file size in bytes; required.\\ +\verb+ "tor_version": "0.2.2.35",+ & +Tor software version; optional.\\ +\verb+ "start": 1365473921.75,+ & +Time when the connection process starts; required.\\ +\verb+ "socket": 1365473921.75,+ & +Time when the socket was created; required.\\ +\verb+ "connect": 1365473921.75,+ & +Time when the socket was connected; required.\\ +\verb+ "negotiate": 1365473921.75,+ & +Time when SOCKS 5 authentication methods have been negotiated; required.\\ +\verb+ "request": 1365473921.75,+ & +Time when the SOCKS request was sent; required.\\ +\verb+ "response": 1365473922.07,+ & +Time when the SOCKS response was received; required.\\ +\verb+ "datarequest": 1365473922.07,+ & +Time when the HTTP request was written; required.\\ +\verb+ "dataresponse": 1365473922.40,+ & +Time when the first response was received; required.\\ +\verb+ "datacomplete": 1365473922.76,+ & +Time when the payload was complete; required.\\ +\verb+ "writebytes": 82,+ & +Total number of bytes written; required.\\ +\verb+ "readbytes": 51323,+ & +Total number of bytes read; required.\\ +\verb+ "didtimeout": False,+ & +True if the request timed out, False otherwise; optional.\\ +\verb+ "dataperc": {+ & +Time when $x\%$ of expected bytes were read for +$x$ = \{ 10, 20, 30, 40, 50, 60, 70, 80, 90 \}; optional.\\ +\verb+ "10": 1365473922.56,+\\ +\verb+ "20": 1365473922.64,+\\ +\verb+ "30": 1365473922.64,+\\ +\verb+ "40": 1365473922.68,+\\ +\verb+ "50": 1365473922.68,+\\ +\verb+ "60": 1365473922.72,+\\ +\verb+ "70": 1365473922.72,+\\ +\verb+ "80": 1365473922.75,+\\ +\verb+ "90": 1365473922.75 },+\\ +\verb+ "stream_guid":+ & +Global unique identifier of the stream used for this measurement; +optional.\\ +\verb+ "ec2_9001_1365473922.76_884"+\\ +\verb+}+ & \\ +\end{longtable} + +\subsection{Stream} + +A stream document contains all data and meta data from opening a stream, +attaching it to a circuit, maybe detaching and re-attaching it, to closing +it. +Any experiment can generate stream documents for the streams created by +its tor process. +In the following, we give a stream example document and define its fields: + +\begin{longtable}{p{8.5cm}p{7cm}} +\verb+{+ \\ +\verb+ "type": "stream",+ & +Document type string; always \verb+"stream"+ for this document type; +required.\\ +\verb+ "guid": "ec2_9001_1365473922.76_884",+ & +Globally unique identifier for a stream document, consisting of service +identifier, tor SOCKS port, and stream creation time; required.\\ +\verb+ "source": "ec2",+ & +Configured service identifier; required.\\ +\verb+ "socks": 9001,+ & +Configured tor SOCKS port; required.\\ +\verb+ "create": 1365473922.76,+ & +Time when the stream was created; required.\\ +\verb+ "stream_id": 884,+ & +Locally unique stream identifier; required.\\ +\verb+ "circuits": [+ & +Sequence of circuits that the stream was attached +to, each identified by its globally unique identifier and optionally +containing reason and remote reason for detaching; optional.\\ +\verb+ { "circuit_guid":+\\ +\verb+ "ec2_9001_1365473682.13_501",+\\ +\verb+ "reason": "END",+\\ +\verb+ "remote_reason": "DONE" } ]+\\ +\verb+}+\\ +\end{longtable} + +\subsection{Circuit} + +A circuit document contains all data and meta data about launching and +creating a circuit. +Any experiment can generate circuit documents for the circuits built by +its tor process. +In the following, we give a circuit example document and define its +fields: + +\begin{longtable}{p{8.5cm}p{7cm}} +\verb+{+ \\ +\verb+ "type": "circuit",+ & +Document type string; always \verb+"circuit"+ for this document type; +required.\\ +\verb+ "guid": "ec2_9001_1365473682.13_501",+ & +Globally unique identifier for a circuit document, consisting of service +identifier, tor SOCKS port, and circuit launch time; required.\\ +\verb+ "source": "ec2",+ & +Configured service identifier; required.\\ +\verb+ "socks": 9001,+ & +Configured tor SOCKS port; required.\\ +\verb+ "launch": 1365473682.13,+ & +Time when the circuit was launched; required.\\ +\verb+ "circ_id": 501,+ & +Locally unique circuit identifier; required.\\ +\verb+ "path": [+ & +Sequence of relays chosen for the circuit including +build times since launching the circuit; optional.\\ +\verb+ { "fingerprint":+\\ +\verb+ "$62680CF0743460E5F836F949E37A6DEC22622F9E",+\\ +\verb+ "buildtime": 0.53 },+\\ +\verb+ { "fingerprint":+\\ +\verb+ "$11064D066F892DC38AAEFDA5EDAE1A227D07D182",+\\ +\verb+ "buildtime": 1.10 },+\\ +\verb+ { "fingerprint":+\\ +\verb+ "$35F51DCE73B988CBAE06B06312B4D1271979FE3B",+\\ +\verb+ "buildtime": 1.32 } ],+\\ +\verb+ "buildtimeout": 4.83,+ & +Circuit build timeout used when building this +circuit; optional.\\ +\verb+ "quantile": 0.800000+ & +Circuit build time quantile used to determine +the circuit build timeout; optional.\\ +\verb+}+\\ +\end{longtable} + \end{document}

1 0

[tech-reports/master] Add Sathya's comments from tor-dev@.
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 6df54da6f97d3c3127f23b066f1596e9ff62558e Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Sep 24 15:10:04 2013 +0200 Add Sathya's comments from tor-dev@. --- 2013/torperf2/torperf2.tex | 39 ++++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex index d67bd62..b8f2e71 100644 --- a/2013/torperf2/torperf2.tex +++ b/2013/torperf2/torperf2.tex @@ -121,13 +121,27 @@ What's not easily possible with this approach is to configure client and server to run on different hosts. Is there an experiment where this might be useful? -\subsubsection{Experiment extensibility considerations} - -It should be easy for a user to implement or install an experiment that isn't -bundled with the core distribution. Ideally, installing an experiment should be as -simple as unzipping a folder or config file into an experiments folder. -Optimally, the Torperf service would detect the new experiment and schedule -it as soon as practical without additional user intervention. +% Commented out Kevin's text on 2013-09-24: +% \subsubsection{Experiment extensibility considerations} +% +% It should be easy for a user to implement or install an experiment that isn't +% bundled with the core distribution. Ideally, installing an experiment should be as +% simple as unzipping a folder or config file into an experiments folder. +% Optimally, the Torperf service would detect the new experiment and schedule +% it as soon as practical without additional user intervention. +% Sathya on tor-dev@: +% "I don't understand how this will work when users just apt-get install +% torperf. Ideally if someone writes a good experiment, they should send +% the patches upstream and get it merged, and then we update torperf to +% include those tests and then the users just update torperf with their +% package managers." +% Karsten on tor-dev@: +% "I don't feel strongly. I'd prefer a design that makes it easy to add +% new experiments, but I'm fine with an approach that requires merging +% patches. We can always add the functionality to drop something in a +% directory and make Torperf magically detect and run the new experiment, +% but that can happen later. Maybe we shouldn't let that distract us from +% getting the first version done. I commented out this section." % Commented out Kevin's text by Karsten on 2013-09-18: % For implementing, this will require a clearly defined interface for developers @@ -179,6 +193,12 @@ signature of new tor versions as they are released. The user could specify if they plan to test stable, beta or alpha versions of tor with their Torperf instance. +This requirement should be implemented in three phases: 1) Torperf uses +the default tor binary that comes with the operating system; 2) Torperf +accepts the path to a tor binary that was previously built by the user; +3) Torperf accepts that path to a Git tag or tor source directory and +downloads, verifies, and builds a tor binary itself. + \subsubsection{User-defined third-party software version or binary} Similar to the previous requirement, it should be easy to specify custom @@ -217,7 +237,9 @@ requests.% Deciding which data to store should be the responsibility of whoever designs a Torperf experiment, though formats should be somewhat uniform between experiments. -A possible results data format could be based on JSON,% +Torperf should store its results in a format that is widely used and +already has libraries (like JSON), so that other applications can use +the results and build on it.% \footnote{In particular, we could use a de-facto standard like the HAR (HTTP Archive) format to store measurements results. Advantages are that such a format is widely used and that there are a lot @@ -227,7 +249,6 @@ not relevant to us, and that we'd have to add tor-specific data fields to the format which wouldn't be processed by existing tools. Note that we could use such a format only for request data, not for tor-specific data like building circuits and attaching streams.} -but other data formats are plausible, too. \subsubsection{Results web interface}

1 0

[tech-reports/master] Add authors to torperf2 report, tweak text a bit.
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 23efccdcecccb0be555f1471decd40340e4167a5 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Oct 29 17:14:02 2013 +0100 Add authors to torperf2 report, tweak text a bit. --- 2013/torperf2/torperf2.tex | 71 +++++++++++++++++++++++++++----------------- 1 file changed, 44 insertions(+), 27 deletions(-) diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex index b8f2e71..8e522bf 100644 --- a/2013/torperf2/torperf2.tex +++ b/2013/torperf2/torperf2.tex @@ -10,13 +10,14 @@ \title{Requirements and Software Design for a\\ Better Tor Performance Measurement Tool} -\author{Karsten Loesing} +\author{Karsten Loesing, Sathyanarayanan Gunasekaran, and Kevin Butler} -\contact{\href{mailto:karsten@torproject.org}{karsten@torproject.org}} -\reportid{} -\date{\emph{\\--- This report has not been published yet, so it has no -number and its URL is going to change once it's published; better not -reference it yet. Work in progress. ---}} +\contact{\href{mailto:karsten@torproject.org}{karsten@torproject.org},% +\href{mailto:gsathya@torproject.org}{gsathya@torproject.org},% +\href{mailto:haqkrs@gmail.com}{haqkrs@gmail.com}} + +\reportid{2013-10-002 (DRAFT)} +\date{October 31, 2013} \maketitle @@ -35,7 +36,7 @@ Also, Torperf consists of a bunch of shell scripts which makes it neither very user-friendly to set up and run, nor extensible to cover new use cases. -For reference, we made an earlier approach 1.5 later that suggested +For reference, we made an earlier approach 1.5 years later that suggested redesigning the Python parts in Torperf, but that redesign never happened.% \footnote{\url{https://trac.torproject.org/projects/tor/ticket/2565}} @@ -49,13 +50,10 @@ After that, we discuss actual functional requirements, so experiments that we want to perform to measure things in the Tor network on a regular and automated basis. Finally we suggest a software design that fulfills all these requirements. -This report is meant to be updated in the process of writing this new -software, so that it can later serve as documentation. -Only then it's going to be published officially. \section{Requirements} -Before we talk about functional requirements, that is, actual experiments +Before we talk about actual experiments the new Torperf is supposed to perform, we want to discuss more general requirements of a Torperf rewrite. For the time being, assume that the major functional requirement is to @@ -87,7 +85,8 @@ The new Torperf should run as OS service, unrelated to a specific user. It should have a config file that is easy to understand, and it should come with scripts to start, restart, and stop the Torperf service. -If the service operator wants to run multiple experiments at once, they'd +If the service operator wants to run multiple experiments in parallel, +they'd simply configure all those experiments in the configuration file or directory, rather than starting multiple instances of Torperf. @@ -118,8 +117,9 @@ single tool: think of how a tor relay does bandwidth self-tests to itself. \end{itemize} What's not easily possible with this approach is to configure client and -server to run on different hosts. Is there an experiment where this -might be useful? +server to run on different hosts. +It's unclear yet whether there is an experiment where this might be +useful. % Commented out Kevin's text on 2013-09-24: % \subsubsection{Experiment extensibility considerations} @@ -162,11 +162,11 @@ might be useful? % speak Torperf's programming language. If people want to add a simple % experiment, they can write a wrapper for using their tool in Torperf. -This requirement is still subject to discussion, because it's unclear how -this will work when users just \verb+apt-get install torperf+. -Ideally if someone writes a good experiment, they should send the patches -upstream and get it merged, and then we update Torperf to include those -tests and then the users just update Torperf with their package managers. +% This requirement is still subject to discussion, because it's unclear how +% this will work when users just \verb+apt-get install torperf+. +% Ideally if someone writes a good experiment, they should send the patches +% upstream and get it merged, and then we update Torperf to include those +% tests and then the users just update Torperf with their package managers. \subsubsection{User-defined tor version or binary} @@ -190,13 +190,13 @@ results. It might be beneficial to provide a mechanism to download and verify the signature of new tor versions as they are released. The user could specify -if they plan to test stable, beta or alpha versions of tor with their +if they plan to test stable, beta, or alpha versions of tor with their Torperf instance. This requirement should be implemented in three phases: 1) Torperf uses the default tor binary that comes with the operating system; 2) Torperf accepts the path to a tor binary that was previously built by the user; -3) Torperf accepts that path to a Git tag or tor source directory and +3) Torperf accepts the path to a Git tag or tor source directory and downloads, verifies, and builds a tor binary itself. \subsubsection{User-defined third-party software version or binary} @@ -220,7 +220,9 @@ But at the same time we want to keep it as easy as possible to process results. Results may come from various sources, e.g., an HTTP/SOCKS client, -Selenium/Firefox, a tor client used to make the request or to answer the +Selenium% +\footnote{\url{http://www.seleniumhq.org/}}% +/Firefox, a tor client used to make the request or to answer the request as hidden service, an HTTP server, etc. Torperf should not store original logs, because that would only shift the issue of processing different log formats to the analysis step. @@ -228,12 +230,14 @@ Such an approach would also generate unnecessarily large results files. Torperf should rather process data from these sources and store them in a custom results format that can be easily processed using tools shipped with Torperf. + Results may include data which appear not immediately relevant to measuring Tor performance, but which may be useful for related purposes. For example, Torperf should include data about circuit failures in its results, even though these circuits may not have been used in actual requests.% \footnote{\url{https://trac.torproject.org/projects/tor/ticket/8662}} + Deciding which data to store should be the responsibility of whoever designs a Torperf experiment, though formats should be somewhat uniform between experiments. @@ -307,7 +311,9 @@ The new Torperf should come with an easy-to-use library to process its results. The library should also make it simple to create Torperf compatible results or clearly highlight areas of non conformance with existing results. Alternatively, this library could be provided and maintained as part of -stem or another parsing library. +stem% +\footnote{\url{https://stem.torproject.org/}} +or another parsing library. \section{Experiments} @@ -445,8 +451,12 @@ deploying Twisted applications, rather than forcing us to solve these problems yet another time. Of course, at the same time it forces us to follow the Twisted model of writing applications, which seems not too bad in this case. -The current prototype also requires twisted-socks as SOCKS client, -txtorcon for communicating with tor clients, and stem for event parsing. +The current prototype also requires twisted-socks% +\footnote{\url{https://github.com/ln5/twisted-socks}} +as SOCKS client, +txtorcon% +\footnote{\url{https://github.com/meejah/txtorcon}} +for communicating with tor clients, and stem for event parsing. The following list outlines tasks that the new Torperf needs to perform. These could be implemented as Python classes, though this list was not @@ -504,6 +514,11 @@ and produce graphs and other aggregate statistics. Could also become part of stem instead, together with a tutorial. \end{description} +\section*{Acknowledgments} + +Lunar, Zack Weinberg, and Rob van der Hoeven provided valuable feedback +for this report. + \bibliography{torperf2} \appendix @@ -596,7 +611,8 @@ Document type string; always \verb+"stream"+ for this document type; required.\\ \verb+ "guid": "ec2_9001_1365473922.76_884",+ & Globally unique identifier for a stream document, consisting of service -identifier, tor SOCKS port, and stream creation time; required.\\ +identifier, tor SOCKS port, and stream creation time, and +locally unique stream identifier; required.\\ \verb+ "source": "ec2",+ & Configured service identifier; required.\\ \verb+ "socks": 9001,+ & @@ -632,7 +648,8 @@ Document type string; always \verb+"circuit"+ for this document type; required.\\ \verb+ "guid": "ec2_9001_1365473682.13_501",+ & Globally unique identifier for a circuit document, consisting of service -identifier, tor SOCKS port, and circuit launch time; required.\\ +identifier, tor SOCKS port, circuit launch time, and +locally unique circuit identifier; required.\\ \verb+ "source": "ec2",+ & Configured service identifier; required.\\ \verb+ "socks": 9001,+ &

1 0

[tech-reports/master] Minor tweaks to torperf2 report.
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 1442202573b6b4ed2fe803e24d6e09f96e3ec391 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Oct 30 19:42:24 2013 +0100 Minor tweaks to torperf2 report. --- 2013/torperf2/torperf2.tex | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex index 8e522bf..bc0d896 100644 --- a/2013/torperf2/torperf2.tex +++ b/2013/torperf2/torperf2.tex @@ -14,10 +14,10 @@ Better Tor Performance Measurement Tool} \contact{\href{mailto:karsten@torproject.org}{karsten@torproject.org},% \href{mailto:gsathya@torproject.org}{gsathya@torproject.org},% -\href{mailto:haqkrs@gmail.com}{haqkrs@gmail.com}} +\href{mailto:kbutler03@qub.ac.uk}{kbutler03@qub.ac.uk}} -\reportid{2013-10-002 (DRAFT)} -\date{October 31, 2013} +\reportid{2013-10-002} +\date{October 30, 2013} \maketitle

1 0

[tech-reports/master] Merge branch 'torperf2'
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 5d742f62e9faea0f8acd303792dd7e7aa938fd6e Merge: 329bd05 1442202 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Oct 30 19:42:39 2013 +0100 Merge branch 'torperf2' 2013/torperf2/.gitignore | 2 + 2013/torperf2/torperf2.bib | 10 + 2013/torperf2/torperf2.tex | 683 ++++++++++++++++++++++++++++++++++++++++++ 2013/torperf2/tortechrep.cls | 1 + 4 files changed, 696 insertions(+)

1 0

[tech-reports/master] Update .gitignore.
by karsten＠torproject.org 30 Oct '13

30 Oct '13

commit 1b318c873516da7ffe5293b513953c519b991da3 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Oct 30 19:43:23 2013 +0100 Update .gitignore. --- 2013/torperf2/.gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/2013/torperf2/.gitignore b/2013/torperf2/.gitignore index 78ee6f6..4092f13 100644 --- a/2013/torperf2/.gitignore +++ b/2013/torperf2/.gitignore @@ -1,2 +1,3 @@ torperf2.pdf +torperf2-2013-10-30.pdf

1 0

[translation/vidalia] Update translations for vidalia
by translation＠torproject.org 30 Oct '13

30 Oct '13

commit c71eaaa09d37d4c2fcc2510378351ac24fbce740 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Oct 30 17:45:13 2013 +0000 Update translations for vidalia --- zh_TW/vidalia_zh_TW.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zh_TW/vidalia_zh_TW.po b/zh_TW/vidalia_zh_TW.po index 570d3e1..8da1591 100644 --- a/zh_TW/vidalia_zh_TW.po +++ b/zh_TW/vidalia_zh_TW.po @@ -10,7 +10,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2012-03-21 17:52+0000\n" -"PO-Revision-Date: 2013-10-30 17:10+0000\n" +"PO-Revision-Date: 2013-10-30 17:20+0000\n" "Last-Translator: wongyinho981 <wongyinho981(a)gmail.com>\n" "Language-Team: Chinese (Taiwan) (http://www.transifex.com/projects/p/torproject/language/zh_TW/)\n" "MIME-Version: 1.0\n"

1 0