[tor-commits] [tech-reports/master] Add data formats.
karsten at torproject.org
karsten at torproject.org
Wed Oct 30 18:44:39 UTC 2013
commit 268ea07f7abf79e6ae39dc6b34b2fd5c9456ce83
Author: Karsten Loesing <karsten.loesing at gmx.net>
Date: Wed Sep 18 12:08:48 2013 +0200
Add data formats.
Based on discussion in #8662.
---
2013/torperf2/torperf2.tex | 161 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 161 insertions(+)
diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex
index abc9c39..9a0b080 100644
--- a/2013/torperf2/torperf2.tex
+++ b/2013/torperf2/torperf2.tex
@@ -3,6 +3,7 @@
\usepackage{graphicx}
\usepackage{enumerate}
\usepackage{hyperref}
+\usepackage{longtable}
\begin{document}
@@ -238,6 +239,9 @@ It would be useful to have the accumulator warn if a remote service
instance becomes stale and doesn't serve recent results anymore.
Note how the service instance name or identifier should become part of
Torperf results meta data.
+The accumulating Torperf instance must verify that all requested data
+contains the expected Torperf service identifier and otherwise discard the
+received data.
\subsubsection{Truncate old results}
@@ -298,6 +302,7 @@ that one that uses optimistic data.%
\footnote{\url{https://trac.torproject.org/projects/tor/ticket/3875}}
\subsubsection{Static file downloads}
+\label{sec:static-file-downloads}
Another major reason for rewriting Torperf is to supersede the existing,
bit-rotting codebase.
@@ -465,5 +470,161 @@ Could also become part of stem instead, together with a tutorial.
\bibliography{torperf2}
+\appendix
+\section{Data formats}
+
+In this section we give examples of the suggested data formats to be
+produced by Torperf.
+
+\subsection{File download}
+
+A file download document contains all data and meta data from making a
+static file download over Tor.
+It will only be used by the experiment specified in
+\ref{sec:static-file-downloads}.
+In the following, we give a file download example document and define its
+fields:
+
+\begin{longtable}{p{8.5cm}p{7cm}}
+\verb+{+ \\
+\verb+ "type": "file_download",+ &
+Document type string; always \verb+"file_download"+ for this document
+type; required.\\
+\verb+ "guid": "ec2_9001_1365473921.75",+ &
+Globally unique identifier for a file download document, consisting of
+service identifier, tor SOCKS port, and request start time; required.\\
+\verb+ "source": "ec2",+ &
+Configured service identifier; required.\\
+\verb+ "socks": 9001,+ &
+Configured tor SOCKS port; required.\\
+\verb+ "filesize": 51200,+ &
+Configured file size in bytes; required.\\
+\verb+ "tor_version": "0.2.2.35",+ &
+Tor software version; optional.\\
+\verb+ "start": 1365473921.75,+ &
+Time when the connection process starts; required.\\
+\verb+ "socket": 1365473921.75,+ &
+Time when the socket was created; required.\\
+\verb+ "connect": 1365473921.75,+ &
+Time when the socket was connected; required.\\
+\verb+ "negotiate": 1365473921.75,+ &
+Time when SOCKS 5 authentication methods have been negotiated; required.\\
+\verb+ "request": 1365473921.75,+ &
+Time when the SOCKS request was sent; required.\\
+\verb+ "response": 1365473922.07,+ &
+Time when the SOCKS response was received; required.\\
+\verb+ "datarequest": 1365473922.07,+ &
+Time when the HTTP request was written; required.\\
+\verb+ "dataresponse": 1365473922.40,+ &
+Time when the first response was received; required.\\
+\verb+ "datacomplete": 1365473922.76,+ &
+Time when the payload was complete; required.\\
+\verb+ "writebytes": 82,+ &
+Total number of bytes written; required.\\
+\verb+ "readbytes": 51323,+ &
+Total number of bytes read; required.\\
+\verb+ "didtimeout": False,+ &
+True if the request timed out, False otherwise; optional.\\
+\verb+ "dataperc": {+ &
+Time when $x\%$ of expected bytes were read for
+$x$ = \{ 10, 20, 30, 40, 50, 60, 70, 80, 90 \}; optional.\\
+\verb+ "10": 1365473922.56,+\\
+\verb+ "20": 1365473922.64,+\\
+\verb+ "30": 1365473922.64,+\\
+\verb+ "40": 1365473922.68,+\\
+\verb+ "50": 1365473922.68,+\\
+\verb+ "60": 1365473922.72,+\\
+\verb+ "70": 1365473922.72,+\\
+\verb+ "80": 1365473922.75,+\\
+\verb+ "90": 1365473922.75 },+\\
+\verb+ "stream_guid":+ &
+Global unique identifier of the stream used for this measurement;
+optional.\\
+\verb+ "ec2_9001_1365473922.76_884"+\\
+\verb+}+ & \\
+\end{longtable}
+
+\subsection{Stream}
+
+A stream document contains all data and meta data from opening a stream,
+attaching it to a circuit, maybe detaching and re-attaching it, to closing
+it.
+Any experiment can generate stream documents for the streams created by
+its tor process.
+In the following, we give a stream example document and define its fields:
+
+\begin{longtable}{p{8.5cm}p{7cm}}
+\verb+{+ \\
+\verb+ "type": "stream",+ &
+Document type string; always \verb+"stream"+ for this document type;
+required.\\
+\verb+ "guid": "ec2_9001_1365473922.76_884",+ &
+Globally unique identifier for a stream document, consisting of service
+identifier, tor SOCKS port, and stream creation time; required.\\
+\verb+ "source": "ec2",+ &
+Configured service identifier; required.\\
+\verb+ "socks": 9001,+ &
+Configured tor SOCKS port; required.\\
+\verb+ "create": 1365473922.76,+ &
+Time when the stream was created; required.\\
+\verb+ "stream_id": 884,+ &
+Locally unique stream identifier; required.\\
+\verb+ "circuits": [+ &
+Sequence of circuits that the stream was attached
+to, each identified by its globally unique identifier and optionally
+containing reason and remote reason for detaching; optional.\\
+\verb+ { "circuit_guid":+\\
+\verb+ "ec2_9001_1365473682.13_501",+\\
+\verb+ "reason": "END",+\\
+\verb+ "remote_reason": "DONE" } ]+\\
+\verb+}+\\
+\end{longtable}
+
+\subsection{Circuit}
+
+A circuit document contains all data and meta data about launching and
+creating a circuit.
+Any experiment can generate circuit documents for the circuits built by
+its tor process.
+In the following, we give a circuit example document and define its
+fields:
+
+\begin{longtable}{p{8.5cm}p{7cm}}
+\verb+{+ \\
+\verb+ "type": "circuit",+ &
+Document type string; always \verb+"circuit"+ for this document type;
+required.\\
+\verb+ "guid": "ec2_9001_1365473682.13_501",+ &
+Globally unique identifier for a circuit document, consisting of service
+identifier, tor SOCKS port, and circuit launch time; required.\\
+\verb+ "source": "ec2",+ &
+Configured service identifier; required.\\
+\verb+ "socks": 9001,+ &
+Configured tor SOCKS port; required.\\
+\verb+ "launch": 1365473682.13,+ &
+Time when the circuit was launched; required.\\
+\verb+ "circ_id": 501,+ &
+Locally unique circuit identifier; required.\\
+\verb+ "path": [+ &
+Sequence of relays chosen for the circuit including
+build times since launching the circuit; optional.\\
+\verb+ { "fingerprint":+\\
+\verb+ "$62680CF0743460E5F836F949E37A6DEC22622F9E",+\\
+\verb+ "buildtime": 0.53 },+\\
+\verb+ { "fingerprint":+\\
+\verb+ "$11064D066F892DC38AAEFDA5EDAE1A227D07D182",+\\
+\verb+ "buildtime": 1.10 },+\\
+\verb+ { "fingerprint":+\\
+\verb+ "$35F51DCE73B988CBAE06B06312B4D1271979FE3B",+\\
+\verb+ "buildtime": 1.32 } ],+\\
+\verb+ "buildtimeout": 4.83,+ &
+Circuit build timeout used when building this
+circuit; optional.\\
+\verb+ "quantile": 0.800000+ &
+Circuit build time quantile used to determine
+the circuit build timeout; optional.\\
+\verb+}+\\
+\end{longtable}
+
\end{document}
More information about the tor-commits
mailing list