October 2013 - tor-commits - lists.torproject.org

[tor/master] Added no_tempfile parameter to write_chunks_to_file to do non-atomic writes. Implements #1376.
by nickm＠torproject.org 11 Oct '13

11 Oct '13

commit 1bdb391ed0df979bc29ed1b62e7c0f3c9494a8d7 Author: Kevin Butler <haqkrs(a)gmail.com> Date: Sun Sep 1 00:24:07 2013 +0100 Added no_tempfile parameter to write_chunks_to_file to do non-atomic writes. Implements #1376. --- changes/bug1376 | 2 ++ src/common/util.c | 13 ++++++++++--- src/common/util.h | 2 +- src/or/dirvote.c | 2 +- src/or/routerlist.c | 4 ++-- 5 files changed, 16 insertions(+), 7 deletions(-) diff --git a/changes/bug1376 b/changes/bug1376 new file mode 100644 index 0000000..631f2af --- /dev/null +++ b/changes/bug1376 @@ -0,0 +1,2 @@ + o Minor bugfixes: + - Added additional argument to write_chunks_to_file to optionally skip using a temp file to do non-atomic writes. Implements ticket #1376. \ No newline at end of file diff --git a/src/common/util.c b/src/common/util.c index 6e14a58..4e84d94 100644 --- a/src/common/util.c +++ b/src/common/util.c @@ -2190,12 +2190,19 @@ write_chunks_to_file_impl(const char *fname, const smartlist_t *chunks, return -1; } -/** Given a smartlist of sized_chunk_t, write them atomically to a file - * fname, overwriting or creating the file as necessary. */ +/** Given a smartlist of sized_chunk_t, write them to a file + * fname, overwriting or creating the file as necessary. + * If no_tempfile is 0 then the file will be written + * atomically. */ int -write_chunks_to_file(const char *fname, const smartlist_t *chunks, int bin) +write_chunks_to_file(const char *fname, const smartlist_t *chunks, int bin, int no_tempfile) { int flags = OPEN_FLAGS_REPLACE|(bin?O_BINARY:O_TEXT); + + if (no_tempfile) { + // O_APPEND stops write_chunks_to_file from using tempfiles + flags |= O_APPEND; + } return write_chunks_to_file_impl(fname, chunks, flags); } diff --git a/src/common/util.h b/src/common/util.h index 090243e..24428ad 100644 --- a/src/common/util.h +++ b/src/common/util.h @@ -365,7 +365,7 @@ typedef struct sized_chunk_t { size_t len; } sized_chunk_t; int write_chunks_to_file(const char *fname, const struct smartlist_t *chunks, - int bin); + int bin, int no_tempfile); int append_bytes_to_file(const char *fname, const char *str, size_t len, int bin); int write_bytes_to_new_file(const char *fname, const char *str, size_t len, diff --git a/src/or/dirvote.c b/src/or/dirvote.c index 12ceba8..456a033 100644 --- a/src/or/dirvote.c +++ b/src/or/dirvote.c @@ -3143,7 +3143,7 @@ dirvote_compute_consensuses(void) }); votefile = get_datadir_fname("v3-status-votes"); - write_chunks_to_file(votefile, votestrings, 0); + write_chunks_to_file(votefile, votestrings, 0, 0); tor_free(votefile); SMARTLIST_FOREACH(votestrings, sized_chunk_t *, c, tor_free(c)); smartlist_free(votestrings); diff --git a/src/or/routerlist.c b/src/or/routerlist.c index 46da17e..3e8b9fb 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -428,7 +428,7 @@ trusted_dirs_flush_certs_to_disk(void) } DIGESTMAP_FOREACH_END; filename = get_datadir_fname("cached-certs"); - if (write_chunks_to_file(filename, chunks, 0)) { + if (write_chunks_to_file(filename, chunks, 0, 0)) { log_warn(LD_FS, "Error writing certificates to disk."); } tor_free(filename); @@ -1048,7 +1048,7 @@ router_rebuild_store(int flags, desc_store_t *store) smartlist_add(chunk_list, c); } SMARTLIST_FOREACH_END(sd); - if (write_chunks_to_file(fname_tmp, chunk_list, 1)<0) { + if (write_chunks_to_file(fname_tmp, chunk_list, 1, 1)<0) { log_warn(LD_FS, "Error writing router store to disk."); goto done; }

1 0

[onionoo/master] Update contact email address.
by karsten＠torproject.org 11 Oct '13

11 Oct '13

commit 78b7d1c4334a6a2317f58df9b232830a68a85587 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Oct 11 18:26:02 2013 +0200 Update contact email address. --- web/index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/index.html b/web/index.html index 5e4e60c..6899c53 100644 --- a/web/index.html +++ b/web/index.html @@ -153,7 +153,7 @@ Protocol changes will be announced here by writing deprecated API parts in red and new parts in blue. Deprecated parts will be removed one month after their announcement. -Let us <a href="mailto:karsten.loesing@gmx.net">know</a> that you're +Let us <a href="mailto:tor-dev@lists.torproject.org">know</a> that you're developing a client, and we're going to inform you of upcoming protocol changes.

1 0

[onionoo/master] Make protocol specification much clearer. Thanks, rndm!
by karsten＠torproject.org 11 Oct '13

11 Oct '13

commit b7bf9ce381e7ee69025c96f8cd937008392b9061 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Oct 11 18:22:01 2013 +0200 Make protocol specification much clearer. Thanks, rndm! --- web/index.html | 1685 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 1284 insertions(+), 401 deletions(-) diff --git a/web/index.html b/web/index.html old mode 100755 new mode 100644 index a648b8d..5e4e60c --- a/web/index.html +++ b/web/index.html @@ -1,108 +1,214 @@ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <html> <head> -<style> -body { background-color: white; font-family: Arial, Helvetica, sans-serif; - font-size: 1em; font-style: normal; color: #000000; padding-top: 0px; - background-color: white; margin: 0px auto 0 auto; width: 85%; - padding: 15px 0 10px 10px; text-indent: 0pt; direction: ltr; - text-align: left; } -h2 { font-size: 1.4em; font-weight: bold; } -h3 { font-size: 1.2em; font-weight: bold; } -li { margin: .2em .2em .2em 1em; } -a.anchor { font-size: 1em; color: black; font-weight: bold; - text-decoration: none; } -</style> <title>Onionoo — a Tor network status protocol</title> +<style type="text/css"> +body { font-family: "Open Sans","lucida grande","Segoe UI",arial,verdana, + "lucida sans unicode",tahoma,sans-serif; background: #fafafa; + font-size: 13px; line-height: 22px; color: #222; } +h1 { font-size: 20px; font-weight: normal; text-align: center; } +h3 { color: #7D4698; } +a { color: #7D4698; text-decoration: none; font-weight: bold; } +ul { list-style: none; padding: 0; margin: 0; } +p { margin: 0; padding: 0; } +a[name] { padding: 0; margin: 0; } +.box { max-width: 850px; width: 100%; margin: 0 auto 30px auto; + padding-bottom: 30px; background: white; border: 1px solid #eee; } +.box > * { margin-left: 30px; margin-right: 30px; } +.box h3 a { visibility: hidden; } +.box:hover h3 a { visibility: visible; } +.api-request { border-bottom: 1px solid #eee; } +.request-url, .request-type, .request-response { padding: 8px 10px; + vertical-align: middle } +.request-type { color: #57145F; display: inline-block; } +.request-url { color: #333; font-size: 18px; } +.request-response { float: right; color: #666; } +.api-urls>li:last-child { border-bottom: 0; } +.required-true, .required-false, .typeof { display: inline-block; + vertical-align: middle; padding: 5px 10px; } +.required-true { color: #1d7508; } +.required-false { color: #aaa; } +.properties { margin-top: 10px; margin-bottom: 10px; + border: 1px solid #eee; } +.properties li { padding: 5px 0; } +.properties li ul { border: 1px solid #eee; margin: 10px 10px 10px 40px; + background: white; } +.properties .properties { margin-left: 10px; } +.properties li:nth-child(even) { background: #fafafa; } +.properties p { padding: 10px 15px; } +.properties b { padding: 5px 10px; display: inline-block; + vertical-align: middle; } +.api-urls{ margin-top: 30px; margin-bottom: 30px; } +</style> <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"> <link href="favicon.ico" type="image/x-icon" rel="shortcut icon"> </head> <body> + +<div class="box"> <h3>Onionoo protocol overview</h3> -The Onionoo service is designed as a RESTful web service. + + <h4>Table of contents</h4> + <ul> + <li><a href="#general">General</a></li> + <li><a href="#methods">Methods</a></li> + <li><a href="#summary">Summary documents</a></li> + <li><a href="#details">Details documents</a></li> + <li><a href="#bandwidth">Bandwidth documents</a></li> + <li><a href="#weights">Weights documents</a></li> + </ul> + +</div> +<div class="box"> + +<a name="general"></a> +<h3>General <a href="#general">#</a></h3> + + +The Onionoo service is designed as a RESTful web service. Onionoo clients send HTTP GET requests to the Onionoo server which -responds with JSON-formatted replies. -Onionoo clients and their developers should follow a few rules: -<ul> -<li>Compression: Clients should include an "Accept-Encoding: -gzip" header in their requests and handle gzip-compressed responses. +responds with JSON-formatted replies. +Onionoo clients and their developers should follow a few rules: + + +<h4>Compression</h4> + +Clients should include an "Accept-Encoding: +gzip" header in their requests and handle gzip-compressed +responses. Only requests starting at a certain size will be compressed by the -server.</li> -<li>Caching: Clients should make use of the "Last-Modified" +server. + + +<h4>Caching</h4> +Clients should make use of the "Last-Modified" header of responses and include that timestamp in a -"If-Modified-Since" header of subsequent requests.</li> -<li>Response codes: Clients should handle response codes by +"If-Modified-Since" header of subsequent requests. + + +<h4>Response codes</h4> + +Clients should handle response codes by distinguishing between client and server errors, and if there's a problem, informing their users about the kind of problem. The following response codes are used: -<ul> -<li>200 OK: The request was processed successfully.</li> -<li>304 Not Modified: Server data has not changed since the -"If-Modified-Since" header included in the request.</li> -<li>400 Bad Request: The request could not be processed either + + +<ul class="properties"> + +<li> +200 OK + +The request was processed successfully. + +</li> + +<li> +304 Not Modified + +Server data has not changed since the +"If-Modified-Since" header included in the request. + +</li> + +<li> +400 Bad Request + +The request could not be processed either because the requested resource could not be found or because of bad syntax. -This is most likely a client problem.</li> -<li>500 Internal Server Error: There is an unspecific problem with +This is most likely a client problem. + +</li> + +<li> +500 Internal Server Error + +There is an unspecific problem with the server which the service operator may not yet be aware of. Please check if there's already a bug report for this problem, and if not, -file one.</li> -<li>503 Service Unavailable: The server is temporarily down for +file one. + +</li> + +<li> +503 Service Unavailable + +The server is temporarily down for maintenance, or there is a temporary problem with the server that the service operator is already aware of. -Only file a bug report if this problem persists.</li> +Only file a bug report if this problem persists. + +</li> + </ul> -<li>Protocol changes: There are plenty of reasons why we may have + +<h4>Protocol changes</h4> + +There are plenty of reasons why we may have to change the protocol described here. Clients should be able to handle all valid JSON responses, ignoring unknown fields and not breaking horribly in case of missing fields. Protocol changes will be announced here by writing deprecated API parts in -red and new parts in -blue. +red and new parts in +blue. Deprecated parts will be removed one month after their announcement. Let us <a href="mailto:karsten.loesing@gmx.net">know</a> that you're developing a client, and we're going to inform you of upcoming protocol changes. -</li> -</ul> + + The Onionoo API is described by resource types and available methods below. - + +</div>  + +<div class="box"> + <a name="methods"></a> -<h3><a href="#methods" class="anchor">Methods</a></h3> -The following methods each return a single document containing zero or -more relay and/or bridge documents. -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr> -<td>GET summary</td> -<td>Return summaries of all relays and bridges that are currently running -or that have been running in the past week. -</td> -</tr> -<tr> -<td>GET details</td> -<td>Return details of all relays and bridges that are currently running -or that have been running in the past week. -</td> -</tr> -<tr> -<td>GET bandwidth</td> -<td>Return bandwidth documents of all relays and bridges that are -currently running or that have been running in the past week. -</td> -</tr> -<tr> -<td>GET weights</td> -<td>Return weights documents of all relays and bridges that are currently -running or that have been running in the past week. -</td> -</tr> -</table> -Each of the methods above can be parameterized to select only a subset +<h3>Methods <a href="#methods">#</a></h3> + + +The following methods each return a single document containing zero or +more objects of relays and/or bridges that are currently running or that +have been running in the past week. + + +<ul class="api-urls"> + +<li class="api-request"> +GET +https://onionoo.torproject.org/summary +returns a <a href="#summary">summary +document</a> +</li> + +<li class="api-request"> +GET +https://onionoo.torproject.org/details +returns a <a href="#details">details +document</a> +</li> + +<li class="api-request"> +GET +https://onionoo.torproject.org/bandwidth +returns a <a href="#bandwidth">bandwidth +document</a> +</li> + +<li class="api-request"> +GET +https://onionoo.torproject.org/weights +returns a <a href="#weights">weights +document</a> +</li> + +</ul> + +<h4>Parameters</h4> + +Each of the methods above can be parameterized to select only a subset of relay and/or bridge documents to be included in the response. If multiple parameters are specified, they are combined using a logical AND operation, meaning that only the intersection of relays and bridges @@ -110,22 +216,34 @@ matching all parameters is returned. If the same parameter is specified more than once, only the first parameter value is considered. -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr><td>type</td><td>Return only relay (parameter value -relay) or only bridge documents (parameter value -bridge). + +<ul class="properties"> + +<li> +type + +Return only relay (parameter value +relay) or only bridge documents (parameter value +bridge). Parameter values are case-insensitive. -</td></tr> -<tr><td>running</td><td>Return only running (parameter value -true) or only non-running relays and/or bridges (paramter value -false). + +</li> + +<li> +running + +Return only running (parameter value +true) or only non-running relays and/or bridges +(paramter value +false). Parameter values are case-insensitive. -</td></tr> -<tr><td>search</td><td>Return only relays with the parameter value + +</li> + +<li> +search + +Return only relays with the parameter value matching (part of a) nickname, (possibly $-prefixed) beginning of a fingerprint, or beginning of an IP address, and bridges with (part of a) nickname or (possibly $-prefixed) beginning of a hashed fingerprint. @@ -138,8 +256,13 @@ of all relays and bridges matching all search terms will be returned. Full fingerprints should always be hashed using SHA-1, regardless of searching for a relay or a bridge, in order to not accidentally leak non-hashed bridge fingerprints in the URL. -</td></tr> -<tr><td>lookup</td><td>Return only the relay with the parameter + +</li> + +<li> +lookup + +Return only the relay with the parameter value matching the fingerprint or the bridge with the parameter value matching the hashed fingerprint. Fingerprints should always be hashed using SHA-1, regardless of looking up @@ -148,34 +271,55 @@ fingerprints in the URL. Lookups only work for full fingerprints or hashed fingerprints consisting of 40 hex characters. Lookups are case-insensitive. -</td></tr> -<tr><td>country</td><td>Return only relays which are located in the + +</li> + +<li> +country + +Return only relays which are located in the given country as identified by a two-letter country code. Filtering by country code is case-insensitive. -</td></tr> -<tr><td>as</td> -<td>Return only relays which are located in the + +</li> + +<li> +as + +Return only relays which are located in the given autonomous system (AS) as identified by the AS number (with or without preceding "AS" part). Filtering by AS number is case-insensitive. -</td></tr> -<tr><td>flag</td> -<td>Return only relays which have the + +</li> + +<li> +flag + +Return only relays which have the given relay flag assigned by the directory authorities. Note that if the flag parameter is specified more than once, only the first parameter value will be considered. Filtering by flag is case-insensitive. -</td></tr> -<tr><td>first_seen_days</td> -<td>Return only relays or bridges which + +</li> + +<li> +first_seen_days + +Return only relays or bridges which have first been seen during the given range of days ago. A parameter value "x-y" with x >= y returns relays or bridges that have first been seen at least x and at most y days ago. Accepted short forms are "x", "x-", and "-y" which are interpreted as "x-x", "x-infinity", and "0-y". -</td></tr> -<tr><td>last_seen_days</td> -<td>Return only relays or bridges which + +</li> + +<li> +last_seen_days + +Return only relays or bridges which have last been seen during the given range of days ago. A parameter value "x-y" with x >= y returns relays or bridges that have last been seen at least x and at most y days ago. @@ -184,8 +328,13 @@ Accepted short forms are "x", "x-", and "-y" which are interpreted as Note that relays and bridges that haven't been running in the past week are never included in results, so that setting x to 8 or higher will always lead to an empty result set. -</td></tr> -<tr><td>contact</td><td>Return only relays with the parameter value + +</li> + +<li> +contact + +Return only relays with the parameter value matching (part of) the contact line. If the parameter value contains spaces, only relays are returned which contain all space-separated parts in their contact line. @@ -193,600 +342,1334 @@ Only printable ASCII characters are permitted in the parameter value, some of which need to be percent-encoded (# as %23, % as %25, & as %26, + as %2B, and / as %2F). Comparisons are case-insensitive. -</td></tr> -</table> -Response documents can be reduced in size by requesting only a subset -of contained fields. -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr><td>fields</td><td>Comma-separated list of fields that will be + +</li> + +</ul> + + +Response documents can be reduced in size by requesting only a subset +of contained fields. + + +<ul class="properties"> + +<li> +fields + +Comma-separated list of fields that will be included in the result. So far, only top-level fields in relay or bridge objects of details -documents can be specified, e.g., nickname,hashed_fingerprint. +documents can be specified, e.g., +nickname,hashed_fingerprint. If the fields parameter is provided, all other fields which are not contained in the provided list will be removed from the result. Field names are case-insensitive. -</td></tr> -</table> -Relay and/or bridge documents in the response can be ordered and + +</li> + + +Relay and/or bridge documents in the response can be ordered and limited by providing further parameters. If the same parameter is specified more than once, only the first -parameter value is considered. -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr><td>order</td><td>Re-order results by a comma-separated list +parameter value is considered. + + +<ul class="properties"> + +<li> +order + +Re-order results by a comma-separated list of fields in ascending or descending order. Results are first ordered by the first list element, then by the second, and so on. -Possible fields for ordering are: consensus_weight. +Possible fields for ordering are: consensus_weight. Field names are case-insensitive. Ascending order is the default; descending order is selected by prepending -fields with a minus sign (-). +fields with a minus sign (-). Relays or bridges which don't have any value for a field to be ordered by are always appended to the end, regardless or sorting order. The ordering is defined independent of the requested document type and does not require the ordering field to be contained in the document. -If no order parameter is given, ordering of results is +If no order parameter is given, ordering of results is undefined. -</td></tr> -<tr><td>offset</td><td>Skip the given number of relays and/or + +</li> + +<li> +offset + +Skip the given number of relays and/or bridges. Relays are skipped first, then bridges. -Non-positive offset values are treated as zero and don't change the +Non-positive offset values are treated as zero and don't +change the result. -</td></tr> -<tr><td>limit</td><td>Limit result to the given number of + +</li> + +<li> +limit + +Limit result to the given number of relays and/or bridges. Relays are kept first, then bridges. -Non-positive limit values are treated as zero and lead to an empty +Non-positive limit values are treated as zero and lead +to an empty result. -When used together with offset, the offsetting step precedes the +When used together with offset, the offsetting step +precedes the limiting step. -</td></tr> -</table> - + +</li> + +</div>  + +<div class="box"> <a name="summary"></a> -<h3><a href="#summary" class="anchor">Summary documents</a></h3> +<h3>Summary documents <a href="#summary">#</a> + +<a href="summary?limit=4">example request</a> + +</h3> + Summary documents contain short summaries of relays with nicknames, fingerprints, IP addresses, and running information as well as bridges with hashed fingerprints and running information. -Summary documents contain the following fields: -<ul> -<li>"relays_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when -the last known relay network status consensus started being valid. +Summary documents contain the following fields: + +<ul class="properties"> + +<li> +relays_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known relay network +status consensus started being valid. Indicates how recent the relay summaries in this document are. -Required field.</li> -<li>"relays": Array of objects representing relay summaries. -Required field. + +</li> + +<li> +relays +<code class="typeof">array of objects</code> +required + +Array of objects representing relay summaries. Each array object contains the following key-value pairs: -<ul> -<li>"n": Relay nickname consisting of 1–19 alphanumerical -characters. -Optional field. -Omitted if the relay nickname is "Unnamed".</li> -<li>"f": Relay fingerprint consisting of 40 upper-case hexadecimal -characters. -Required field.</li> -<li>"a": Array of IPv4 or IPv6 addresses where the relay accepts + + +<ul class="properties"> + +<li> +n +<code class="typeof">string</code> +optional + +Relay nickname consisting of 1–19 alphanumerical characters. +Omitted if the relay nickname is "Unnamed". + +</li> + +<li> +f +<code class="typeof">string</code> +required + +Relay fingerprint consisting of 40 upper-case hexadecimal characters. + +</li> + +<li> +a +<code class="typeof">array of strings</code> +required + +Array of IPv4 or IPv6 addresses where the relay accepts onion-routing connections or which the relay used to exit to the Internet in the past 24 hours. The first address is the primary onion-routing address that the relay used to register in the network, subsequent addresses are in arbitrary order. IPv6 hex characters are all lower-case. -Required field.</li> -<li>"r": Boolean field saying whether this relay was listed as + +</li> + +<li> +r +<code class="typeof">boolean</code> +required + +Boolean field saying whether this relay was listed as running in the last relay network status consensus. -Required field.</li> + +</li> + </ul> </li> -<li>"bridges_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when + +<li> +bridges_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known bridge network status was published. Indicates how recent the bridge summaries in this document are. -Required field.</li> -<li>"bridges": Array of objects representing bridge summaries. -Required field. + +</li> + +<li> +bridges +<code class="typeof">array of objects</code> +required + +Array of objects representing bridge summaries. Each array object contains the following key-value pairs: -<ul> -<li>"n": Bridge nickname consisting of 1–19 alphanumerical -characters. -Optional field. -Omitted if the bridge nickname is "Unnamed".</li> -<li>"h": SHA-1 hash of the bridge fingerprint consisting of 40 + + +<ul class="properties"> + +<li> +n +<code class="typeof">string</code> +optional + +Bridge nickname consisting of 1–19 alphanumerical characters. +Omitted if the bridge nickname is "Unnamed". + +</li> + +<li> +h +<code class="typeof">string</code> +required + +SHA-1 hash of the bridge fingerprint consisting of 40 upper-case hexadecimal characters. -Required field.</li> -<li>"r": Boolean field saying whether this bridge was listed as + +</li> + +<li> +r +<code class="typeof">boolean</code> +required + +Boolean field saying whether this bridge was listed as running in the last bridge network status. -Required field.</li> + +</li> + </ul> + +</li> + </ul> - + +</div>  + +<div class="box"> <a name="details"></a> -<h3><a href="#details" class="anchor">Detail documents</a></h3> -Detail documents contain all known details of relays and bridges that +<h3>Details documents <a href="#details">#</a> + +<a href="details?limit=4">example request</a> + +</h3> + + +Details documents contain all known details of relays and bridges that have been running in the past week. -Detail documents are based on the network statuses published by the Tor +Details documents are based on the network statuses published by the Tor directories and the server descriptors published by relays and bridges. -Detail documents contain the following fields: -<ul> -<li>"relays_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when +Details documents contain the following fields: + + +<ul class="properties"> + +<li> +relays_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known relay network status consensus started being valid. Indicates how recent the relay details in this document are. -Required field.</li> -<li>"relays": Array of objects representing relay details. -Required field. + +</li> + +<li> +relays +<code class="typeof">array of objects</code> +required + +Array of objects representing relay details. Each array object contains the following key-value pairs: -<ul> -<li>"nickname": Relay nickname consisting of 1–19 -alphanumerical characters. -Optional field. -Omitted if the relay nickname is "Unnamed".</li> -<li>"fingerprint": Relay fingerprint consisting of 40 upper-case + + +<ul class="properties"> + +<li> +nickname +<code class="typeof">string</code> +optional + +Relay nickname consisting of 1–19 alphanumerical characters. +Omitted if the relay nickname is "Unnamed". + +</li> + +<li> +fingerprint +<code class="typeof">string</code> +required + +Relay fingerprint consisting of 40 upper-case hexadecimal characters. -Required field.</li> -<li>"or_addresses": Array of IPv4 or IPv6 addresses and TCP ports + +</li> + +<li> +or_addresses +<code class="typeof">array of strings</code> +required + +Array of IPv4 or IPv6 addresses and TCP ports or port lists where the relay accepts onion-routing connections. The first address is the primary onion-routing address that the relay used to register in the network, subsequent addresses are in arbitrary order. IPv6 hex characters are all lower-case. -Required field.</li> -<li>"exit_addresses": Array of IPv4 or IPv6 addresses that the + +</li> + +<li> +exit_addresses +<code class="typeof">array of strings</code> +optional + +Array of IPv4 or IPv6 addresses that the relay used to exit to the Internet in the past 24 hours. IPv6 hex characters are all lower-case. Only those addresses are listed that are different from onion-routing addresses. -Optional field. -Omitted if array is empty.</li> -<li>"dir_address": IPv4 address and TCP port where the relay +Omitted if array is empty. + +</li> + +<li> +dir_address +<code class="typeof">string</code> +optional + +IPv4 address and TCP port where the relay accepts directory connections. -Optional field. -Omitted if the relay does not accept directory connections.</li> -<li>"last_seen": UTC timestamp (YYYY-MM-DD hh:mm:ss) when this +Omitted if the relay does not accept directory connections. + +</li> + +<li> +last_seen +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when this relay was last seen in a network status consensus. -Required field.</li> -<li>"last_changed_address_or_port": + +</li> + +<li> +last_changed_address_or_port +<code class="typeof">string</code> +required + UTC timestamp (YYYY-MM-DD hh:mm:ss) when this relay last stopped announcing an IPv4 or IPv6 address or TCP port where it previously accepted onion-routing or directory connections. This timestamp can serve as indicator whether this relay would be a suitable fallback directory. -Required field.</li> -<li>"first_seen": UTC timestamp (YYYY-MM-DD hh:mm:ss) when this + +</li> + +<li> +first_seen +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when this relay was first seen in a network status consensus. -Required field.</li> -<li>"running": Boolean field saying whether this relay was listed as + +</li> + +<li> +running +<code class="typeof">boolean</code> +required + +Boolean field saying whether this relay was listed as running in the last relay network status consensus. -Required field.</li> -<li>"flags": Array of relay flags that the directory authorities + +</li> + +<li> +flags +<code class="typeof">array of strings</code> +optional + +Array of relay flags that the directory authorities assigned to this relay. -Optional field. -Omitted if empty.</li> -<li>"country": Two-letter lower-case country code as found in a +Omitted if empty. + +</li> + +<li> +country +<code class="typeof">string</code> +optional + +Two-letter lower-case country code as found in a GeoIP database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the GeoIP -database.</li> -<li>"country_name": Country name as found in a GeoIP database by +database. + +</li> + +<li> +country_name +<code class="typeof">string</code> +optional + +Country name as found in a GeoIP database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the GeoIP -database, or if the GeoIP database did not contain a country name.</li> -<li>"region_name": Region name as found in a GeoIP database by +database, or if the GeoIP database did not contain a country name. + +</li> + +<li> +region_name +<code class="typeof">string</code> +optional + +Region name as found in a GeoIP database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the GeoIP -database, or if the GeoIP database did not contain a region name.</li> -<li>"city_name": City name as found in a +database, or if the GeoIP database did not contain a region name. + +</li> + +<li> +city_name +<code class="typeof">string</code> +optional + +City name as found in a GeoIP database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the GeoIP -database, or if the GeoIP database did not contain a city name.</li> -<li>"latitude": Latitude as found in a GeoIP database by resolving +database, or if the GeoIP database did not contain a city name. + +</li> + +<li> +latitude +<code class="typeof">number</code> +optional + +Latitude as found in a GeoIP database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the GeoIP -database.</li> -<li>"longitude": Longitude as found in a GeoIP database by +database. + +</li> + +<li> +longitude +<code class="typeof">number</code> +optional + +Longitude as found in a GeoIP database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the GeoIP -database.</li> -<li>"as_number": AS number as found in an AS database by +database. + +</li> + +<li> +as_number +<code class="typeof">number</code> +optional + +AS number as found in an AS database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the AS -database.</li> -<li>"as_name": AS name as found in an AS database by resolving the +database. + +</li> + +<li> +as_name +<code class="typeof">string</code> +optional + +AS name as found in an AS database by resolving the relay's first onion-routing IP address. -Optional field. Omitted if the relay IP address could not be found in the AS -database.</li> -<li>"consensus_weight": Weight assigned to this relay by the +database. + +</li> + +<li> +consensus_weight +<code class="typeof">number</code> +required + +Weight assigned to this relay by the directory authorities that clients use in their path selection algorithm. The unit is arbitrary; currently it's kilobytes per second, but that might change in the future. -Required field.</li> -<li>"host_name": Host name as found in a reverse DNS lookup of the + +</li> + +<li> +host_name +<code class="typeof">string</code> +optional + +Host name as found in a reverse DNS lookup of the relay IP address. This field is updated at most once in 12 hours, unless the relay IP address changes. -Optional field. Omitted if the relay IP address was not looked up or if no lookup request -was successful yet.</li> -<li>"last_restarted": UTC timestamp (YYYY-MM-DD hh:mm:ss) when the +was successful yet. + +</li> + +<li> +last_restarted +<code class="typeof">string</code> +optional + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the relay was last (re-)started. -Optional field. Missing if router descriptor containing this information cannot be -found.</li> -<li>"bandwidth_rate": Average bandwidth +found. + +</li> + +<li> +bandwidth_rate +<code class="typeof">number</code> +optional + +Average bandwidth in bytes per second that this relay is willing to sustain over long periods. -Optional field. Missing if router descriptor containing this information cannot be -found.</li> -<li>"bandwidth_burst": Bandwidth in bytes +found. + +</li> + +<li> +bandwidth_burst +<code class="typeof">number</code> +optional + +Bandwidth in bytes per second that this relay is willing to sustain in very short intervals. -Optional field. Missing if router descriptor containing this information cannot be -found.</li> -<li>"observed_bandwidth": Bandwidth +found. + +</li> + +<li> +observed_bandwidth +<code class="typeof">number</code> +optional + +Bandwidth estimate in bytes per second of the capacity this relay can handle. The relay remembers the maximum bandwidth sustained output over any ten second period in the past day, and another sustained input. The "observed_bandwidth" value is the lesser of these two numbers. -Optional field. Missing if router descriptor containing this information cannot be -found.</li> -<li>"advertised_bandwidth": Bandwidth in bytes per second that this +found. + +</li> + +<li> +advertised_bandwidth +<code class="typeof">number</code> +optional + +Bandwidth in bytes per second that this relay is willing and capable to provide. -Optional field. Missing if router descriptor containing this information cannot be -found.</li> -<li>"exit_policy": Array of exit-policy lines. -Optional field. +found. + +</li> + +<li> +exit_policy +<code class="typeof">array of strings</code> +optional + +Array of exit-policy lines. Missing if router descriptor containing this information cannot be -found.</li> -<li>"exit_policy_summary": Summary +found. + +</li> + +<li> +exit_policy_summary +<code class="typeof">object</code> +optional + +Summary version of the relay's exit policy containing a dictionary with either an "accept" or a "reject" element. If there is an "accept" ("reject") element, the relay accepts (rejects) all TCP ports or port ranges in the given list for most IP addresses and rejects (accepts) all other ports. -Optional field.</li> -<li>"contact": Contact address of the relay operator. + +</li> + +<li> +contact +<code class="typeof">string</code> +optional + +Contact address of the relay operator. Omitted if empty or if descriptor containing this information cannot be -found.</li> -<li>"platform": Platform string containing operating system and Tor +found. + +</li> + +<li> +platform +<code class="typeof">string</code> +optional + +Platform string containing operating system and Tor version details. -Optional field. Omitted if empty or if descriptor containing this information cannot be -found.</li> -<li>"family": Array of fingerprints or nicknames of relays in the +found. + +</li> + +<li> +family +<code class="typeof">array of strings</code> +optional + +Array of fingerprints or nicknames of relays in the same family as this relay. -Optional field. Omitted if empty or if descriptor containing this information cannot be -found.</li> -<li>"advertised_bandwidth_fraction": +found. + +</li> + +<li> +advertised_bandwidth_fraction +<code class="typeof">number</code> +optional + Relative advertised bandwidth of this relay compared to the total advertised bandwidth in the network. If there were no bandwidth authorities, this fraction would be a very rough approximation of the probability of this relay to be selected by clients. -Optional field. Omitted if the relay is not running, or router descriptor containing this -information cannot be found.</li> -<li>"consensus_weight_fraction": +information cannot be found. + +</li> + +<li> +consensus_weight_fraction +<code class="typeof">number</code> +optional + Fraction of this relay's consensus weight compared to the sum of all consensus weights in the network. This fraction is a very rough approximation of the probability of this relay to be selected by clients. -Optional field. -Omitted if the relay is not running.</li> -<li>"guard_probability": +Omitted if the relay is not running. + +</li> + +<li> +guard_probability +<code class="typeof">number</code> +optional + Probability of this relay to be selected for the guard position. This probability is calculated based on consensus weights, relay flags, and bandwidth weights in the consensus. Path selection depends on more factors, so that this probability can only be an approximation. -Optional field. Omitted if the relay is not running, or the consensus does not contain -bandwidth weights.</li> -<li>"middle_probability": +bandwidth weights. + +</li> + +<li> +middle_probability +<code class="typeof">number</code> +optional + Probability of this relay to be selected for the middle position. This probability is calculated based on consensus weights, relay flags, and bandwidth weights in the consensus. Path selection depends on more factors, so that this probability can only be an approximation. -Optional field. Omitted if the relay is not running, or the consensus does not contain -bandwidth weights.</li> -<li>"exit_probability": +bandwidth weights. + +</li> + +<li> +exit_probability +<code class="typeof">number</code> +optional + Probability of this relay to be selected for the exit position. This probability is calculated based on consensus weights, relay flags, and bandwidth weights in the consensus. Path selection depends on more factors, so that this probability can only be an approximation. -Optional field. Omitted if the relay is not running, or the consensus does not contain -bandwidth weights.</li> +bandwidth weights. + +</li> + </ul> + </li> -<li>"bridges_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when +<li> +bridges_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known bridge network status was published. Indicates how recent the bridge details in this document are. -Required field.</li> -<li>"bridges": Array of objects representing bridge summaries. -Required field. + +</li> + +<li> +bridges +<code class="typeof">array of objects</code> +required + +Array of objects representing bridge summaries. Each array object contains the following key-value pairs: -<ul> -<li>"nickname": Bridge nickname consisting of 1–19 + + +<ul class="properties"> + +<li> +nickname +<code class="typeof">string</code> +optional + +Bridge nickname consisting of 1–19 alphanumerical characters. -Optional field. -Omitted if the bridge nickname is "Unnamed".</li> -<li>"hashed_fingerprint": SHA-1 hash of the bridge fingerprint +Omitted if the bridge nickname is "Unnamed". + +</li> + +<li> +hashed_fingerprint +<code class="typeof">string</code> +required + +SHA-1 hash of the bridge fingerprint consisting of 40 upper-case hexadecimal characters. -Required field.</li> -<li>"or_addresses": Array of sanitized IPv4 or IPv6 addresses and + +</li> + +<li> +or_addresses +<code class="typeof">array of strings</code> +required + +Array of sanitized IPv4 or IPv6 addresses and TCP ports or port lists where the bridge accepts onion-routing connections. The first address is the primary onion-routing address that the bridge used to register in the network, subsequent addresses are in arbitrary order. IPv6 hex characters are all lower-case. -Sanitized IP addresses are always in 10/8 or -[fd9f:2e19:3bcf/48] IP networks and are only useful to learn which +Sanitized IP addresses are always in 10/8 or +[fd9f:2e19:3bcf/48] IP networks and are only useful to +learn which IP version the bridge uses and to detect whether the bridge changed its address. Sanitized IP addresses always change on the 1st of every month at 00:00:00 UTC, regardless of the bridge actually changing its IP address. TCP ports are not sanitized. -Required field.</li> -<li>"last_seen": UTC timestamp (YYYY-MM-DD hh:mm:ss) when this + +</li> + +<li> +last_seen +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when this bridge was last seen in a bridge network status. -Required field.</li> -<li>"first_seen": UTC timestamp (YYYY-MM-DD hh:mm:ss) when this + +</li> + +<li> +first_seen +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when this bridge was first seen in a bridge network status. -Required field.</li> -<li>"running": Boolean field saying whether this bridge was listed + +</li> + +<li> +running +<code class="typeof">boolean</code> +required + +Boolean field saying whether this bridge was listed as running in the last bridge network status. -Required field.</li> -<li>"flags": Array of relay flags that the bridge authority + +</li> + +<li> +flags +<code class="typeof">array of strings</code> +optional + +Array of relay flags that the bridge authority assigned to this bridge. -Optional field. -Omitted if empty.</li> -<li>"last_restarted": UTC timestamp (YYYY-MM-DD hh:mm:ss) when the +Omitted if empty. + +</li> + +<li> +last_restarted +<code class="typeof">string</code> +optional + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the bridge was last (re-)started. -Optional field. Missing if router descriptor containing this information cannot be -found.</li> -<li>"advertised_bandwidth": Bandwidth in bytes per second that this +found. + +</li> + +<li> +advertised_bandwidth +<code class="typeof">number</code> +optional + +Bandwidth in bytes per second that this bridge is willing and capable to provide. -Optional field. Missing if router descriptor containing this information cannot be -found.</li> -<li>"platform": Platform string containing operating system and Tor +found. + +</li> + +<li> +platform +<code class="typeof">string</code> +optional + +Platform string containing operating system and Tor version details. -Optional field. Omitted if not provided by the bridge or if descriptor containing this -information cannot be found.</li> -<li>"pool_assignment": Information of the pool that BridgeDB +information cannot be found. + +</li> + +<li> +pool_assignment +<code class="typeof">string</code> +optional + +Information of the pool that BridgeDB assigned this bridge to, including further assignment information if available. -Optional field.</li> + +</li> + </ul> + </li> + </ul> - + +</div>  + +<div class="box"> <a name="bandwidth"></a> -<h3><a href="#bandwidth" class="anchor">Bandwidth documents</a></h3> -Bandwidth documents contain aggregate statistics of a relay's or +<h3>Bandwidth documents <a href="#bandwidth">#</a> + +<a href="bandwidth?limit=4">example request</a> + +</h3> + + +Bandwidth documents contain aggregate statistics of a relay's or bridge's consumed bandwidth for different time intervals. Bandwidth documents are available for all relays and bridges that have been running in the past week. Bandwidth documents contain the following fields: -<ul> -<li>"relays_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when + + +<ul class="properties"> + +<li> +relays_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known relay network status consensus started being valid. Indicates how recent the relay bandwidth documents in this document are. -Required field.</li> -<li>"relays": Array of objects representing relay bandwidth + +</li> + +<li> +relays +<code class="typeof">array of objects</code> +required + +Array of objects representing relay bandwidth documents. -Required field. Each array object contains the following key-value pairs: -<ul> -<li>"fingerprint": Relay fingerprint consisting of 40 upper-case + + +<ul class="properties"> + +<li> +fingerprint +<code class="typeof">string</code> +required + +Relay fingerprint consisting of 40 upper-case hexadecimal characters. -Required field.</li> -<li>"write_history": Object containing bandwidth history objects + +</li> + +<li> +write_history +<code class="typeof">object</code> +required + +Object containing bandwidth history objects for different time periods. -Required field. Keys are string representation of the time period covered by the bandwidth history object. -Keys are fixed strings "3_days", "1_week", "1_month", -"3_months", "1_year", and "5_years". +Keys are fixed strings "3_days", +"1_week", "1_month", +"3_months", "1_year", and +"5_years". Keys refer to the last known bandwidth history of a relay, not to the time when the bandwidth document was published. A bandwidth history object is only contained if the time period it covers is not already contained in another bandwidth history object with shorter time period and higher data resolution. Each bandwidth history object contains the following key-value pairs: -<ul> -<li>"first": UTC timestamp (YYYY-MM-DD hh:mm:ss) of the first data + + +<ul class="properties"> + +<li> +first +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the first data data point in the bandwidth history. -Required field.</li> -<li>"last": UTC timestamp (YYYY-MM-DD hh:mm:ss) of the last data + +</li> + +<li> +last +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the last data data point in the bandwidth history. -Required field.</li> -<li>"interval": Time interval between two data points in seconds. -Required field.</li> -<li>"factor": Factor by which subsequent bandwidth values need to + +</li> + +<li> +interval +<code class="typeof">number</code> +required + +Time interval between two data points in seconds. + +</li> + +<li> +factor +<code class="typeof">number</code> +required + +Factor by which subsequent bandwidth values need to be multiplied to get a unit of bytes per second. The idea is that contained bandwidth values are normalized to a range from 0 to 999 to reduce document size while still providing sufficient detail for both slow and fast relays. -Required field.</li> -<li>"count": Number of provided data points, included mostly for + +</li> + +<li> +count +<code class="typeof">number</code> +optional + +Number of provided data points, included mostly for debugging purposes. Can also be derived from the number of elements in the subsequent array. -Optional field.</li> -<li>"values": Array of normalized bandwidth values in bytes per + +</li> + +<li> +values +<code class="typeof">array of numbers</code> +required + +Array of normalized bandwidth values in bytes per second. May contain null values if the relay did not provide any bandwidth data or only data for less than 20% of a given time period. Only includes non-null values for series of at least two subsequent data points to enable drawing of line graphs. -Required field.</li> + +</li> + </ul> + </li> -<li>"read_history": Object containing bandwidth history objects + +<li> +read_history +<code class="typeof">object</code> +required + +Object containing bandwidth history objects for different time periods. -Required field. The specification of bandwidth history objects is similar to those in the -write_history field.</li> +write_history field. + +</li> + </ul> + </li> -<li>"bridges_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when +<li> +bridges_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known bridge network status was published. Indicates how recent the bridge bandwidth documents in this document are. -Required field.</li> -<li>"bridges": Array of objects representing bridge bandwidth + +</li> + +<li> +bridges +<code class="typeof">array of objects</code> +required + +Array of objects representing bridge bandwidth documents. -Required field. Each array object contains the following key-value pairs: -<ul> -<li>"fingerprint": SHA-1 hash of the bridge fingerprint consisting + + +<ul class="properties"> + +<li> +fingerprint +<code class="typeof">string</code> +required + +SHA-1 hash of the bridge fingerprint consisting of 40 upper-case hexadecimal characters. -Required field.</li> -<li>"write_history": Object containing bandwidth history objects + +</li> + +<li> +write_history +<code class="typeof">object</code> +required + +Object containing bandwidth history objects for different time periods. -Required field. The specification of bandwidth history objects is similar to those in the -write_history field of relays.</li> -<li>"read_history": Object containing bandwidth history objects +write_history field of relays. + +</li> + +<li> +read_history +<code class="typeof">object</code> +required + +Object containing bandwidth history objects for different time periods. -Required field. The specification of bandwidth history objects is similar to those in the -write_history field of relays.</li> +write_history field of relays. + +</li> + </ul> + </li> + </ul> - + +</div>  + +<div class="box"> <a name="weights"></a> -<h3><a href="#weights" class="anchor">Weights documents</a></h3> -Weights documents contain aggregate statistics of a relay's probability +<h3>Weights documents <a href="#weights">#</a> + +<a href="weights?limit=4">example request</a> + +</h3> + + +Weights documents contain aggregate statistics of a relay's probability to be selected by clients for building paths. Weights documents contain different time intervals and are available for all relays that have been running in the past week. Weights documents contain the following fields: -<ul> -<li>"relays_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when + + +<ul class="properties"> + +<li> +relays_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known relay network status consensus started being valid. Indicates how recent the relay weights documents in this document are. -Required field.</li> -<li>"relays": Array of objects representing relay weights + +</li> + +<li> +relays +<code class="typeof">array of objects</code> +required + +Array of objects representing relay weights documents. -Required field. Each array object contains the following key-value pairs: -<ul> -<li>"fingerprint": Relay fingerprint consisting of 40 upper-case + + +<ul class="properties"> + +<li> +fingerprint +<code class="typeof">string</code> +required + +Relay fingerprint consisting of 40 upper-case hexadecimal characters. -Required field.</li> -<li>"advertised_bandwidth_fraction": History object containing + +</li> + +<li> +advertised_bandwidth_fraction +<code class="typeof">object</code> +optional + +History object containing relative advertised bandwidth of this relay compared to the total advertised bandwidth in the network. If there were no bandwidth authorities, this fraction would be a very rough approximation of the probability of this relay to be selected by clients. -Optional field. Keys are string representation of the time period covered by the weights history object. -Keys are fixed strings "1_week", "1_month", -"3_months", "1_year", and "5_years". +Keys are fixed strings "1_week", +"1_month", "3_months", +"1_year", and "5_years". Keys refer to the last known weights history of a relay, not to the time when the weights document was published. A weights history object is only contained if the time period it covers is not already contained in another weights history object with shorter time period and higher data resolution. Each weights history object contains the following key-value pairs: -<ul> -<li>"first": UTC timestamp (YYYY-MM-DD hh:mm:ss) of the first data + + +<ul class="properties"> + +<li> +first +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the first data data point in the weights history. -Required field.</li> -<li>"last": UTC timestamp (YYYY-MM-DD hh:mm:ss) of the last data + +</li> + +<li> +last +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) of the last data data point in the weights history. -Required field.</li> -<li>"interval": Time interval between two data points in seconds. -Required field.</li> -<li>"factor": Factor by which subsequent weights values need to + +</li> + +<li> +interval +<code class="typeof">number</code> +required + +Time interval between two data points in seconds. + +</li> + +<li> +factor +<code class="typeof">number</code> +required + +Factor by which subsequent weights values need to be multiplied to get the path-selection probability. The idea is that contained weights values are normalized to a range from 0 to 999 to reduce document size while still providing sufficient detail for both slow and fast relays. -Required field.</li> -<li>"count": Number of provided data points, included mostly for + +</li> + +<li> +count +<code class="typeof">number</code> +optional + +Number of provided data points, included mostly for debugging purposes. Can also be derived from the number of elements in the subsequent array. -Optional field.</li> -<li>"values": Array of normalized weights values. + +</li> + +<li> +values +<code class="typeof">array of numbers</code> +required + +Array of normalized weights values. May contain null values if the relay was running less than 20% of a given time period. Only includes non-null values for series of at least two subsequent data points to enable drawing of line graphs. -Required field.</li> + +</li> + </ul> + </li> -<li>"consensus_weight_fraction": History object containing the +<li> +consensus_weight_fraction +<code class="typeof">object</code> +optional + +History object containing the fraction of this relay's consensus weight compared to the sum of all consensus weights in the network. This fraction is a very rough approximation of the probability of this relay to be selected by clients. -Optional field. The specification of this history object is similar to that in the -advertised_bandwidth_fraction field above.</li> -<li>"guard_probability": History object containing the probability +advertised_bandwidth_fraction field above. + +</li> + +<li> +guard_probability +<code class="typeof">object</code> +optional + +History object containing the probability of this relay to be selected for the guard position. This probability is calculated based on consensus weights, relay flags, and bandwidth weights in the consensus. Path selection depends on more factors, so that this probability can only be an approximation. -Optional field. The specification of this history object is similar to that in the -advertised_bandwidth_fraction field above.</li> -<li>"middle_probability": History object containing the probability +advertised_bandwidth_fraction field above. + +</li> + +<li> +middle_probability +<code class="typeof">object</code> +optional + +History object containing the probability of this relay to be selected for the middle position. This probability is calculated based on consensus weights, relay flags, and bandwidth weights in the consensus. Path selection depends on more factors, so that this probability can only be an approximation. -Optional field. The specification of this history object is similar to that in the -advertised_bandwidth_fraction field above.</li> -<li>"exit_probability": History object containing the probability +advertised_bandwidth_fraction field above. + +</li> + +<li> +exit_probability +<code class="typeof">object</code> +optional + +History object containing the probability of this relay to be selected for the exit position. This probability is calculated based on consensus weights, relay flags, and bandwidth weights in the consensus. Path selection depends on more factors, so that this probability can only be an approximation. -Optional field. The specification of this history object is similar to that in the -advertised_bandwidth_fraction field above.</li> +advertised_bandwidth_fraction field above. + +</li> + </ul> + </li> -<li>"bridges_published": UTC timestamp (YYYY-MM-DD hh:mm:ss) when + +<li> +bridges_published +<code class="typeof">string</code> +required + +UTC timestamp (YYYY-MM-DD hh:mm:ss) when the last known bridge network status was published. Only included for compatibility reasons with the other document types. -Required field.</li> -<li>"bridges": Empty array of objects that would represent bridge + +</li> + +<li> +bridges +<code class="typeof">array of objects</code> +required + +Empty array of objects that would represent bridge weights documents. Only included for compatibility reasons with the other document types. -Required field.</li> + +</li> + </ul> + +</div>  + </body> </html>

1 0

[onionoo/master] Move methods part above document specifications.
by karsten＠torproject.org 11 Oct '13

11 Oct '13

commit 9c788a8897fa394baf4c66b00b1703d7ca84b8b3 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Oct 11 18:14:39 2013 +0200 Move methods part above document specifications. --- web/index.html | 366 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 183 insertions(+), 183 deletions(-) diff --git a/web/index.html b/web/index.html index f663280..a648b8d 100755 --- a/web/index.html +++ b/web/index.html @@ -68,6 +68,189 @@ changes. The Onionoo API is described by resource types and available methods below. +<a name="methods"></a> +<h3><a href="#methods" class="anchor">Methods</a></h3> +The following methods each return a single document containing zero or +more relay and/or bridge documents. +<table border="0" cellpadding="4" cellspacing="0" summary=""> +<colgroup> +<col width="150"> +<col width="850"> +</colgroup> +<tr> +<td>GET summary</td> +<td>Return summaries of all relays and bridges that are currently running +or that have been running in the past week. +</td> +</tr> +<tr> +<td>GET details</td> +<td>Return details of all relays and bridges that are currently running +or that have been running in the past week. +</td> +</tr> +<tr> +<td>GET bandwidth</td> +<td>Return bandwidth documents of all relays and bridges that are +currently running or that have been running in the past week. +</td> +</tr> +<tr> +<td>GET weights</td> +<td>Return weights documents of all relays and bridges that are currently +running or that have been running in the past week. +</td> +</tr> +</table> +Each of the methods above can be parameterized to select only a subset +of relay and/or bridge documents to be included in the response. +If multiple parameters are specified, they are combined using a logical +AND operation, meaning that only the intersection of relays and bridges +matching all parameters is returned. +If the same parameter is specified more than once, only the first +parameter value is considered. + +<table border="0" cellpadding="4" cellspacing="0" summary=""> +<colgroup> +<col width="150"> +<col width="850"> +</colgroup> +<tr><td>type</td><td>Return only relay (parameter value +relay) or only bridge documents (parameter value +bridge). +Parameter values are case-insensitive. +</td></tr> +<tr><td>running</td><td>Return only running (parameter value +true) or only non-running relays and/or bridges (paramter value +false). +Parameter values are case-insensitive. +</td></tr> +<tr><td>search</td><td>Return only relays with the parameter value +matching (part of a) nickname, (possibly $-prefixed) beginning of a +fingerprint, or beginning of an IP address, and bridges with (part of a) +nickname or (possibly $-prefixed) beginning of a hashed fingerprint. +Searches for beginnings of IP addresses are performed on textual +representations of canonical IP address forms, so that searches using CIDR +notation or non-canonical forms will return empty results. +Searches are case-insensitive. +If multiple search terms are given, separated by spaces, the intersection +of all relays and bridges matching all search terms will be returned. +Full fingerprints should always be hashed using SHA-1, regardless of +searching for a relay or a bridge, in order to not accidentally leak +non-hashed bridge fingerprints in the URL. +</td></tr> +<tr><td>lookup</td><td>Return only the relay with the parameter +value matching the fingerprint or the bridge with the parameter value +matching the hashed fingerprint. +Fingerprints should always be hashed using SHA-1, regardless of looking up +a relay or a bridge, in order to not accidentally leak non-hashed bridge +fingerprints in the URL. +Lookups only work for full fingerprints or hashed fingerprints consisting +of 40 hex characters. +Lookups are case-insensitive. +</td></tr> +<tr><td>country</td><td>Return only relays which are located in the +given country as identified by a two-letter country code. +Filtering by country code is case-insensitive. +</td></tr> +<tr><td>as</td> +<td>Return only relays which are located in the +given autonomous system (AS) as identified by the AS number (with or +without preceding "AS" part). +Filtering by AS number is case-insensitive. +</td></tr> +<tr><td>flag</td> +<td>Return only relays which have the +given relay flag assigned by the directory authorities. +Note that if the flag parameter is specified more than once, only the +first parameter value will be considered. +Filtering by flag is case-insensitive. +</td></tr> +<tr><td>first_seen_days</td> +<td>Return only relays or bridges which +have first been seen during the given range of days ago. +A parameter value "x-y" with x >= y returns relays or bridges that have +first been seen at least x and at most y days ago. +Accepted short forms are "x", "x-", and "-y" which are interpreted as +"x-x", "x-infinity", and "0-y". +</td></tr> +<tr><td>last_seen_days</td> +<td>Return only relays or bridges which +have last been seen during the given range of days ago. +A parameter value "x-y" with x >= y returns relays or bridges that have +last been seen at least x and at most y days ago. +Accepted short forms are "x", "x-", and "-y" which are interpreted as +"x-x", "x-infinity", and "0-y". +Note that relays and bridges that haven't been running in the past week +are never included in results, so that setting x to 8 or higher will +always lead to an empty result set. +</td></tr> +<tr><td>contact</td><td>Return only relays with the parameter value +matching (part of) the contact line. +If the parameter value contains spaces, only relays are returned which +contain all space-separated parts in their contact line. +Only printable ASCII characters are permitted in the parameter value, +some of which need to be percent-encoded (# as %23, % as %25, & as +%26, + as %2B, and / as %2F). +Comparisons are case-insensitive. +</td></tr> +</table> +Response documents can be reduced in size by requesting only a subset +of contained fields. +<table border="0" cellpadding="4" cellspacing="0" summary=""> +<colgroup> +<col width="150"> +<col width="850"> +</colgroup> +<tr><td>fields</td><td>Comma-separated list of fields that will be +included in the result. +So far, only top-level fields in relay or bridge objects of details +documents can be specified, e.g., nickname,hashed_fingerprint. +If the fields parameter is provided, all other fields which are not +contained in the provided list will be removed from the result. +Field names are case-insensitive. +</td></tr> +</table> +Relay and/or bridge documents in the response can be ordered and +limited by providing further parameters. +If the same parameter is specified more than once, only the first +parameter value is considered. +<table border="0" cellpadding="4" cellspacing="0" summary=""> +<colgroup> +<col width="150"> +<col width="850"> +</colgroup> +<tr><td>order</td><td>Re-order results by a comma-separated list +of fields in ascending or descending order. +Results are first ordered by the first list element, then by the second, +and so on. +Possible fields for ordering are: consensus_weight. +Field names are case-insensitive. +Ascending order is the default; descending order is selected by prepending +fields with a minus sign (-). +Relays or bridges which don't have any value for a field to be ordered by +are always appended to the end, regardless or sorting order. +The ordering is defined independent of the requested document type and +does not require the ordering field to be contained in the document. +If no order parameter is given, ordering of results is +undefined. +</td></tr> +<tr><td>offset</td><td>Skip the given number of relays and/or +bridges. +Relays are skipped first, then bridges. +Non-positive offset values are treated as zero and don't change the +result. +</td></tr> +<tr><td>limit</td><td>Limit result to the given number of +relays and/or bridges. +Relays are kept first, then bridges. +Non-positive limit values are treated as zero and lead to an empty +result. +When used together with offset, the offsetting step precedes the +limiting step. +</td></tr> +</table> + <a name="summary"></a> <h3><a href="#summary" class="anchor">Summary documents</a></h3> Summary documents contain short summaries of relays with nicknames, @@ -604,189 +787,6 @@ weights documents. Only included for compatibility reasons with the other document types. Required field.</li> </ul> - -<a name="methods"></a> -<h3><a href="#methods" class="anchor">Methods</a></h3> -The following methods each return a single document containing zero or -more relay and/or bridge documents. -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr> -<td>GET summary</td> -<td>Return summaries of all relays and bridges that are currently running -or that have been running in the past week. -</td> -</tr> -<tr> -<td>GET details</td> -<td>Return details of all relays and bridges that are currently running -or that have been running in the past week. -</td> -</tr> -<tr> -<td>GET bandwidth</td> -<td>Return bandwidth documents of all relays and bridges that are -currently running or that have been running in the past week. -</td> -</tr> -<tr> -<td>GET weights</td> -<td>Return weights documents of all relays and bridges that are currently -running or that have been running in the past week. -</td> -</tr> -</table> -Each of the methods above can be parameterized to select only a subset -of relay and/or bridge documents to be included in the response. -If multiple parameters are specified, they are combined using a logical -AND operation, meaning that only the intersection of relays and bridges -matching all parameters is returned. -If the same parameter is specified more than once, only the first -parameter value is considered. - -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr><td>type</td><td>Return only relay (parameter value -relay) or only bridge documents (parameter value -bridge). -Parameter values are case-insensitive. -</td></tr> -<tr><td>running</td><td>Return only running (parameter value -true) or only non-running relays and/or bridges (paramter value -false). -Parameter values are case-insensitive. -</td></tr> -<tr><td>search</td><td>Return only relays with the parameter value -matching (part of a) nickname, (possibly $-prefixed) beginning of a -fingerprint, or beginning of an IP address, and bridges with (part of a) -nickname or (possibly $-prefixed) beginning of a hashed fingerprint. -Searches for beginnings of IP addresses are performed on textual -representations of canonical IP address forms, so that searches using CIDR -notation or non-canonical forms will return empty results. -Searches are case-insensitive. -If multiple search terms are given, separated by spaces, the intersection -of all relays and bridges matching all search terms will be returned. -Full fingerprints should always be hashed using SHA-1, regardless of -searching for a relay or a bridge, in order to not accidentally leak -non-hashed bridge fingerprints in the URL. -</td></tr> -<tr><td>lookup</td><td>Return only the relay with the parameter -value matching the fingerprint or the bridge with the parameter value -matching the hashed fingerprint. -Fingerprints should always be hashed using SHA-1, regardless of looking up -a relay or a bridge, in order to not accidentally leak non-hashed bridge -fingerprints in the URL. -Lookups only work for full fingerprints or hashed fingerprints consisting -of 40 hex characters. -Lookups are case-insensitive. -</td></tr> -<tr><td>country</td><td>Return only relays which are located in the -given country as identified by a two-letter country code. -Filtering by country code is case-insensitive. -</td></tr> -<tr><td>as</td> -<td>Return only relays which are located in the -given autonomous system (AS) as identified by the AS number (with or -without preceding "AS" part). -Filtering by AS number is case-insensitive. -</td></tr> -<tr><td>flag</td> -<td>Return only relays which have the -given relay flag assigned by the directory authorities. -Note that if the flag parameter is specified more than once, only the -first parameter value will be considered. -Filtering by flag is case-insensitive. -</td></tr> -<tr><td>first_seen_days</td> -<td>Return only relays or bridges which -have first been seen during the given range of days ago. -A parameter value "x-y" with x >= y returns relays or bridges that have -first been seen at least x and at most y days ago. -Accepted short forms are "x", "x-", and "-y" which are interpreted as -"x-x", "x-infinity", and "0-y". -</td></tr> -<tr><td>last_seen_days</td> -<td>Return only relays or bridges which -have last been seen during the given range of days ago. -A parameter value "x-y" with x >= y returns relays or bridges that have -last been seen at least x and at most y days ago. -Accepted short forms are "x", "x-", and "-y" which are interpreted as -"x-x", "x-infinity", and "0-y". -Note that relays and bridges that haven't been running in the past week -are never included in results, so that setting x to 8 or higher will -always lead to an empty result set. -</td></tr> -<tr><td>contact</td><td>Return only relays with the parameter value -matching (part of) the contact line. -If the parameter value contains spaces, only relays are returned which -contain all space-separated parts in their contact line. -Only printable ASCII characters are permitted in the parameter value, -some of which need to be percent-encoded (# as %23, % as %25, & as -%26, + as %2B, and / as %2F). -Comparisons are case-insensitive. -</td></tr> -</table> -Response documents can be reduced in size by requesting only a subset -of contained fields. -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr><td>fields</td><td>Comma-separated list of fields that will be -included in the result. -So far, only top-level fields in relay or bridge objects of details -documents can be specified, e.g., nickname,hashed_fingerprint. -If the fields parameter is provided, all other fields which are not -contained in the provided list will be removed from the result. -Field names are case-insensitive. -</td></tr> -</table> -Relay and/or bridge documents in the response can be ordered and -limited by providing further parameters. -If the same parameter is specified more than once, only the first -parameter value is considered. -<table border="0" cellpadding="4" cellspacing="0" summary=""> -<colgroup> -<col width="150"> -<col width="850"> -</colgroup> -<tr><td>order</td><td>Re-order results by a comma-separated list -of fields in ascending or descending order. -Results are first ordered by the first list element, then by the second, -and so on. -Possible fields for ordering are: consensus_weight. -Field names are case-insensitive. -Ascending order is the default; descending order is selected by prepending -fields with a minus sign (-). -Relays or bridges which don't have any value for a field to be ordered by -are always appended to the end, regardless or sorting order. -The ordering is defined independent of the requested document type and -does not require the ordering field to be contained in the document. -If no order parameter is given, ordering of results is -undefined. -</td></tr> -<tr><td>offset</td><td>Skip the given number of relays and/or -bridges. -Relays are skipped first, then bridges. -Non-positive offset values are treated as zero and don't change the -result. -</td></tr> -<tr><td>limit</td><td>Limit result to the given number of -relays and/or bridges. -Relays are kept first, then bridges. -Non-positive limit values are treated as zero and lead to an empty -result. -When used together with offset, the offsetting step precedes the -limiting step. -</td></tr> -</table> </body> </html>

1 0

[tor/master] Make --version, --help, etc incremement quiet level, never decrease it
by nickm＠torproject.org 11 Oct '13

11 Oct '13

commit 6f9584b3fd5346bfc7ee58dedee2f1c292bf0354 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Oct 11 12:32:59 2013 -0400 Make --version, --help, etc incremement quiet level, never decrease it Fixes other case of #9578 --- changes/bug9578 | 6 ++++++ src/or/main.c | 6 ++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/changes/bug9578 b/changes/bug9578 new file mode 100644 index 0000000..96d66fe --- /dev/null +++ b/changes/bug9578 @@ -0,0 +1,6 @@ + o Minor bugfixes: + - When a command-line option such as --version or --help that ordinarily + implies --hush appears on the command line along with --quiet, obey + --quiet. Previously, we obeyed --quiet only if it appeared later on the + command line. Fixes bug 9578; bugfix on 0.2.5.1-alpha. + diff --git a/src/or/main.c b/src/or/main.c index ac756de..4d691bb 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -2355,8 +2355,10 @@ tor_init(int argc, char *argv[]) if (!strcmp(cl->key, "--version") || !strcmp(cl->key, "--digests") || !strcmp(cl->key, "--list-torrc-options") || !strcmp(cl->key, "--library-versions") || - !strcmp(cl->key, "-h") || !strcmp(cl->key, "--help")) - quiet = 1; + !strcmp(cl->key, "-h") || !strcmp(cl->key, "--help")) { + if (quiet < 1) + quiet = 1; + } } config_free_lines(opts); config_free_lines(cmdline_opts);

1 0

[metrics-tasks/master] Add code to count bridges by transport (#9187).
by karsten＠torproject.org 11 Oct '13

11 Oct '13

commit 4a2472ef566f5be4f9f1589c22ff0eeca2ee2ca4 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Fri Oct 11 17:47:22 2013 +0200 Add code to count bridges by transport (#9187). --- task-9187/.gitignore | 8 ++ task-9187/R/plot-pt-bridges.R | 14 +++ task-9187/src/CountBridgesByTransport.java | 172 ++++++++++++++++++++++++++++ 3 files changed, 194 insertions(+) diff --git a/task-9187/.gitignore b/task-9187/.gitignore new file mode 100644 index 0000000..5694d63 --- /dev/null +++ b/task-9187/.gitignore @@ -0,0 +1,8 @@ +Rplots.pdf +bin/ +in/ +pt-bridges.csv +pt-bridges.png +.classpath +.project + diff --git a/task-9187/R/plot-pt-bridges.R b/task-9187/R/plot-pt-bridges.R new file mode 100644 index 0000000..b7c380f --- /dev/null +++ b/task-9187/R/plot-pt-bridges.R @@ -0,0 +1,14 @@ +# Usage (from parent directory): +# R --slave -f R/plot-pt-bridges.R +require(ggplot2) +b <- read.csv("pt-bridges.csv", stringsAsFactors = FALSE) +b <- b[substr(b$published, 1, 10) > substr(min(b$published), 1, 10), ] +ggplot(b, aes(x = as.POSIXct(published), y = bridges, + colour = transport)) + +geom_line() + +scale_x_datetime("") + +scale_y_continuous("") + +scale_colour_hue("") + +ggtitle("Running bridges by pluggable transport\n") +ggsave("pt-bridges.png", width = 8, height = 5, dpi = 100) + diff --git a/task-9187/src/CountBridgesByTransport.java b/task-9187/src/CountBridgesByTransport.java new file mode 100644 index 0000000..3ed22d6 --- /dev/null +++ b/task-9187/src/CountBridgesByTransport.java @@ -0,0 +1,172 @@ +import java.io.BufferedWriter; +import java.io.File; +import java.io.FileWriter; +import java.text.SimpleDateFormat; +import java.util.Date; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; +import java.util.SortedMap; +import java.util.TimeZone; +import java.util.TreeMap; + +import org.torproject.descriptor.BridgeNetworkStatus; +import org.torproject.descriptor.Descriptor; +import org.torproject.descriptor.DescriptorFile; +import org.torproject.descriptor.DescriptorReader; +import org.torproject.descriptor.DescriptorSourceFactory; +import org.torproject.descriptor.ExtraInfoDescriptor; +import org.torproject.descriptor.NetworkStatusEntry; +import org.torproject.descriptor.ServerDescriptor; + +/* + * Parses sanitized bridge descriptors to count bridges by transport. + * + * Usage: download and extract sanitized bridge descriptors to these + * subdirectories: + * + * in/statuses/ (bridge network statuses) + * in/server-descriptors/ (bridge server descriptors) + * in/extra-infos/ (bridge extra-info descriptors) + * + * Alternatively, if you're too lazy to extract tarballs, just put them in + * one of the three directories and make symbolic links in the other two. + * This will take somewhat longer to execute, because we're going through + * all tarballs three times. + * + * Run with a few GB heap space, e.g., java -Xmx4g ... + * + * Results will be written to pt-bridges.csv. + */ +public class CountBridgesByTransport { + public static void main(String[] args) throws Exception { + + System.out.println(new Date().toString() + " Starting."); + + /* Parse extra-info descriptors to get a mapping from "extra-info + * descriptor identifier" to "list of transports". */ + System.out.println(new Date().toString() + " Parsing extra-info " + + "descriptors."); + Map<String, List<String>> extraInfos = + new HashMap<String, List<String>>(); + DescriptorReader descriptorReader = + DescriptorSourceFactory.createDescriptorReader(); + descriptorReader.addDirectory(new File("in/extra-infos")); + Iterator<DescriptorFile> descriptorFiles = + descriptorReader.readDescriptors(); + while (descriptorFiles.hasNext()) { + DescriptorFile descriptorFile = descriptorFiles.next(); + if (descriptorFile.getDescriptors() == null) { + continue; + } + for (Descriptor descriptor : descriptorFile.getDescriptors()) { + if (!(descriptor instanceof ExtraInfoDescriptor)) { + continue; + } + ExtraInfoDescriptor extraInfoDescriptor = + (ExtraInfoDescriptor) descriptor; + if (extraInfoDescriptor.getTransports() == null || + extraInfoDescriptor.getTransports().isEmpty()) { + continue; + } + String extraInfoDigest = extraInfoDescriptor.getExtraInfoDigest(); + List<String> transports = extraInfoDescriptor.getTransports(); + extraInfos.put(extraInfoDigest, transports); + } + } + + /* Parse server descriptors to get a mapping from "server descriptor + * identifier" to "extra-info descriptor", but store it as "list of + * transports". */ + System.out.println(new Date().toString() + " Parsing server " + + "descriptors."); + Map<String, List<String>> serverDescriptors = + new HashMap<String, List<String>>(); + descriptorReader = DescriptorSourceFactory.createDescriptorReader(); + descriptorReader.addDirectory(new File("in/server-descriptors")); + descriptorFiles = descriptorReader.readDescriptors(); + while (descriptorFiles.hasNext()) { + DescriptorFile descriptorFile = descriptorFiles.next(); + if (descriptorFile.getDescriptors() == null) { + continue; + } + for (Descriptor descriptor : descriptorFile.getDescriptors()) { + if (!(descriptor instanceof ServerDescriptor)) { + continue; + } + ServerDescriptor serverDescriptor = (ServerDescriptor) descriptor; + if (serverDescriptor.getExtraInfoDigest() == null) { + continue; + } + String serverDescriptorDigest = + serverDescriptor.getServerDescriptorDigest(); + String extraInfoDigest = serverDescriptor.getExtraInfoDigest(); + if (extraInfos.containsKey(extraInfoDigest)) { + List<String> transports = extraInfos.get(extraInfoDigest); + serverDescriptors.put(serverDescriptorDigest, transports); + } + } + } + + /* Parse statuses, look up server descriptor identifiers and save + * number of running bridges by transport to disk as result. */ + System.out.println(new Date().toString() + " Parsing statuses."); + BufferedWriter bw = new BufferedWriter(new FileWriter( + "pt-bridges.csv")); + bw.write("published,transport,bridges\n"); + SimpleDateFormat dateTimeFormat = new SimpleDateFormat( + "yyyy-MM-dd HH:mm:ss"); + dateTimeFormat.setLenient(false); + dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); + descriptorReader = DescriptorSourceFactory.createDescriptorReader(); + descriptorReader.addDirectory(new File("in/statuses")); + descriptorFiles = descriptorReader.readDescriptors(); + while (descriptorFiles.hasNext()) { + DescriptorFile descriptorFile = descriptorFiles.next(); + if (descriptorFile.getDescriptors() == null) { + continue; + } + for (Descriptor descriptor : descriptorFile.getDescriptors()) { + if (!(descriptor instanceof BridgeNetworkStatus)) { + continue; + } + BridgeNetworkStatus status = (BridgeNetworkStatus) descriptor; + SortedMap<String, Integer> bridgesByTransport = + new TreeMap<String, Integer>(); + for (NetworkStatusEntry entry : + status.getStatusEntries().values()) { + if (!entry.getFlags().contains("Running")) { + continue; + } + if (serverDescriptors.containsKey(entry.getDescriptor())) { + List<String> transports = + serverDescriptors.get(entry.getDescriptor()); + for (String transport : transports) { + if (!bridgesByTransport.containsKey(transport)) { + bridgesByTransport.put(transport, 0); + } + bridgesByTransport.put(transport, + bridgesByTransport.get(transport) + 1); + } + } + } + if (bridgesByTransport.isEmpty()) { + continue; + } + String publishedString = dateTimeFormat.format( + status.getPublishedMillis()); + for (Map.Entry<String, Integer> e : + bridgesByTransport.entrySet()) { + String transport = e.getKey(); + int bridges = e.getValue(); + bw.write(publishedString + "," + transport + "," + bridges + + "\n"); + } + } + } + bw.close(); + + System.out.println(new Date().toString() + " Terminating."); + } +}

1 0

[tor/master] Merge remote-tracking branch 'karsten/geoip-manual-update-oct2013'
by nickm＠torproject.org 11 Oct '13

11 Oct '13

commit 3e3b9219ec968f21ab0956dd785c17541a908d5b Merge: 6429313 f4ef7c7 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Oct 11 11:17:25 2013 -0400 Merge remote-tracking branch 'karsten/geoip-manual-update-oct2013' src/config/geoip-manual | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-)

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.4'
by nickm＠torproject.org 11 Oct '13

11 Oct '13

commit 64293135df7611b7caa939eee57a8f0b9d29e617 Merge: f96a8d5 7ef2939 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Oct 11 11:17:18 2013 -0400 Merge remote-tracking branch 'origin/maint-0.2.4' changes/geoip-october2013 | 3 + src/config/geoip | 2019 ++++++++++++++++++++++++++++----------------- 2 files changed, 1245 insertions(+), 777 deletions(-)

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.3' into maint-0.2.4
by nickm＠torproject.org 11 Oct '13

11 Oct '13

commit 7ef2939e5a902c6159227de176622ee9388e34a4 Merge: eb5d75e 82d8944 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Oct 11 11:16:59 2013 -0400 Merge remote-tracking branch 'origin/maint-0.2.3' into maint-0.2.4 changes/geoip-october2013 | 3 + src/config/geoip | 2019 ++++++++++++++++++++++++++++----------------- 2 files changed, 1245 insertions(+), 777 deletions(-)

1 0

[tor/master] Merge remote-tracking branch 'origin/maint-0.2.2' into maint-0.2.3
by nickm＠torproject.org 11 Oct '13

11 Oct '13

commit 82d8944928daf868d12797e59a3a58ce4cb4f205 Merge: 004a9c6 3b02651 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Oct 11 11:16:45 2013 -0400 Merge remote-tracking branch 'origin/maint-0.2.2' into maint-0.2.3 changes/geoip-october2013 | 3 + src/config/geoip | 2019 ++++++++++++++++++++++++++++----------------- 2 files changed, 1245 insertions(+), 777 deletions(-)

1 0