- tor-dev - lists.torproject.org

Proposal 248: Remove all RSA identity keys
by Nick Mathewson 22 Jul '15

22 Jul '15

Filename: 248-removing-rsa-identities.txt Title: Remove all RSA identity keys Authors: Nick Mathewson Created: 15 August 2015 Status: Draft 1. Summary With 0.2.7.2-alpha, all relays will have Ed25519 identity keys. Old identity keys are 1024-bit RSA, which should not really be considered adequate. In proposal 220, we describe a migration path to start using Ed25519 keys. This proposal describes an additional migration path, for finally removing our old Ed25519 keys. See also proposal 245, which describes a migration path away from the old TAP RSA1024-based circuit extension protocol. 1.1. Steps of migration Phase 1. Prepare for routers that do not advertise their RSA identities, by teaching clients and relays and other dependent software how to handle them. Reject such routers at the authority level. Phase 2. Once all supported routers and clients are updated to phase 1, we can accept routers at the authority level which lack RSA keys. Phase 3. Once all authorities accept routers without RSA keys, we can finally remove RSA keys from relays. 2. Accepting descriptors without RSA identities We make the following changes to the descriptor format: If an ed25519 key and signature are present, then these elements may be omitted: "fignerprint", "signing-key", "router-signature". They must either be all present or all absent. If they are all absent, then the router has no RSA identity key. Authorities MUST NOT accept routers descriptors of this form in phase 1. 3. Accepting handshakes without RSA identities When performing a new version of our link handshake, only the Ed25519 key and certificates and authentication need to be performed. If the link handshake is performed this way, it is accepted as authenticating the route with an ed25519 key but no RSA key. A circuit extension EXTEND2 cell may contain an Ed25519 identity but not an RSA identity. In this case, the relay should connect the circuit to any connection with the correct ed25519 identity, regardless of RSA identity. If an EXTEND2 cell contains an RSA identity fingerprint, however, its the relay receiving it should not connect to any relay that has a different RSA identity or that has no identity, even if the Ed25519 identity does match. 4. UI updates In phase 1 we can update our UIs to refer to all relays that have Ed25519 keys by their Ed25519 keys. We can update our configuration and control port interfaces so that they accept Ed keys as well as RSA keys. During phase 1, we should warn about identifying any dual-identity relays by their Ed identity alone. For backward compatibility, we should consider a default that refers to referring to Ed25519 relays by the first 160 bits of their key. This would allow many controller-based tools to work transparently with the new key types. 5. Changes to external tools This is the big one. We need a relatively comprehensive list of tools we can break with the above changes. Anything that refers to relays by SHA1(RSA1024_id) will need to be able to remember and use an Ed25519 key instead. 5. Testing Before going forward with phase 2 and phase 3, we need to verify that we did phase 1 correctly. To do so, we should create a small temporary testing network, and verify that it works correctly as we make the phase 2 and phase 3 changes.

6 6

BOINC-based Tor wrapper
by Serg 21 Jul '15

21 Jul '15

Greetings to all developers! What do you think? BOINC - The Berkeley Open Infrastructure for Network Computing. https://boinc.berkeley.edu/ Technical wiki: https://boinc.berkeley.edu/trac/wiki/ProjectMain BOINC -> VirtualBox -> operating system with running Tor relay. Real working solutions? Yes: http://atlasathome.cern.ch/ http://lhcathome2.cern.ch/vLHCathome/ Platform diversity in Tor network may be solved by preparing images with different operating systems. The power of the Tor network may grow very quickly. Fastes reaction to Security vulnerabilities. All client will upgrade Tor images by administrator command. (all images should be signed ofcourse. It is the BOINC principle). You are doing a great job. Thank you! -- Best regards, Sergey

4 6

[Measurement Team] Next IRC meeting happens tomorrow, Jul 22, 14:00 UTC in #tor-project
by Karsten Loesing 21 Jul '15

21 Jul '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 [I hope this message is not considered noise by people on this list, but we haven't yet decided where these announcements should be sent. Until we have, I'm planning to send these messages to this list and tag them with [Measurement Team], so that people can filter these messages if they absolutely don't care. Thanks for understanding.] Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team tomorrow on Wednesday, July 22, 2015, 14:00 UTC in #tor-project http://www.timeanddate.com/worldclock/fixedtime.html?iso=20150722T14 Two weeks ago we started with introductions, and last week we created a list of measurement-related products and started writing fact sheets for some of them. Logs are available here: http://meetbot.debian.net/tor-project/2015/tor-project.2015-07-08-14.06.log… http://meetbot.debian.net/tor-project/2015/tor-project.2015-07-15-14.00.log… This week I suggest the following agenda: - Introductions, 5 min (only if someone didn't introduce themselves yet) - Problems, 5-10 min: did anything else come up since last week - Products, 5 min: what are we doing with fact sheets - Priorities, 30 min: should we focus more on a certain aspect of measurements in the Tor network, while temporarily suspending or even permanently dropping other aspects? - Misc, 5 min: anything else you come up with Regarding the Priorities item, I'd want us to do another collaborative round involving a pad where we temporarily assume that no single Tor measurement-related tool exists or ever existed and where we brainstorm what tools we should build to measure (passively collect, actively probe, sanitize, archive, pre-process, feed back, aggregate, visualize) things in the Tor network. Maybe we can conclude that brainstorming session by deriving some suggestions for prioritizing our work, given the fact that some of the tools already exist. Hope that sounds interesting to you, but please remember that this discussion will happen tomorrow, not on this mailing list. If you want to participate, please join us at the meeting! Talk to you tomorrow! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJVrjlvAAoJEJD5dJfVqbCrJp4IAMiq9Wp4N+ZEjaMdnXPPkMJS 5ItvmsxPxPZkhckrJafZtayE+3x+xfIZG44shgps53DKACxrtDxGdVruZRuA8o3f FGGLZ/YXQj7JJBiVGDn/vIXCfC4fuZSU5K4QYZeG9nDdspKl6HiB/xyavcF76mZw bQNyLU1870On1iQc7Ka9lP8+wO5WyfkJl0jspEs2GrRNMxK68hHakG8rFMVuE9z8 BQnqV113TQtSGJcpoWkfMeNQo0vf/BgbWVgE5/FA9x27WqQtS06oRfwxVMhgGYoW KNWVHyw0R7AG/1WEG/DTR8B/2M7iYVUmr4BrIjaaBo7GLYhySvo/z90fSiFz4Jg= =x3uM -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] [tor-commits] [tor/master] Remove the HidServDirV2 and VoteOnHidServDirectoriesV2 options
by Roger Dingledine 21 Jul '15

21 Jul '15

Isis: I'd like to highlight this change for you, since it means that the bridge authority's networkstatus files are now going to have HSDir flags on the bridge status lines. I don't know if this is going to be a problem for any of your parsing code in any way (hopefully not), but I figured now's a great time to let you know it's coming. --Roger On Thu, Jul 16, 2015 at 07:47:05PM +0000, nickm(a)torproject.org wrote: > --- a/src/or/networkstatus.c > +++ b/src/or/networkstatus.c > @@ -1678,7 +1678,7 @@ networkstatus_getinfo_by_purpose(const char *purpose_string, time_t now) > if (bridge_auth && ri->purpose == ROUTER_PURPOSE_BRIDGE) > dirserv_set_router_is_running(ri, now); > /* then generate and write out status lines for each of them */ > - set_routerstatus_from_routerinfo(&rs, node, ri, now, 0, 0); > + set_routerstatus_from_routerinfo(&rs, node, ri, now, 0); > smartlist_add(statuses, networkstatus_getinfo_helper_single(&rs)); > } SMARTLIST_FOREACH_END(ri);

2 2

Per-transport bridge bandwidth and bridge counts
by David Fifield 21 Jul '15

21 Jul '15

I made some graphs that show the count and total bandwidth of all bridges, broken down by transport. https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwi… https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-count.… The top part of the graph is non-default bridges, and the bottom is default bridges (the ones that ship with Tor Browser). Note the varying vertical scales. Source code and data are here: https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11.tar.gz https://people.torproject.org/~dcf/graphs/pt-bandwidth-2015-07-11/pt-bandwi… I have a question about getting bandwidth numbers. What I want is something like available bandwidth capacity; i.e., how much headroom transports have given their level of use. I think that the bandwidth numbers I'm using are partially conflated with usage. Notice how parts of pt-count.png are flat while pt-bandwidth.png has a weekly pattern. The bandwidth number I'm using is the "w Bandwidth=" line from a bridge-network-status document. It looks like this: r starman qgM+62FgGytzEtibYqqiPcPtijQ mdOOBxVOTpw8loBezhSDZxLIcXs 2015-07-03 21:39:31 10.174.163.60 9002 0 s Fast Guard Running Stable Valid w Bandwidth=2646 p reject 1-65535 dir-spec.txt says: An estimate of the bandwidth of this relay, in an arbitrary unit (currently kilobytes per second). Used to weight router selection. See section 3.4.2 for details on how the value of Bandwidth is determined in a consensus." Section 3.4.2 says: When we speak of a router's bandwidth in this section, we mean either its measured bandwidth, or its advertised bandwidth.... The bandwidth in a "w" line should be taken as the best estimate of the router's actual capacity that the authority has. For now, this should be the lesser of the observed bandwidth and bandwidth rate limit from the server descriptor. It is given in kilobytes per second, and capped at some arbitrary value (currently 10 MB/s). I don't know where observed bandwidth comes from. Is there some kind of external test that measures it, or does it come from measuring user traffic? Another option for getting bandwidth is the "bandwidth" line in a bridge-server-descriptor document. It looks like this: bandwidth 1073741824 1073741824 2646895 The three fields are bandwidth-avg bandwidth-burst bandwidth-observed. dir-spec.txt says: Estimated bandwidth for this router, in bytes per second. The "average" bandwidth is the volume per second that the OR is willing to sustain over long periods; the "burst" bandwidth is the volume that the OR is willing to sustain in very short intervals. The "observed" value is an estimate of the capacity this relay can handle. The relay remembers the max bandwidth sustained output over any ten second period in the past day, and another sustained input. The "observed" value is the lesser of these two numbers. So it seems that bandwidth-observed is not what we want, because it's derived from current usage. bandwidth-avg and bandwidth-burst aren't useful because they are operator-controlled and often left at the default of 1 GB/s. Yet another option is the speed of serving consensuses, the "dirreq-v3-tunneled-dl" line in a bridge-extra-info document: dirreq-v3-tunneled-dl complete=13624,timeout=1456,running=4,min=1938,d1=21008,d2=43181,q1=53397,d3=63519,d4=88911,md=117048,d6=146982,d7=180109,q3=199979,d8=223313,d9=298860,max=1558627 You can take the "md=" field, for example, to get median B/s for serving consensuses.

3 3

Commit broken code to otherwsie working master
by l.m 20 Jul '15

20 Jul '15

Hi, Is it normal for a core developer to want to commit broken code to master? I mean if the code is known to be completely broken. Wouldn't it be better to fix the code that is broken before commit. I mean master is a basis for working code isn't it? --leeroy

2 1

4th status report for OnioNS
by Jesse V 19 Jul '15

19 Jul '15

Hello again everyone, This report covers the period of time that I spent in Washington, DC at the hidden service meetings. I made excellent progress on this project. On the 10th, for example, I pushed nearly 30 commits. I fixed many significant bugs and improved many areas that would make the software easier to understand and to configure. * Created a separate and fully-fledged manpage for the hidden service, server, and client aspects of the software. * Client-side and HS-side are now complete and working reliably. The hidden service can generate a Record (a claim on a domain name) and transmit it over a Tor circuit to a remote server. The client now has a binary that launches the Tor executable, the onions-client binary, and a Stem script when the Tor Browser opens, and can now shut them all down properly now. * Fixed a show-stopping bug that prevented the Stem script from launching automatically with the Tor Browser. (George, this bug was why I had to launch Stem manually when I showed the software to you.) The error thrown for this bug was large and unhelpful, but the fix was amusingly simple: https://github.com/Jesse-V/OnioNS-client/commit/9e80691b02e2a843b26ac21b6b0… * Many bug fixes and enhancement updates, including some points of confusion. * Improved flexibility for hosting a server, including the capability to bind to a custom TCP port. * Finalized event logging and a flag to specify where the log goes. Logs on the client-side now go in an OnioNS folder inside the Tor Browser directory. * Migrated flag-parsing from the tclap library I found long ago on SourceForge to Unix popt. * Added a system installation of Tor as a dependency for the hidden service code. * Minor code cleanup. Global sources of randomness were also discussed at the hidden service meetings. I need a source of timestamped archivable randomness that the whole network agrees on in order to securely build OnioNS into a distributed system. The global randomness also has implications for the next generation of hidden service protocols, so there was an overall need to get something together. We made good progress towards this goal. I was able to show Roger and George a demonstration of OnioNS. I registered example.tor and arma.example.tor, pointed example.tor at a HS that I was hosting on my laptop and arma.example.tor at Roger's hidden service (duskgytldkxiuqc6.onion) and uploaded the Record over a Tor circuit to a remote machine, Server A. I then manually transfered the JSON-encoded data structures from Server A to Server B, another machine. Then I launched the Tor Browser (with my binary substitution in place) so that all of the client-side programs were running. I entered "example.tor" into the Tor Browser and as expected arrived at my hidden service, and then typed arma.example.tor into the browser and as expected loaded duskgytldkxiuqc6.onion. This was a great test. The server-to-server communication needs a few bug fixes, but most of that code is in place. As soon as that is complete, I should be about ready for a beta test. Jesse V.

2 1

Proposal 246: Defending Against Guard Discovery Attacks using Vanguards
by George Kadianakis 19 Jul '15

19 Jul '15

Hello, I'm attaching a proposal draft that should help us defend against guard discovery attacks. There are a few pieces left unfinished (see the XXXs) but I decided to release early and release often for the sake of moving forward with this. I consider this issue very important and any feedback is greatly appreciated. Thanks! ---- Filename: 246-hs-guard-discovery.txt Title: Defending Against Guard Discovery Attacks using Vanguards Author: George Kadianakis Created: 2015-07-10 Status: Draft 0. Motivation A guard discovery attack allow attackers to determine the guard node of a Tor client. The hidden service rendezvous protocol provides an attack vector for a guard discovery attack since anyone can force an HS to construct a 3-hop circuit to a relay (#9001). Following the guard discovery attack with a compromise or coercion of the guard node can lead to the deanonymization of a hidden service. 1. Overview This document tries to make the above guard discovery + coersion attack harder to launch. It introduces an optional configuration option which makes the hidden service also pin the second and third hops of its circuits for a longer duration. With this new path selection, we force the adversary to perform a Sybil attack and two coercion attacks before succeeding. This is an improvement over the current state where the Sybil attack is trivial to pull off, and only a single coercion attack is required. With this new path selection, an attacker is forced to do a coercion attack before learning the guard node of a hidden service. This increases the uncertainty of the attacker, since he will need to perform a coercion attack before he learns the identity of the guard node and whether he can compromise it. Coercion attacks are costly and potentially detectable, so an attacker will have to think twice before beginning a chain of coercion attacks that he might not be able to complete. 1.1. Visuals Here is how a hidden service rendezvous circuit currently looks like: -> middle_1 -> middle_A -> middle_2 -> middle_B -> middle_3 -> middle_C -> middle_4 -> middle_D HS -> guard -> middle_5 -> middle_E -> Rendezvous Point -> middle_6 -> middle_F -> middle_7 -> middle_G -> middle_8 -> middle_H -> ... -> ... -> middle_n -> middle_n this proposal pins the two middles nodes to a much more restricted set, as follows: -> guard_3_A -> guard_3_B HS -> guard_1 -> guard_2_A -> guard_3_C -> Rendezvous Point -> guard_2_B -> guard_3_D -> guard_3_E -> guard_3_F 2. Design This feature requires the HiddenServiceGuardDiscovery torrc option to be enabled. When a hidden service picks its guard nodes, it also picks two additional sets of guard nodes `second_guard_set` and `third_guard_set` of size NUM_SECOND_GUARDS and NUM_THIRD_GUARDS respectively. When a hidden service needs to establish a circuit to an HSDir, introduction point or a rendezvous point, it uses nodes from `second_guard_set` as the second hop of the circuit and nodes from `third_guard_set` as third hops of the circuit. A hidden service rotates 'second_guard_set' every SECOND_GUARD_ROTATION hours, and 'third_guard_set' every THIRD_GUARD_ROTATION hours. These extra guard nodes should be picked with the same path selection procedure that is used for regular guard nodes. Care should be taken such that guard sets do not share any common relays. XXX or simply that they are not used in the same circuit? XXX maybe pick the second and third layer guards from the set of middle nodes but also enforce some kind of uptime requirement? that should greatly help our load balancing. XXX maybe we should also introduce consensus flags for the extra guard layers? Vanguard? XXX how should proposal 241 ("Resisting guard-turnover attacks") be applied here? 2.1. Security parameters We set NUM_SECOND_GUARDS to 2 nodes and NUM_THIRD_GUARDS to 6 nodes. We set SECOND_GUARD_ROTATION to 2 weeks and THIRD_GUARD_ROTATION to 1 day. See the discussion section for more analysis on these constants. 3. Discussion 3.1 How were these security parameters chosen? Consider an adversary with the following powers: - Can launch a Sybil guard discovery attack against any node of a rendezvous circuit. The slower the rotation period of the node, the longer the attack takes. - Can compromise any node on the network. We assume that the adversary cannot compromise too many nodes, otherwise Tor's security would be breached anyhow. We now calculate the time it takes for the adversary to launch a Sybil attack with 50% success assuming 5% network control. This depends solely on how frequently the hidden service rotates that node: +-------------------+-------------------------------+------------------------+----------------------------+ | Rotation period | Sybil attack with 50% success | Sybil attack (2 guards)| Sybil attack (6 guards) | +-------------------+-------------------------------+------------------------+----------------------------+ | 1 hour | 14 hours | 7 hours | 2.5 hours | | 1 day | 14 days | 7 days | 2.5 days | | 1 week | 3.5 months | 6 weeks | 2.5 weeks | | 2 weeks | 7 months | 3.5 months | 5 weeks | | 1 month | 1 year and 2 months | 6 months | 2.5 months | | 3 months | 3.5 years | 1.7 years | 6 months | +-------------------+-------------------------------+------------------------+----------------------------+ Required time for Sybil attack by a 5% adversary Our security parameters were selected so that the first two layers of guards should be hard to attack using a Sybil guard discovery attack and hence require a coercion attack. On the other hand, the outmost layer of guards should rotate fast enough to _require_ a Sybil attack. XXX does this security model even make sense? what about a network adversary, or an adversary that can launch congestion attacks etc.???? 3.2. Distinguishing new HS circuits from normal HS circuits By pinning the middle nodes of rendezvous circuits, we make it easier for all hops of the circuit to detect that they are part of a special hidden service circuit. XXX hm how does the outermost guard knows? Compare this to the current Tor network, where only the guard and the third hop of the HS circuit can trivially distinguish whether it's part of an HS circuit. 3.3. Circuit nodes can now be linked to specific hidden services With this proposal the hops of hidden service circuits will be static, and hence an adversrary will be able to map them to specific hidden services. For example, a middle node that sees two hidden service circuits with the same guard node in different times, can assume with non-negligible probability that both circuits correspond to the same hidden service. That said, this is also partially the case for the current Tor network, where the middle node can associate a guard with a specific hidden service. 3.4 Why is the torrc setting disabled by default, if it's so good? We suggest this torrc option to be optional because it puts additional load on guard nodes and we are not sure if the network will be able to handle it. However, by having this setting be disabled by default, we make hidden services who use it stand out a lot. For this reason, we should eventually fix our hidden service circuit load balancing so that we can enable this globally. XXX But hidden services traffic is only 6% of the total network, so maybe it's not that big of a problem and we should just enable this feature by default anyway 3.4.1. How should we load balance to enable this feature globally? The load balancing issue with this feature is that hidden service traffic that would otherwise be passing through middle nodes, will now be passing through guard nodes. Furthermore, this additional traffic is not accounted for in our bandwidth weights. This means that a guard node that had 1% probability of being chosen as a guard for normal circuits, will still have the same probability of being chosen as a guard even though the hidden service traffic that it pushes increases. To improve the load balancing here, we could have each relay report the amount of hidden service traffic it pushes every day (#15254), and have the authorities take this into account when they calculate bandwidth weights. The idea here would be that the dirauths would know that N% of the network is hidden services traffic, hence they would tweak the bandwidth weights such that guards would reserve some N% of their bandwidth for hidden service purposes. 4. Future directions Here are some more ideas for improvements that should be done sooner or later: - Maybe we should make the size and rotation period of secondary/third guard sets to be configurable by the user. - To make it harder for an adversary, a hidden service MAY extend the path length of its circuits by an additional static hop. This forces the adversary to use another coercion attack to walk the chain up to the hidden service. 5. Acknowledgements Thanks to Aaron Johnson, John Brooks, Mike Perry and everyone else who helped with this idea.

7 12

Some statistics on introduction point stability and correctness
by George Kadianakis 18 Jul '15

18 Jul '15

Hello, during the past months we have been working on evaluating and confirming the stability and correctness of Tor hidden services. The hidden services protocol has multiple steps, and its soundness depends on various components of the Tor network. Throughout this document, we assume that the reader is familiar with the hidden services protocol. For people who want to get started now, we suggest you read the hidden services protocol description on our website [0]. During our evaluation, we continuously fetched the descriptors of a few hidden services, and collected statistics on the introduction points they picked. The hidden services we tested were selected arbitrarily so that they have decent uptime and are moderately used. We have labeled them alphabetically from (a) to (e). The data was collected over a period of 90 days using tor-hs-descriptor-fetcher [1]. With our experiment we were trying to answer questions like: Q: Do hidden services publish descriptors correctly and in the intended time schedule? Q: Is the introduction point codebase working properly? Are hidden service introduction points as stable as we want them to be? Q: How many introduction points do hidden services expose themselves to? Q: Normally, hidden services will use 3 intro points. However if they think they are getting too much traffic they will dynamically adjust the number of their introduction points according to a self-evaluation of their popularity. In this case, very popular hidden services may use up to 10 introduction points. Does this algorithm work as intended? Let's start our analysis by looking at some graphs: ----- [*] https://people.torproject.org/~asn/desc_stats/lifetimes-2015-07-14-b.png This graph shows the number of descriptors published per hour by each hidden service. Including descriptor replicas, a normal hidden service would publish 2 descriptors per hour which indeed seems to be the case most of the time according to the graphs. This is good since it shows that our system works properly. However, we also observe that all HSes will publish more than 2 descriptors every hour at least 15% of the time. We believe this occurs when they republish their descriptor because of an expired or dead intro point. ---- [*] https://people.torproject.org/~asn/desc_stats/lifetimes-2015-07-14-a.png Normally hidden services will keep their introduction points for a random lifetime between 18 and 24 hours. This graph shows how long the measured hidden services kept their introduction points for. Looking at the three hidden services (a), (e) and (f) it seems that introduction points indeed rotate as intended most of the time. Specifically, we see that about 75% of the intro points of those three hidden services indeed stay up for more than 18 hours. This reassured us that the introduction point rotation code works well. However, we can see that hidden services (b), (c) and (d) have lower intro point lifetimes. We believe that this is caused by the dynamic intro point formula which adjusts the number of introduction points for popular hidden services. Consider a hidden service with 9 intro points that starts getting less traffic and needs to go down to 5 intro points; in that case, the HS will discard 4 IPs reducing the average intro circuit lifetime. If the above procedure happens multiple times, it will drastically reduce the average lifetime of intro circuits. ---- [*] https://people.torproject.org/~asn/desc_stats/lifetimes-2015-07-14-g.png In this graph we present the number of introduction points of the hidden services over time. As mentioned previously, hidden services normally use 3 intro points, but they may increase that number if they believe they are too popular. Although this self-evaluation sounds like a neat feature, it also means that an attacker can estimate the popularity of a hidden service just by looking at the number of its introduction points. While this might not sound as a dangerous information leak, we think that there are legitimate use cases where the popularity of a hidden service should be hidden [2]. Because we want to avoid this popularity leak and also because we think that the number of introduction points is not that important for scalability, we have disabled the dynamic intro point formula completely (#4862). Now, hidden services establish 3 intro points by default but operators have the option to tune that number using a torrc parameter. ---- [*] https://people.torproject.org/~asn/desc_stats/lifetimes-2015-07-14-h.png In this graph, we see the total number of relays that were used as the introduction points of each hidden service over the whole measurement period. This data is interesting because introduction points can estimate the popularity of their hidden services, so a service should ideally not expose itself to too many of them. Looking at the last graph we see that the three normal hidden services have used approximately 250 distinct relays as intro points. In other words, about 250 relays had the chance to measure the popularity of those hidden services. This seems reasonable given the 90 days measurement period and if we assume an average lifetime of 20 hours per introduction point and about 3 intro points per HS, which gives us some confidence for the correctness of the whole system. In the meanwhile, hidden service (b) used about 2000 relays as IPs! This was again caused by the dynamic intro point formula which forced it to rotate introduction continuously. We have also heard rumors that (b) got attacked by a DoS in the beginning of our measurement period, which caused it to rotate introduction points even more. We also see that hidden service (e) starts using more introduction points from the 15th of May and onwards. This seems to be caused because that hidden service started using two hidden service instances for load balancing and each instance advertises a different introduction point set. With regards to load balancing, it's worth mentioning that we are currently developing a tool called 'onionbalance' which will become the better way of load-balancing hidden services. Donncha, the author, released an alpha version just a few weeks ago which is worth trying [3]. ---- And this sums up our short analysis for today. All in all, it seems that the system works properly most of the time. Descriptors get published in the intended frequency and introduction points get rotated as they should. The popularity leakage we found for popular hidden services was also interesting, and we are happy that it has since been fixed. We hope you had fun reading our analysis, and please let us know if there are any other privacy-preserving experiments you would like to see on the hidden services world. Footnotes: [0]: https://www.torproject.org/docs/hidden-services.html.en [1]: https://github.com/DonnchaC/tor-hs-descriptor-fetcher [2]: https://lists.torproject.org/pipermail/tor-dev/2015-April/008597.html [3]: https://github.com/DonnchaC/onionbalance/ https://lists.torproject.org/pipermail/tor-talk/2015-July/038312.html

1 0

Finding location metadata in large "dark market" datasets
by Griffin Boyce 18 Jul '15

18 Jul '15

Hello all, I came across a blog post that might interest you all. @techdad did a quick analysis of public images from online black markets (such as Silk Road et al)[2] from 2011-2015, and came to the following conclusion: "After parsing hundreds of thousands of images, I came across about 37 unique images that were not properly sanitized."[1] That's surprisingly low -- 0.00037% if one assumes 100k images analyzed. Given the number of high-profile cases [4] where this location information led to arrests, it's not very surprising that some people likely took the time to remove the EXIF data, but I'm curious whether a given website may have stripped the metadata for uploaded images. The images that tested positive are shown on the blog post, and 8/37 were clearly from the same individual. When mapped out, the location data is primarily in the US (5 locations), along with 1 location in France and Australia. Incidentally, the full 1.6TB dataset from 2011-2015 is available on the Internet Archive [3], just in case the Hacking Team disclosures haven't used up all your hard drive space. ;-) This data on its own is a rather interesting look into the workings of black markets -- many of which no longer exist. Curious to see what you all think and what analyses you'd like to see from this kind of data. best, Griffin [1] http://atechdad.com/Deanonymizing-Darknet-Data/ [2] http://www.gwern.net/Black-market%20archives [3] https://archive.org/details/dnmarchives [4] https://www.eff.org/deeplinks/2012/04/picture-worth-thousand-words-includin…

1 0