- tor-relays - lists.torproject.org

Sizing, Pricing, Effect
by Dennis Ljungmark 06 Aug '12

06 Aug '12

Hi! [Hypothetical part below] We have been involved with our ISP and there may ( discussions will have to wait until after summer vacation is over when we can find real numbers and answers) but it looks as if it'd cost us ~800€ / month to go from ~1GBit to 10GBit connection. If funding was available, this would be 9.5 GBit dedicated to Tor as Exit node + relays depending on what needs people have. [End hypothetical] Even without funding, we are still looking at transforming (some) of our existing relay nodes into Exit nodes, and there's a few practical questions here. Would you rather have several smaller ones, spread out via DHCP allocated ranges, or is it preferrable to have one or a handful of "larger" exit nodes? Regards, D.S. Ljungmark

1 0

tcmalloc in FreeBSD
by Jordi Espasa Clofent 06 Aug '12

06 Aug '12

Hi all, From time to time my tor node (minibofh) stopped to work properly; since it's a tiny FreeBSD VPS, the root origin was some CPU over-consumption. I'm taking some actions to fix it (more swap, freeing CPU from another services... etc). The FreeBSD tor port offers to you the tcmalloc compilation option, so I had some fun reading this interesting paper: http://goog-perftools.sourceforge.net/doc/tcmalloc.html I recompiled the tor port but just using the tcmalloc option. Actually, now I got the google-perftool package and, obviously, the tcmalloc library: mb# pkg_info | grep google google-perftools-1.8.3 Fast, multi-threaded malloc() and nifty performance analysi mb# pkg_info -L google-perftools-1.8.3 | grep malloc.so /usr/local/lib/libtcmalloc.so /usr/local/lib/libtcmalloc.so.2 But, when I'm checking which libraries the tor binary is using, I get: mb# ldd /usr/local/bin/tor /usr/local/bin/tor: libz.so.5 => /lib/libz.so.5 (0x281aa000) libm.so.5 => /lib/libm.so.5 (0x281bc000) libevent-2.0.so.6 => /usr/local/lib/event2/libevent-2.0.so.6 (0x2844e000) libssl.so.7 => /usr/local/lib/libssl.so.7 (0x28489000) libcrypto.so.7 => /usr/local/lib/libcrypto.so.7 (0x284d7000) libthr.so.3 => /lib/libthr.so.3 (0x281df000) libc.so.7 => /lib/libc.so.7 (0x28090000) So... I understand that the tor binary is actually NOT using the new tcmalloc library. Am I wrong? PS. I adding a CC to tor FreeBSD port mantainer.

2 2

Re: [tor-relays] [tor-assistants] Call for discussion: turning funding into more exit relays
by Andrew Beveridge 05 Aug '12

05 Aug '12

---------- Forwarded message ---------- From: TechFix LTD <andrew(a)techfixuk.com> Date: Wed, Aug 1, 2012 at 8:25 AM Subject: Re: [tor-assistants] Call for discussion: turning funding into more exit relays To: Roger Dingledine <arma(a)mit.edu> Cc: tor-relays(a)lists.torproject.org On Wed, Aug 1, 2012 at 5:03 AM, Roger Dingledine <arma(a)mit.edu> wrote: > Sounds like you should bump it up to 100mbit then. ;) > Heh, I've just done exactly that - well, I've raised the bandwidth limits in my torrc to values higher than what my ISP caps my connection at. Just to clarify though, is there any technical reason why it would be better practice to have a realistic configured limit (like 12.5MB) as the limit rather than something higher than that which I know will likely never actually be possible? I simply want to use all the bandwidth available. I'll monitor the websites I'm hosting from this server closely for a while and see if they are noticeably affected by this at all. They are hosted from a separate VM and I'm not sure how the OpenVZ host schedules packets going through the virtual network adapters, but I'm hoping that tor can run at maximum capacity most of the time while only adding perhaps a second or two of load time onto the websites. > You can see on http://atlas.torproject.org/#details/FA02311AF49EB663CA2685A8604C403A9E10E6… that there are periods where your rate limiting is bottlenecking traffic. I presume you're referring to the busy periods where the graph clips? Sure, but as my company pays for the server entirely, if it were to not function for its intended purpose (basic hosting of small-time client websites) I would have a problem. I was worried that allowing tor to use all my bandwidth would cause that, but I hadn't actually experimented with it to see - I'm doing that now! > > I definitely don't want Tor to be in the middle of the transactions -- if Tor pays the bills directly, that's too much like being the relay operator. One nice situation would be for you to produce receipts showing expenses, and then we reimburse those costs. It requires fronting a bit of money on your part, but that's part of saving the world, yes? :) > Ultimately, we're also going to want to look into reducing overhead on Tor's side from sending out money. If we have to write and mail 50 checks every month, that's going to waste a lot of somebody's time. Maybe that means Paypal is the way to go. Maybe it means we send some money in bulk to Zwiebelfreunde, and they do intra-Europe wire transfers to the other Europeans (though I admit maybe that just shifts the time-wasting). Lots of options there. What would be best for you? Good point. I guess the solution which seems logical to me would be a standing order from Tor into my company account for the amount mentioned above, with my OVH invoice emailed to you as justification. As the amount would not change from month to month, a standing order could be set up once and never require any more attention unless I got cut off by OVH. Presumably that would require a european bank account under Tor's control - I'm not sure about how your connection with the Wau Holland Foundation works, but are they perhaps able to help with european transfers? If that is too awkward, then sure, PayPal (or one of several other e-Payment systems) is a fairly easy way to go about it, with the loss of a few % of the payment amount. My business currently uses PayPal for all payments from US entities, simply because it is easy. > All of this said, don't go out and start spending money quite yet. We should figure out these logistics first. And Tor should get a bit more of a handle on what this diversity thing should mean. And I should get buy-in from other Tor people for my plans here. :) > --Roger > I'm going to try my best to run my current node as a full 100Mbit node from now on anyway, even if that means paying for more traffic. As soon as things are ready at your end, I'm ready to start a second identical exit as described in my previous mail. I'm going to try and start looking for other providers, but I don't want to reinvent the wheel - is https://trac.torproject.org/projects/tor/wiki/doc/GoodBadISPs kept up to date by many people here? Perhaps we could develop a slightly better system for tracking good/bad ISP experiences than a flat wiki page with countries? So my node has been running without any bandwidth restriction for a few days now, and I'm not noticing any critical speed problems from the websites I host. The latency drops significantly when the node is pushing traffic at max bandwidth, but that doesn't seem to be very often ( https://atlas.torproject.org/#details/FA02311AF49EB663CA2685A8604C403A9E10E… ). So, as it stands right now, I'm happy to run that node for free as I already have been. If the OVH manager decides I am very close to my initial 10TB traffic cap, I will have to limit the node again unless I can get some funding (£9-£18) for another 20TB each month. Out of curiosity, can anyone explain to me why my bandwidth /hasn't/ been maxed out the past few days? (at least, according to Atlas) -- * Andrew Beveridge, **Tor lurker / sympathizer / advocate, exit node maintainer.* Director / Computer Repair Technician @ TechFix LTD [ http://www.techfixuk.com] Freelance Web Developer @ [http://www.andrewbeveridge.co.uk] Tel/Fax: +44 (0) 1383269735 | * Email: tor(a)techfixuk.com <andrew(a)techfixuk.com> * * 5 Drum Road, Kelty, KY4 0DQ, Scotland *

1 0

Re: [tor-relays] task-6329 / tor relays stats patch
by Karsten Loesing 03 Aug '12

03 Aug '12

Hi Michael, I'm cc'ing tor-relays for the discussion here, because I figured if you're okay with sharing the patch, you're probably also okay with a public discussion of it. :) On 7/21/12 4:19 PM, Michael Zeltner wrote: > Hi Karsten, > > On my way out, and I would otherwise forget but here you go, maybe you'd like > this. > > Best from Metalab/Vienna, > Michael Thanks for sending the patch! Most of it looks good, but I took out the --family-for-fp option which, I think, isn't entirely correct. This option looks for relays having the given fingerprint in their family line, but it doesn't cross-check if the given relay has those relays in its family line, too. The option also doesn't look at nicknames (of relays with the Named flag) in family lines, which means it might miss some family members. Oh, and it would be really cool to have a -F or --by-family option to aggregate by families. I wonder if we should do the whole family-calculating business inside Onionoo and add a family_number field which has the same number for relays in the same family. The script could then look up the family number of the given relay and display all relays with that family number. Would that make sense? Want to extend the script once Onionoo has such a field? Thanks! Karsten

2 4

Re: [tor-relays] Call for discussion: turning funding into more exit relays
by Andrew Beveridge 02 Aug '12

02 Aug '12

On Mon, Jul 23, 2012 at 9:22 PM, Roger Dingledine <arma(a)mit.edu> wrote: > Hello Andrew, > > I wanted to draw your attention to a thread I've started on the tor-relays > list: > https://lists.torproject.org/pipermail/tor-relays/2012-July/001433.html > > Hi Roger, I'm not too familiar with mailing lists, so hopefully I'm not top posting or replying in the wrong place here. I'm simply replying to your email in Gmail with a CC to the tor-relays list. Sorry for the late reply, I've been away with no internet access for the past week. > In short, we have a funder who wants to sponsor more and faster Tor > exits, and we're brainstorming about how to use the money in a way that > makes the network stronger but also doesn't screw up the "community" > side of the Tor relay operator community. The first step is collecting > facts about the current fast Tor exit relays. > > Great! I've not been vocal in the Tor community before now, but I'd like to start, and I'm certainly happy to share my experience / thoughts. > - What do you currently pay for hosting/bandwidth, and how much bandwidth > do you get for that? > This is a complicated question, because I run a single Tor exit in a VPS on my company dedicated server. I run a local company doing computer repair and web development, and lease a single dedicated server from OVH (more specifically, Kimsufi) for a total of £64 a month (inc. VAT). That gets me the Kimsufi 16G dedicated server, a RIPE block of 4 extra IPs, and an external 2TB HDD. 100Mbit pipe, 10TB/month bandwidth. £0.87/TB if I go over that, so if I were to max out the bandwidth for an entire month, using around 30TB traffic, I would have to pay about £18 on top for the extra bandwidth. However, according to the OVH manager I never seem to go anywhere near the traffic limit, despite having had the exit set to use 50Mbit/s constantly for the past 3 months. As far as I'm concerned, it costs me nothing to run this exit node - my company needs the dedicated server regardless, and none of the ~50 websites I host use enough traffic to be affected by the fact that my server is using half it's available bandwidth for Tor. In an ideal world, I would rent a second Kimsufi server just for Tor purposes, which would cost £36/month (Kimsufi 16G) + £4/month (RIPE block). Therefore, if I were to participate in this experiment, I would say *£40 GBP / month* would get *10TB of 100 Mbit/s* exit traffic. Additional *20TB* traffic could be purchased for *£18 / month*, which would bring the maximum cost to *$92 USD / month.* > - Is it a stable hosting situation? For example, how do they handle > abuse complaints so far? > This is an important one, because OVH are not Tor-friendly. In fact, they aren't very friendly in general! Several people told me it was very foolish to run an exit node on an OVH server, since as soon as OVH get even a whiff of a complaint, they are quick to suspend. All my company-hosted websites are hosted on a second redundant server with another ISP so I'm not worried if OVH do find out and take that route and cut me off. I'm also not worried about legal issues, as I have positioned myself as a web-hosting customer of my own PLC, with terms and conditions absolving my PLC of any legal risks. In the off chance some legal authority were to come to my flat in britain, they would find no electrical equipment to seize as I own only a single laptop and it usually lives in my car or partners house anyway! As I have only been running this exit for 3 months, I am far from an authoritative voice on the issue of abuse complaints, but the most important thing is definitely SWIP as far as I can see it - the IP address I use for the exit is one from the RIPE block I lease and as such the abuse email is my own. I've had about 20 abuse report emails so far, all automated (by the looks of things) from some system within the brazilian government, following an Sqlmap SQL injection scan/attack on a few government sites. I replied to the first one with the standard template, got no reply, haven't worried about it since. I reckon if OVH found out I was running an exit they would be likely to cut me off fairly swiftly, but they don't seem to pay much attention to Kimsufi customers since it is their budget range with very slow (week+ per ticket) support and presumably low margins. I think that and me as the primary abuse contact for the IP mean it's likely to stay up for a while. If I do get any hassle, I'll defend it as far as I can from a business perspective, but if they don't give in I'll likely just cancel it and open a new Kimsufi lease - I very much doubt anyone is checking new invoices for similar details to past customers. > - Is your hosting situation one where it could make sense for us to > reimburse your bandwidth costs? (Some people have a deal through their > employer, friend, etc where they don't pay for hosting.) > I don't think so - as explained above, at present I don't pay a penny, but I can only offer about half of my available bandwidth as the server is used for many purposes. If I were to participate in this experiment, it would probably mean purchasing another Kimsufi just for this, and the cost of the server itself would be what I would be looking for financial help with. > - Are you in a position to get more bandwidth if you pay more? At what > rates? We're most interested in sponsoring >=100mbit relays. > Unfortunately the Kimsufi servers are capped at 100Mbit regardless of whether you want to pay more. OVH obviously have Gbit and 10Gbit servers available, but they are too expensive for this. There are obviously far better alternatives for higher bandwidth servers - a quick look tells me I could get a 1Gbit dedicated server with 100TB traffic from Leaseweb for €99 ($121) / month, so obviously if the money is there, more bandwidth and traffic can be had. I guess it boils down to how many people you can get interested in this - if plenty, lots of 100Mbit servers is presumably better than a few 10Gbit ones for the money as it aids network diversity, even if (worst case) they are all hosted by the same provider. > - Do you have other locations in mind where you would run another exit > relay if you didn't have to pay for it? > Definitely! As far as I'm concerned, I am not worried about legal issues as long as I can purchase hosting through my business and SWIP the IP, and I have plenty of free time to spend configuring servers and responding to abuse emails, so if I had the money I would happily be running exit nodes in any country I could find a hosting provider in - money is the hurdle for me. > - What else should we be asking here? :) > One thing which I haven't seen discussed yet is how funding would actually be connected with operators - I'm not sure if you were thinking about the funder(s) directly sending money to operators, or if The Tor Project Inc would be acting as a middleman? What money transfer mechanisms would be safe to use, how would you verify that the money was going to the right person, would The Tor Project Inc receive invoices directly from hosting companies or would operators email copies of invoices to someone and then some money would turn up in their bank accounts? What about PayPal, etc? Just a few thoughts :) Thanks! > --Roger Thanks for inviting me to share my thoughts on this! -Andrew

8 10

Domain name based policies (was: Call for discussion: turning funding into more exit relays)
by Nicolas Braud-Santoni 01 Aug '12

01 Aug '12

2012/8/1 Roger Dingledine <arma(a)mit.edu>: > On Tue, Jul 31, 2012 at 11:21:01AM +0100, mick wrote: >> Question for tor developers. How hard would it be to change the logic >> (and syntax) of exit policy in tor to allow domain based formulations >> like: >> >> reject *.gmail.com >> reject *aol.com > > Very hard. > > https://trac.torproject.org/projects/tor/wiki/doc/TorFAQ#Exitpoliciesshould… Hi, While I see how allowing wildcards and domains in policies would be more than challenging, wouldn't it be possible to : - resolve domain-names at Tor startup, and get all associated A and AAAA records - Repeat when record's TTL is reached. Of course, it wouldn't work for sites that don't advertise all their IPs. It would also require the Exit node's operator to run some DNS resolver (or trust an external one), but locally running unbound (for example) is quite simple. Moreover, the risk evoked in the FAQ is already present : if I poison an exit node's DNS resolver, wouldn't I be able to replace nytimes.com A record with some bogon, like 0.0.0.0 ? Nicolas

2 1

Re: [tor-relays] Call for discussion: turning funding into more exit relays
by Scott Bennett 01 Aug '12

01 Aug '12

On Fri, 27 Jul 2012 01:23:57 -0400 grarpamp <grarpamp(a)gmail.com> wrote: >>> 4) What exactly do we mean by diversity? >> >> I would look at this almost entirely from a jurisdictional and ISP level. I >> believe the biggest "sudden impact" threats to the tor network are going to >> be from legal changes (jurisdictional, i.e. "save the children, nullroute >> the nodes") and local business policy changes ("sorry tor customers, no more >> tor egress from our DC due to complaints"). > >I'm not sure which thread I mentioned this on so I'll put it here to be sure. >I think one main thing needed is a project to catalog all the current >exits as to their diversity... >Box: ISP/hoster, AS, datacenter, country, upstream AS/Tier-n path, >relay-operator >Relay-operator: country > >Without that, seems like placing nodes amounts to, 'Well, >we don't have any in Iran, let's go there'. If it turns out that >IP is more or less fed as a courtesy from UAE across the >gulf, there's not much gain. Repeat analysis for any of the >above parameters. > >More nodes are probably good, just not all as USA, Equinix, >Level3, with whatever hoster has a rack in all the DC's. I agree completely. But I would also like to add that, aside from Brasil, most of South America is still dark. Central America is not much better either. Many of those states are not especially cooperative with each other, politically speaking, yet they all need the benefits of commerce associated with the Internet. That combination strikes me as beneficial to placement of tor relays in as many of those countries as possible. Much of Africa may be worth closer examination for the same reasons. We really need to keep political diversity in view, especially given the large fractions of the tor network currently concentrated inside a mere handful of politically allied states. The Dictator of the U.S., for example, has already made the threat of shutting down the entire U.S. portion of the Internet, including relaying between other countries, which would certainly have a severely disruptive effect upon tor users all around the globe were it to happen under today's distribution of tor relays. Even more drastic would be if any of, for example, the U.K., France, Germany, or the Netherlands were to follow suit. Having countries like Venezuela, Bolivia, Peru, and Ecuador more tor-populated looks to me like a good thing. Another point I'd like to make is that I don't see why having one 100 MB/s relay is somehow better than having ten 10 MB/s relays or 20 5 MB/s relays. The superhigh-speed relays push operating system limits on the number of connections. Due to tor's design, distributing the workload of such relays across multiple CPU cores is problematic. Olaf Selke got around that problem by running four nodes on a quad-core machine with two IP addresses, but that meant that each node usually ran at less than 15 MB/s. For a superfast setup today, it might mean running multiple 25 MB/s nodes in similar fashion to what Olaf did, rather than a single 100 MB/s node. The benefit to tor users would seem to me to be the same either way, but the multinode method would not satisfy the demand of the funding source, as I understood it. Either way, though, the operating system limits may place keep a lid on the actual tor capacity of a very fast setup. From an infrastructure standpoint, I acknowledge that there can be problems in setting up really fast relays in Latin American countries. However, adding a few relays on the order of 500 - 5,000 KB/s in each Latin American country could probably be done, even if it meant they could only be set up in national capitals, which are mostly megalopolises of several million people, and might involve making special arrangements with the ISPs. Other major cities in some countries may also have the infrastructure to make moderately fast relays possible. Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at cs.niu.edu * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * **********************************************************************

1 0

How to protect yourself from network scanning
by amki 01 Aug '12

01 Aug '12

Hiho, I am hosting a 3-5MB/s tor exit relay but as of today my hoster has closed my server because of network scanning. Is there a known proper way to protect yourself from being used as a network scan relay? I've thought about constructing iptables rules to limit the number of SYN packets for the same host per second or such, but I'm not sure if this is allowed or will get me flagged as a bad exit node. My hoster is quite ok with us generating some abuse complaints per month, but does not want to route network scanning traffic since it is a severe load to their routers. Any help would be appreciated Cheers amki -- "Whatever the reason you're on Mars, I'm glad you're there, and I wish I was with you." - Carl Sagan

5 5

Re: [tor-relays] Call for discussion: turning funding into more exit relays
by Steve Snyder 01 Aug '12

01 Aug '12

Roger Dingledine arma at mit.edu Mon Jul 23 18:58:54 UTC 2012 [snip] >At the same time, much of our performance improvement comes from better >load balancing -- that is, concentrating traffic on the relays that can >handle it better. The result though is a direct tradeoff with relay >diversity: on today's network, clients choose one of the fastest 5 exit >relays around 25-30% of the time, and 80% of their choices come from a >pool of 40-50 relays. [snip] From what I see on the TorStatus pages (torstatus.all.de, blutmagie.de) about a third of the roughly 3000 relays listed are at or below 64KB/sec of demonstrated bandwidth. No doubt some of these are soon-to-be-high-bandwidth servers that are just ramping up, and some are nodes having transitory networking problems. It seems reasonable to assume, though, that most of these low-bandwidth nodes are intentionally low-bandwidth, perhaps on the basis of the Tor doc stating a 20KB/sec minimum. With "80% of their choices come from a pool of 40-50 relays" that leaves a 20% chance for the remaining 2950 nodes. A case for low-bandwidth nodes can be made as a means to dissuade anticipated routing (due to pool size), but it seems from the stats quoted above that there is little chance that 2000+ of these 3000 nodes will ever carry Tor traffic, and thus can be ignored for purposes of traffic analysis. Is there any justification for a low-bandwidth Tor node? And if so, what is the practical minimum bandwidth needed to actually see any traffic?

4 3

Re: [tor-relays] Call for discussion: turning funding into more exit relays
by Roger Dingledine 01 Aug '12

01 Aug '12

On Fri, Jul 27, 2012 at 05:49:34AM -0400, Motoko Kusanagi wrote: > I am very interested in running 100 Mbit (maybe even more) exit nodes at 100$/month, however, a question immediately comes to mind: > When we say "100Mbit exit node", do we imply "really unmetered" traffic at 100 Mbit, or do we mean "semi-reasonable traffic limit at 100Mbit speed". > "Really unlimited", in my experience, oftentimes turns out to be somewhat problematic. I don't think we necessarily need to have the 100mbit all to ourselves (see the other discussions here about shared connections), but I think it needs to be able to push a good fraction of 100mbit throughout the month. So something that can handle 20-30TB/mo is in the right ballpark. Note that many of our current 100mbit nodes are actually gbit nodes that try to limit themselves to a long-term average (95th percentile) of 100mbit. > I could arrange for 50-100 100Mbit exit nodes running 24/7, with a >traffic limit somewhere between 1 and 4 terabytes (will need to scrutinize >my current contract and do some math). Arranging for such a number of >"honestly unlimited traffic at 100Mbit speed" would be...significantly >more problematic. 4TB/mo is good for a 10mbit connection, but not good for a 100mbit connection. You'll end up either needing to hibernate for 80% of the month, or rate limit your relays down a lot. That said, we don't need 50-100 100mbit exit relays from any single group. Just 5-10 would be nice. Maybe that makes the capacity issue easier? --Roger

1 0