On Tue, Nov 4, 2014 at 12:25 PM, Libertas libertas@mykolab.com wrote:
I think it would be a good idea to add OpenBSD to doc/TUNING because [...] promoting OpenBSD relays benefits the Tor network's security.
Absolutely. Not just due to OpenBSD's security positioning, but moreso from network diversity. Windows is its own world. But if you're a Unix admin there's no reason Linux should be deployed 20x more often than [Free/Open]BSD. It's ridiculously counter to meeting diversity goals, especially with bandwith weighting if one platform is getting grossly disproportionate traffic than another. Just pick one of the two BSD's and run it instead. FreeBSD in particular is well suited to the OS and network needs of Tor. And knowing how to admin more Unixes will serve any admin well.
5950 Linux 1593 Windows 173 FreeBSD 55 Darwin 44 OpenBSD 7 NetBSD 6 SunOS 4 Bitrig 2 GNU/kFreeBSD 1 DragonFly
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Agreed. Thanks for pulling together the statistics, too. However, I'd like to make an argument for OpenBSD specifically.
I openly acknowledge that, at least for non-experts (and I'm one of them), OpenBSD isn't ideal for many uses. It isn't used much because of its conservative/cautious philosophy and its lack of bells and whistles. It doesn't have the greatest hardware support, it's a little slower than FreeBSD and Linux, and it isn't very inviting for people that don't know at least intermediate Unix.
However, there is at least one field in which OpenBSD has a big market share: firewalls. It's perfect for this use because of its simplicity, its great networking software (pf, etc.) and its bulletproof out-of-the-box security. These same features make it excellent for Tor relays as well.
It's possible that governments like China's are trying to hack Tor relays in an attempt to deanonymize users. It's almost definite that malicious hackers try to break into exit nodes to troll traffic. Even an up-to-date, hardened Linux or FreeBSD system probably can't weather all such attacks. For such a simple, single-use, security-critical application, something as sturdy and impenetrable as OpenBSD is the best option.
I would love to start a larger conversation about running Tor on OpenBSD. I've been considering making a guide describing the process. However, that violates the OpenBSD philosophy to some extent. They tend to only help those who help themselves - in the long term, only those who want to learn Unix and who RTFMs continue using OpenBSD.[1] Hopefully, though, we can spark enough interest that node operators will take that initiative. I know there's been a lot more interest in OpenBSD on Hacker News et al. since the surveillance revelations.
[1] I hope this doesn't sound pretentious. I recognize that a lot of people are busy or distracted, or simply don't want to make the time commitment. That's reasonable.
Thanks for reading another rambling email, Libertas
On 11/05/2014 04:04 AM, grarpamp wrote:
On Tue, Nov 4, 2014 at 12:25 PM, Libertas libertas@mykolab.com wrote:
I think it would be a good idea to add OpenBSD to doc/TUNING because [...] promoting OpenBSD relays benefits the Tor network's security.
Absolutely. Not just due to OpenBSD's security positioning, but moreso from network diversity. Windows is its own world. But if you're a Unix admin there's no reason Linux should be deployed 20x more often than [Free/Open]BSD. It's ridiculously counter to meeting diversity goals, especially with bandwith weighting if one platform is getting grossly disproportionate traffic than another. Just pick one of the two BSD's and run it instead. FreeBSD in particular is well suited to the OS and network needs of Tor. And knowing how to admin more Unixes will serve any admin well.
5950 Linux 1593 Windows 173 FreeBSD 55 Darwin 44 OpenBSD 7 NetBSD 6 SunOS 4 Bitrig 2 GNU/kFreeBSD 1 DragonFly _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 11/05/2014 10:35 AM, Libertas wrote:
I would love to start a larger conversation about running Tor on OpenBSD. I've been considering making a guide describing the process. However, that violates the OpenBSD philosophy to some extent. They tend to only help those who help themselves - in the long term, only those who want to learn Unix and who RTFMs continue using OpenBSD.[1] Hopefully, though, we can spark enough interest that node operators will take that initiative. I know there's been a lot more interest in OpenBSD on Hacker News et al. since the surveillance revelations.
As a node operator and as someone who has been a small-time sysadmin for *something* with the Unix nature since 1996, I have to say that the main reason I run my nodes on Linux is that I don't feel I know my way around any modern *BSD enough to lock them down properly. A thorough guide to setting up -- and maintaining -- OpenBSD for a node would help with that a great deal.
zw
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Thanks for the quick response, Zack!
I'm hesitant to give too much advice, as I've been using OpenBSD for all of four months, and I've been a small-time sysadmin for all of three years. So, take all this with a grain of salt.
My ultimate concern is that OpenBSD is just far less wizard-friendly and tutorial-rich than Linux etc. are. The man pages are excellent, but there inevitably comes times when one needs to do some research (sometimes even in the source code - check out bxr.su) to solve something properly. People maintain, and I agree with them, that this is ultimately a more stable and fast means of administration, but it takes a commitment to doing the necessary reading. Many OpenBSD people therefore dislike ad hoc guides, as they just delay the frustration that is inevitable for some people.
Come to think of it, my email which started this discussion ("[tor-dev] OpenBSD in doc/TUNING") is a good example of what I'm talking about. I had to do some digging and man-page-reading to change the maximum number of file descriptors for the daemon. In return, though, an unprivileged user can't choke my system out by opening files en masse on the default install.
Maybe the best solution is to just put such a disclaimer on the guide. Most OpenBSD introductions make it very explicit. On that note, the single venerated beginner's guide to OpenBSD is _Absolute OpenBSD 2nd Ed._ by Michael Lucas. You should check it out if you're interested. It's a fantastic, colorful book, and it focuses on what's unique about OpenBSD.
I appreciate your interest! Also, I hope I'm not speaking with too much authority. If anyone here has more OpenBSD experience than me, please send addendums or corrections.
Libertas
On 11/05/2014 10:47 AM, Zack Weinberg wrote:
On 11/05/2014 10:35 AM, Libertas wrote:
I would love to start a larger conversation about running Tor on OpenBSD. I've been considering making a guide describing the process. However, that violates the OpenBSD philosophy to some extent. They tend to only help those who help themselves - in the long term, only those who want to learn Unix and who RTFMs continue using OpenBSD.[1] Hopefully, though, we can spark enough interest that node operators will take that initiative. I know there's been a lot more interest in OpenBSD on Hacker News et al. since the surveillance revelations.
As a node operator and as someone who has been a small-time sysadmin for *something* with the Unix nature since 1996, I have to say that the main reason I run my nodes on Linux is that I don't feel I know my way around any modern *BSD enough to lock them down properly. A thorough guide to setting up -- and maintaining -- OpenBSD for a node would help with that a great deal.
zw _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 2014-11-05 10:47, Libertas wrote:
I appreciate your interest! Also, I hope I'm not speaking with too much authority. If anyone here has more OpenBSD experience than me, please send addendums or corrections.
Maybe call this an addendum? Some version of the following work in progress is going onto our local documentation store for others maintaining our OpenBSD relays.
It's a bit long-winded for inclusion in doc/TUNING per https://trac.torproject.org/projects/tor/ticket/13702 , as it's intended to educate *BSD and Linux sysadmins about a smidgen of the why behind the tuning recommendations, as well as point at further exploration.
Richard
------- Our OpenBSD tuning for Tor involves: 1) Certainly boosting the per-process allowed number of open files per Tor relay to accommodate Tor circuits, 2) Probably increasing the system maximum number of open files to accommodate Tor circuits, 3) Probably boosting the pf state table capacity to accommodate Tor circuits, guard, and exit traffic, and 4) Probably running more than one relay on a host to use available CPU cores and bandwidth.
1) Per-process open files (openfiles-max)
If you leave the per-process default of 1024 open files in place, even a moderately busy Tor relay will eventually log messages telling you it's using all available file descriptors, meaning relay throughput is constrained artificially.
Here are a couple ways that work for boosting openfiles-max for Tor without allowing other processes to run away.
a) At the end of /etc/login.conf: -----8<----- # For Tor, set an openfiles-max to override probable default openfiles-max # of 1024 tor:\ :openfiles-max=8192:\ :tc=daemon: -----8<----- The name 'tor' for this stanza comes from the pkg_scripts entry in /etc/rc.conf.local.
b) Before the line "rc_cmd $1" in /etc/rc.d/tor: -----8<----- ulimit -n 8192 -----8<----- This line will need to be re-added when|if the /etc/rc.d/tor script is replaced during the course a package upgrade.
2) System maximum open files (kern.maxfiles)
The system maximum number of open files allowed (kern.maxfiles) has a default calculated per architecture (7030 for amd64 [1]). This default may be exceeded on a moderately busy Tor relay. It can be adjusted with sysctl(8).
Check 'sysctl kern.nfiles' under load to see how many file descriptors are in use. We regularly find > 7000 open files in use on a long-running relay that does nothing but Tor.
Roughly doubling the amd64 default kern.maxfiles to 2^14=16384 might be good if you're hitting the limit.
To change it until reboot: sudo sysctl -w kern.maxfiles=16384
kern.maxfiles can be set persistently by adding a line to /etc/sysctl.conf as follows: -----8<----- kern.maxfiles=16384 # bump kern.maxfiles from ${ARCH} default for Tor -----8<-----
--- [1] Here's one description of the amd64 default kern.maxfiles calculation: http://unix.stackexchange.com/questions/104929/does-openbsd-have-a-limit-to-...
3) pf state table capacity
pf is on by default in OpenBSD, keeping state for network flows. Rather than turning it off or writing rules with 'quick' and 'no state' for your Tor ports (the state table will be faster than rule parsing), you may want to adjust its capacity. This will be especially useful if you're running a popular Tor relay or set of relays.
Examine your pf resource limits: sudo pfctl -sm Examine your pf usage under load: sudo pfctl -si
If your 'state-limit', 'src-limit', and/or 'memory' counters are incrementing, your host is almost certainly dropping packets from time to time due to resource limits on state table storage.
For example, this kind of output from pfctl -si is probably bad for your throughput: memory 8986091 0.9/s
If your relay is used widely, especially as a guard, or you have multiple relays sharing pf's limit, you may also find the "current entries" in your state table (per pfctl -si) getting close to the default limit of 10k (per pfctl -sm).
Boost the limits as needed. One suggestion for the options setting section of your /etc/pf.conf is to boost the defaults by 10x or so: -----8<----- set limit { states 100000, src-nodes 100000, frags 8000 } -----8<----- Then continue monitoring the usage
If you're worried about memory use for the extra state table entries, 256 MB should be more than enough for 500k entries. If your kernel is severely memory constrained (e.g., you're running on older i386 hardware with a total of 512 MB RAM), you can also explore expiring TCP states faster (see sudo pfctl -st, maybe drop tcp.closed from 90s to 10s or the like).
For more info, here are writeups (the second with RRDTool monitoring suggestions): http://serverascode.com/2011/09/12/openbsd-pf-set-limit-states.html http://www.skeptech.org/blog/2013/01/15/pf-limits-in-openbsd/
4) Loading more CPU cores
If you have one of your CPUs maxed out running a Tor relay, with the other CPU(s) mostly idle (see top(1)), yet you have bandwidth to spare still, you can run additional Tor instances to sop some of it up.
The sanest way to handle this is to make each relay a stand-alone entity with a naming scheme to keep them straight. Here, we'll use "tor#" for every relay past the first.
Make per-relay directories in /var owned by _tor:_tor mode 700 drwx------ 5 _tor _tor 512 Jan 13 18:52 /var/tor/ drwx------ 5 _tor _tor 512 Jan 13 22:39 /var/tor2/ drwx------ 5 _tor _tor 512 Jan 13 22:39 /var/tor3/ ... Copy the tor startup script /etc/rc.d/tor to match the naming scheme. /etc/rc.d/tor2 /etc/rc.d/tor3 ... Copy the torrc from /etc/tor/torrc. /etc/tor/torrc2 /etc/tor/torrc3 ... Modify /etc/tor/torrc2, /etc/tor/torrc3, ... so they refer to their appropriate private DataDirectory and PidFile, listen on the appropriate ports and IP addresses, and have the appropriate exit policies. (Remember that the public Tor network will by design ignore more than two relays per IP address.) DataDirectory /var/tor2 PidFile /var/tor2/pid ControlPort 9222 Address 10.2.2.2 ORPort 8222 DirPort 7222 ... DataDirectory /var/tor3 PidFile /var/tor3/pid ControlPort 9333 Address 10.3.3.3 ORPort 8333 DirPort 7222 ... Set each relay to launch at system startup via the named /etc/rc.d scripts in /etc/rc.conf.local's pkg_scripts. tor_flags="${tor_flags} -f /etc/tor/torrc" tor2_flags="${tor2_flags} -f /etc/tor/torrc2" tor3_flags="${tor3_flags} -f /etc/tor/torrc3" ... pkg_scripts=" ... tor tor2 tor3 ..." Set openfiles-max for each named pkg_script from /etc/rc.conf.log in /etc/login.conf. tor:\ :openfiles-max=8192:\ :tc=daemon: tor2:\ :openfiles-max=8192:\ :tc=daemon: tor3:\ :openfiles-max=8192:\ :tc=daemon: ... Remember to allow inbound traffic to the additional ports set in /etc/tor/torrc[#] in your /etc/pf.conf.
Is there much of a difference between setting up Tor on OpenBSD vs. Linux or other Unix(like) systems?
Or is this just about setting up OpenBSD in general, or additional security for relays (disk encryption, memory protection) whose use isn't common on most general servers?
I would love to start a larger conversation about running Tor on OpenBSD. I've been considering making a guide describing the process. However, that violates the OpenBSD philosophy to some extent. They tend to only help those who help themselves - in the long term, only those who want to learn Unix and who RTFMs continue using OpenBSD.[1] Hopefully, though, we can spark enough interest that node operators will take that initiative. I know there's been a lot more interest in OpenBSD on Hacker News et al. since the surveillance revelations.
On Wed, Nov 5, 2014 at 11:20 AM, Niklas Kielblock niklas@spiderschwe.in wrote:
Is there much of a difference between setting up Tor on OpenBSD vs. Linux or other Unix(like) systems?
Or is this just about setting up OpenBSD in general, or additional security for relays (disk encryption, memory protection) whose use isn't common on most general servers?
Well, the thing *I* don't feel I have the least idea even where to begin with, with *BSD in general, is reliable automatic installation of security updates for both the base system and the ports. I can figure everything else out once and write it into /etc and be done with it. But if I have to manually monitor for bug fixes in all the installed software, and manually update local source code copies and recompile every time, well, that's three chores that computers are better at than I am.
(Actually, the ports system has blown up in my face often enough that I'm convinced it has fundamental design flaws -- and this was in the much less demanding environment of a development VM. I would be much more comfortable with a BSD that accepted the maxim that there can be only one package manager and nothing may escape its gaze.)
zw
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
I don't want to spam this list with OS discussion, but I think yours is an important point, so I'll give my perspective briefly.
This is one of the main aspects of OpenBSD that make it better suited for firewalls etc. than for desktops. One of the main tenets of OpenBSD is "secure by default". There have only been two remote holes in the default install in the last seventeen or so years, so there generally isn't a serious need for base updates. Keeping up with the occasional patches is a good idea, which is done manually by default. This generally takes a file download and about three copied-and-pasted commands. The announce mailing list lets you know about these, and there are scripts to apply them automatically.
If you're running a Tor relay,, Tor might be the only thing you install. I also have Vim, SSHGuard, and possibly a library or two for Arm, but that's it. Hopefully, all relay operators keep up with the Tor community enough to stay on a supported version, if not the newest one. The updates are rare enough that I haven't found manual compilation an issue. My OpenBSD node is currently on 0.2.5.10.
If compilation is considered tedious, though, I or someone like me could start more aggressively maintaining the Tor port. I was actually considering this recently, although I have no prior experience with port development. There are almost 9,000 ports, and they're only updated as quickly as they're developed.
Libertas
On 11/05/2014 12:07 PM, Zack Weinberg wrote:
On Wed, Nov 5, 2014 at 11:20 AM, Niklas Kielblock niklas@spiderschwe.in wrote:
Is there much of a difference between setting up Tor on OpenBSD vs. Linux or other Unix(like) systems?
Or is this just about setting up OpenBSD in general, or additional security for relays (disk encryption, memory protection) whose use isn't common on most general servers?
Well, the thing *I* don't feel I have the least idea even where to begin with, with *BSD in general, is reliable automatic installation of security updates for both the base system and the ports. I can figure everything else out once and write it into /etc and be done with it. But if I have to manually monitor for bug fixes in all the installed software, and manually update local source code copies and recompile every time, well, that's three chores that computers are better at than I am.
(Actually, the ports system has blown up in my face often enough that I'm convinced it has fundamental design flaws -- and this was in the much less demanding environment of a development VM. I would be much more comfortable with a BSD that accepted the maxim that there can be only one package manager and nothing may escape its gaze.)
zw _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
I don't want to spam this list with OS discussion, but I think yours is an important point, so I'll give my perspective briefly.
This is one of the main aspects of OpenBSD that make it better suited for firewalls etc. than for desktops. One of the main tenets of OpenBSD is "secure by default". There have only been two remote holes in the default install in the last seventeen or so years, so there generally isn't a serious need for base updates. Keeping up with the occasional patches is a good idea, which is done manually by default. This generally takes a file download and about three copied-and-pasted commands. The announce mailing list lets you know about these, and there are scripts to apply them automatically.
If you're running a Tor relay,, Tor might be the only thing you install. I also have Vim, SSHGuard, and possibly a library or two for Arm, but that's it. Hopefully, all relay operators keep up with the Tor community enough to stay on a supported version, if not the newest one. The updates are rare enough that I haven't found manual compilation an issue. My OpenBSD node is currently on 0.2.5.10.
If compilation is considered tedious, though, I or someone like me could start more aggressively maintaining the Tor port. I was actually considering this recently, although I have no prior experience with port development. There are almost 9,000 ports, and they're only updated as quickly as they're developed.
Libertas
On 11/05/2014 12:07 PM, Zack Weinberg wrote:
On Wed, Nov 5, 2014 at 11:20 AM, Niklas Kielblock niklas@spiderschwe.in wrote:
Is there much of a difference between setting up Tor on OpenBSD vs. Linux or other Unix(like) systems?
Or is this just about setting up OpenBSD in general, or additional security for relays (disk encryption, memory protection) whose use isn't common on most general servers?
Well, the thing *I* don't feel I have the least idea even where to begin with, with *BSD in general, is reliable automatic installation of security updates for both the base system and the ports. I can figure everything else out once and write it into /etc and be done with it. But if I have to manually monitor for bug fixes in all the installed software, and manually update local source code copies and recompile every time, well, that's three chores that computers are better at than I am.
(Actually, the ports system has blown up in my face often enough that I'm convinced it has fundamental design flaws -- and this was in the much less demanding environment of a development VM. I would be much more comfortable with a BSD that accepted the maxim that there can be only one package manager and nothing may escape its gaze.)
zw _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On Wed, 05 Nov 2014 15:17:29 -0500, Libertas libertas@mykolab.com wrote:
My OpenBSD node is currently on 0.2.5.10.
Which is from the -current ports tree. On -stable it's tor-0.2.4.23 but I doubt that you want to run your relay on -current.
If compilation is considered tedious, though, I or someone like me could start more aggressively maintaining the Tor port.
You're wrong because you're mistaking port and package (http://www.openbsd.org/faq/faq15.html).
I was actually considering this recently, although I have no prior experience with port development.
Have you only even tried to mail the maintener?
Quoting another mail from you for the lulz:
On Wed, 05 Nov 2014 14:04:43 -0500, Libertas libertas@mykolab.com wrote:
Another example that I forgot to mention earlier: encrypting swap on Debian or Ubuntu involves apt-getting and such. On OpenBSD, it involves changing two characters of /etc/sysctl.conf.
You mean activating something which is activated by default since 3.8 which has been released in Nov. 2005?
Cheers, Vigdis
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
My OpenBSD node is currently on 0.2.5.10.
Which is from the -current ports tree. On -stable it's tor-0.2.4.23 but I doubt that you want to run your relay on -current.
Huh? I compiled it from non-port source because the port at the time was old. For my first manual compilation, I think I was on 0.2.4.23. I'm now on 0.2.5.10 which, if I'm not mistaken, is the most recent stable Tor release. This was also compiled from downloaded, non-port source.
My base and ports are on -stable, not -current.
You're wrong because you're mistaking port and package (http://www.openbsd.org/faq/faq15.html).
No, I know the difference. However, the ports are compiled into the packages, no? Are the two not maintained together? I was focusing on the port because it's a little "upstream" of the package. Also, the port includes the necessary system changes like adding a daemon user, so aside from the pretty minimal compilation time, it's no different from a package install.
Have you only even tried to mail the maintener?
That was going to come after reading some docs on port development and writing a patch or two. I don't like establishing contact until I have something worthwhile to offer.
You mean activating something which is activated by default since 3.8 which has been released in Nov. 2005?
You got me on this one. The below line in /etc/sysctl.conf confused me:
#vm.swapencrypt.enable=0 # 0=Do not encrypt pages that go to swap
In hindsight, it's pretty obvious.
There's no need to be so aggressive, by the way. I like the generally friendly tone here.
Libertas
On 11/05/2014 04:40 PM, Daniel Jakots wrote:
On Wed, 05 Nov 2014 15:17:29 -0500, Libertas libertas@mykolab.com wrote:
My OpenBSD node is currently on 0.2.5.10.
Which is from the -current ports tree. On -stable it's tor-0.2.4.23 but I doubt that you want to run your relay on -current.
If compilation is considered tedious, though, I or someone like me could start more aggressively maintaining the Tor port.
You're wrong because you're mistaking port and package (http://www.openbsd.org/faq/faq15.html).
I was actually considering this recently, although I have no prior experience with port development.
Have you only even tried to mail the maintener?
Quoting another mail from you for the lulz:
On Wed, 05 Nov 2014 14:04:43 -0500, Libertas libertas@mykolab.com wrote:
Another example that I forgot to mention earlier: encrypting swap on Debian or Ubuntu involves apt-getting and such. On OpenBSD, it involves changing two characters of /etc/sysctl.conf.
You mean activating something which is activated by default since 3.8 which has been released in Nov. 2005?
Cheers, Vigdis _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
It's a little bit of both. The OpenBSD port and package of Tor were out of date last time I checked, so the first difference is that you want to build from source. If you do so, though, you have to make the unprivileged user yourself. This is covered in _Absolute OpenBSD 2nd Ed._ and on the web. It's a technique that OpenBSD popularized, so it's easy to do.
The general sysadmin experience with OpenBSD involves more manual config file editing and man page reading, as I've mentioned previously. Some things (like having tons of open files) have to be enabled by default, as they're a complication and a potential security risk. However, most users find it very convenient and elegant in the long run.
Another example that I forgot to mention earlier: encrypting swap on Debian or Ubuntu involves apt-getting and such. On OpenBSD, it involves changing two characters of /etc/sysctl.conf.
I don't want to beat a dead horse, so I'll leave most of the specifics out. Also, the beginning of _Absolute OpenBSD 2nd Ed._ explains the differences better than I can.
Libertas
On 11/05/2014 11:20 AM, Niklas Kielblock wrote:
Is there much of a difference between setting up Tor on OpenBSD vs. Linux or other Unix(like) systems?
Or is this just about setting up OpenBSD in general, or additional security for relays (disk encryption, memory protection) whose use isn't common on most general servers?
I would love to start a larger conversation about running Tor on OpenBSD. I've been considering making a guide describing the process. However, that violates the OpenBSD philosophy to some extent. They tend to only help those who help themselves - in the long term, only those who want to learn Unix and who RTFMs continue using OpenBSD.[1] Hopefully, though, we can spark enough interest that node operators will take that initiative. I know there's been a lot more interest in OpenBSD on Hacker News et al. since the surveillance revelations.
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
On 11/05/2014 08:20 AM, Niklas Kielblock wrote:
Is there much of a difference between setting up Tor on OpenBSD vs. Linux or other Unix(like) systems?
Not really. Partitioning the disk is a little different, but that's about it. I'll admit to needing to refer to the manpage for installing from ports, but I typically install nothing I don't absolutely need on systems (i.e., only what I need to run Tor).
Or is this just about setting up OpenBSD in general, or additional security for relays (disk encryption, memory protection) whose use isn't common on most general servers?
I get the impression that this is what was being referred to.
- -- The Doctor [412/724/301/703] [ZS] Developer, Project Byzantium: http://project-byzantium.org/
PGP: 0x807B17C1 / 7960 1CDC 85C9 0B63 8D9F DD89 3BD8 FF2B 807B 17C1 WWW: https://drwho.virtadpt.net/
May you be half the person your hamster believes you to be.
On Wed, Nov 5, 2014 at 11:20 AM, Niklas Kielblock niklas@spiderschwe.in wrote:
Is there much of a difference between setting up Tor on OpenBSD vs. Linux or other Unix(like) systems?
Tor itself? ...
https://dist.torproject.org/ tar -xzf torball.tar.gz cd tor ; ./configure ; make ; cd src ; ./tor
Nope, absolutely no difference whatsoever ;)
Pity the fool who endeavours to learn any system dependant packages/ports system before such Unix basics, lest ye cast your lot as a poor and dependant admin forever.
On Wed, 05 Nov 2014 10:35:01 -0500, Libertas libertas@mykolab.com wrote:
Agreed. Thanks for pulling together the statistics, too. However, I'd like to make an argument for OpenBSD specifically.
It isn't very inviting for people that don't know at least intermediate Unix.
You're wrong, OpenBSD's documentation (and other BSDs' too) is awesome. I learn to use Unix systems with OpenBSD.
It's possible that governments like China's are trying to hack Tor relays in an attempt to deanonymize users. It's almost definite that malicious hackers try to break into exit nodes to troll traffic. Even an up-to-date, hardened Linux or FreeBSD system probably can't weather all such attacks. For such a simple, single-use, security-critical application, something as sturdy and impenetrable as OpenBSD is the best option.
You have to find OS vulnerabilities when the sysadmin does the job correctly. You think that all the relays have their (for instance) sshd configured correctly? (like PermitRootLogin set to no, no password and so on). And that's only one daemon.
I would love to start a larger conversation about running Tor on OpenBSD. I've been considering making a guide describing the process. However, that violates the OpenBSD philosophy to some extent.
What? One of the point of OpenBSD is to provide a correct documentation. The only problem is people asking for stuff which is already written down in the FAQ or in the man page.
Just write the guide, I'd be happy to review it. You can even ask for help on the Tor-BSD mailing list[1].
[1]: http://lists.nycbug.org/mailman/listinfo/tor-bsd
Cheers, Vigdis
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
You're wrong, OpenBSD's documentation (and other BSDs' too) is awesome. I learn to use Unix systems with OpenBSD.
I never said the docs are bad - one of my previous emails mentioned how great the man pages are. What I meant was that there are less wizards, tutorials, guides, and autoconfigs - you're responsible for actually editing raw /etc config files and understanding what you're doing.
You have to find OS vulnerabilities when the sysadmin does the job correctly. You think that all the relays have their (for instance) sshd configured correctly? (like PermitRootLogin set to no, no password and so on). And that's only one daemon.
This is indeed a problem. I'm actually working on a website to identify these vulnerabilities, warn operators of them, and show people how to fix them. OpenBSD does come with more sane default options for these kinds of things, though. For example, PermitRootLogin is set to no by default if you add a user during install.
What? One of the point of OpenBSD is to provide a correct documentation. The only problem is people asking for stuff which is already written down in the FAQ or in the man page.
Ad hoc guides aren't documentation, though. Everything is already in the FAQ and man pages. What we're discussing is a more specific and user-friendly guide.
Libertas
On 11/05/2014 12:28 PM, Daniel Jakots wrote:
On Wed, 05 Nov 2014 10:35:01 -0500, Libertas libertas@mykolab.com wrote:
Agreed. Thanks for pulling together the statistics, too. However, I'd like to make an argument for OpenBSD specifically.
It isn't very inviting for people that don't know at least intermediate Unix.
You're wrong, OpenBSD's documentation (and other BSDs' too) is awesome. I learn to use Unix systems with OpenBSD.
It's possible that governments like China's are trying to hack Tor relays in an attempt to deanonymize users. It's almost definite that malicious hackers try to break into exit nodes to troll traffic. Even an up-to-date, hardened Linux or FreeBSD system probably can't weather all such attacks. For such a simple, single-use, security-critical application, something as sturdy and impenetrable as OpenBSD is the best option.
You have to find OS vulnerabilities when the sysadmin does the job correctly. You think that all the relays have their (for instance) sshd configured correctly? (like PermitRootLogin set to no, no password and so on). And that's only one daemon.
I would love to start a larger conversation about running Tor on OpenBSD. I've been considering making a guide describing the process. However, that violates the OpenBSD philosophy to some extent.
What? One of the point of OpenBSD is to provide a correct documentation. The only problem is people asking for stuff which is already written down in the FAQ or in the man page.
Just write the guide, I'd be happy to review it. You can even ask for help on the Tor-BSD mailing list[1].
Cheers, Vigdis _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 5 November 2014 03:04, grarpamp grarpamp@gmail.com wrote:
On Tue, Nov 4, 2014 at 12:25 PM, Libertas libertas@mykolab.com wrote:
I think it would be a good idea to add OpenBSD to doc/TUNING because [...] promoting OpenBSD relays benefits the Tor network's security.
Absolutely. Not just due to OpenBSD's security positioning, but moreso from network diversity. Windows is its own world.
I tried installing OpenBSD once... it was tough, heh.
Coming from a Windows background, I like the idea of running more nodes on (up-to-date, maintained) Windows servers.
I'll also throw out the obvious that if we're talking about diversity for the purposes of security, the network-accessible parts of tor rely on OpenSSL, which would probably be difficult to swap out, but might be worth it as an experiment. Even if it's to LibreSSL. Maybe the zlib library also, but that one's had a lot fewer problems than OpenSSL.
-tom
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
I hope I don't sound too pompous saying this, but I really don't think relays should run on Windows. Windows is the primary target of weaponized and general exploits, and it's less secure than a properly configured Unix distribution. People running nodes, especially exit nodes, have a responsibility to their users, and I just don't think Windows is the best choice in that regard.
This is especially relevant with potential adversaries like the Chinese government, who can buy Windows exploits that can't be prevented by user configuration, and can't be recognized by public auditors because of the closed source code. Market *nix exploits also exist, but (IIRC) they're much rarer and less expensive.
It's possible that I'm wrong, though. Let me know if Windows is more secure than I think.
Libertas
On 11/05/2014 11:15 AM, Tom Ritter wrote:
On 5 November 2014 03:04, grarpamp grarpamp@gmail.com wrote:
On Tue, Nov 4, 2014 at 12:25 PM, Libertas libertas@mykolab.com wrote:
I think it would be a good idea to add OpenBSD to doc/TUNING because [...] promoting OpenBSD relays benefits the Tor network's security.
Absolutely. Not just due to OpenBSD's security positioning, but moreso from network diversity. Windows is its own world.
I tried installing OpenBSD once... it was tough, heh.
Coming from a Windows background, I like the idea of running more nodes on (up-to-date, maintained) Windows servers.
I'll also throw out the obvious that if we're talking about diversity for the purposes of security, the network-accessible parts of tor rely on OpenSSL, which would probably be difficult to swap out, but might be worth it as an experiment. Even if it's to LibreSSL. Maybe the zlib library also, but that one's had a lot fewer problems than OpenSSL.
-tom _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
I'd agree simply because Windows presents a much larger attack surface. The amount of code running on a minimal Unix installation plus Tor is a lot less than a Windows system, especially network facing code.
On 05/11/2014 18:55, Libertas wrote:
I hope I don't sound too pompous saying this, but I really don't think relays should run on Windows. Windows is the primary target of weaponized and general exploits, and it's less secure than a properly configured Unix distribution. People running nodes, especially exit nodes, have a responsibility to their users, and I just don't think Windows is the best choice in that regard.
On 5 November 2014 11:55, Libertas libertas@mykolab.com wrote:
I hope I don't sound too pompous saying this, but I really don't think relays should run on Windows. Windows is the primary target of weaponized and general exploits,
Windows desktops, yes. Where users are browsing websites on IE, with plugins and Flash Player and old versions of Adobe Reader and Java. Windows Servers have none of those things, most importantly users fiddling around on them regularly.
and it's less secure than a properly configured Unix distribution.
Are you comparing a Linux Server to a Windows Desktop? Or a Linux Server to a Windows Server? If it's the latter - I'm going to disagree and try and provide supporting evidence...
This is especially relevant with potential adversaries like the Chinese government, who can buy Windows exploits that can't be prevented by user configuration,
I'm confused by this - what bugs are you talking about? The only bugs that 'can't be prevented by user configuration' would be in the networking stack. And that applies just as much to Linux as it does to Windows.
Now yes, you can patch your kernel yourself on Linux, which you can't on Windows - but when Shellshock came out, were you going into Bash to patch it yourself? Or were you waiting for bash itself to provide patches?
Also, they can't buy Linux exploits?
and can't be recognized by public auditors because of the closed source code.
That's true, it's definitely easier to audit open source than Windows. But from a "is this bug serious" point of view - MSFT gives pretty good insight into what they're patching and the impact of it. "Public Auditors" (like myself) have a good deal of confidence in understanding risk based on this information. For example [0] [1] last month, You've got: 1 RCE in IE 1 RCE in .Net WebApps with understanding about how to determine if you're vulnerable 3 CE if you phish a user into opening a document or browsing a website (two of them in office, not windows) 1 UXSS if you phish someone 1 Local EOP in default config 1 Local EOP if it's not a default configuration
None of these are realistically exploitable on a Windows Server.
On a tor relay on a Windows Server you've got (maybe) IIS running, the Windows networking stack, and maybe but usually not RDP open to the world.
I can only think of two or three bugs in the last 3 years that _could_ have been exploitable in that configuration. The weak point is (as usual) whatever random web application the user has running on the relay. (Ideally, none. But I expect most relays that run on servers pull double duty.)
Market *nix exploits also exist, but (IIRC) they're much rarer and less expensive.
I'm not sure 'rarer' and 'less expensive' go together, did you mean more expensive? (I'll assume yes.) I don't like arguing economics because I don't think either of us buys or sells exploits, so everything is just hearsay. But it's definitely easier to write exploits for open source code than it is closed source. That would push the price down.
They're also more common. I can point to several remotely exploitable bugs in Linux-land. I have a hard time pointing to equivalent bugs in the equivalent Windows subsystem. Big bugs are remotely exploitable, and they get remotely exploited, and have easy-to-use attack tools - regardless of platform. So going by that yardstick:
nginx RCE (2013) vs IIS RCE (any?)
several rails RCEs vs .Net Framework RCE (can't think of any, but maybe one or two somewhere)
OpenSSL, which runs on Windows in Tor also, but I'm going to count as 'Linux' because Windows has its own SSL stack: SRTP DoS last month, Heartbleed, EarlyCCS vs MSFT SSL stack bugs (can't think of any)
Linux networking stack (can't think of any) vs Windows (there was that one bug a couple years ago, can't recall all the details, but iirc no one managed to make an exploit out of it)
OpenSSH (none) vs RDP (again, one a couple years ago, but it required open RDP, without Client Certificates, and while I think someone may have pulled off an exploit, I don't think it was public.)
It's possible that I'm wrong, though. Let me know if Windows is more secure than I think.
I think it is more secure than you think.
On 5 November 2014 12:20, Niklas Kielblock niklas@spiderschwe.in wrote:
I'd agree simply because Windows presents a much larger attack surface. The amount of code running on a minimal Unix installation plus Tor is a lot less than a Windows system, especially network facing code.
Running code, or network accessible code? Either way I don't see how you came to that calculation. 'Minimal' Unix + Tor + SSH restricted by SSH Key vs 'Minimal' Windows + Tor + RDP restricted by Client Certificate. I also don't know what you mean by 'minimal' as very few people actually configure their kernels themselves - most use debian/ubuntu. On the face, I'm not thinking Ubuntu is any more 'minimal' than Windows.
I'm going off of my experience, which comes across in the form of some data here, some data there. It's not comprehensive, by any means. It's a subjective argument at its core, but I'm trying to bring out why I think the way I do.
I think a Windows Server, properly configured, is roughly as secure as a properly configured Linux Server. (I think OpenBSD probably beats both of them, but I have very little experience with OpenBSD.) I think there have been more bugs that result in RCE on production Linux servers running SSH and a webserver in the past 4 years than there have been in production Windows servers running RDP and IIS. I think if you're pointing fingers at China and the NSA, you should assume they have RCE in both Windows and Linux. I think running relays on Windows Servers is no worse than running relays on Linux Servers, and therefore it is good to do, because it adds diversity to the network. I think diversity is a good thing, because it reduces the likelihood that a bug (security or not) will result in a disruption for a majority of the network at once.
-tom
[0] http://blogs.technet.com/b/msrc/ [1] https://technet.microsoft.com/library/security/ms14-oct
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Also, they can't buy Linux exploits?
Of course. You clipped out the part where I acknowledged this.
I'm not sure 'rarer' and 'less expensive' go together, did you mean more expensive? (I'll assume yes.)
I did in fact mean "more expensive".
I'm confused by this - what bugs are you talking about? The only bugs that 'can't be prevented by user configuration' would be in the networking stack. And that applies just as much to Linux as it does to Windows.
As you guessed, I was referring mostly to networking stack bugs. It does apply to all networked OSs, but I've generally been told that Microsoft has had a worse history of this, and that it's made even worse by the lack of open source code. As you pointed out, maybe that was mistaken.
I think that a lot of the publicity of Linux bugs is because of how much of the Internet runs on it. Based on the government surveillance documents I've read, it seems that they have a very easy time getting access to Windows servers, and that they have a big corporate pipeline for Windows exploits. I've also read that Windows is the primary target of exploit markets, whereas Linux exploits tend to be much more publicly documented or high-profile (NSA trade secrets, etc.). The second tier of hackers (non-Five Eyes governments and big commercial blackhats) are probably the biggest threat to Tor relays, and they seem to have more access to Windows exploits than Linux exploits.
I don't have enough knowledge or experience to comment on this much more. I will point out, however, that I'm promoting OpenBSD rather than Linux as an alternative. I think almost no one would argue that Windows is more secure than OpenBSD for this sort of application. I suspect most would side with Linux over Windows as well.
I think it is more secure than you think.
Fair point, I think you're right.
Libertas
On 11/05/2014 01:53 PM, Tom Ritter wrote:
On 5 November 2014 11:55, Libertas libertas@mykolab.com wrote:
I hope I don't sound too pompous saying this, but I really don't think relays should run on Windows. Windows is the primary target of weaponized and general exploits,
Windows desktops, yes. Where users are browsing websites on IE, with plugins and Flash Player and old versions of Adobe Reader and Java. Windows Servers have none of those things, most importantly users fiddling around on them regularly.
and it's less secure than a properly configured Unix distribution.
Are you comparing a Linux Server to a Windows Desktop? Or a Linux Server to a Windows Server? If it's the latter - I'm going to disagree and try and provide supporting evidence...
This is especially relevant with potential adversaries like the Chinese government, who can buy Windows exploits that can't be prevented by user configuration,
I'm confused by this - what bugs are you talking about? The only bugs that 'can't be prevented by user configuration' would be in the networking stack. And that applies just as much to Linux as it does to Windows.
Now yes, you can patch your kernel yourself on Linux, which you can't on Windows - but when Shellshock came out, were you going into Bash to patch it yourself? Or were you waiting for bash itself to provide patches?
Also, they can't buy Linux exploits?
and can't be recognized by public auditors because of the closed source code.
That's true, it's definitely easier to audit open source than Windows. But from a "is this bug serious" point of view - MSFT gives pretty good insight into what they're patching and the impact of it. "Public Auditors" (like myself) have a good deal of confidence in understanding risk based on this information. For example [0] [1] last month, You've got: 1 RCE in IE 1 RCE in .Net WebApps with understanding about how to determine if you're vulnerable 3 CE if you phish a user into opening a document or browsing a website (two of them in office, not windows) 1 UXSS if you phish someone 1 Local EOP in default config 1 Local EOP if it's not a default configuration
None of these are realistically exploitable on a Windows Server.
On a tor relay on a Windows Server you've got (maybe) IIS running, the Windows networking stack, and maybe but usually not RDP open to the world.
I can only think of two or three bugs in the last 3 years that _could_ have been exploitable in that configuration. The weak point is (as usual) whatever random web application the user has running on the relay. (Ideally, none. But I expect most relays that run on servers pull double duty.)
Market *nix exploits also exist, but (IIRC) they're much rarer and less expensive.
I'm not sure 'rarer' and 'less expensive' go together, did you mean more expensive? (I'll assume yes.) I don't like arguing economics because I don't think either of us buys or sells exploits, so everything is just hearsay. But it's definitely easier to write exploits for open source code than it is closed source. That would push the price down.
They're also more common. I can point to several remotely exploitable bugs in Linux-land. I have a hard time pointing to equivalent bugs in the equivalent Windows subsystem. Big bugs are remotely exploitable, and they get remotely exploited, and have easy-to-use attack tools - regardless of platform. So going by that yardstick:
nginx RCE (2013) vs IIS RCE (any?)
several rails RCEs vs .Net Framework RCE (can't think of any, but maybe one or two somewhere)
OpenSSL, which runs on Windows in Tor also, but I'm going to count as 'Linux' because Windows has its own SSL stack: SRTP DoS last month, Heartbleed, EarlyCCS vs MSFT SSL stack bugs (can't think of any)
Linux networking stack (can't think of any) vs Windows (there was that one bug a couple years ago, can't recall all the details, but iirc no one managed to make an exploit out of it)
OpenSSH (none) vs RDP (again, one a couple years ago, but it required open RDP, without Client Certificates, and while I think someone may have pulled off an exploit, I don't think it was public.)
It's possible that I'm wrong, though. Let me know if Windows is more secure than I think.
I think it is more secure than you think.
On 5 November 2014 12:20, Niklas Kielblock niklas@spiderschwe.in wrote:
I'd agree simply because Windows presents a much larger attack surface. The amount of code running on a minimal Unix installation plus Tor is a lot less than a Windows system, especially network facing code.
Running code, or network accessible code? Either way I don't see how you came to that calculation. 'Minimal' Unix + Tor + SSH restricted by SSH Key vs 'Minimal' Windows + Tor + RDP restricted by Client Certificate. I also don't know what you mean by 'minimal' as very few people actually configure their kernels themselves - most use debian/ubuntu. On the face, I'm not thinking Ubuntu is any more 'minimal' than Windows.
I'm going off of my experience, which comes across in the form of some data here, some data there. It's not comprehensive, by any means. It's a subjective argument at its core, but I'm trying to bring out why I think the way I do.
I think a Windows Server, properly configured, is roughly as secure as a properly configured Linux Server. (I think OpenBSD probably beats both of them, but I have very little experience with OpenBSD.) I think there have been more bugs that result in RCE on production Linux servers running SSH and a webserver in the past 4 years than there have been in production Windows servers running RDP and IIS. I think if you're pointing fingers at China and the NSA, you should assume they have RCE in both Windows and Linux. I think running relays on Windows Servers is no worse than running relays on Linux Servers, and therefore it is good to do, because it adds diversity to the network. I think diversity is a good thing, because it reduces the likelihood that a bug (security or not) will result in a disruption for a majority of the network at once.
-tom
[0] http://blogs.technet.com/b/msrc/ [1] https://technet.microsoft.com/library/security/ms14-oct _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
I'd agree simply because Windows presents a much larger attack surface. The amount of code running on a minimal Unix installation plus Tor is a lot less than a Windows system, especially network facing code.
... Running code, or network accessible code? Either way I don't see how you came to that calculation. 'Minimal' Unix + Tor + SSH restricted by SSH Key vs 'Minimal' Windows + Tor + RDP restricted by Client Certificate. I also don't know what you mean by 'minimal' as very few ... I think a Windows Server, properly configured, is roughly as secure as a properly configured Linux Server. ... I think there have been more bugs that result in RCE on production Linux servers running SSH and a webserver in the past 4 years than there have been in production Windows servers running RDP and IIS. ... I think if you're pointing fingers at China and the NSA, you should assume they have RCE in both Windows and Linux. ... I think running relays on Windows Servers is no worse than running relays on Linux Servers, and therefore it is good to do, because it adds diversity to the network.
Attack surface on a well adminned relay comes down to three things: - Network stack itself (kernel) - Daemon software itself (tor + remote admin) - Their respective use of other kernel/library/shell provided resources.
I might suggest the current proportion of Windows to Linux is roughly ideal. This is primarily because, all other things set equal at 'minimal' (= tor + remote admin), good adminning, and good control of corporate secrets (or moles)... Windows still has one huge strategic weakness at that point... the magic packet. It's the whole binary vs. opensource argument. So essentially, the correct ratio of the two might be the odds you place that a binary OS has a magic packet. Today's node count shows 73% to opensource platforms. I'd suspect 73% is about where a lot of analysts might bet on Windows being magical, whether by/for the NSA, or any other reason or source.
(Remember this... https://en.wikipedia.org/wiki/NSAKEY That was just from running 'strings'. Good luck trolling all of Windows with a disassembler... a nice fat payoff if you do. And the number of disassembling vs. opensource auditors is probably even more heavily skewed. And Windows is 'trusted' by buyers, nor can you replicate their binaries from any 'source code sharing agreements'. Then it's Patch Tuesday again... so it could be no one has or ever will disassemble audit it. So odds end up being pitched instead. And for many applications, that's good enough.)
The real problem below is the 96% allocation of opensource to Linux and 4% to Other opensource. That's something that should be fixed. For these purposes, it doesn't matter which BSD/Other you pick... once you get the security odds there back towards say 50:50 Linux:Other, then you can debate userland and relative security amongst them all you want.
Here's some links to get you started, including two other main branches of the Unix Kernel family tree at the bottom...
5939 Linux 1591 Windows 173 FreeBSD http://www.freebsd.org/ 56 Darwin 44 OpenBSD http://www.openbsd.org/ 7 NetBSD http://netbsd.org/ 6 SunOS 4 Bitrig https://www.bitrig.org/ 2 GNU/kFreeBSD https://www.debian.org/ports/kfreebsd-gnu/ 2 DragonFly http://www.dragonflybsd.org/ 0 Illumos (OpenSolaris) http://wiki.illumos.org/display/illumos/Distributions 0 Minix http://www.minix3.org/
Official metrics... https://metrics.torproject.org/network.html
Someone should really do an analysis of platform vs. exit bandwidth as well. Anyone?
Also, isn't there some project out there that is counting the historical number of kernel bugs+severity per OS over time?
[To cpunks to cover all the other volunteer node based networks out there that could benefit from tuning their platform ratios.]
On 2014-11-05 23:58:43 (-0500), grarpamp wrote:
The real problem below is the 96% allocation of opensource to Linux and 4% to Other opensource.
Someone should really do an analysis of platform vs. exit bandwidth as well. Anyone?
Here ya go. Observed bandwidth per OS in relays having the exit flag:
93.62% 4459816582 Linux 4.51% 214639363 FreeBSD 1.25% 59672066 Windows 0.25% 11754598 Darwin 0.17% 7896687 Bitrig 0.15% 6964863 OpenBSD 0.06% 3091495 SunOS
On Thu, Nov 6, 2014 at 2:43 AM, David Serrano tor@dserrano5.es wrote:
On 2014-11-05 23:58:43 (-0500), grarpamp wrote:
The real problem below is the 96% allocation of opensource to Linux and 4% to Other opensource.
Someone should really do an analysis of platform vs. exit bandwidth as well. Anyone?
Here ya go. Observed bandwidth per OS in relays having the exit flag:
93.62% 4459816582 Linux 4.51% 214639363 FreeBSD 1.25% 59672066 Windows 0.25% 11754598 Darwin 0.17% 7896687 Bitrig 0.15% 6964863 OpenBSD 0.06% 3091495 SunOS
This excessive Linux dominance in both node count and bandwidth really should be balanced out, like why not? I'd expect if some of the big relays switch to any other OS that would flatten out the bandwidth part pretty easily. You'd have to check say the top 10, 25, 50 or so relays to see to what extent they are part of this mess, I'm sure it's similar.
On 2014-11-07 02:46:40 (-0500), grarpamp wrote:
You'd have to check say the top 10, 25, 50 or so relays to see to what extent they are part of this mess, I'm sure it's similar.
Top 200: 94.90% 2850128913 Linux 4.86% 146110227 FreeBSD 0.24% 7156736 Darwin
Top 50: 93.08% 1317726797 Linux 6.92% 97980759 FreeBSD
Top 20: 91.93% 678898278 Linux 8.07% 59598665 FreeBSD
Top 10: 100.00% 444716242 Linux
The first non-Linux is a FreeBSD on 13th place, then the next new one is Darwin on 185th, and the next is Windows on 404th. Same data as yesterday, grabbed from onionoo.tpo/details.
grarpamp schreef op 07/11/14 08:46:
On Thu, Nov 6, 2014 at 2:43 AM, David Serrano tor@dserrano5.es wrote:
On 2014-11-05 23:58:43 (-0500), grarpamp wrote:
The real problem below is the 96% allocation of opensource to Linux and 4% to Other opensource.
Someone should really do an analysis of platform vs. exit bandwidth as well. Anyone?
Here ya go. Observed bandwidth per OS in relays having the exit flag:
93.62% 4459816582 Linux 4.51% 214639363 FreeBSD 1.25% 59672066 Windows 0.25% 11754598 Darwin 0.17% 7896687 Bitrig 0.15% 6964863 OpenBSD 0.06% 3091495 SunOS
This excessive Linux dominance in both node count and bandwidth really should be balanced out, like why not? I'd expect if some of the big relays switch to any other OS that would flatten out the bandwidth part pretty easily. You'd have to check say the top 10, 25, 50 or so relays to see to what extent they are part of this mess, I'm sure it's similar.
Hi,
I run a bunch of top50 relays (about 5.5% of global exit traffic), I'll have a look at converting my setup to OpenBSD - preferably without too much downtime.
Tom
On 11/5/14, grarpamp grarpamp@gmail.com wrote:
... 1 DragonFly
kudos, whoever you are!
(i love this flavor more than most :)
best regards,
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 11/05/2014 04:04 AM, grarpamp wrote:
On Tue, Nov 4, 2014 at 12:25 PM, Libertas libertas@mykolab.com wrote:
I think it would be a good idea to add OpenBSD to doc/TUNING because [...] promoting OpenBSD relays benefits the Tor network's security.
Absolutely. Not just due to OpenBSD's security positioning, but moreso from network diversity.
This would be a lot more work than just adding varieties of Unix to the pool, but if someone wants to do something _major_ for diversity and platform hardening at the same time, they might like to think about writing a relay daemon for MirageOS: see http://www.openmirage.org/ I say "writing", not "porting", because the whole point of Mirage is that everything is done in OCaml.
On Wed, Nov 05, 2014 at 04:04:41AM -0500, grarpamp wrote:
173 FreeBSD
FreeBSD still seems to use globally incrementing IP IDs by default. That's an issue as it leaks fine-grained information about how many packets a relay's networking stack processes. (However, nobody investigated the exact impact on Tor relays so far, which makes this a FUD-heavy topic.) It looks like approximately 50 out of the 131 FreeBSD relays I tested (38%) use global IP IDs.
There's a sysctl variable called "net.inet.ip.random_id" which makes a FreeBSD's IP ID behaviour random. FreeBSD relay operators should set this to "1".
Note that this issue was already discussed earlier this year in a thread called "Lots of tor relays send out sequential IP IDs; please fix that!".
Cheers, Philipp
On Thu, Nov 6, 2014 at 8:52 AM, Philipp Winter phw@nymity.ch wrote:
On Wed, Nov 05, 2014 at 04:04:41AM -0500, grarpamp wrote:
173 FreeBSD
FreeBSD still seems to use globally incrementing IP IDs by default. That's an issue as it leaks fine-grained information about how many packets a relay's networking stack processes. (However, nobody investigated the exact impact on Tor relays so far, which makes this a FUD-heavy topic.) It looks like approximately 50 out of the 131 FreeBSD relays I tested (38%) use global IP IDs.
There's a sysctl variable called "net.inet.ip.random_id" which makes a FreeBSD's IP ID behaviour random. FreeBSD relay operators should set this to "1".
Note that this issue was already discussed earlier this year in a thread called "Lots of tor relays send out sequential IP IDs; please fix that!".
It's been default off since before it was a sysctl over a decade ago. Anyone know what the deal is with that? Some objection, or forgotten flag day, or oversight that really should be set to 1? https://svnweb.freebsd.org/base?view=revision&revision=133720
tor-relays@lists.torproject.org
-
coderman -
Daniel Jakots -
David Serrano -
grarpamp -
Libertas -
Niklas Kielblock -
Philipp Winter -
Richard Johnson -
The Doctor -
Tom Ritter -
Tom van der Woerdt -
Zack Weinberg