Hi everyone,
I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project.
The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596.
#2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it.
#2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint.
There are a couple of different ways to solve this problem; Karsten suggested that we could modify OpenWrt to stop creating a /tmp partition. This probably means that we will have to ship our own OpenWrt image. I'm thinking that another option would be to modify the Tor-OpenWrt-package to use / as the data directory instead of /tmp. However, I wonder if 64M (no matter how it's partitioned) of space on the Buffalo router is insufficient as long as #2334 remains open.
#2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt.
#2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions?
The Torouter project also has some open questions (some are mentioned on https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter as well):
1. How can we make sure that the version of Tor in OpenWrt is always up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable?
2. We want to collect statistics, which means that we need to ship a GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts?
3. Do we want one Tor process for bridging and one for the transparent wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too.
Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know.
Thanks,
On Tue, 19 Apr 2011 11:17:17 +0100 "Runa A. Sandvik" runa.sandvik@gmail.com wrote:
I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project.
My general thoughts are that we figure out the web ui, a config that is a bridge by default, and start sending some out to users to get some real feedback.
#2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it.
Shipping a beta test router with a 1 gb disk stuck in it isn't so bad. Fixing tor to handle such a constrained environment is better, but clearly longer term.
#2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint.
I don't see this as a real problem, actually.
#2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt.
A tor-ui for openwrt sounds good.
- How can we make sure that the version of Tor in OpenWrt is always
up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable?
A tor openwrt package sounds good.
- We want to collect statistics, which means that we need to ship a
GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts?
A tor-geoipdb package sounds good.
- Do we want one Tor process for bridging and one for the transparent
wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too.
I think starting with a bridge config is good. Starting with transparent tor wifi may be too much to start. The original goal of the router was to provide automatically configured bridges and relays for people who want to help.
On 04/19/2011 03:17 AM, Runa A. Sandvik wrote:
Hi everyone,
I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project.
The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596.
#2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it.
This is only a problem on smaller hardware - so if we really want the Buffalo router, we'll need to fix this or create a work around that isn't a total PITA. For now, I think we can just add a USB disk and deal with it later.
If we find that the router can't handle being a bridge, I'm not really concerned with this bug.
#2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint.
I commented on the bug - it's easy enough to change this by submitting a patch to openwrt-devel@lists.openwrt.org
There are a couple of different ways to solve this problem; Karsten suggested that we could modify OpenWrt to stop creating a /tmp partition. This probably means that we will have to ship our own OpenWrt image. I'm thinking that another option would be to modify the Tor-OpenWrt-package to use / as the data directory instead of /tmp. However, I wonder if 64M (no matter how it's partitioned) of space on the Buffalo router is insufficient as long as #2334 remains open.
If we're shipping our own image, we can do a bunch of stuff. Is that what we want to do?
#2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt.
The web interface should go into version control, it should be added to the tor-alpha package in the OpenWRT svn (that's why I created it), and it should be used by some people.
#2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions?
I think torouter is a perfectly fine name until we actually have a shipping prototype.
The Torouter project also has some open questions (some are mentioned on https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter as well):
- How can we make sure that the version of Tor in OpenWrt is always
up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable?
I think we should contribute back to OpenWRT (as I've been doing) and for our router, we should consider adding a feed for the specific hardware we intend to support. Still, we'll need to ensure the versions of _everything_ are up to date and not just Tor.
So what's our general update strategy for any platform we ship?
- We want to collect statistics, which means that we need to ship a
GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts?
There's a tor-geoip package created by the tor package in the OpenWRT svn repo (from looking at the Makefile).
- Do we want one Tor process for bridging and one for the transparent
wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too.
I think we should run both in a single Tor for now but I think we should only enable the bridge by default. The web UI can enable the transparent stuff if we want it.
Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know.
We need to make packages for the libnatpmp and miniupnpc libraries because we need tor-fw-helper support in tor-alpha.
All the best, Jake
On Thu, Apr 21, 2011 at 9:26 AM, Jacob Appelbaum jacob@appelbaum.net wrote:
On 04/19/2011 03:17 AM, Runa A. Sandvik wrote:
Hi everyone,
I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project.
The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596.
#2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it.
This is only a problem on smaller hardware - so if we really want the Buffalo router, we'll need to fix this or create a work around that isn't a total PITA. For now, I think we can just add a USB disk and deal with it later.
If we find that the router can't handle being a bridge, I'm not really concerned with this bug.
As far as I know, the router can handle being a bridge as long as it has enough disk space.
#2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint.
I commented on the bug - it's easy enough to change this by submitting a patch to openwrt-devel@lists.openwrt.org
Great.
There are a couple of different ways to solve this problem; Karsten suggested that we could modify OpenWrt to stop creating a /tmp partition. This probably means that we will have to ship our own OpenWrt image. I'm thinking that another option would be to modify the Tor-OpenWrt-package to use / as the data directory instead of /tmp. However, I wonder if 64M (no matter how it's partitioned) of space on the Buffalo router is insufficient as long as #2334 remains open.
If we're shipping our own image, we can do a bunch of stuff. Is that what we want to do?
I don't think we should ship our own image, for the simple reason that we already have enough stuff to maintain.
#2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt.
The web interface should go into version control, it should be added to the tor-alpha package in the OpenWRT svn (that's why I created it), and it should be used by some people.
Ok, do you want to take care of this? That is, putting the web interface into version control and packaging it for OpenWrt.
#2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions?
I think torouter is a perfectly fine name until we actually have a shipping prototype.
Sure, that works too.
The Torouter project also has some open questions (some are mentioned on https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter as well):
- How can we make sure that the version of Tor in OpenWrt is always
up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable?
I think we should contribute back to OpenWRT (as I've been doing) and for our router, we should consider adding a feed for the specific hardware we intend to support. Still, we'll need to ensure the versions of _everything_ are up to date and not just Tor.
So what's our general update strategy for any platform we ship?
- We want to collect statistics, which means that we need to ship a
GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts?
There's a tor-geoip package created by the tor package in the OpenWRT svn repo (from looking at the Makefile).
So, it turns out that packages in OpenWrt are not updated after a release. That explains why I got 0.2.1.26 when 0.2.1.30 is available in the OpenWrt SVN.
The best way to ensure that users can easily upgrade Tor and related packages is to set up a torproject.org repository for OpenWrt packages. This repository would then have to be added to /etc/opkg.conf.
- Do we want one Tor process for bridging and one for the transparent
wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too.
I think we should run both in a single Tor for now but I think we should only enable the bridge by default. The web UI can enable the transparent stuff if we want it.
Sounds good.
Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know.
We need to make packages for the libnatpmp and miniupnpc libraries because we need tor-fw-helper support in tor-alpha.
Is this something you can / want to take care of?
On 04/21/2011 07:24 AM, Runa A. Sandvik wrote:
On Thu, Apr 21, 2011 at 9:26 AM, Jacob Appelbaum jacob@appelbaum.net wrote:
On 04/19/2011 03:17 AM, Runa A. Sandvik wrote:
Hi everyone,
I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project.
The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596.
#2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it.
This is only a problem on smaller hardware - so if we really want the Buffalo router, we'll need to fix this or create a work around that isn't a total PITA. For now, I think we can just add a USB disk and deal with it later.
If we find that the router can't handle being a bridge, I'm not really concerned with this bug.
As far as I know, the router can handle being a bridge as long as it has enough disk space.
What tests indicate this? How many clients can it handle and at what rate?
#2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint.
I commented on the bug - it's easy enough to change this by submitting a patch to openwrt-devel@lists.openwrt.org
Great.
There are a couple of different ways to solve this problem; Karsten suggested that we could modify OpenWrt to stop creating a /tmp partition. This probably means that we will have to ship our own OpenWrt image. I'm thinking that another option would be to modify the Tor-OpenWrt-package to use / as the data directory instead of /tmp. However, I wonder if 64M (no matter how it's partitioned) of space on the Buffalo router is insufficient as long as #2334 remains open.
If we're shipping our own image, we can do a bunch of stuff. Is that what we want to do?
I don't think we should ship our own image, for the simple reason that we already have enough stuff to maintain.
I generally agree but I think we're going to have to deal with this issue one way or another.
If we're shipping people a router, I'd like to harden it. For example - it does not appear that all of the gcc hardening features are enabled by default.
We can commit to a simple, fixed release cycle for the OS and a constant stream of updates for Tor.
#2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt.
The web interface should go into version control, it should be added to the tor-alpha package in the OpenWRT svn (that's why I created it), and it should be used by some people.
Ok, do you want to take care of this? That is, putting the web interface into version control and packaging it for OpenWrt.
Not really? :-)
In an ideal world, I think we should make these changes in the tor-alpha package. Currently, I'm not familiar with the webui at all.
I'd prefer that someone involved in developing the webui actually built a stand alone package.
#2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions?
I think torouter is a perfectly fine name until we actually have a shipping prototype.
Sure, that works too.
The Torouter project also has some open questions (some are mentioned on https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter as well):
- How can we make sure that the version of Tor in OpenWrt is always
up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable?
I think we should contribute back to OpenWRT (as I've been doing) and for our router, we should consider adding a feed for the specific hardware we intend to support. Still, we'll need to ensure the versions of _everything_ are up to date and not just Tor.
So what's our general update strategy for any platform we ship?
- We want to collect statistics, which means that we need to ship a
GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts?
There's a tor-geoip package created by the tor package in the OpenWRT svn repo (from looking at the Makefile).
So, it turns out that packages in OpenWrt are not updated after a release. That explains why I got 0.2.1.26 when 0.2.1.30 is available in the OpenWrt SVN.
Makes sense.
The best way to ensure that users can easily upgrade Tor and related packages is to set up a torproject.org repository for OpenWrt packages. This repository would then have to be added to /etc/opkg.conf.
That's similar to what I said on IRC; it seems reasonable to keep the Tor package updated in OpenWRT. We need to decide if we want to build regular packages for installation and also if we want to host them. I think this is mostly an Erinn question - helix?
- Do we want one Tor process for bridging and one for the transparent
wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too.
I think we should run both in a single Tor for now but I think we should only enable the bridge by default. The web UI can enable the transparent stuff if we want it.
Sounds good.
What's the default config that we want to ship?
Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know.
We need to make packages for the libnatpmp and miniupnpc libraries because we need tor-fw-helper support in tor-alpha.
Is this something you can / want to take care of?
I'm a little more interested in making these packages. Do you want to open a set of bugs and we can continue deeper discussion in bugs?
All the best, Jake
* Jacob Appelbaum jacob@appelbaum.net [2011:04:21 09:17 -0700]:
The best way to ensure that users can easily upgrade Tor and related packages is to set up a torproject.org repository for OpenWrt packages. This repository would then have to be added to /etc/opkg.conf.
That's similar to what I said on IRC; it seems reasonable to keep the Tor package updated in OpenWRT. We need to decide if we want to build regular packages for installation and also if we want to host them. I think this is mostly an Erinn question
I'm not sure if this is actually a question for me, since I'm not involved in the Tor router stuff and don't currently have any plans to maintain a new package. Give me more information about the format of the package and repository?
On 04/21/2011 10:57 AM, Erinn Clark wrote:
- Jacob Appelbaum jacob@appelbaum.net [2011:04:21 09:17 -0700]:
The best way to ensure that users can easily upgrade Tor and related packages is to set up a torproject.org repository for OpenWrt packages. This repository would then have to be added to /etc/opkg.conf.
That's similar to what I said on IRC; it seems reasonable to keep the Tor package updated in OpenWRT. We need to decide if we want to build regular packages for installation and also if we want to host them. I think this is mostly an Erinn question
I'm not sure if this is actually a question for me, since I'm not involved in the Tor router stuff and don't currently have any plans to maintain a new package. Give me more information about the format of the package and repository?
It's a question for what we as a project can handle supporting - when a new Tor is released, we'll need to build it unless we rely on upstream builds. Runa and I suggest that we (Tor) may want our own OpenWRT repository - that by default seems to fall directly on our main build person, I think.
All the best, Jake
* Jacob Appelbaum jacob@appelbaum.net [2011:04:21 11:54 -0700]:
It's a question for what we as a project can handle supporting - when a new Tor is released, we'll need to build it unless we rely on upstream builds. Runa and I suggest that we (Tor) may want our own OpenWRT repository - that by default seems to fall directly on our main build person, I think.
Jake and I discussed this on IRC and the basic summary is that for now we'll wait and see -- probably longer term we can support maintaining a repository, if that turns out to be the right route, but my role is going to be mainly infrastructure related so I can help make sure people are able to do what they need without blocking on me.
On 04/23/2011 04:32 PM, Erinn Clark wrote:
- Jacob Appelbaum jacob@appelbaum.net [2011:04:21 11:54 -0700]:
It's a question for what we as a project can handle supporting - when a new Tor is released, we'll need to build it unless we rely on upstream builds. Runa and I suggest that we (Tor) may want our own OpenWRT repository - that by default seems to fall directly on our main build person, I think.
Jake and I discussed this on IRC and the basic summary is that for now we'll wait and see -- probably longer term we can support maintaining a repository, if that turns out to be the right route, but my role is going to be mainly infrastructure related so I can help make sure people are able to do what they need without blocking on me.
One other important point made in that discussion is that no one seems to have time for supporting an entirely new platform for every Tor release. So while The Tor Project may support it - we have no one willing to bell the cat today.
What this means practically is that as we've seen with Android, we're going to seriously lag releases as it won't be the responsibility of any single person or group of people. This won't work if we ship our own OS (such as a custom OpenWRT image) and it will simply be difficult if we're just shipping Tor (with or without supporting libraries).
All the best, Jake
On 04/23/2011 10:55 PM, Jacob Appelbaum wrote:
What this means practically is that as we've seen with Android, we're going to seriously lag releases as it won't be the responsibility of any single person or group of people.
As a note on this, I think I and/or whoever wants to be responsible for Android builds, needs to just understand how to get into the workflow for updating releases with the latest Tor binaries. In addition, it would be good to understand when a release is optional, vs recommended vs. absolutely necessary. I don't quite have my head around that yet.
At some point, we had a good flow going b/c Erinn had Android as part of her critical path of updates to do, and I was regularly pushing new ready to ship builds, instead of getting sidetracked on unshippable code to handle esoteric Android issues.
Lastly, I think what we did with 1.0.4 made sense, where we just updated the Tor binary and not the Android code, smoked test that, then shipped, made a lot of sense.
+n
On Sun, Apr 24, 2011 at 3:55 AM, Jacob Appelbaum jacob@appelbaum.net wrote:
On 04/23/2011 04:32 PM, Erinn Clark wrote:
- Jacob Appelbaum jacob@appelbaum.net [2011:04:21 11:54 -0700]:
It's a question for what we as a project can handle supporting - when a new Tor is released, we'll need to build it unless we rely on upstream builds. Runa and I suggest that we (Tor) may want our own OpenWRT repository - that by default seems to fall directly on our main build person, I think.
Jake and I discussed this on IRC and the basic summary is that for now we'll wait and see -- probably longer term we can support maintaining a repository, if that turns out to be the right route, but my role is going to be mainly infrastructure related so I can help make sure people are able to do what they need without blocking on me.
One other important point made in that discussion is that no one seems to have time for supporting an entirely new platform for every Tor release. So while The Tor Project may support it - we have no one willing to bell the cat today.
What this means practically is that as we've seen with Android, we're going to seriously lag releases as it won't be the responsibility of any single person or group of people. This won't work if we ship our own OS (such as a custom OpenWRT image) and it will simply be difficult if we're just shipping Tor (with or without supporting libraries).
We already know that we can't rely on upstream builds. If we want to our users to have the latest version of Tor, we need to set up an okpg repository ourselves.
Jake; it was my impression that you wanted to do this. Is that not the case anymore?
On 04/24/2011 01:14 AM, Runa A. Sandvik wrote:
On Sun, Apr 24, 2011 at 3:55 AM, Jacob Appelbaum jacob@appelbaum.net wrote:
On 04/23/2011 04:32 PM, Erinn Clark wrote:
- Jacob Appelbaum jacob@appelbaum.net [2011:04:21 11:54 -0700]:
It's a question for what we as a project can handle supporting - when a new Tor is released, we'll need to build it unless we rely on upstream builds. Runa and I suggest that we (Tor) may want our own OpenWRT repository - that by default seems to fall directly on our main build person, I think.
Jake and I discussed this on IRC and the basic summary is that for now we'll wait and see -- probably longer term we can support maintaining a repository, if that turns out to be the right route, but my role is going to be mainly infrastructure related so I can help make sure people are able to do what they need without blocking on me.
One other important point made in that discussion is that no one seems to have time for supporting an entirely new platform for every Tor release. So while The Tor Project may support it - we have no one willing to bell the cat today.
What this means practically is that as we've seen with Android, we're going to seriously lag releases as it won't be the responsibility of any single person or group of people. This won't work if we ship our own OS (such as a custom OpenWRT image) and it will simply be difficult if we're just shipping Tor (with or without supporting libraries).
We already know that we can't rely on upstream builds. If we want to our users to have the latest version of Tor, we need to set up an okpg repository ourselves.
I'm of a mixed feeling here - we can easily rely on upstream packaging work but we need to have a commitment inside of Tor to actually support a repository, if we need to run our own. It's probably the case that for rapid development, we'll need to do so. Stuff like x-wrt are a hybrid example where we may be able to have regular builds of Tor. I haven't really understood the process by which a package is actually ever compiled by OpenWRT or x-wrt and then shipped to users; the exception is when OpenWRT cuts a release...
Jake; it was my impression that you wanted to do this. Is that not the case anymore?
I want a lot of things. After talking with Erinn, I'm a little more enlightened on build issues. No one will take our work and cut a new Tor release as part of their work flow unless we somehow allocate resources or indicate that this is a priority.
With that said - I'm happy to handle packaging of Tor on OpenWRT as I've been working on already. However, that is not enough - we have to actually have a task that is going to be done regularly - no matter what OS or hardware choice we make. Android is a good example, we have repeatedly dropped the ball for a number of (good and bad) reasons. We should not repeat those mistakes - one of the biggest was simply that we did lacked a clear support plan - when a security release for Tor is tagged, Orbot needs to have at least a new Tor binary in a reasonable amount of time. We have utterly failed at this in a few cases - we should avoid re-creating this problem with Torouter.
We're adding a new "product" to The Tor Project - one of the things we need to do is actually plan for the software maintenance phase of that product. As it stands, I don't believe we have a build machine (see bug #2969) that either you (Runa) or I have access to. That makes it hard to build an OpenWRT image or even have a system where we can co-work on packages together but also where we trust the compiler for cutting a release. Speaking of which, we also lack a plan for actually cutting releases - for a real beta test, I believe we'll really need to solve this issue. It's not reasonable to ship the Torouter project without having a good way forward and that includes a solid commitment from someone or someones that will ensure Tor builds kick off for each major or security important release.
All the best, JAke
Sorry for jumping in late, see my comments below.
On Thu, Apr 21, 2011 at 9:26 AM, Jacob Appelbaum <jacob at appelbaum.net> wrote:
On 04/19/2011 03:17 AM, Runa A. Sandvik wrote:
The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596.
#2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it.
This is only a problem on smaller hardware - so if we really want the Buffalo router, we'll need to fix this or create a work around that isn't a total PITA. For now, I think we can just add a USB disk and deal with it later.
If we find that the router can't handle being a bridge, I'm not really concerned with this bug.
Buffalo with a 1Gb USB flash works as a bridge/client but the bridge is not very busy so I don't know what the limits are. See my stats I've just posted to #2334. I don't really see a quick solution that would work without adding extra storage. Fortunately, the flash drives can be small and barely visible. But one thing that must be improved here is formatting it with a usb-friendly filesystem as I'm using ext2 which was the easiest to install.
#2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint.
I commented on the bug - it's easy enough to change this by submitting a patch to openwrt-devel at lists.openwrt.org
I believe the only files that can stay unchanged over router's lifetime is secret_id_key in (the fingerprint file is recalculated from it on each start). In my patch I moved this one file to /etc/tor. Should I be worried about getting new descriptors on each reboot which hopefully happens only once every few weeks?
#2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt.
The web interface should go into version control, it should be added to the tor-alpha package in the OpenWRT svn (that's why I created it), and it should be used by some people.
I can package it that way and then you can submit. As discussed in #2370, I'm of opinion that once we have a mature release, the UI should be made into a separate package. Vidalia is separate on other linux systems, and this situation is similar. Maybe the Luci guys should express themselves on that as i think they maintain all of the UI code, but i may be wrong.
#2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions?
I think torouter is a perfectly fine name until we actually have a shipping prototype.
I agree but if you want something corny: RoutOr - there's no Tor in it ;)
- We want to collect statistics, which means that we need to ship a
GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts?
There's a tor-geoip package created by the tor package in the OpenWRT svn repo (from looking at the Makefile).
If we're including the UI in the package, and the UI uses geoip should we include geoipdb as well?
- Do we want one Tor process for bridging and one for the transparent
wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too.
I think we should run both in a single Tor for now but I think we should only enable the bridge by default. The web UI can enable the transparent stuff if we want it.
My current package enables both by default. Easy to change.
Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know.
There are a few things there that I'm not entirely happy and I am planning to work on them. The config situation should be clarified, for example. I'll get back to you on this.
All in all, I know I'll be working on this project so I can take the responsibility of creating packages once it's been decided what goes in to what repo and such. I could probably help with a build machine, too.
Cheers, Daniel
On Thu, Apr 21, 2011 at 5:17 PM, Jacob Appelbaum jacob@appelbaum.net wrote:
On 04/21/2011 07:24 AM, Runa A. Sandvik wrote:
On Thu, Apr 21, 2011 at 9:26 AM, Jacob Appelbaum jacob@appelbaum.net wrote:
On 04/19/2011 03:17 AM, Runa A. Sandvik wrote:
Hi everyone,
I am trying to figure out how we can move the Torouter project forward. In this email, I will try to summarize the current status of this project.
The Torouter project has a total of four bugs in Trac. These are #2334, #2376, #2370 and #2596.
#2334: Torouter on Buffalo breaks with large cached-descriptors[.new] files. The quick and easy solution here is to attach a USB stick to the router and use it as Tor's data directory. However, it would be great if someone could take a look at this and figure out a way to solve it.
This is only a problem on smaller hardware - so if we really want the Buffalo router, we'll need to fix this or create a work around that isn't a total PITA. For now, I think we can just add a USB disk and deal with it later.
If we find that the router can't handle being a bridge, I'm not really concerned with this bug.
As far as I know, the router can handle being a bridge as long as it has enough disk space.
What tests indicate this? How many clients can it handle and at what rate?
My mistake. I thought someone had a bridge running and that the only problem was disk space. I'll see if I can set up my router as a bridge this weekend and get things working.
#2376: Torouter on OpenWrt shouldn't have its data directory in /tmp/. The problem with having Tor's data directory in /tmp is that whenever Torouter is rebooted, Tor generates new keys and gets a new fingerprint.
I commented on the bug - it's easy enough to change this by submitting a patch to openwrt-devel@lists.openwrt.org
Great.
There are a couple of different ways to solve this problem; Karsten suggested that we could modify OpenWrt to stop creating a /tmp partition. This probably means that we will have to ship our own OpenWrt image. I'm thinking that another option would be to modify the Tor-OpenWrt-package to use / as the data directory instead of /tmp. However, I wonder if 64M (no matter how it's partitioned) of space on the Buffalo router is insufficient as long as #2334 remains open.
If we're shipping our own image, we can do a bunch of stuff. Is that what we want to do?
I don't think we should ship our own image, for the simple reason that we already have enough stuff to maintain.
I generally agree but I think we're going to have to deal with this issue one way or another.
If we're shipping people a router, I'd like to harden it. For example - it does not appear that all of the gcc hardening features are enabled by default.
Are you talking about hardening Tor or OpenWrt? Or both?
We can commit to a simple, fixed release cycle for the OS and a constant stream of updates for Tor.
Sure, that would work. I believe users will have to re-flash their routers to install a new image, though. Or maybe there's a nice way to handle upgrades in OpenWrt?
#2370: Torouter basic Web UI for OpenWrt. I haven't tried the web interface myself, but development seems to be moving along nicely. I'm not sure what the remaining steps are, other than packaging it as tor-ui (or similar) for OpenWrt.
The web interface should go into version control, it should be added to the tor-alpha package in the OpenWRT svn (that's why I created it), and it should be used by some people.
Ok, do you want to take care of this? That is, putting the web interface into version control and packaging it for OpenWrt.
Not really? :-)
In an ideal world, I think we should make these changes in the tor-alpha package. Currently, I'm not familiar with the webui at all.
I'd prefer that someone involved in developing the webui actually built a stand alone package.
I've sent Daniel an email asking if he wants to package the GUI.
#2596: Figure out a better name than "torouter". Andrew thinks we should come up with a better name for this project, one that does not have "Tor" in the name. Suggestions?
I think torouter is a perfectly fine name until we actually have a shipping prototype.
Sure, that works too.
The Torouter project also has some open questions (some are mentioned on https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter as well):
- How can we make sure that the version of Tor in OpenWrt is always
up to date? Should we set up our own OpenWrt repository? Right now, the version of Tor in OpenWrt is 0.2.1.26. Also, should we offer packages for both stable and unstable?
I think we should contribute back to OpenWRT (as I've been doing) and for our router, we should consider adding a feed for the specific hardware we intend to support. Still, we'll need to ensure the versions of _everything_ are up to date and not just Tor.
So what's our general update strategy for any platform we ship?
- We want to collect statistics, which means that we need to ship a
GeoIP file as well. I'm thinking we should create a tor-geoipdb package for OpenWrt. Thoughts?
There's a tor-geoip package created by the tor package in the OpenWRT svn repo (from looking at the Makefile).
So, it turns out that packages in OpenWrt are not updated after a release. That explains why I got 0.2.1.26 when 0.2.1.30 is available in the OpenWrt SVN.
Makes sense.
The best way to ensure that users can easily upgrade Tor and related packages is to set up a torproject.org repository for OpenWrt packages. This repository would then have to be added to /etc/opkg.conf.
That's similar to what I said on IRC; it seems reasonable to keep the Tor package updated in OpenWRT. We need to decide if we want to build regular packages for installation and also if we want to host them. I think this is mostly an Erinn question - helix?
- Do we want one Tor process for bridging and one for the transparent
wifi network? I think this sounds good, if the router can handle it. If not, then just running a bridge is ok too.
I think we should run both in a single Tor for now but I think we should only enable the bridge by default. The web UI can enable the transparent stuff if we want it.
Sounds good.
What's the default config that we want to ship?
I was thinking about a config similar to the one in the bridge-by-default bundle.
Have I missed anything important? Are there any other packages that we should include in OpenWrt? Other comments or suggestions? If you have more information or would like to help out, please let me know.
We need to make packages for the libnatpmp and miniupnpc libraries because we need tor-fw-helper support in tor-alpha.
Is this something you can / want to take care of?
I'm a little more interested in making these packages. Do you want to open a set of bugs and we can continue deeper discussion in bugs?
Sure, I can do that.
On Thu, Apr 21, 2011 at 07:15:17PM +0100, runa.sandvik@gmail.com wrote 7.4K bytes in 195 lines about: : My mistake. I thought someone had a bridge running and that the only : problem was disk space. I'll see if I can set up my router as a bridge : this weekend and get things working.
I did, on the buffalo. It worked fine with a 1gb usb drive stuck in the router.
: Sure, that would work. I believe users will have to re-flash their : routers to install a new image, though. Or maybe there's a nice way to : handle upgrades in OpenWrt?
99% of users are not going to reflash their router. If it isn't a point and click automatic update, it won't happen. I hate reflashing openwrt on the buffalo.
: > That's similar to what I said on IRC; it seems reasonable to keep the : > Tor package updated in OpenWRT. We need to decide if we want to build : > regular packages for installation and also if we want to host them. I : > think this is mostly an Erinn question - helix?
We can barely keep up with the current build load, nevermind adding new operating systems.
Torouter based on openwrt is an experiment. It seems it's going to cost us more time, effort, and people than we have to spare. The entire torouter, or bridge/relay-by-default in hardware, is an experiment.
I'd much rather see a debian-based torouter exist. We can more easily integrate debian packages of the necessary architecture, likely ARM, into our build farm than we can an entirely new OS and build environment.
The openwrt-based torouter can be a community-run and maintained project. I'd rather Tor Project spend its time and effort on making tor work on debian-compatible low-cost hardware, like a dreamplug or excito, than trying to force tor into a new OS and platform.
On Sun, Apr 24, 2011 at 11:36 PM, andrew@torproject.org wrote:
On Thu, Apr 21, 2011 at 07:15:17PM +0100, runa.sandvik@gmail.com wrote 7.4K bytes in 195 lines about: : My mistake. I thought someone had a bridge running and that the only : problem was disk space. I'll see if I can set up my router as a bridge : this weekend and get things working.
I did, on the buffalo. It worked fine with a 1gb usb drive stuck in the router.
: Sure, that would work. I believe users will have to re-flash their : routers to install a new image, though. Or maybe there's a nice way to : handle upgrades in OpenWrt?
99% of users are not going to reflash their router. If it isn't a point and click automatic update, it won't happen. I hate reflashing openwrt on the buffalo.
The most user-friendly way now is to upload an upgrade image via UI. It shouldn't be very difficult to make LuCI download the image from a website automatically.
: > That's similar to what I said on IRC; it seems reasonable to keep the : > Tor package updated in OpenWRT. We need to decide if we want to build : > regular packages for installation and also if we want to host them. I : > think this is mostly an Erinn question - helix?
We can barely keep up with the current build load, nevermind adding new operating systems.
Torouter based on openwrt is an experiment. It seems it's going to cost us more time, effort, and people than we have to spare. The entire torouter, or bridge/relay-by-default in hardware, is an experiment.
I'd much rather see a debian-based torouter exist. We can more easily integrate debian packages of the necessary architecture, likely ARM, into our build farm than we can an entirely new OS and build environment.
The openwrt-based torouter can be a community-run and maintained project. I'd rather Tor Project spend its time and effort on making tor work on debian-compatible low-cost hardware, like a dreamplug or excito, than trying to force tor into a new OS and platform.
I understand that, if you're trying to provide Tor on a router as a packaged product, it would certainly be better to have one that can handle it well without hardware modifications. And apparently we haven't seen it with OpenWRT. But i hope that someone will help make Tor run better and friendlier on OpenWRT as it's probably the most popular system on home routers that people actually use with Tor and it would be of benefit to everybody to convert many of these clients into stable bridges.
Daniel.
On Sun, 2011-04-24 at 19:36 -0400, andrew@torproject.org wrote:
Torouter based on openwrt is an experiment. It seems it's going to
cost
us more time, effort, and people than we have to spare. The entire torouter, or bridge/relay-by-default in hardware, is an experiment.
I'd much rather see a debian-based torouter exist. We can more easily integrate debian packages of the necessary architecture, likely ARM, into our build farm than we can an entirely new OS and build environment.
I recently came across:
DebWrt is all about running Debian GNU/Linux on embedded devices, for example wireless routers. DebWrt connects two very powerful technologies: Debian and OpenWrt.
http://dev.debwrt.net/wiki/TableOfSupportedHardware
Cheers, Rob van der Hoeven.
-
Andrew Lewman -
andrew＠torproject.org
-
Daniel Bryg -
Erinn Clark -
Jacob Appelbaum -
Nathan Freitas -
Rob van der Hoeven -
Runa A. Sandvik