As we wrap up the GSOC, I wanted to do a retrospective on the Tor Browser development process for new people.
Basically, building Tor Browser is difficult from a setup perspective (we're always working on making it easier, but it's still not trivial) and it's difficult from a resource perspective - it can take a long time to build Tor Browser. You can easy have a day where if you get two compiles in - that's a good day. Couple that with the trial and error process that new people usually have with a software project - and you have a pretty high barrier to entry.
The experience with nmago this past summer illustrated how important this is. I estimate he would have been able to accomplish 33% to 50% more than he was able to accomplish if he had both a faster machine and a pre-set-up build environment.
What I'd like to do to improve this situation is to provide a development instance on EC2. We can publish an image that anyone can run on their own and do development in. They would have to pay their own money to run the machine, while the host org (Tor or me) would pay a small cost each month to provide the image (I think <$3/month).
Furthermore, I'd like Tor Project to _fund_ this development image for future GSOC people and potentially others on a case by case basis. GSOC is finishing, so this isn't an actual request to pay any money now. But I think we should in the future.
Right now, the cost to reserve a single machine for an entire year is 2248.28. If we only did on-demand it would be 825.72 for 3 months.
There is no reason we couldn't have multiple people using a single machine; aside from if two people compile at the same time they'll be fighting for resources. Depending on how often it is used this may not matter or it may matter a lot.
Obviously we don't trust cloud images, but this is for developing patches which will receive code review. There is no real trust here, unless the user does something silly like put their ssh keys on it, which they should not.
-tom
On Aug 18, 2017, at 9:00 AM, Tom Ritter tom@ritter.vg wrote:
As we wrap up the GSOC, I wanted to do a retrospective on the Tor Browser development process for new people.
Basically, building Tor Browser is difficult from a setup perspective (we're always working on making it easier, but it's still not trivial) and it's difficult from a resource perspective - it can take a long time to build Tor Browser. You can easy have a day where if you get two compiles in - that's a good day. Couple that with the trial and error process that new people usually have with a software project - and you have a pretty high barrier to entry.
The experience with nmago this past summer illustrated how important this is. I estimate he would have been able to accomplish 33% to 50% more than he was able to accomplish if he had both a faster machine and a pre-set-up build environment.
What I'd like to do to improve this situation is to provide a development instance on EC2. We can publish an image that anyone can run on their own and do development in. They would have to pay their own money to run the machine, while the host org (Tor or me) would pay a small cost each month to provide the image (I think <$3/month).
Furthermore, I'd like Tor Project to _fund_ this development image for future GSOC people and potentially others on a case by case basis. GSOC is finishing, so this isn't an actual request to pay any money now. But I think we should in the future.
Right now, the cost to reserve a single machine for an entire year is 2248.28. If we only did on-demand it would be 825.72 for 3 months.
There is no reason we couldn't have multiple people using a single machine; aside from if two people compile at the same time they'll be fighting for resources. Depending on how often it is used this may not matter or it may matter a lot.
Obviously we don't trust cloud images, but this is for developing patches which will receive code review. There is no real trust here, unless the user does something silly like put their ssh keys on it, which they should not.
-tom
Hey Tom. Can you be more specific about the actual costs? Are you saying that for all of our GSOC interns combined, we'd need to pay one single charge of $825.72 to get 3 months plus the approximately $3/month over the course of the year? Or would the $825.72 charge be for each GSOC intern? We could probably manage the former but not the latter. Shari
On 18 August 2017 at 11:20, Shari Steele ssteele@riseup.net wrote:
Can you be more specific about the actual costs?
Kinda!
Are you saying that for all of our GSOC interns combined, we'd need to pay one single charge of $825.72 to get 3 months plus the approximately $3/month over the course of the year?
That is an option (and cheaper).
Or would the $825.72 charge be for each GSOC intern?
This is also an option.
The cheaper one means they'd all share the same machine. It'd still be an improvement over what they'd have otherwise, but it wouldn't be as ideal as giving each their own machine.
But in any case, I think this is really most valuable for Tor Browser GSOC students. Other GSOC students (working on other projects) probably don't need a machine at all (or could get similar efficiency improvements with a less powerful (and cheaper) machine. And we may not even have a TB GSOC student next year, it all depends.
-tom
Hey Tom. Thanks for bringing this to my attention. I'm going to respond to you privately to help clarify this some more. Shari
On Aug 18, 2017, at 11:05 AM, Tom Ritter tom@ritter.vg wrote:
On 18 August 2017 at 11:20, Shari Steele ssteele@riseup.net wrote:
Can you be more specific about the actual costs?
Kinda!
Are you saying that for all of our GSOC interns combined, we'd need to pay one single charge of $825.72 to get 3 months plus the approximately $3/month over the course of the year?
That is an option (and cheaper).
Or would the $825.72 charge be for each GSOC intern?
This is also an option.
The cheaper one means they'd all share the same machine. It'd still be an improvement over what they'd have otherwise, but it wouldn't be as ideal as giving each their own machine.
But in any case, I think this is really most valuable for Tor Browser GSOC students. Other GSOC students (working on other projects) probably don't need a machine at all (or could get similar efficiency improvements with a less powerful (and cheaper) machine. And we may not even have a TB GSOC student next year, it all depends.
-tom _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
On Fri, Aug 18, 2017 at 01:45:58PM -0700, Shari Steele wrote:
Thanks for bringing this to my attention. I'm going to respond to you privately to help clarify this some more.
The alternative is that we get dedicated computers from Hetzner, and pay ~$100/mo for Tor Browser build machines. I think we have something like two of those that we pay for right now: one for Windows builds, and one for Tor Messenger builds, and we've been talking about getting a third to help the Tor Browser team do builds.
It's not clear to me whether getting the Hetzner computer is the better or worse idea. There's the price tradeoff; also maybe we're looking for different security goals, e.g. between sandbox developer computers vs official build machines; and maybe the Cloud image is easier for people to work with. But in an ideal world, we would pick the better idea and stop needing to do the worse one. :)
--Roger
On 18 August 2017 at 17:15, Roger Dingledine arma@mit.edu wrote:
On Fri, Aug 18, 2017 at 01:45:58PM -0700, Shari Steele wrote:
Thanks for bringing this to my attention. I'm going to respond to you privately to help clarify this some more.
The alternative is that we get dedicated computers from Hetzner, and pay ~$100/mo for Tor Browser build machines. I think we have something like two of those that we pay for right now: one for Windows builds, and one for Tor Messenger builds, and we've been talking about getting a third to help the Tor Browser team do builds.
It's not clear to me whether getting the Hetzner computer is the better or worse idea. There's the price tradeoff; also maybe we're looking for different security goals, e.g. between sandbox developer computers vs official build machines; and maybe the Cloud image is easier for people to work with. But in an ideal world, we would pick the better idea and stop needing to do the worse one. :)
I don't think we should trust these for security much at all, only for testing or untrusted development. But you can do a lot on an untrusted development box.
Price is then the one axis, the other access is accessibility. The advantage of EC2 is that when we publish the development image, anyone who wants to contribute to Tor Browser and is willing to pay their own cost can get a pre-set up dev environment that they know will compile correctly (and quickly). That's advantageous. I don't know if Hetzner has anything similar. (And we can spin up more with zero configuration, and revert with zero effort.)
That said, I tried to do this a long time ago before reproducible builds with a Windows AMI, and I don't think anyone used it. Maybe because it was Windows, maybe I'm just overestimating the willingness of the community to pay for a machine to get a working dev env.
-tom
On 22 Aug 2017, at 01:51, Tom Ritter tom@ritter.vg wrote:
Price is then the one axis, the other access is accessibility. The advantage of EC2 is that when we publish the development image, anyone who wants to contribute to Tor Browser and is willing to pay their own cost can get a pre-set up dev environment that they know will compile correctly (and quickly). That's advantageous. I don't know if Hetzner has anything similar. (And we can spin up more with zero configuration, and revert with zero effort.)
Another advantage of AWS is you can spin up the machine when you want to compile, and then turn it off again when you're done testing.
So you don't need to pay for the machine to be up all day for the whole 3 months - it only needs to be up when it's being used.
This might be a way we can afford to give each GSOC / TBB dev their own machine.
T
-- Tim Wilson-Brown (teor)
teor2345 at gmail dot com PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B ricochet:ekmygaiu4rzgsk6n xmpp: teor at torproject dot org ------------------------------------------------------------------------
Tom Ritter:
On 18 August 2017 at 17:15, Roger Dingledine arma@mit.edu wrote:
On Fri, Aug 18, 2017 at 01:45:58PM -0700, Shari Steele wrote:
Thanks for bringing this to my attention. I'm going to respond to you privately to help clarify this some more.
The alternative is that we get dedicated computers from Hetzner, and pay ~$100/mo for Tor Browser build machines. I think we have something like two of those that we pay for right now: one for Windows builds, and one for Tor Messenger builds, and we've been talking about getting a third to help the Tor Browser team do builds.
It's not clear to me whether getting the Hetzner computer is the better or worse idea. There's the price tradeoff; also maybe we're looking for different security goals, e.g. between sandbox developer computers vs official build machines; and maybe the Cloud image is easier for people to work with. But in an ideal world, we would pick the better idea and stop needing to do the worse one. :)
I don't think we should trust these for security much at all, only for testing or untrusted development. But you can do a lot on an untrusted development box.
Price is then the one axis, the other access is accessibility. The advantage of EC2 is that when we publish the development image, anyone who wants to contribute to Tor Browser and is willing to pay their own cost can get a pre-set up dev environment that they know will compile correctly (and quickly). That's advantageous. I don't know if Hetzner has anything similar. (And we can spin up more with zero configuration, and revert with zero effort.)
That said, I tried to do this a long time ago before reproducible builds with a Windows AMI, and I don't think anyone used it. Maybe because it was Windows, maybe I'm just overestimating the willingness of the community to pay for a machine to get a working dev env.
What you experienced is one of the things that make me skeptical as well.
However, I think I'd like to make a more general point as well. There are actually only two reasons why one wants to have such an elaborate development setup
1) Making sure the code builds reproducibly 2) Making sure it compiles with all our toolchains
Thus, "normal" patch development should be largely unaffected by having such a developmen environment. We try to point that out on our Tor Browser hacking page by explaining how to just work with the Firefox part and test the patch in an Tor Browser environment. One of the ideas behind that was to make the entry barrier to browser development as low as possible: no special AWS stuff that one needs to use, no need to wait on a ton of patch-unrelated things to get built before seeing the results of the patch, no need to setup the reproducible build environment.
I think I like to keep that approach for the casual Tor Browser patch dev. The question then boils down what to do with the final test of the patch both regarding 1) and 2) mentioned above.
For the GSoC case you mentioned it is perfectly fine for me if one of us gives the proposed patches a final whirl on the dev environments we already have to test that they compile and don't touch the reproduciblity property. Yes, i agree we should not bother the student with setting up a reproducible builds environment on a machine they have access to. But spinning up, and maintaining such an image and paying for these services just for that case seems a bit overkill to me.
Now what about the casual Tor Browser patch writer? I am not sure but recalling past experiences (which are not many) I've never heard that writing Tor Browser patches failed because someone did not have access to a reproducible build system. What we did was basically the GSoC-student related approach mentioned above which happened during code review and it worked out so far. Sure it's hard to estimate how many potential devs got scared away by seeing the complex build system and might have stayed had they access to some preconfigured image. But the solution to that should be, in the first place, to make our wiki clearer about the barriers to development being much lower.
So, all in all I am a bit skeptical about whether the time and money we would spend are spent best the way you envision.
Georg
Amazon also provides research grants, which this could potentially fit well under: https://aws.amazon.com/grants/.
Basically you explain to them what you are trying to do, how many AWS credits you need and they give you some credits to spend in our AWS account.
As OONI we did this last year and got 10k in credits for OONI (that we spent all of). We have currently run out and have applied for another grant that we hope to receive.
The form to fill out is not very time consuming and it generally take 1-2 months to get it approved.
~ Arturo
On 18 August 2017 at 20:06:32, Tom Ritter (tom@ritter.vg) wrote:
On 18 August 2017 at 11:20, Shari Steele ssteele@riseup.net wrote:
Can you be more specific about the actual costs?
Kinda!
Are you saying that for all of our GSOC interns combined, we'd need to pay one single charge of $825.72 to get 3 months plus the approximately $3/month over the course of the year?
That is an option (and cheaper).
Or would the $825.72 charge be for each GSOC intern?
This is also an option.
The cheaper one means they'd all share the same machine. It'd still be an improvement over what they'd have otherwise, but it wouldn't be as ideal as giving each their own machine.
But in any case, I think this is really most valuable for Tor Browser GSOC students. Other GSOC students (working on other projects) probably don't need a machine at all (or could get similar efficiency improvements with a less powerful (and cheaper) machine. And we may not even have a TB GSOC student next year, it all depends.
-tom _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
tor-project@lists.torproject.org
-
Arturo Filastò -
Georg Koppen -
Roger Dingledine -
Shari Steele -
Shari Steele -
teor -
Tom Ritter