Ready to Integrate/Review New Marionette Version into Tor
Gentlemen and Ladies, We are now ready to integrate Marionette, or at least have it evaluated, as a bridge for the Tor browser in its Pythonless form. At the Tor meeting in March, we successfully operated Marionette as a bridge by implementing the PT v2.0 specification (Thanks ahf!). Now we have a new version of Marionette which operates as a stand-alone binary (NO PYTHON!). I checked that it still forms a bridge, like at the Tor meeting. We also have a wider variety of transports enabled. We are in the process of writing the documentation for Marionette, but the documentation on the web page should be sufficient for at least getting a full evaluation started. We'd like to have the evaluation complete by the end of next month, hopefully the middle of next month, and stand ready to make any and all changes necessary. A full set of documentation will also be written for designing your own protocols. This is in process. Please let us know what you need. John Helmsen
On Fri, Jul 20, 2018 at 04:12:21PM -0400, John Helmsen wrote:
We are in the process of writing the documentation for Marionette, but the documentation on the web page should be sufficient for at least getting a full evaluation started. We'd like to have the evaluation complete by the end of next month, hopefully the middle of next month, and stand ready to make any and all changes necessary.
A full set of documentation will also be written for designing your own protocols. This is in process.
Please let us know what you need.
The Tor Browser developers may have more specific requests, but I can suggest some steps to get started. Open a ticket at https://trac.torproject.org/ for discussion and to track progress. Type: project Component: Applications/Tor Browser Keywords: marionette The old ticket for FTE is a good reference: https://bugs.torproject.org/10362 And then it would help if you port your build process to the Tor Browser build system. General information: https://trac.torproject.org/projects/tor/wiki/doc/TorBrowser/Hacking First, just build git clone https://git.torproject.org/builders/tor-browser-build.git cd tor-browser-build git checkout tbb-8.0a9-build3 make testbuild # or, e.g., testbuild-linux-x86_64 Then you'll have to add a new project (consisting of a "build" and "config" file) for Marionette and each of its dependencies. You can copy from existing projects as templates. Here is the meek project, for example: https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/m... You'll also need to add bridge lines to: https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/t... To build just one project, not an entire release, do e.g.: rbm/rbm build gmp --target testbuild --target torbrowser-linux-x86_64 rbm/rbm build marionette --target testbuild --target torbrowser-linux-x86_64
David, Thank you, I have created the ticket as #26920. https://trac.torproject.org/projects/tor/ticket/26920#ticket. Having downloaded the git project, it seems that this work cannot be performed on a Mac, since it doesn't run 'runc'. Is that right? Ben, I am currently trying to create a virtual machine using Ubuntu 16.04 for development. Unless I am mistaken, this work cannot be done on a Mac. Please do the same, so that we can put this thing to bed. On Mon, Jul 23, 2018 at 10:05 PM, David Fifield <david@bamsoftware.com> wrote:
On Fri, Jul 20, 2018 at 04:12:21PM -0400, John Helmsen wrote:
We are in the process of writing the documentation for Marionette, but the documentation on the web page should be sufficient for at least getting a full evaluation started. We'd like to have the evaluation complete by the end of next month, hopefully the middle of next month, and stand ready to make any and all changes necessary.
A full set of documentation will also be written for designing your own protocols. This is in process.
Please let us know what you need.
The Tor Browser developers may have more specific requests, but I can suggest some steps to get started.
Open a ticket at https://trac.torproject.org/ for discussion and to track progress. Type: project Component: Applications/Tor Browser Keywords: marionette The old ticket for FTE is a good reference: https://bugs.torproject.org/ 10362
And then it would help if you port your build process to the Tor Browser build system. General information: https://trac.torproject.org/projects/tor/wiki/doc/TorBrowser/Hacking First, just build git clone https://git.torproject.org/ builders/tor-browser-build.git cd tor-browser-build git checkout tbb-8.0a9-build3 make testbuild # or, e.g., testbuild-linux-x86_64 Then you'll have to add a new project (consisting of a "build" and "config" file) for Marionette and each of its dependencies. You can copy from existing projects as templates. Here is the meek project, for example: https://gitweb.torproject.org/builders/tor-browser-build. git/tree/projects/meek You'll also need to add bridge lines to: https://gitweb.torproject.org/builders/tor-browser-build. git/tree/projects/tor-browser/Bundle-Data/PTConfigs/bridge_prefs.js To build just one project, not an entire release, do e.g.: rbm/rbm build gmp --target testbuild --target torbrowser-linux-x86_64 rbm/rbm build marionette --target testbuild --target torbrowser-linux-x86_64
-- John Helmsen john.helmsen@redjack.com C: (240) 899-5676
On Tue, Jul 24, 2018 at 11:42:08AM -0400, John Helmsen wrote:
Thank you, I have created the ticket as #26920. https://trac.torproject.org/ projects/tor/ticket/26920#ticket. Having downloaded the git project, it seems that this work cannot be performed on a Mac, since it doesn't run 'runc'. Is that right?
Right, the README says "To build Tor Browser, you need a Linux distribution that has support for runc (such as Debian jessie, Ubuntu 16.04, Fedora 20, etc ...)." The tag I suggested, tbb-8.0a9-build3, is the tag of the most recent alpha release (I think). I haven't tried building it myself--if it doesn't work, you may just need to try a different tag. One of the Tor Browser devs could suggest an alternative if it doesn't work on the first try.
Okay, I have generated a VM using VirtualBox of Ubuntu version 16. I've had to restart the build process a couple of times, since the hard drive was 10GB, then 20GB. Now I am using a 50GB box, so it may work this time. Should we create the branch using the -b command? ('git checkout -b tbb-8.0a9-build3'?) Otherwise, it complains about being headless. On Tue, Jul 24, 2018 at 12:46 PM, David Fifield <david@bamsoftware.com> wrote:
On Tue, Jul 24, 2018 at 11:42:08AM -0400, John Helmsen wrote:
Thank you, I have created the ticket as #26920. https://trac. torproject.org/ projects/tor/ticket/26920#ticket. Having downloaded the git project, it seems that this work cannot be performed on a Mac, since it doesn't run 'runc'. Is that right?
Right, the README says "To build Tor Browser, you need a Linux distribution that has support for runc (such as Debian jessie, Ubuntu 16.04, Fedora 20, etc ...)."
The tag I suggested, tbb-8.0a9-build3, is the tag of the most recent alpha release (I think). I haven't tried building it myself--if it doesn't work, you may just need to try a different tag. One of the Tor Browser devs could suggest an alternative if it doesn't work on the first try.
-- John Helmsen john.helmsen@redjack.com C: (240) 899-5676
On Tue, Jul 24, 2018 at 01:57:36PM -0400, John Helmsen wrote:
Okay, I have generated a VM using VirtualBox of Ubuntu version 16. I've had to restart the build process a couple of times, since the hard drive was 10GB, then 20GB. Now I am using a 50GB box, so it may work this time.
Should we create the branch using the -b command? ('git checkout -b tbb-8.0a9-build3'?) Otherwise, it complains about being headless.
It doesn't matter, just for the sake of doing a testbuild and priming the cache of dependencies (you'll notice the git_clones directory get filled in among others). But yes, you'll want a branch for integration, something like git checkout -b marionette-integration tbb-8.0a9-build3
David, Ben hit the following error while running 'make testbuild': --2018-07-26 08:33:09-- https://downloads.sourceforge.net/stixfonts/STIXv1.1.1-latex.zip Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.105.38.13 Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.105.38.13|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-07-26 08:33:09 ERROR 404: Not Found. When looking for the file, I found this on the website: As of April 2018 STIX Fonts has moved to GitHub ( https://github.com/stipub/stixfonts/). All releases of STIX are available only through GitHub and any new issues should be reported on the new site. On Tue, Jul 24, 2018 at 2:09 PM, David Fifield <david@bamsoftware.com> wrote:
On Tue, Jul 24, 2018 at 01:57:36PM -0400, John Helmsen wrote:
Okay, I have generated a VM using VirtualBox of Ubuntu version 16. I've had to restart the build process a couple of times, since the hard drive was 10GB, then 20GB. Now I am using a 50GB box, so it may work this time.
Should we create the branch using the -b command? ('git checkout -b tbb-8.0a9-build3'?) Otherwise, it complains about being headless.
It doesn't matter, just for the sake of doing a testbuild and priming the cache of dependencies (you'll notice the git_clones directory get filled in among others). But yes, you'll want a branch for integration, something like git checkout -b marionette-integration tbb-8.0a9-build3
-- John Helmsen john.helmsen@redjack.com C: (240) 899-5676
* John Helmsen:
David,
Ben hit the following error while running 'make testbuild':
--2018-07-26 08:33:09-- https://downloads.sourceforge.net/stixfonts/STIXv1.1.1-latex.zip Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.105.38.13 Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.105.38.13|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-07-26 08:33:09 ERROR 404: Not Found.
(https://trac.torproject.org/projects/tor/ticket/26949 tracks this issue.) If you want to fix it for now, you can apply this and it should work: diff --git a/projects/fonts/build b/projects/fonts/build index 9b33da9..a1e066b 100644 --- a/projects/fonts/build +++ b/projects/fonts/build @@ -13,8 +13,8 @@ mkdir -p $distdir END; %] [% IF c("var/linux") || c("var/osx") %] - unzip -o STIXv1.1.1-latex.zip -d STIX - cp "STIX/Fonts/fonts/opentype/public/stix/STIXMath-Regular.otf" $distdir/ + unzip -o 2.0.0.zip -d STIX + cp "STIX/stixfonts-2.0.0/archive/STIXv1.1.1/Fonts/STIX-Word/STIXMath-Regular.otf" $distdir/ [% END %] [% IF c("var/linux") %] cp NotoEmoji-Regular.ttf $distdir/ diff --git a/projects/fonts/config b/projects/fonts/config index 1547403..9d11d2c 100644 --- a/projects/fonts/config +++ b/projects/fonts/config @@ -102,6 +102,6 @@ input_files: - URL: https://github.com/googlei18n/noto-cjk/raw/f36eda03dfa5582a6d49abbfb5c83d020... sha256sum: e6b82f7d3dab605c428161124ceb5e169cde93de632d800297b167cdd88e7baa enable: '[% c("var/linux") %]' - - URL: https://downloads.sourceforge.net/stixfonts/STIXv1.1.1-latex.zip - sha256sum: e3b0f712e2644438eee2d0dcd2b10b2d54f1b972039de95b2f8e800bae1adbd8 + - URL: https://github.com/stipub/stixfonts/archive/2.0.0.zip + sha256sum: 4327a16797dabebedce28a9075671730e22c7f74831b24b1fb91e27faec5a235 enable: '[% c("var/linux") || c("var/osx") %]'
On Thu, Jul 26, 2018 at 12:26:53PM -0400, Sukhbir Singh wrote:
Ben hit the following error while running 'make testbuild':
--2018-07-26 08:33:09-- https://downloads.sourceforge.net/stixfonts/STIXv1.1.1-latex.zip Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.105.38.13 Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.105.38.13|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-07-26 08:33:09 ERROR 404: Not Found.
(https://trac.torproject.org/projects/tor/ticket/26949 tracks this issue.)
If you want to fix it for now, you can apply this and it should work:
Thanks, Sukhbir. I think you can also work around it by downloading STIXv1.1.1-latex.zip manually and copying it to out/fonts/STIXv1.1.1-latex.zip.
David Fifield:
On Fri, Jul 20, 2018 at 04:12:21PM -0400, John Helmsen wrote:
We are in the process of writing the documentation for Marionette, but the documentation on the web page should be sufficient for at least getting a full evaluation started. We'd like to have the evaluation complete by the end of next month, hopefully the middle of next month, and stand ready to make any and all changes necessary.
A full set of documentation will also be written for designing your own protocols. This is in process.
Please let us know what you need.
The Tor Browser developers may have more specific requests, but I can suggest some steps to get started.
Open a ticket at https://trac.torproject.org/ for discussion and to track progress. Type: project Component: Applications/Tor Browser Keywords: marionette The old ticket for FTE is a good reference: https://bugs.torproject.org/10362
And then it would help if you port your build process to the Tor Browser build system. General information: https://trac.torproject.org/projects/tor/wiki/doc/TorBrowser/Hacking First, just build git clone https://git.torproject.org/builders/tor-browser-build.git cd tor-browser-build git checkout tbb-8.0a9-build3 make testbuild # or, e.g., testbuild-linux-x86_64 Then you'll have to add a new project (consisting of a "build" and "config" file) for Marionette and each of its dependencies. You can copy from existing projects as templates. Here is the meek project, for example: https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/m... You'll also need to add bridge lines to: https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/t... To build just one project, not an entire release, do e.g.: rbm/rbm build gmp --target testbuild --target torbrowser-linux-x86_64 rbm/rbm build marionette --target testbuild --target torbrowser-linux-x86_64
Thanks David. Yes, those are good first steps to get started. Experience shows that getting things running for Linux is the easiest part, thus I'd suggest to start with that one first. There is no need to have everything ready for all supported platforms to get your PT integrated. It's fine to ship it for the alpha for some platforms only to further shake out bugs and make it more robust. Thus, don't hesitate to put things into review state early on. Finally, don't hesitate as well to ask in #tor-dev or by mail or on the trac ticket (I'll have it on my radar) in case you are stuck or are running into issues, we are happy to help. Georg
participants (4)
-
David Fifield -
Georg Koppen -
John Helmsen -
Sukhbir Singh