[tor-commits] [tor-browser-bundle/master] Update documentation and TODO file.

Fri Apr 25 13:43:11 UTC 2014

commit e4d79c85701de21e6b15d6a0aba39b4931b4ba71
Author: Mike Perry <mikeperry-git at torproject.org>
Date:   Fri Apr 25 15:40:23 2014 +0200

    Update documentation and TODO file.
    
    Some of the TODO file may still be out of date. I think we have fixed most of
    the hardening issues, for example, but we should double check that.
---
 gitian/README.build |   75 +++++++++++++++++++++++++++++----------------------
 gitian/TODO         |    7 -----
 2 files changed, 43 insertions(+), 39 deletions(-)

diff --git a/gitian/README.build b/gitian/README.build
index ece4cb6..47a8f57 100644
--- a/gitian/README.build
+++ b/gitian/README.build
@@ -48,18 +48,18 @@ Detailed Explanation of Scripts:
  0. Makefile: The main Makefile. It has six main commands:
      - prep: Check OS prerequisites and download source dependency inputs
      - build: Build localized bundles for Linux, Windows, and Mac
-     - clean: Remove prior partial build stages (Tor and Firefox)
+     - clean: Remove prior partial build stages (see 'Partial Rebuilds' below)
      - vmclean: Remove VM base images
      - distclean: Remove source dependency inputs, and run clean and vmclean
      - all: The default. It calls clean, prep, and then build.
-     - sign: Signs your build output and uplodas it to people.torproject.org
+     - sign: Signs your build output and uploads it to people.torproject.org
      - match: Checks your build output against public signed hashes
-    To build alpha bundles, alternate targets are provided:
-     - alpha: The equivalent to the 'all' rule for alpha packages
-     - build-alpha: The equivalent to the 'build' rule for alpha packages
-     - prep-alpha: The equivalent to the 'prep' rule for alpha packages
-     - sign-alpha: Signs your build output and uplodas it to people.torproject.org
-     - match-alpha: Checks your build output against public signed hashes
+    To build beta/alpha/nightly bundles, alternate targets are provided:
+     - beta: The equivalent to the 'all' rule for beta packages
+     - build-beta: The equivalent to the 'build' rule for beta packages
+     - prep-beta: The equivalent to the 'prep' rule for beta packages
+     - sign-beta: Signs your build output and uploads it to people.torproject.org
+     - match-beta: Checks your build output against public signed hashes
  
  1. check-prerequisites.sh: This script checks if your system is capable of
     running Gitian, and if it is not, it tells you what you need to do.
@@ -95,31 +95,47 @@ Detailed Explanation of Scripts:
     call the appropriate descriptors to build a 32bit Mac bundle. It also
     will create build VM images for you if you haven't done that before.
 
+ 9. record-inputs.sh: This script records all of the hashes and bundle
+    inputs for inclusion in the 'Docs/sources' subdirectory of the bundles
+    themselves.
+ 
+ 10. hash-bundles.sh: This script generates a 'sha256sums.txt' file in sorted,
+     reproducible order.
 
-Partial Rebuilds:
+ 11. check-match.sh: This script checks your 'sha256sums.txt' file against
+     any signed, published builds.
 
-  The mkbundle shell scripts each have three Gitian stages.
+ 12. upload-signature.sh: This script signs and uploads your 'sha256sums.txt'
+     file (for use if you are an official builder).
 
-  First, using their platform's gitian-tor.yml Gitian descriptor, they build
-  tor and the tor dependencies (libevent, zlib, and openssl). Second, using
-  the gitian-firefox.yml descriptor, they build Firefox with the Tor Browser
-  patches and preferences applied. Third, in gitian-bundle.yml, they assemble
-  the bundles using the addons and localization packs.
 
-  The Tor and Firefox stages end up as a *gbuilt*.zip files in
-  ../../gitian-builder/inputs/, and those zip files are used as inputs into
-  the final bundling and localization stage. If those files exist in that
-  directory, the corresponding bundle will *not* be built.
+Partial Rebuilds:
+
+  The mkbundle shell scripts each have five Gitian stages:
+    1. Build support utilities (gitian-utils.yml)
+    2. Tor (gitian-tor.yml)
+    3. Tor Browser (gitian-firefox.yml)
+    4. Pluggable Transports (gitian-pluggable-transports.yml)
+    5. Addon and bundle package assembly (gitian-bundle.yml)
+
+  Each stage leaves build artifacts in ../../gitian-builder/inputs/ for use
+  in later stages. If the artifact files for a given stage are present, that
+  stage is skipped by a 'make build'.
 
   No other dependency checking on build stages is performed.
 
-  This means that if any of the source inputs, or the Ubuntu buildchain tools,
-  or the build tools' dependencies upgrade, your Tor and Firefox inputs will
-  no longer match those produced by someone else's build from a fresher VM
-  that downloaded those newer toolchain packages.
+  If you wish to rebuild only a given stage, there are several 'make clean-'
+  targets to remove the artifacts for that stage. Running a subsequent
+  'make build' after that will rebuild only those targets that have been
+  cleaned, and then re-package everything again as well.
+
+  Note that if any of the source inputs, or the Ubuntu buildchain tools, or
+  the build tools' dependencies upgrade, your Tor and Firefox inputs will
+  no longer match those produced by someone else's build from a fresh build
+  that downloaded those newer input, dependency, and toolchain packages.
 
-  This means for official builds, you should always run 'make clean' (which
-  is called by the default Makefile target).
+  This means for official builds, it is a good idea to always run a full
+  'make clean' first, or just run 'make'.
 
 
 Known Issues and Quirks:
@@ -138,7 +154,8 @@ Known Issues and Quirks:
      VMs) or 'rm ../../gitian-builder/*lucid*' (to remove the Linux VMs).
 
      You probably want to make sure you have no stray qemu processes before
-     rebuilding the VMs or starting a new build, too.
+     rebuilding the VMs or starting a new build, too. 'killall qemu-kvm' is
+     your friend.
 
      Once you get a working set of base VMs (in ../../gitian-builder/base-*),
      you should probably copy them somewhere safe if you want to avoid VM
@@ -166,12 +183,6 @@ Known Issues and Quirks:
      SSH keys of the previous instances' VM, and VM startup will either hang
      or prompt you for an SSH password).
 
-  6. The number of cpus in the VM images is currently hardcoded at 4. If you
-     need a different number, please let me know and I will write a patch to 
-     support configuring the number of CPUs easily.
-
-  7. LXC-built bundles are still not deterministic and may run into other issues.
-     See TODO file for details.
 
 Diagnosing Problems:
 
diff --git a/gitian/TODO b/gitian/TODO
index cda7f32..47a1b6f 100644
--- a/gitian/TODO
+++ b/gitian/TODO
@@ -20,7 +20,6 @@ Short Term:
      - Refuse to build if `git diff --raw` is non-zero?
  - Detect arch on Linux
    - https://trac.torproject.org/projects/tor/ticket/3841
- - Too many libevent libs are included.. Only need a couple
 
 Mid Term:
  - Write a VM test+rebuild script
@@ -36,8 +35,6 @@ Mid Term:
  - Clean up old paths (requires Tor Launcher commits)
    - Remove FirefoxPortable paths for Windows
    - Relocate torrc to Library/Tor for MacOS
- - DMGs for Mac
-   https://trac.torproject.org/projects/tor/ticket/4261
  - Universal binaries for Mac?
    - Usability win, but gettor loss (we're at 23M right now)
    - https://developer.mozilla.org/en-US/docs/Mac_OS_X_Universal_Binaries
@@ -48,10 +45,6 @@ Mid Term:
  - Fix windows issues:
    - Link to DirectX via Wine (for WebGL)
      - Weird C++ stdext namespace issues remain :/
-   - Accesibility still fails :/
-     - https://bugzilla.mozilla.org/show_bug.cgi?id=815581
-       - Has conflicts with FF17-ESR :/
-       - Can we just pull the whole accessibility dir?
  - If Tor Launcher fails to load, the browser still can launch
    - Add browser code to detect missing addons?
    - Missing/Partial Localization strings can cause this..

