January 2017 - tor-commits - lists.torproject.org

[torspec/master] prop224: Fix tabs issue
by dgoulet＠torproject.org 31 Jan '17

31 Jan '17

commit 7e807c15aee409b68930a616d0e37b10a6f6f147 Author: David Goulet <dgoulet(a)torproject.org> Date: Tue Jan 31 09:39:34 2017 -0500 prop224: Fix tabs issue Signed-off-by: David Goulet <dgoulet(a)torproject.org> --- proposals/224-rend-spec-ng.txt | 102 ++++++++++++++++++++--------------------- 1 file changed, 51 insertions(+), 51 deletions(-) diff --git a/proposals/224-rend-spec-ng.txt b/proposals/224-rend-spec-ng.txt index 4a685e5..6f71b32 100644 --- a/proposals/224-rend-spec-ng.txt +++ b/proposals/224-rend-spec-ng.txt @@ -2137,80 +2137,80 @@ Appendix E. Managing authorized client data [CLIENT-AUTH-MGMT] E.1.1. Hidden Service side - A hidden service that wants to perform client authorization, adds a new - option HiddenServiceAuthorizeClient to its torrc file: + A hidden service that wants to perform client authorization, adds a new + option HiddenServiceAuthorizeClient to its torrc file: - HiddenServiceAuthorizeClient auth-type client-name,client-name,... + HiddenServiceAuthorizeClient auth-type client-name,client-name,... - The only recognized auth-type value is "basic" which describes the scheme in - section [CLIENT-AUTH]. The rest of the line is a comma-separated list of - human-readable authorized client names. + The only recognized auth-type value is "basic" which describes the scheme in + section [CLIENT-AUTH]. The rest of the line is a comma-separated list of + human-readable authorized client names. - Let's consider that one of the listed client names is "alice". In this - case, Tor checks the "client_authorized_pubkeys" file for any entries - with client_name being "alice". If an "alice" entry is found, we use the - relevant pubkeys to authenticate Alice. + Let's consider that one of the listed client names is "alice". In this + case, Tor checks the "client_authorized_pubkeys" file for any entries + with client_name being "alice". If an "alice" entry is found, we use the + relevant pubkeys to authenticate Alice. - If no "alice" entry is found in the "client_authorized_pubkeys" file, Tor - is tasked with generating public/private keys for Alice. To do so, Tor - generates x25519 and ed25519 keypairs for Alice, then makes a - "client_authorized_privkeys/alice.privkey" file and writes the private - keys inside; it also adds an entry for alice to the - "client_authorized_pubkeys" file. + If no "alice" entry is found in the "client_authorized_pubkeys" file, Tor + is tasked with generating public/private keys for Alice. To do so, Tor + generates x25519 and ed25519 keypairs for Alice, then makes a + "client_authorized_privkeys/alice.privkey" file and writes the private + keys inside; it also adds an entry for alice to the + "client_authorized_pubkeys" file. - In this last case, the hidden service operator has the responsibility to - pass the .key file to Alice in a secure out-of-band way. After the file - is passed to Alice, it can be shredded from the filesystem, as only the - public keys are required for the hidden service to function. + In this last case, the hidden service operator has the responsibility to + pass the .key file to Alice in a secure out-of-band way. After the file + is passed to Alice, it can be shredded from the filesystem, as only the + public keys are required for the hidden service to function. E.1.2. Client side - A client who wants to register client authorization data for a hidden service - needs to add the following line to their torrc: + A client who wants to register client authorization data for a hidden service + needs to add the following line to their torrc: - HidServAuth onion-address x25519-private-key ed25519-private-key + HidServAuth onion-address x25519-private-key ed25519-private-key - The keys above are either generated by Alice using a key generation utility, - or they are extracted from a .key file provided by the hidden service. + The keys above are either generated by Alice using a key generation utility, + or they are extracted from a .key file provided by the hidden service. - In the former case, the client is also tasked with transfering the public - keys to the hidden service in a secure out-of-band way. + In the former case, the client is also tasked with transfering the public + keys to the hidden service in a secure out-of-band way. E.2. Configuring client authorization using the control port E.2.1. Service side - A hidden service also has the option to configure authorized clients - using the control port. The idea is that hidden service operators can use - controller utilities that manage their access control instead of using - the filesystem to register client keys. + A hidden service also has the option to configure authorized clients + using the control port. The idea is that hidden service operators can use + controller utilities that manage their access control instead of using + the filesystem to register client keys. - Specifically, we require a new control port command ADD_ONION_CLIENT_AUTH - which is able to register x25519/ed25519 public keys tied to a specific - authorized client. + Specifically, we require a new control port command ADD_ONION_CLIENT_AUTH + which is able to register x25519/ed25519 public keys tied to a specific + authorized client. [XXX figure out control port command format] - Hidden services who use the control port interface for client auth need - to perform their own key management. + Hidden services who use the control port interface for client auth need + to perform their own key management. E.2.2. Client side - There should also be a control port interface for clients to register - authorization data for hidden services without having to use the - torrc. It should allow both generation of client authorization private - keys, and also to import client authorization data provided by a hidden - service + There should also be a control port interface for clients to register + authorization data for hidden services without having to use the + torrc. It should allow both generation of client authorization private + keys, and also to import client authorization data provided by a hidden + service - This way, Tor Browser can present "Generate client auth keys" and "Import - client auth keys" dialogs to users when they try to visit a hidden service - that is protected by client authorization. + This way, Tor Browser can present "Generate client auth keys" and "Import + client auth keys" dialogs to users when they try to visit a hidden service + that is protected by client authorization. - Specifically, we require two new control port commands: - IMPORT_ONION_CLIENT_AUTH_DATA - GENERATE_ONION_CLIENT_AUTH_DATA - which import and generate client authorization data respectively. + Specifically, we require two new control port commands: + IMPORT_ONION_CLIENT_AUTH_DATA + GENERATE_ONION_CLIENT_AUTH_DATA + which import and generate client authorization data respectively. - [XXX how does key management work here?] - [XXX what happens when people use both the control port interface and the - filesystem interface?] + [XXX how does key management work here?] + [XXX what happens when people use both the control port interface and the + filesystem interface?]

1 0

[torspec/master] prop224: Specify rough directory format of hidden services.
by asn＠torproject.org 31 Jan '17

31 Jan '17

commit 009fd7195a0a302d04b4d7d98b213c69a3d76a7f Author: George Kadianakis <desnacked(a)riseup.net> Date: Tue Jan 31 14:57:14 2017 +0200 prop224: Specify rough directory format of hidden services. --- proposals/224-rend-spec-ng.txt | 80 ++++++++++++++++++++++++++++-------------- 1 file changed, 53 insertions(+), 27 deletions(-) diff --git a/proposals/224-rend-spec-ng.txt b/proposals/224-rend-spec-ng.txt index 814fd74..4a685e5 100644 --- a/proposals/224-rend-spec-ng.txt +++ b/proposals/224-rend-spec-ng.txt @@ -2096,15 +2096,46 @@ Appendix E. Reserved numbers Note: The value "0A" is skipped because it's reserved for the onion key cross-certifying ntor identity key from proposal 228. -Appendix F. Managing authorized client data [CLIENT-AUTH-MGMT] +Appendix F. Hidden service directory format [HIDSERVDIR-FORMAT] + + This appendix section specifies the contents of the HiddenServiceDir directory: + + - "hostname" [FILE] + + This file contains the onion address of the onion service. + + - "private_key_ed25519" [FILE] + + This file contains the private master ed25519 key of the onion service. + [TODO: Offline keys] + + - "client_authorized_pubkeys" [FILE] + + If client authorization is _enabled_, this is a newline-separated file of + "<client name> <pubkeys> entries for authorized clients. You can think of it + as the ~/.ssh/authorized_keys of onion services. See [CLIENT-AUTH-MGMT] for + more details. + + - "./client_authorized_privkeys/" [DIRECTORY] + "./client_authorized_privkeys/alice.privkey" [FILE] + "./client_authorized_privkeys/bob.privkey" [FILE] + "./client_authorized_privkeys/charlie.privkey" [FILE] + + If client authorization is _enabled_ _AND_ if the hidden service is + responsible for generating and distributing private keys for its clients, + then this directory contains files with client's private keys. See + [CLIENT-AUTH-MGMT] for more details. + +Appendix E. Managing authorized client data [CLIENT-AUTH-MGMT] Hidden services and clients can configure their authorized client data either using the torrc, or using the control port. This section presents a suggested - scheme for configuring client authorization. + scheme for configuring client authorization. Please see appendix + [HIDSERVDIR-FORMAT] for more information about relevant hidden service files. - F.1. Configuring client authorization using torrc + E.1. Configuring client authorization using torrc - F.1.1. Hidden Service side + E.1.1. Hidden Service side A hidden service that wants to perform client authorization, adds a new option HiddenServiceAuthorizeClient to its torrc file: @@ -2115,29 +2146,24 @@ Appendix F. Managing authorized client data [CLIENT-AUTH-MGMT] section [CLIENT-AUTH]. The rest of the line is a comma-separated list of human-readable authorized client names. - Let's consider that one of the listed client names is "alice". In this case, - Tor checks the directory at "DataDirectory/hidden_service/client_auth/" for - any files with filename alice.key or alice.pub . - - Files ending in .key contain private keys for authorized clients, whereas - .pub files contain public keys for authorized clients. In general, authorized - clients should send their public keys to the hidden service operator, and the - operator should place them in the filesystem as .pub files. Only .pub files - are useful to the hidden service, whereas .key files are there only if the - hidden service had to generate its client's keypairs as described below. + Let's consider that one of the listed client names is "alice". In this + case, Tor checks the "client_authorized_pubkeys" file for any entries + with client_name being "alice". If an "alice" entry is found, we use the + relevant pubkeys to authenticate Alice. - If no alice.key or alice.pub files exist, Tor is tasked with generating - client keys for Alice. To do so, Tor generates x25519 and ed25519 keypairs - for Alice, then makes an alice.key file and writes the private keys inside; - it also makes an alice.pub file and writes the public keys inside. - [XXX what format? it should be convenient so that client can just copy-paste] + If no "alice" entry is found in the "client_authorized_pubkeys" file, Tor + is tasked with generating public/private keys for Alice. To do so, Tor + generates x25519 and ed25519 keypairs for Alice, then makes a + "client_authorized_privkeys/alice.privkey" file and writes the private + keys inside; it also adds an entry for alice to the + "client_authorized_pubkeys" file. - In this case, the hidden service operator has the responsibility to pass the - .key file to Alice in a secure out-of-band way. After the file is passed to - Alice, it can be shredded from the filesystem, as only the public keys are - required for the hidden service to function. + In this last case, the hidden service operator has the responsibility to + pass the .key file to Alice in a secure out-of-band way. After the file + is passed to Alice, it can be shredded from the filesystem, as only the + public keys are required for the hidden service to function. - F.1.2. Client side + E.1.2. Client side A client who wants to register client authorization data for a hidden service needs to add the following line to their torrc: @@ -2150,9 +2176,9 @@ Appendix F. Managing authorized client data [CLIENT-AUTH-MGMT] In the former case, the client is also tasked with transfering the public keys to the hidden service in a secure out-of-band way. - F.2. Configuring client authorization using the control port + E.2. Configuring client authorization using the control port - F.2.1. Service side + E.2.1. Service side A hidden service also has the option to configure authorized clients using the control port. The idea is that hidden service operators can use @@ -2167,7 +2193,7 @@ Appendix F. Managing authorized client data [CLIENT-AUTH-MGMT] Hidden services who use the control port interface for client auth need to perform their own key management. - F.2.2. Client side + E.2.2. Client side There should also be a control port interface for clients to register authorization data for hidden services without having to use the

1 0

[torspec/master] prop224: Specify onion address encoding
by asn＠torproject.org 31 Jan '17

31 Jan '17

commit 80e443aac029a454c1737e309569674b8eb0e2f3 Author: George Kadianakis <desnacked(a)riseup.net> Date: Tue Jan 31 14:52:32 2017 +0200 prop224: Specify onion address encoding --- proposals/224-rend-spec-ng.txt | 52 ++++++++++++++++++++++++++++++------------ 1 file changed, 37 insertions(+), 15 deletions(-) diff --git a/proposals/224-rend-spec-ng.txt b/proposals/224-rend-spec-ng.txt index 4ef1ae3..814fd74 100644 --- a/proposals/224-rend-spec-ng.txt +++ b/proposals/224-rend-spec-ng.txt @@ -67,7 +67,8 @@ Table of contents: 4.2.1. Key expansion 4.3. Using legacy hosts as rendezvous points 5. Encrypting data between client and host - 6. Open Questions: + 6. Encoding onion addresses [ONIONADDRESS] + 7. Open Questions: -1. Draft notes @@ -404,9 +405,10 @@ Table of contents: 1.2. In more detail: naming hidden services [NAMING] - A hidden service's name is its long term master identity key. This - is encoded as a hostname by encoding the entire key in Base 32, and - adding the string ".onion" at the end. + A hidden service's name is its long term master identity key. This is + encoded as a hostname by encoding the entire key in Base 32, including a + version byte and a checksum, and then appending the string ".onion" at the + end. The result is a 56-character domain name. (This is a change from older versions of the hidden service protocol, where we used an 80-bit truncated SHA1 hash of a 1024 bit RSA key.) @@ -419,17 +421,9 @@ Table of contents: And a new name following this specification might look like: - a1uik0w1gmfq3i5ievxdm9ceu27e88g6o7pe0rffdw9jmntwkdsd.onion + l5satjgud6gucryazcyvyvhuxhr74u6ygigiuyixe3a6ysis67ororad.onion - Note that since master keys are 32 bytes long, and 52 bytes of base - 32 encoding can hold 260 bits of information, we have four unused - bits in each of these names. - - [TODO: Alternatively, we could require that the first bit of the - master key always be zero, and use a 51-byte encoding. Or we could - require that the first two bits be zero, and use a 51-byte encoding - and reserve the first bit. Or we could require that the first nine - bits, or ten bits be zero, etc.] + Please see section [ONIONADDRESS] for the encoding specification. 1.3. In more detail: Access control [IMD:AC] @@ -1874,7 +1868,31 @@ Table of contents: decryption. The client encrypts with Kf and decrypts with Kb; the service host does the opposite. -6. Open Questions: +6. Encoding onion addresses [ONIONADDRESS] + + The onion address of a hidden service includes its identity public key, a + version field and a basic checksum. All this information is then base32 + encoded as shown below: + + onion_address = base32(PUBKEY || CHECKSUM || VERSION) + ".onion" + CHECKSUM = H(".onion checksum" || PUBKEY || VERSION) + + where: + - PUBKEY is the 32 bytes ed25519 master pubkey of the hidden service. + - VERSION is an one byte version field (default value '\x03') + - ".onion checksum" is a constant string + - CHECKSUM is truncated to two bytes before inserting it in onion_address + + Here are a few example addresses (with broken checksum): + + l5satjgud6gucryazcyvyvhuxhr74u6ygigiuyixe3a6ysis67ororad.onion + btojiu7nu5y5iwut64eufevogqdw4wmqzugnoluw232r4t3ecsfv37ad.onion + vckjr6bpchiahzhmtzslnl477hdfvwhzw7dmymz3s5lp64mwf6wfeqad.onion + + For more information about this encoding, please see our discussion thread + at [ONIONADDRESS-REFS]. + +7. Open Questions: Scaling hidden services is hard. There are on-going discussions that you might be able to help with. See [SCALING-REFS]. @@ -1941,6 +1959,9 @@ References: [VANITY-REFS]: https://github.com/Yawning/horse25519 +[ONIONADDRESS-REFS]: + https://lists.torproject.org/pipermail/tor-dev/2017-January/011816.html + Appendix A. Signature scheme with key blinding [KEYBLIND] As described in [IMD:DIST] and [SUBCRED] above, we require a "key @@ -2166,3 +2187,4 @@ Appendix F. Managing authorized client data [CLIENT-AUTH-MGMT] [XXX how does key management work here?] [XXX what happens when people use both the control port interface and the filesystem interface?] +

1 0

[tor-browser-bundle/master] Bug 21343: Remove unused FTE related parts for macOS
by gk＠torproject.org 31 Jan '17

31 Jan '17

commit fc71d02f70b8bd05fbd4cd09650b175ea29fe186 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Jan 30 12:46:01 2017 +0000 Bug 21343: Remove unused FTE related parts for macOS --- Bundle-Data/PTConfigs/mac/torrc-defaults-appendix | 3 - .../mac/gitian-pluggable-transports.yml | 148 --------------------- gitian/descriptors/mac/gitian-utils.yml | 21 --- gitian/mkbundle-mac.sh | 18 +-- 4 files changed, 4 insertions(+), 186 deletions(-) diff --git a/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix b/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix index f5eb5e4..271b36c 100644 --- a/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix +++ b/Bundle-Data/PTConfigs/mac/torrc-defaults-appendix @@ -1,6 +1,3 @@ -## fteproxy configuration -ClientTransportPlugin fte exec PluggableTransports/fteproxy.bin --managed - ## obfs4proxy configuration ClientTransportPlugin obfs2,obfs3,obfs4,scramblesuit exec PluggableTransports/obfs4proxy diff --git a/gitian/descriptors/mac/gitian-pluggable-transports.yml b/gitian/descriptors/mac/gitian-pluggable-transports.yml index 8fc4df7..5f47b47 100644 --- a/gitian/descriptors/mac/gitian-pluggable-transports.yml +++ b/gitian/descriptors/mac/gitian-pluggable-transports.yml @@ -7,21 +7,9 @@ architectures: - "i386" packages: - "unzip" -- "python-setuptools" -- "python-dev" - "zip" reference_datetime: "2000-01-01 00:00:00" remotes: -- "url": "https://git.torproject.org/pluggable-transports/pyptlib.git" - "dir": "pyptlib" -- "url": "https://git.torproject.org/pluggable-transports/obfsproxy.git" - "dir": "obfsproxy" -- "url": "https://github.com/kpdyer/libfte.git" - "dir": "libfte" -- "url": "https://github.com/kpdyer/fteproxy.git" - "dir": "fteproxy" -- "url": "https://github.com/habnabit/txsocksx.git" - "dir": "txsocksx" - "url": "https://git.torproject.org/pluggable-transports/goptlib.git" "dir": "goptlib" - "url": "https://git.torproject.org/pluggable-transports/meek.git" @@ -38,20 +26,11 @@ remotes: "dir": "obfs4" files: - "versions" -- "pycrypto.tar.gz" -- "argparse.tar.gz" -- "pyyaml.tar.gz" -- "zope.interface.zip" -- "twisted.tar.bz2" -- "parsley.tar.gz" - "go14.tar.gz" - "go.tar.gz" -- "apple-uni-sdk-10.6_20110407-0.flosoft1_i386.deb" - "MacOSX10.7.sdk.tar.gz" - "multiarch-darwin11-cctools127.2-gcc42-5666.3-llvmgcc42-2336.1-Linux-120724.tar.xz" - "dzip.sh" -- "gmp-mac64-utils.zip" -- "openssl-mac64-utils.zip" script: | INSTDIR="$HOME/install" source versions @@ -60,13 +39,10 @@ script: | PTDIR="$TBDIR/Contents/MacOS/Tor/PluggableTransports" DOCSDIR="$TBDIR/Contents/Resources/TorBrowser/Docs" TORBINDIR="$TBDIR/Contents/MacOS/Tor" - # TODO: Fix packaging of fte so it can be included. See bug 18495. - INCLUDE_FTE=0 else PTDIR="$TBDIR/TorBrowser/Tor/PluggableTransports" DOCSDIR="$TBDIR/TorBrowser/Docs" TORBINDIR="$TBDIR/TorBrowser/Tor" - INCLUDE_FTE=1 fi export LIBRARY_PATH="$INSTDIR/lib" export REFERENCE_DATETIME @@ -77,29 +53,14 @@ script: | mkdir -p $PTDIR/ mkdir -p $OUTDIR/ # - # dpkg requires sbin directories in the PATH - export PATH="/usr/sbin:/sbin:$PATH" - sudo dpkg -i *.deb tar xaf multiarch-darwin*tar.xz # The 10.7 SDK is needed for Go: https://bugs.torproject.org/20023#comment:6 tar xaf MacOSX10.7.sdk.tar.gz export PATH="$PATH:$HOME/build/apple-osx/bin/" - if [ "z$INCLUDE_FTE" = "z1" ]; then - unzip -d $INSTDIR gmp-mac64-utils.zip - unzip -d $INSTDIR openssl-mac64-utils.zip - # FTE only needs libgmp.10.dylib and no libgmpxx anymore. - cp $INSTDIR/gmp/lib/libgmp.10.dylib $TORBINDIR/ - FTE_EXTRA_CFLAGS=" -I$INSTDIR/gmp/include -L$INSTDIR/gmp/lib" - else - FTE_EXTRA_CFLAGS="" - fi - # export AR=$HOME/build/apple-osx/bin/i686-apple-darwin11-ar export CC=$HOME/build/apple-osx/bin/i686-apple-darwin11-gcc export CXX=$HOME/build/apple-osx/bin/i686-apple-darwin11-g++ - # http://bugs.python.org/issue9437 - export LDSHARED="$CC -pthread -shared" # Building go 1.4.x # This is needed to bootstrap the go that we actually use @@ -134,115 +95,6 @@ script: | cd ../.. export PATH="$PATH:$PWD/go/bin" - if [ "z$INCLUDE_FTE" = "z1" ]; then - - # Building pyptlib - cd pyptlib - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - python setup.py build --build-lib build - cp -a build/pyptlib $PTDIR/ - cd .. - - # Building PyCrypto - tar xzf pycrypto.tar.gz - cd pycrypto-* - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - # https://bugs.launchpad.net/pycrypto/+bug/1096207 for ac_cv_func_malloc_0_nonnull. - ac_cv_func_malloc_0_nonnull=yes sh configure --host=i686-apple-darwin11 - LDSHARED="$LDSHARED -framework Python" python setup.py build_ext --build-lib build -I/usr/lib/apple/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/Python.framework/Headers - python setup.py build --build-lib build - cp -a build/Crypto $PTDIR/ - cd .. - - # Building argparse - tar xzf argparse.tar.gz - cd argparse-* - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - python setup.py build --build-lib build - cp -a build/argparse.py $PTDIR/ - cd .. - - # Building Zope interfaces - unzip zope.interface.zip - cd zope.interface-* - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - python setup.py build --build-lib build - cp -a build/zope $PTDIR/ - cd .. - - # Building PyYAML - tar xzf pyyaml.tar.gz - cd PyYAML-* - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - python setup.py build --build-lib build - cp -a build/yaml $PTDIR/ - cd .. - - # Building Twisted - tar xjf twisted.tar.bz2 - cd Twisted-* - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - LDSHARED="$LDSHARED -framework Python" python setup.py build_ext --build-lib build -I/usr/lib/apple/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/Python.framework/Headers - python setup.py build --build-lib build - cp -a build/twisted $PTDIR/ - cd .. - - # Building Parsley - tar xzf parsley.tar.gz - cd Parsley-* - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - python setup.py build --build-lib build - cp -a build/parsley.py $PTDIR/ - cp -a build/ometa $PTDIR/ - cp -a build/terml $PTDIR/ - cd .. - - # Building txsocksx - cd txsocksx - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - # Let's pretend we have the setup dependency already as we don't want to get - # it downloaded during building. Just pretending and thus avoiding another - # dependency should be fine here as txsocksx catches the exception due to - # missing __version__ and __sha__ . - mkdir vcversioner-1.14.1.1-py2.7.egg - python setup.py build --build-lib build - cp -a build/txsocksx $PTDIR/ - cd .. - - # Building obfsproxy - cd obfsproxy - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - python setup.py build --build-lib build - cp -a build/obfsproxy $PTDIR/ - cp -a bin/obfsproxy $PTDIR/obfsproxy.bin - mkdir -p $DOCSDIR/Obfsproxy - cp -a {LICENSE,README} $DOCSDIR/Obfsproxy - # XXX: Workaround for bug 11190 - sed 's/python2/python/' -i $PTDIR/obfsproxy.bin - cd .. - - # Building libfte - cd libfte - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - CFLAGS="-L$INSTDIR/Tor -I$INSTDIR/gmp/include $CFLAGS" CXXFLAGS="-L$INSTDIR/Tor -I$INSTDIR/gmp/include $CXXFLAGS" make - cp -ra fte $PTDIR/ - mkdir -p $DOCSDIR/libfte - cp -a {LICENSE,README.md} $DOCSDIR/libfte - cp -an thirdparty/re2/LICENSE $DOCSDIR/libfte/LICENSE.re2 - cd .. - - # Building fteproxy - cd fteproxy - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - cp -a bin/fteproxy $PTDIR/fteproxy.bin - cp -ra fteproxy $PTDIR/ - mkdir -p $DOCSDIR/fteproxy - cp -a {COPYING,README.md} $DOCSDIR/fteproxy - cd .. - - fi - # end of 'INCLUDE_FTE' section - # Building goptlib cd goptlib find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" diff --git a/gitian/descriptors/mac/gitian-utils.yml b/gitian/descriptors/mac/gitian-utils.yml index f54d5cb..55d6c7f 100644 --- a/gitian/descriptors/mac/gitian-utils.yml +++ b/gitian/descriptors/mac/gitian-utils.yml @@ -27,7 +27,6 @@ files: - "apple-uni-sdk-10.6_20110407-0.flosoft1_i386.deb" - "multiarch-darwin11-cctools127.2-gcc42-5666.3-llvmgcc42-2336.1-Linux-120724.tar.xz" - "openssl.tar.gz" -- "gmp.tar.bz2" - "versions" - "dzip.sh" - "libfaketime.patch" @@ -78,8 +77,6 @@ script: | export PATH="$PATH:$HOME/build/apple-osx/bin/" # For OpenSSL ln -s ~/build/apple-osx/bin/apple-osx-gcc ~/build/apple-osx/bin/i686-apple-darwin11-cc - #For gmp, need to trick it so it knows we're doing a 64 bit build - for i in ~/build/apple-osx/bin/i686-apple-darwin11-*; do j=`echo $i | sed 's/i686/x86_64/'`; ln -s $i $j; done; # XXX Clean up these flags? export CFLAGS="-m64 -I/usr/lib/apple/SDKs/MacOSX10.6.sdk/usr/include/ -I/usr/lib/gcc/i686-apple-darwin10/4.2.1/include/ -I. -L/usr/lib/apple/SDKs/MacOSX10.6.sdk/usr/lib/ -L/usr/lib/apple/SDKs/MacOSX10.6.sdk/usr/lib/system/ -mmacosx-version-min=10.5" @@ -109,27 +106,9 @@ script: | make install cd .. - # Building GMP - tar xjf gmp.tar.bz2 - cd gmp-* - # |configure| can't cope with nano seconds faked. And even if we would revert - # that feature it would hang sometimes for unknown but to libfaketime related - # reasons. - export LD_PRELOAD="" - find -type f -print0 | xargs -0 touch --date="$REFERENCE_DATETIME" - # Even if we are not shipping libgmpxx anymore we still need --enable-xcc - # during compile time. - ./configure --host=x86_64-apple-darwin11 --prefix=$INSTDIR/gmp --disable-static --enable-shared --enable-cxx - export LD_PRELOAD="$INSTDIR/faketime/usr/local/lib/faketime/libfaketime.so.1" - make - make install - cd .. - # Grabbing the results cd $INSTDIR ~/build/dzip.sh openssl-$OPENSSL_VER-mac64-utils.zip openssl ~/build/dzip.sh libevent-${LIBEVENT_TAG#release-}-mac64-utils.zip libevent - ~/build/dzip.sh gmp-$GMP_VER-mac64-utils.zip gmp - cp *utils.zip $OUTDIR/ fi diff --git a/gitian/mkbundle-mac.sh b/gitian/mkbundle-mac.sh index 3536553..789cc80 100755 --- a/gitian/mkbundle-mac.sh +++ b/gitian/mkbundle-mac.sh @@ -55,17 +55,10 @@ rm -f $GITIAN_DIR/inputs/tbb-docs.zip $WRAPPER_DIR/build-helpers/dzip.sh $GITIAN_DIR/inputs/tbb-docs.zip ./Docs/ rm -f $GITIAN_DIR/inputs/TorBrowser.app.meek-http-helper.zip (cd PTConfigs/mac && $WRAPPER_DIR/build-helpers/dzip.sh $GITIAN_DIR/inputs/TorBrowser.app.meek-http-helper.zip TorBrowser.app.meek-http-helper) -if [ "z$DATA_OUTSIDE_APP_DIR" = "z1" ]; then -# FTE is temporarily disabled due to bug 18495 and snowflake is Linux-only for +cp PTConfigs/mac/torrc-defaults-appendix $GITIAN_DIR/inputs/torrc-defaults-appendix-mac +# FTE is temporarily removed due to bug 18495 and snowflake is Linux-only for # now. - grep -v 'fteproxy' PTConfigs/mac/torrc-defaults-appendix > $GITIAN_DIR/inputs/torrc-defaults-appendix-mac grep -Ev 'default_bridge\.fte|default_bridge\.snowflake' PTConfigs/bridge_prefs.js > $GITIAN_DIR/inputs/bridge_prefs.js -else - cp PTConfigs/mac/torrc-defaults-appendix $GITIAN_DIR/inputs/torrc-defaults-appendix-mac - # We don't have snowflake available on macOS yet. - grep -v 'default_bridge\.snowflake' PTConfigs/bridge_prefs.js > $GITIAN_DIR/inputs/bridge_prefs.js - -fi cp PTConfigs/meek-http-helper-user.js $GITIAN_DIR/inputs/ cp mac-tor.sh $GITIAN_DIR/inputs/ @@ -144,8 +137,7 @@ cd $GITIAN_DIR if [ ! -f inputs/clang-$CLANG_VER-linux64-wheezy-utils.zip -o \ ! -f inputs/openssl-$OPENSSL_VER-mac64-utils.zip -o \ - ! -f inputs/libevent-${LIBEVENT_TAG_ORIG#release-}-mac64-utils.zip -o \ - ! -f inputs/gmp-$GMP_VER-mac64-utils.zip ]; + ! -f inputs/libevent-${LIBEVENT_TAG_ORIG#release-}-mac64-utils.zip ]; then echo echo "****** Starting Utilities Component of Mac Bundle (1/5 for Mac) ******" @@ -162,7 +154,6 @@ then ln -sf clang-$CLANG_VER-linux64-wheezy-utils.zip clang-linux64-wheezy-utils.zip ln -sf openssl-$OPENSSL_VER-mac64-utils.zip openssl-mac64-utils.zip ln -sf libevent-${LIBEVENT_TAG_ORIG#release-}-mac64-utils.zip libevent-mac64-utils.zip - ln -sf gmp-$GMP_VER-mac64-utils.zip gmp-mac64-utils.zip cd .. #cp -a result/utils-mac-res.yml inputs/ else @@ -176,7 +167,6 @@ else ln -sf clang-$CLANG_VER-linux64-wheezy-utils.zip clang-linux64-wheezy-utils.zip ln -sf openssl-$OPENSSL_VER-mac64-utils.zip openssl-mac64-utils.zip ln -sf libevent-${LIBEVENT_TAG_ORIG#release-}-mac64-utils.zip libevent-mac64-utils.zip - ln -sf gmp-$GMP_VER-mac64-utils.zip gmp-mac64-utils.zip cd .. fi @@ -229,7 +219,7 @@ then echo "****** Starting Pluggable Transports Component of Mac Bundle (4/5 for Mac) ******" echo - ./bin/gbuild -j $NUM_PROCS -m $VM_MEMORY --commit pyptlib=$PYPTLIB_TAG,obfsproxy=$OBFSPROXY_TAG,libfte=$LIBFTE_TAG,fteproxy=$FTEPROXY_TAG,txsocksx=$TXSOCKSX_TAG,goptlib=$GOPTLIB_TAG,meek=$MEEK_TAG,ed25519=$GOED25519_TAG,siphash=$GOSIPHASH_TAG,goxcrypto=$GO_X_CRYPTO_TAG,goxnet=$GO_X_NET_TAG,obfs4=$OBFS4_TAG $DESCRIPTOR_DIR/mac/gitian-pluggable-transports.yml + ./bin/gbuild -j $NUM_PROCS -m $VM_MEMORY --commit goptlib=$GOPTLIB_TAG,meek=$MEEK_TAG,ed25519=$GOED25519_TAG,siphash=$GOSIPHASH_TAG,goxcrypto=$GO_X_CRYPTO_TAG,goxnet=$GO_X_NET_TAG,obfs4=$OBFS4_TAG $DESCRIPTOR_DIR/mac/gitian-pluggable-transports.yml if [ $? -ne 0 ]; then #mv var/build.log ./firefox-fail-mac.log.`date +%Y%m%d%H%M%S`

1 0

[tor-browser-bundle/master] Bug 21348: Make snowflake only available on Linux for now
by gk＠torproject.org 31 Jan '17

31 Jan '17

commit ff0756e4778736cb2e05f260bf277c20bcd2d31a Author: Georg Koppen <gk(a)torproject.org> Date: Mon Jan 30 14:25:18 2017 +0000 Bug 21348: Make snowflake only available on Linux for now --- gitian/mkbundle-mac.sh | 9 ++++++--- gitian/mkbundle-windows.sh | 3 ++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/gitian/mkbundle-mac.sh b/gitian/mkbundle-mac.sh index 16140b1..3536553 100755 --- a/gitian/mkbundle-mac.sh +++ b/gitian/mkbundle-mac.sh @@ -56,12 +56,15 @@ $WRAPPER_DIR/build-helpers/dzip.sh $GITIAN_DIR/inputs/tbb-docs.zip ./Docs/ rm -f $GITIAN_DIR/inputs/TorBrowser.app.meek-http-helper.zip (cd PTConfigs/mac && $WRAPPER_DIR/build-helpers/dzip.sh $GITIAN_DIR/inputs/TorBrowser.app.meek-http-helper.zip TorBrowser.app.meek-http-helper) if [ "z$DATA_OUTSIDE_APP_DIR" = "z1" ]; then -# FTE is temporarily disabled due to bug 18495. +# FTE is temporarily disabled due to bug 18495 and snowflake is Linux-only for +# now. grep -v 'fteproxy' PTConfigs/mac/torrc-defaults-appendix > $GITIAN_DIR/inputs/torrc-defaults-appendix-mac - grep -v 'default_bridge\.fte' PTConfigs/bridge_prefs.js > $GITIAN_DIR/inputs/bridge_prefs.js + grep -Ev 'default_bridge\.fte|default_bridge\.snowflake' PTConfigs/bridge_prefs.js > $GITIAN_DIR/inputs/bridge_prefs.js else cp PTConfigs/mac/torrc-defaults-appendix $GITIAN_DIR/inputs/torrc-defaults-appendix-mac - cp PTConfigs/bridge_prefs.js $GITIAN_DIR/inputs/ + # We don't have snowflake available on macOS yet. + grep -v 'default_bridge\.snowflake' PTConfigs/bridge_prefs.js > $GITIAN_DIR/inputs/bridge_prefs.js + fi cp PTConfigs/meek-http-helper-user.js $GITIAN_DIR/inputs/ cp mac-tor.sh $GITIAN_DIR/inputs/ diff --git a/gitian/mkbundle-windows.sh b/gitian/mkbundle-windows.sh index 1afa840..6254937 100755 --- a/gitian/mkbundle-windows.sh +++ b/gitian/mkbundle-windows.sh @@ -55,7 +55,8 @@ cd $WRAPPER_DIR/../Bundle-Data/ rm -f $GITIAN_DIR/inputs/tbb-docs.zip $WRAPPER_DIR/build-helpers/dzip.sh $GITIAN_DIR/inputs/tbb-docs.zip ./Docs/ cp PTConfigs/windows/torrc-defaults-appendix $GITIAN_DIR/inputs/torrc-defaults-appendix-windows -cp PTConfigs/bridge_prefs.js $GITIAN_DIR/inputs/ +# We don't have snowflake available on Windows yet. +grep -v 'default_bridge\.snowflake' PTConfigs/bridge_prefs.js > $GITIAN_DIR/inputs/bridge_prefs.js cp PTConfigs/meek-http-helper-user.js $GITIAN_DIR/inputs/ cd windows

1 0

[atlas/master] Uses pre elements for fingerprints in details view, not tt (Fixes: #21350)
by irl＠torproject.org 31 Jan '17

31 Jan '17

commit 4627254904953d0ba9c02d0cf767472d10b78acc Author: Iain R. Learmonth <irl(a)fsfe.org> Date: Tue Jan 31 00:21:47 2017 +0000 Uses pre elements for fingerprints in details view, not tt (Fixes: #21350) Comment from cypherpunks: "It reverts the solution from #12685 because <pre> tags wrap their content automatically. However, using the <pre> tag changes the fingerprint to look like the fingerprints of the family members (except with black text instead of blue or orange). IMO this is preferable because it makes displaying fingerprints consistent but thought I'd mention it anyway." In testing, I found this to actually be a better solution for responsiveness as instead of wrapping at a fixed location, everything can still be viewable even with a 160px wide screen. This does lead to having different length lines when wrapping, and I guess user feedback will let us know if this was a dreadful idea, but I don't see it causing disasterous breakage for anyone. --- templates/details/router.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/templates/details/router.html b/templates/details/router.html index d94f5cf..f87001f 100644 --- a/templates/details/router.html +++ b/templates/details/router.html @@ -72,7 +72,7 @@ <% } %> <dt><span class="tip" data-content="20-byte unique identifier of the relay." data-original-title="Fingerprint">Fingerprint</span></dt> - <dd><tt><%= relay.get('fingerprint').replace(/(.{20})/g,"$1") %></tt></dd> + <dd><pre><%= relay.get('fingerprint') %></pre></dd> <dt><span class="tip" data-content="Flags that the directory authorities assigned to this relay." data-original-title="Relay flags">Flags</span></dt> <dd><% _.each(relay.get('flags'), function(flag) { %>

1 0

[atlas/master] Removes duplicate id attributes on SVG graphs (Fixes: #21351)
by irl＠torproject.org 31 Jan '17

31 Jan '17

commit 849712efdf623af8cfcde73bc0a44ad2b70a4e96 Author: Iain R. Learmonth <irl(a)fsfe.org> Date: Tue Jan 31 00:16:43 2017 +0000 Removes duplicate id attributes on SVG graphs (Fixes: #21351) Comment from cypherpunks: "A quick grep shows there is no CSS that uses the bw_month id. The id is only used by the JS code that generates the SVG graphs to know where to inject the graphs. This code also injects the duplicate id for some unknown reason. The inline patch fixes the issue by simply removing the injection of the duplicate id." I would agree with this comment. --- js/views/details/main.js | 1 - 1 file changed, 1 deletion(-) diff --git a/js/views/details/main.js b/js/views/details/main.js index e73ac4a..97846be 100644 --- a/js/views/details/main.js +++ b/js/views/details/main.js @@ -32,7 +32,6 @@ define([ width = 550 - margin.left - margin.right, height = 342 - margin.top - margin.bottom; var svg = d3.select("#" + g).append("svg:svg") - .attr("id", g) .attr("version", 1.1) .attr("xmlns", "http://www.w3.org/2000/svg") .attr("width", width + margin.left + margin.right)

1 0

[stem/master] Speed up auth's test_all_use_cases unit test
by atagar＠torproject.org 30 Jan '17

30 Jan '17

commit 9fa91542525e021ea11f7889293fae35d85a7f8c Author: Damian Johnson <atagar(a)torproject.org> Date: Thu Jan 5 10:43:01 2017 -0800 Speed up auth's test_all_use_cases unit test This test isn't overly burdensome and it's neat that it tests all combinations, but this one unit test alone takes around as much as every other test put together. We can speed it up without losing coverage by having it raise some parent exceptions rather than each child exception individually. On my 'lil netbook this drops the test runtime from 5s to 1s (80% faster). --- test/unit/connection/authentication.py | 28 +++++++++------------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/test/unit/connection/authentication.py b/test/unit/connection/authentication.py index b12efeb..70de6d7 100644 --- a/test/unit/connection/authentication.py +++ b/test/unit/connection/authentication.py @@ -65,35 +65,25 @@ class TestAuthenticate(unittest.TestCase): # exceptions that the authentication functions are documented to raise - all_auth_none_exc = (None, stem.connection.OpenAuthRejected(None)) + all_auth_none_exc = ( + None, + stem.connection.OpenAuthRejected(None), + stem.ControllerError(None)) all_auth_password_exc = ( None, stem.connection.PasswordAuthRejected(None), - stem.connection.IncorrectPassword(None)) + stem.connection.IncorrectPassword(None), + stem.ControllerError(None)) all_auth_cookie_exc = ( None, - stem.connection.IncorrectCookieSize(None, False, None), - stem.connection.UnreadableCookieFile(None, False, None), stem.connection.CookieAuthRejected(None, False, None), stem.connection.IncorrectCookieValue(None, False, None), - stem.connection.UnrecognizedAuthChallengeMethod(None, None, None), + stem.connection.IncorrectCookieSize(None, False, None), + stem.connection.UnreadableCookieFile(None, False, None), stem.connection.AuthChallengeFailed(None, None), - stem.connection.AuthSecurityFailure(None, None), - stem.connection.InvalidClientNonce(None, None)) - - # authentication functions might raise a controller error when - # 'suppress_ctl_errors' is False, so including those - - control_exc = ( - stem.ProtocolError(None), - stem.SocketError(None), - stem.SocketClosed(None)) - - all_auth_none_exc += control_exc - all_auth_password_exc += control_exc - all_auth_cookie_exc += control_exc + stem.ControllerError(None)) auth_method_combinations = mocking.get_all_combinations([ stem.connection.AuthMethod.NONE,

1 0

[stem/master] Speed up base controller's test_asynchronous_event_handling
by atagar＠torproject.org 30 Jan '17

30 Jan '17

commit 921955e540cb9ca6018129ab7eea1f3747ff677e Author: Damian Johnson <atagar(a)torproject.org> Date: Thu Jan 19 10:41:51 2017 -0800 Speed up base controller's test_asynchronous_event_handling Replacing BW events with triggered events so we can speed this up by three seconds. --- test/integ/control/base_controller.py | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/test/integ/control/base_controller.py b/test/integ/control/base_controller.py index 48dd3df..5beffc2 100644 --- a/test/integ/control/base_controller.py +++ b/test/integ/control/base_controller.py @@ -2,6 +2,8 @@ Integration tests for the stem.control.BaseController class. """ +import hashlib +import os import re import threading import time @@ -150,15 +152,11 @@ class TestBaseController(unittest.TestCase): with test.runner.get_runner().get_tor_socket() as control_socket: controller = ControlledListener(control_socket) - controller.msg('SETEVENTS BW') + controller.msg('SETEVENTS CONF_CHANGED') - # Wait for a couple events for events to be enqueued. Doing a bunch of - # GETINFO queries while waiting to better exercise the asynchronous event - # handling. - - start_time = time.time() - - while (time.time() - start_time) < 3: + for i in range(10): + random_fingerprint = hashlib.sha1(os.urandom(20)).hexdigest().upper() + controller.msg('SETCONF NodeFamily=%s' % random_fingerprint) test.runner.exercise_controller(self, controller) # Concurrently shut down the controller. We need to do this in another @@ -177,10 +175,10 @@ class TestBaseController(unittest.TestCase): self.assertTrue(len(controller.received_events) >= 2) - for bw_event in controller.received_events: - self.assertTrue(re.match('BW [0-9]+ [0-9]+', str(bw_event))) - self.assertTrue(re.match('650 BW [0-9]+ [0-9]+\r\n', bw_event.raw_content())) - self.assertEqual(('650', ' '), bw_event.content()[0][:2]) + for conf_changed_event in controller.received_events: + self.assertTrue(re.match('CONF_CHANGED\nNodeFamily=*', str(conf_changed_event))) + self.assertTrue(conf_changed_event.raw_content().startswith('650-CONF_CHANGED\r\n650-NodeFamily=')) + self.assertEqual(('650', '-'), conf_changed_event.content()[0][:2]) @require_controller def test_get_latest_heartbeat(self):

1 0

[stem/master] Lower work done by couple BaseController tests
by atagar＠torproject.org 30 Jan '17

30 Jan '17

commit 30ea552a92346cc733bcac82e7d16cdc8ac1efa8 Author: Damian Johnson <atagar(a)torproject.org> Date: Fri Jan 20 10:42:28 2017 -0800 Lower work done by couple BaseController tests Reducing the amount our test_connect_repeatedly and test_msg_repeatedly tests hammer our controller so the tests don't take so long (drops from 1.6s to 0.3s on my netbook). --- test/integ/control/base_controller.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/test/integ/control/base_controller.py b/test/integ/control/base_controller.py index 5beffc2..2507293 100644 --- a/test/integ/control/base_controller.py +++ b/test/integ/control/base_controller.py @@ -54,7 +54,7 @@ class TestBaseController(unittest.TestCase): with test.runner.get_runner().get_tor_socket() as control_socket: controller = stem.control.BaseController(control_socket) - for _ in range(250): + for _ in range(50): controller.connect() controller.close() @@ -105,7 +105,7 @@ class TestBaseController(unittest.TestCase): controller = stem.control.BaseController(control_socket) def run_getinfo(): - for _ in range(150): + for _ in range(50): try: controller.msg('GETINFO version') controller.msg('GETINFO blarg') @@ -121,7 +121,7 @@ class TestBaseController(unittest.TestCase): msg_thread.setDaemon(True) msg_thread.start() - for index in range(100): + for index in range(50): controller.connect() controller.close()

1 0