April 2018 - tor-commits - lists.torproject.org

[tor-browser/tor-browser-52.7.3esr-8.0-1] Bug 20283: Tor Browser should run without a `/proc` filesystem.
by gk＠torproject.org 16 Apr '18

16 Apr '18

commit 90e16dd25b6eb199c016fc8b5e9478a3454d36e1 Author: Richard Pospesel <richard(a)torproject.org> Date: Mon Apr 9 11:40:32 2018 -0700 Bug 20283: Tor Browser should run without a `/proc` filesystem. Firefox uses the current stack frame address and the stack size as a sort of heuristic for various things in the javascript engine. The js::GetNativeStackBaseImpl() function is used to get the base stack address (ie the address from which the stack grows, so this can be either the first or last memory address of the stack memory space depending on the CPU architecture). On Linux, this function is implemented using the pthreads APIs. For non-main threads, the queried thread info is stored in memory. The main thread does not have this information on hand, so it gets the stack memory range via the /proc/self/maps file ( see glibc's pthread_get_attr_np.c ). Fortunately (per discussions with the firefox devs in #jsapi) the base address only needs to be approximation. In reality, environment variables, args, and other things are stored in space between the end/beginning of the mapped stack memory and the 'top' of the stack space used by stack frames. We can get the top of this usable stack from __libc_stack_end, which is a void* set by glibc during program initialization. Non-main threads still get their stack-base through the usual pthreads APIs. This patch integrates the latest jsnativestack.cpp from mozilla- central, and creates a specific implementation of js::GetNativeStackBaseImpl() for non-android Linux using the described __libc_stack_end read. --- js/src/jsnativestack.cpp | 88 +++++++++++++++++++++++++++++------------------- 1 file changed, 54 insertions(+), 34 deletions(-) diff --git a/js/src/jsnativestack.cpp b/js/src/jsnativestack.cpp index 05928ea3dff3..0f4dae156c2b 100644 --- a/js/src/jsnativestack.cpp +++ b/js/src/jsnativestack.cpp @@ -8,22 +8,17 @@ #ifdef XP_WIN # include "jswin.h" - #elif defined(XP_DARWIN) || defined(DARWIN) || defined(XP_UNIX) # include <pthread.h> - # if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__DragonFly__) # include <pthread_np.h> # endif - -# if defined(ANDROID) +# if defined(ANDROID) && !defined(__aarch64__) # include <sys/types.h> # include <unistd.h> # endif - #else # error "Unsupported platform" - #endif #if defined(XP_WIN) @@ -31,32 +26,8 @@ void* js::GetNativeStackBaseImpl() { -# if defined(_M_IX86) && defined(_MSC_VER) - /* - * offset 0x18 from the FS segment register gives a pointer to - * the thread information block for the current thread - */ - NT_TIB* pTib; - __asm { - MOV EAX, FS:[18h] - MOV pTib, EAX - } - return static_cast<void*>(pTib->StackBase); - -# elif defined(_M_X64) - PNT_TIB64 pTib = reinterpret_cast<PNT_TIB64>(NtCurrentTeb()); - return reinterpret_cast<void*>(pTib->StackBase); - -# elif defined(_M_ARM) PNT_TIB pTib = reinterpret_cast<PNT_TIB>(NtCurrentTeb()); return static_cast<void*>(pTib->StackBase); - -# elif defined(_WIN32) && defined(__GNUC__) - NT_TIB* pTib; - asm ("movl %%fs:0x18, %0\n" : "=r" (pTib)); - return static_cast<void*>(pTib->StackBase); - -# endif } #elif defined(SOLARIS) @@ -88,6 +59,55 @@ js::GetNativeStackBaseImpl() context.uc_stack.ss_size; } +#elif defined(XP_LINUX) && !defined(ANDROID) + +# include <dlfcn.h> +# include <sys/syscall.h> +static pid_t +gettid() +{ + return syscall(__NR_gettid); +} + +void* +js::GetNativeStackBaseImpl() +{ + // main thread, get stack base from __libc_stack_end rather than pthread APIs + // to avoid filesystem calls /proc/self/maps + if(gettid() == getpid()) { + void** pLibcStackEnd = (void**)dlsym(RTLD_DEFAULT, "__libc_stack_end"); + // if __libc_stack_end is not found, architecture specific frame pointer hopping will need + // to be implemented + MOZ_ASSERT(pLibcStackEnd); + void* stackBase = *pLibcStackEnd; + MOZ_ASSERT(stackBase); + // we don't need to fix stackBase, as it already roughly points to beginning of the stack + return stackBase; + } + // non-main threads have the required info stored in memory, no filesystem calls are made + else { + pthread_t thread = pthread_self(); + pthread_attr_t sattr; + pthread_attr_init(&sattr); + pthread_getattr_np(thread, &sattr); + // stackBase will be the lowest address on all architectures + void* stackBase = nullptr; + size_t stackSize = 0; + int rc = pthread_attr_getstack(&sattr, &stackBase, &stackSize); + if(rc) { + MOZ_CRASH(); + } + MOZ_ASSERT(stackBase); + pthread_attr_destroy(&sattr); + +# if JS_STACK_GROWTH_DIRECTION > 0 + return stackBase; +# else + return static_cast<char*>(stackBase) + stackSize; +# endif + } +} + #else /* XP_UNIX */ void* @@ -120,11 +140,11 @@ js::GetNativeStackBaseImpl() rc = pthread_stackseg_np(pthread_self(), &ss); stackBase = (void*)((size_t) ss.ss_sp - ss.ss_size); stackSize = ss.ss_size; -# elif defined(ANDROID) +# elif defined(ANDROID) && !defined(__aarch64__) if (gettid() == getpid()) { - // bionic's pthread_attr_getstack doesn't tell the truth for the main - // thread (see bug 846670). So we scan /proc/self/maps to find the - // segment which contains the stack. + // bionic's pthread_attr_getstack prior to API 21 doesn't tell the truth + // for the main thread (see bug 846670). So we scan /proc/self/maps to + // find the segment which contains the stack. rc = -1; // Put the string on the stack, otherwise there is the danger that it

1 0

[translation/liveusb-creator_completed] Update translations for liveusb-creator_completed
by translation＠torproject.org 16 Apr '18

16 Apr '18

commit 56aa1ad79811dc5672c6f8428eef820301515131 Author: Translation commit bot <translation(a)torproject.org> Date: Mon Apr 16 09:45:46 2018 +0000 Update translations for liveusb-creator_completed --- de/de.po | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/de/de.po b/de/de.po index 9be5e7214..326060d04 100644 --- a/de/de.po +++ b/de/de.po @@ -31,7 +31,8 @@ # Sacro <Scion(a)T-Online.de>, 2013 # Sacro <Scion(a)T-Online.de>, 2012 # Sebastian Fiele <rawsta(a)rawsta.de>, 2017 -# spriver <spriver(a)autistici.org>, 2015 +# spriver, 2018 +# spriver, 2015 # stefanf <inactive+stefanf(a)transifex.com>, 2015 # Tobias Bannert <tobannert(a)gmail.com>, 2013 # Tobias Bannert <tobannert(a)gmail.com>, 2013-2014 @@ -44,8 +45,8 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2017-11-10 15:57+0100\n" -"PO-Revision-Date: 2018-04-12 18:59+0000\n" -"Last-Translator: IDRASSI Mounir <mounir.idrassi(a)idrix.fr>\n" +"PO-Revision-Date: 2018-04-16 09:45+0000\n" +"Last-Translator: spriver\n" "Language-Team: German (http://www.transifex.com/otf/torproject/language/de/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -381,7 +382,7 @@ msgid "" "The USB stick \"%(pretty_name)s\" is configured as non-removable by its " "manufacturer and Tails will fail to start on it. Please try installing on a " "different model." -msgstr "Der USB-Stick \"%(pretty_name)s\" wurde vom Herrsteller als nicht entfernbar konfiguriert und somit würde Tails davon nicht starten. Bitte versuch es auf einem anderen Moddel zu installiern." +msgstr "Der USB-Stick \"%(pretty_name)s\" wurde vom Hersteller als nicht entfernbar konfiguriert und somit würde Tails nicht davon starten. Bitte versuchen Sie es auf einem anderen Moddel zu installieren." #: ../tails_installer/gui.py:567 #, python-format

1 0

[translation/liveusb-creator] Update translations for liveusb-creator
by translation＠torproject.org 16 Apr '18

16 Apr '18

commit 3aafe00e09b24509a722a2e4529ee5719bcd594a Author: Translation commit bot <translation(a)torproject.org> Date: Mon Apr 16 09:45:41 2018 +0000 Update translations for liveusb-creator --- de/de.po | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/de/de.po b/de/de.po index 9be5e7214..326060d04 100644 --- a/de/de.po +++ b/de/de.po @@ -31,7 +31,8 @@ # Sacro <Scion(a)T-Online.de>, 2013 # Sacro <Scion(a)T-Online.de>, 2012 # Sebastian Fiele <rawsta(a)rawsta.de>, 2017 -# spriver <spriver(a)autistici.org>, 2015 +# spriver, 2018 +# spriver, 2015 # stefanf <inactive+stefanf(a)transifex.com>, 2015 # Tobias Bannert <tobannert(a)gmail.com>, 2013 # Tobias Bannert <tobannert(a)gmail.com>, 2013-2014 @@ -44,8 +45,8 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: \n" "POT-Creation-Date: 2017-11-10 15:57+0100\n" -"PO-Revision-Date: 2018-04-12 18:59+0000\n" -"Last-Translator: IDRASSI Mounir <mounir.idrassi(a)idrix.fr>\n" +"PO-Revision-Date: 2018-04-16 09:45+0000\n" +"Last-Translator: spriver\n" "Language-Team: German (http://www.transifex.com/otf/torproject/language/de/)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" @@ -381,7 +382,7 @@ msgid "" "The USB stick \"%(pretty_name)s\" is configured as non-removable by its " "manufacturer and Tails will fail to start on it. Please try installing on a " "different model." -msgstr "Der USB-Stick \"%(pretty_name)s\" wurde vom Herrsteller als nicht entfernbar konfiguriert und somit würde Tails davon nicht starten. Bitte versuch es auf einem anderen Moddel zu installiern." +msgstr "Der USB-Stick \"%(pretty_name)s\" wurde vom Hersteller als nicht entfernbar konfiguriert und somit würde Tails nicht davon starten. Bitte versuchen Sie es auf einem anderen Moddel zu installieren." #: ../tails_installer/gui.py:567 #, python-format

1 0

[metrics-lib/master] Create ServerDescriptor#isHiddenServiceDir.
by karsten＠torproject.org 16 Apr '18

16 Apr '18

commit 368deb83efd2e8492070593d10268a1a3e2ff397 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Apr 4 22:04:03 2018 +0200 Create ServerDescriptor#isHiddenServiceDir. Turns out that Tor has never supported versions in the hidden-service-dir descriptor line. This commit replaces ServerDescriptor#getHiddenServiceDirVersions with ServerDescriptor#isHiddenServiceDir. Fixes #25712. --- CHANGELOG.md | 7 +++++- .../torproject/descriptor/ServerDescriptor.java | 10 ++++++++ .../descriptor/impl/ServerDescriptorImpl.java | 29 ++++++++-------------- .../descriptor/impl/ServerDescriptorImplTest.java | 11 +++----- 4 files changed, 31 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 58235e3..d64d58d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,11 @@ # Changes in version 2.3.0 - 2018-0?-?? - * Minor changes. + * Medium changes + - Replace ServerDescriptor#getHiddenServiceDirVersions with + ServerDescriptor#isHiddenServiceDir, because Tor has never + supported versions in the hidden-service-dir descriptor line. + + * Minor changes - Override logLines() method from LogDescriptor in WebServerAccessLog. - Use 1-minute connect and read timeouts for fetching CollecTor's diff --git a/src/main/java/org/torproject/descriptor/ServerDescriptor.java b/src/main/java/org/torproject/descriptor/ServerDescriptor.java index ac9c645..1b91ec8 100644 --- a/src/main/java/org/torproject/descriptor/ServerDescriptor.java +++ b/src/main/java/org/torproject/descriptor/ServerDescriptor.java @@ -318,11 +318,21 @@ public interface ServerDescriptor extends Descriptor { * this server stores and serves, or null if it doesn't store and serve * any hidden service descriptors. * + * @deprecated Replaced with {@link #isHiddenServiceDir}, because Tor has + * never supported versions in the hidden-service-dir descriptor line. + * * @since 1.0.0 */ public List<Integer> getHiddenServiceDirVersions(); /** + * Return whether this server stores and serves hidden service descriptors. + * + * @since 2.3.0 + */ + public boolean isHiddenServiceDir(); + + /** * Return the list of link protocol versions that this server * supports. * diff --git a/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java index ca79cd2..9c9d47a 100644 --- a/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java +++ b/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java @@ -10,6 +10,7 @@ import org.torproject.descriptor.ServerDescriptor; import java.io.File; import java.util.ArrayList; import java.util.Arrays; +import java.util.Collections; import java.util.EnumSet; import java.util.List; import java.util.Scanner; @@ -462,21 +463,8 @@ public abstract class ServerDescriptorImpl extends DescriptorImpl } private void parseHiddenServiceDirLine(String line, String lineNoOpt, - String[] partsNoOpt) throws DescriptorParseException { - if (partsNoOpt.length == 1) { - this.hiddenServiceDirVersions = new Integer[] { 2 }; - } else { - try { - Integer[] result = new Integer[partsNoOpt.length - 1]; - for (int i = 1; i < partsNoOpt.length; i++) { - result[i - 1] = Integer.parseInt(partsNoOpt[i]); - } - this.hiddenServiceDirVersions = result; - } catch (NumberFormatException e) { - throw new DescriptorParseException("Illegal value in line '" - + line + "'."); - } - } + String[] partsNoOpt) { + this.hiddenServiceDir = true; } private void parseProtocolsLine(String line, String lineNoOpt, @@ -854,12 +842,17 @@ public abstract class ServerDescriptorImpl extends DescriptorImpl return this.extraInfoDigestSha256; } - private Integer[] hiddenServiceDirVersions; + private boolean hiddenServiceDir; + + @Override + public boolean isHiddenServiceDir() { + return this.hiddenServiceDir; + } @Override + @SuppressWarnings("deprecation") public List<Integer> getHiddenServiceDirVersions() { - return this.hiddenServiceDirVersions == null ? null - : Arrays.asList(this.hiddenServiceDirVersions); + return this.hiddenServiceDir ? null : Collections.singletonList(2); } private Integer[] linkProtocolVersions; diff --git a/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java b/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java index f121cb3..b81c3fb 100644 --- a/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java +++ b/src/test/java/org/torproject/descriptor/impl/ServerDescriptorImplTest.java @@ -482,8 +482,7 @@ public class ServerDescriptorImplTest { assertEquals(53470, (int) descriptor.getBandwidthObserved()); assertEquals("1469D1550738A25B1E7B47CDDBCD7B2899F51B74", descriptor.getExtraInfoDigestSha1Hex()); - assertEquals(Arrays.asList(new Integer[] {2}), - descriptor.getHiddenServiceDirVersions()); + assertTrue(descriptor.isHiddenServiceDir()); assertEquals("Random Person <nobody AT example dot com>", descriptor.getContact()); assertEquals(Arrays.asList(new String[] {"reject *:*"}), @@ -1014,7 +1013,7 @@ public class ServerDescriptorImplTest { throws DescriptorParseException { ServerDescriptor descriptor = DescriptorBuilder .createWithHiddenServiceDirLine(null); - assertNull(descriptor.getHiddenServiceDirVersions()); + assertFalse(descriptor.isHiddenServiceDir()); } @Test @@ -1022,8 +1021,7 @@ public class ServerDescriptorImplTest { throws DescriptorParseException { ServerDescriptor descriptor = DescriptorBuilder .createWithHiddenServiceDirLine("hidden-service-dir"); - assertEquals(Arrays.asList(new Integer[] {2}), - descriptor.getHiddenServiceDirVersions()); + assertTrue(descriptor.isHiddenServiceDir()); } @Test @@ -1031,8 +1029,7 @@ public class ServerDescriptorImplTest { throws DescriptorParseException { ServerDescriptor descriptor = DescriptorBuilder .createWithHiddenServiceDirLine("hidden-service-dir 2 3"); - assertEquals(Arrays.asList(new Integer[] {2, 3}), - descriptor.getHiddenServiceDirVersions()); + assertTrue(descriptor.isHiddenServiceDir()); } @Test

1 0

[onionoo/master] Make war-task depend on jar-task.
by karsten＠torproject.org 16 Apr '18

16 Apr '18

commit 59eb04b1df1e0e885566188be25152117f24617d Author: iwakeh <iwakeh(a)torproject.org> Date: Fri Apr 6 09:53:35 2018 +0000 Make war-task depend on jar-task. This makes the git-revision available even when running 'ant clean war'. --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index f6b7e2f..9c5ea3e 100644 --- a/build.xml +++ b/build.xml @@ -74,7 +74,7 @@ </patternset> <target name="war" - depends="compile,gitrev"> + depends="jar"> <war destfile="${dist}/${warfile}" webxml="${resources}/web.xml" basedir="${classes}">

1 0

[metrics-web/master] Make war-task depend on jar-task.
by karsten＠torproject.org 16 Apr '18

16 Apr '18

commit b1e05125fa3442e17cf78994e0818ba540b38571 Author: iwakeh <iwakeh(a)torproject.org> Date: Fri Apr 6 09:02:03 2018 +0000 Make war-task depend on jar-task. This makes the git-revision available. --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 17160ee..9b29452 100644 --- a/build.xml +++ b/build.xml @@ -137,7 +137,7 @@  <target name="war" - depends="submoduleupdate,compile"> + depends="submoduleupdate,jar"> <echo message="Removing problematic service definitions from"/> <echo message="${libs}/jetty9-apache-jsp${jetty.version}.jar and using "/> <echo message="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar below." />

1 0

[snowflake/master] Link to Snowflake wiki page rather than mailing list post.
by dcf＠torproject.org 16 Apr '18

16 Apr '18

commit 9f1c7b12c8a85179f059005c28810899234a06ae Author: David Fifield <david(a)bamsoftware.com> Date: Sun Apr 15 17:36:20 2018 -0700 Link to Snowflake wiki page rather than mailing list post. The wiki page in turn links to the post. --- proxy/static/index.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proxy/static/index.html b/proxy/static/index.html index e454086..c78004f 100644 --- a/proxy/static/index.html +++ b/proxy/static/index.html @@ -43,7 +43,7 @@ and should not yet be relied upon!! </p> See -<a href="https://lists.torproject.org/pipermail/tor-dev/2016-January/010310.html" +<a href="https://trac.torproject.org/projects/tor/wiki/doc/Snowflake" target="_blank"> here</a> for more information.

1 0

[snowflake/master] Declare utf-8 in index.html.
by dcf＠torproject.org 16 Apr '18

16 Apr '18

commit 933c117a484ef0365cc9aff5626520b20b8b8f36 Author: David Fifield <david(a)bamsoftware.com> Date: Sun Apr 15 17:31:47 2018 -0700 Declare utf-8 in index.html. --- proxy/static/index.html | 1 + 1 file changed, 1 insertion(+) diff --git a/proxy/static/index.html b/proxy/static/index.html index 5767593..e454086 100644 --- a/proxy/static/index.html +++ b/proxy/static/index.html @@ -3,6 +3,7 @@ <head> <title>Snowflake</title> +<meta content="text/html;charset=utf-8" http-equiv="Content-Type"> <style> body { text-align: center; margin: auto;

1 0

[snowflake/master] Update "last updated" date; remove "~Serene" signoff.
by dcf＠torproject.org 16 Apr '18

16 Apr '18

commit 5a1dffccebd42dd6dea11b793cccba3caa0bcb72 Author: David Fifield <david(a)bamsoftware.com> Date: Sun Apr 15 17:24:49 2018 -0700 Update "last updated" date; remove "~Serene" signoff. --- proxy/static/index.html | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/proxy/static/index.html b/proxy/static/index.html index ebd6a79..5767593 100644 --- a/proxy/static/index.html +++ b/proxy/static/index.html @@ -71,9 +71,7 @@ href="https://gitweb.torproject.org/pluggable-transports/snowflake.git/"> gitweb.torproject.org/pluggable-transports/snowflake.git </a> <br/> -Last update: 2016-02-10 -<br/> -~Serene +Last update: <time>2018-04-16</time> </body> <html>

1 0

[snowflake/master] Add <title> to index.html and snowflake.html.
by dcf＠torproject.org 16 Apr '18

16 Apr '18

commit bcef2ee7fb9d155e4655eed3f8ef187f6be878ae Author: David Fifield <david(a)bamsoftware.com> Date: Sun Apr 15 17:22:25 2018 -0700 Add <title> to index.html and snowflake.html. --- proxy/static/index.html | 1 + proxy/static/snowflake.html | 1 + 2 files changed, 2 insertions(+) diff --git a/proxy/static/index.html b/proxy/static/index.html index f0beced..ebd6a79 100644 --- a/proxy/static/index.html +++ b/proxy/static/index.html @@ -2,6 +2,7 @@ <html> <head> +<title>Snowflake</title> <style> body { text-align: center; margin: auto; diff --git a/proxy/static/snowflake.html b/proxy/static/snowflake.html index c2bcef5..3a1cdb9 100644 --- a/proxy/static/snowflake.html +++ b/proxy/static/snowflake.html @@ -1,6 +1,7 @@ <!doctype html> <html> <head> + <title>Snowflake</title> <meta content="text/html;charset=utf-8" http-equiv="Content-Type"> <script type="text/javascript" src="modernizr.js"></script> <script type="text/javascript" src="snowflake.js"></script>

1 0