August 2016 - tor-commits - lists.torproject.org

[collector/master] Removed the obsolete 'Ignore' annotations from ConfigurationTest.
by karsten＠torproject.org 11 Aug '16

11 Aug '16

commit 9bf140110be14aa302cfbebfe82787c355ed6dcc Author: iwakeh <iwakeh(a)torproject.org> Date: Thu Aug 11 14:07:48 2016 +0200 Removed the obsolete 'Ignore' annotations from ConfigurationTest. --- src/test/java/org/torproject/collector/conf/ConfigurationTest.java | 3 --- 1 file changed, 3 deletions(-) diff --git a/src/test/java/org/torproject/collector/conf/ConfigurationTest.java b/src/test/java/org/torproject/collector/conf/ConfigurationTest.java index 2f727aa..45ab5ed 100644 --- a/src/test/java/org/torproject/collector/conf/ConfigurationTest.java +++ b/src/test/java/org/torproject/collector/conf/ConfigurationTest.java @@ -11,7 +11,6 @@ import org.torproject.collector.MainTest; import org.torproject.collector.cron.CollecTorMain; import org.torproject.collector.cron.Dummy; -import org.junit.Ignore; import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; @@ -173,7 +172,6 @@ public class ConfigurationTest { conf.setWatchableSourceAndLoad(Paths.get("/tmp/phantom.path")); } - @Ignore("Test fails: Update was not called.") @Test() public void testConfigChange() throws Exception { Configuration conf = new Configuration(); @@ -194,7 +192,6 @@ public class ConfigurationTest { assertFalse("Update was called.", called.get()); } - @Ignore("Test fails: expected:<0> but was:<2>") @Test() public void testConfigUnreadable() throws Exception { Configuration conf = new Configuration();

1 0

[collector/master] Avoid code duplication in SchedulerTest, i.e. use method from MainTest instead.
by karsten＠torproject.org 11 Aug '16

11 Aug '16

commit 2fc5e232c1fefcfe9449c0b03dcbd3dd97294f11 Author: iwakeh <iwakeh(a)torproject.org> Date: Thu Aug 11 14:08:39 2016 +0200 Avoid code duplication in SchedulerTest, i.e. use method from MainTest instead. --- src/test/java/org/torproject/collector/cron/SchedulerTest.java | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/test/java/org/torproject/collector/cron/SchedulerTest.java b/src/test/java/org/torproject/collector/cron/SchedulerTest.java index 6314a8d..cec182a 100644 --- a/src/test/java/org/torproject/collector/cron/SchedulerTest.java +++ b/src/test/java/org/torproject/collector/cron/SchedulerTest.java @@ -5,6 +5,7 @@ package org.torproject.collector.cron; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; +import org.torproject.collector.MainTest; import org.torproject.collector.conf.Configuration; import org.torproject.collector.conf.Key; @@ -97,9 +98,7 @@ public class SchedulerTest { schedulerField.get(Scheduler.getInstance()); Scheduler.getInstance().scheduleModuleRuns(ctms, conf); long sysNow = System.currentTimeMillis(); - while (System.currentTimeMillis() - sysNow < 180_000) { - try { Thread.sleep(10_000);} catch (Exception e) { /* ignored */ } - } + MainTest.waitSec(180); assertEquals(15, Broken.count.intValue()); } }

1 0

[collector/master] Added example for changing the logging configuration.
by karsten＠torproject.org 11 Aug '16

11 Aug '16

commit ae785f0a3663118991dba06ace80124f5e1bbe9f Author: iwakeh <iwakeh(a)torproject.org> Date: Thu Aug 11 14:06:58 2016 +0200 Added example for changing the logging configuration. --- INSTALL.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/INSTALL.md b/INSTALL.md index e941759..1b0920e 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -74,3 +74,17 @@ to watch out for WARNING level and especially ERROR level log statements. These often will point to problems you can do something about, e.g. a full disk or missing file system permissions. +Logging Configuration +--------------------- + +Some hints for those who are familiar with Logback: + +If you want to use your own logging configuration for Logback you can simply +create your own logback.xml or logback.groovy and start CollecTor in the +following way: + +java -cp /folder/with/logback:collector-1.0.0.jar org.torproject.collector.Main + </place/of/collector.properties> + +The default configuration can be found in the tar-ball you downloaded, in +the subdirectory collector-1.0.0/src/main/resources. \ No newline at end of file

1 0

[tor/master] Add some how-to documentation for ht.h. Closes #19896
by nickm＠torproject.org 11 Aug '16

11 Aug '16

commit d5c47f043665d4513bad10e3220dc9b5f088c543 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Aug 11 10:18:51 2016 -0400 Add some how-to documentation for ht.h. Closes #19896 --- src/ext/ht.h | 90 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/src/ext/ht.h b/src/ext/ht.h index 1b6cbe6..a441d0b 100644 --- a/src/ext/ht.h +++ b/src/ext/ht.h @@ -5,6 +5,96 @@ /* Based on ideas by Christopher Clark and interfaces from Niels Provos. */ +/* + These macros provide an intrustive implementation for a typesafe chaining + hash table, loosely based on the BSD tree.h and queue.h macros. Here's + how to use them. + + First, pick a the structure that you'll be storing in the hashtable. Let's + say that's "struct dinosaur". To this structure, you add an HT_ENTRY() + member, as such: + + struct dinosaur { + HT_ENTRY(dinosaur) node; // The name inside the () must match the + // struct. + + // These are just fields from the dinosaur structure... + long dinosaur_id; + char *name; + long age; + int is_ornithischian; + int is_herbivorous; + }; + + You can declare the hashtable itself as: + + HT_HEAD(dinosaur_ht, dinosaur); + + This declares a new 'struct dinosaur_ht' type. + + Now you need to declare two functions to help implement the hashtable: one + compares two dinosaurs for equality, and one computes the hash of a + dinosaur. Let's say that two dinosaurs are equal if they have the same ID + and name. + + int + dinosaurs_equal(const struct dinosaur *d1, const struct dinosaur *d2) + { + return d1->dinosaur_id == d2->dinosaur_id && + 0 == strcmp(d1->name, d2->name); + } + + unsigned + dinosaur_hash(const struct dinosaur *d) + { + // This is a very bad hash function. Use siphash24g instead. + return (d->dinosaur_id + d->name[0] ) * 1337 + d->name[1] * 1337; + } + + Now you'll need to declare the functions that manipulate the hash table. + To do this, you put this declaration either in a header file, or inside + a regular module, depending on what visibility you want. + + HT_PROTOTYPE(dinosaur_ht, // The name of the hashtable struct + dinosaur, // The name of the element struct, + node, // The name of HT_ENTRY member + dinosaur_hash, dinosaurs_equal); + + Later, inside a C function, you use this macro to declare the hashtable + functions. + + HT_GENERATE2(dinosaur_ht, dinosaur, node, dinosaur_hash, dinosaurs_equal, + 0.6, tor_reallocarray, tor_free_); + + Note the use of tor_free_, not tor_free. The 0.6 is magic. + + Now you can use the hashtable! You can initialize one with + + struct dinosaur_ht my_dinos = HT_INITIALIZER(); + + Or create one in core with + + struct dinosaur_ht *dinos = tor_malloc(sizeof(dinosaur_ht)); + HT_INIT(dinosaur_ht, dinos); + + To the hashtable, you use the HT_FOO(dinosaur_ht, ...) macros. For + example, to put new_dino into dinos, you say: + + HT_REPLACE(dinosaur_ht, dinos, new_dino); + + If you're searching for an element, you need to use a dummy 'key' element in + the search. For example. + + struct dinosaur dino_key; + dino_key.dinosaur_id = 12345; + dino_key.name = tor_strdup("Atrociraptor"); + + struct dinosaur *found = HT_FIND(dinosaurs_ht, dinos, &dino_key); + + Have fun with your hash table! + + */ + #ifndef HT_H_INCLUDED_ #define HT_H_INCLUDED_

1 0

[tor-browser-bundle/maint-6.0] Bug 19854: fix URLs in downloads.json file
by gk＠torproject.org 11 Aug '16

11 Aug '16

commit f9d27acb00da3a44206024a6ee3549173d61e6fa Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Aug 8 12:14:23 2016 +0200 Bug 19854: fix URLs in downloads.json file --- tools/update-responses/config.yml | 14 +++++++------- tools/update-responses/update_responses | 15 +++++++++------ 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/tools/update-responses/config.yml b/tools/update-responses/config.yml index 36192fc..ff17d28 100644 --- a/tools/update-responses/config.yml +++ b/tools/update-responses/config.yml @@ -3,6 +3,8 @@ releases_dir: ../../gitian download: archive_url: https://archive.torproject.org/tor-package-archive/torbrowser gpg_keyring: ../../gitian/gpg/torbrowser.gpg + bundles_url: https://dist.torproject.org/torbrowser + mars_url: https://cdn.torproject.org/aus1/torbrowser build_targets: linux32: Linux_x86-gcc3 linux64: Linux_x86_64-gcc3 @@ -13,13 +15,12 @@ build_targets: osx32: Darwin_x86-gcc3 osx64: Darwin_x86_64-gcc3 channels: - alpha: 6.0a5 + alpha: 6.5a2 release: 6.0.3 versions: 6.0.3: platformVersion: 45.3.0 detailsURL: https://blog.torproject.org/blog/tor-browser-603-released - download_url: https://cdn.torproject.org/aus1/torbrowser/6.0.3 incremental_from: - 6.0.2 migrate_archs: @@ -27,12 +28,11 @@ versions: osx32: minSupportedOSVersion: 10.8 detailsURL: https://blog.torproject.org/blog/end-life-plan-tor-browser-32-bit-macs#upda… - 6.0a5: - platformVersion: 45.1.0 - detailsURL: https://blog.torproject.org/blog/tor-browser-60a5-released - download_url: https://cdn.torproject.org/aus1/torbrowser/6.0a5 + 6.5a2: + platformVersion: 45.3.0 + detailsURL: https://blog.torproject.org/blog/tor-browser-65a2-released incremental_from: - - 6.0a4 + - 6.5a1 migrate_archs: osx32: osx64 osx32: diff --git a/tools/update-responses/update_responses b/tools/update-responses/update_responses index cf307da..faa1037 100755 --- a/tools/update-responses/update_responses +++ b/tools/update-responses/update_responses @@ -84,6 +84,7 @@ sub get_version_files { return if $config->{versions}{$version}{files}; my $files = {}; my $vdir = "$releases_dir/$version"; + my $download_url = "$config->{download}{mars_url}/$version"; opendir(my $d, $vdir) or exit_error "Error opening directory $vdir"; foreach my $file (readdir $d) { next unless -f "$vdir/$file"; @@ -91,7 +92,7 @@ sub get_version_files { my ($os, $lang) = ($1, $2); $files->{$os}{$lang}{complete} = { type => 'complete', - URL => "$config->{versions}{$version}{download_url}/$file", + URL => "$download_url/$file", size => -s "$vdir/$file", hashFunction => 'SHA512', hashValue => get_sha512_hex_of_file("$vdir/$file"), @@ -102,7 +103,7 @@ sub get_version_files { my ($os, $from_version, $lang) = ($1, $2, $3); $files->{$os}{$lang}{partial}{$from_version} = { type => 'partial', - URL => "$config->{versions}{$version}{download_url}/$file", + URL => "$download_url/$file", size => -s "$vdir/$file", hashFunction => 'SHA512', hashValue => get_sha512_hex_of_file("$vdir/$file"), @@ -117,6 +118,7 @@ sub get_version_downloads { my ($config, $version) = @_; my $downloads = {}; my $vdir = "$releases_dir/$version"; + my $download_url = "$config->{download}{bundles_url}/$version"; opendir(my $d, $vdir) or exit_error "Error opening directory $vdir"; foreach my $file (readdir $d) { next unless -f "$vdir/$file"; @@ -131,8 +133,8 @@ sub get_version_downloads { next; } $downloads->{$os}{$lang} = { - binary => "$config->{versions}{$version}{download_url}/$file", - sig => "$config->{versions}{$version}{download_url}/$file.asc", + binary => "$download_url/$file", + sig => "$download_url/$file.asc", }; } closedir $d; @@ -175,12 +177,13 @@ sub create_incremental_mar { my $mar_file = "tor-browser-$os-${from_version}-${new_version}_$lang.incremental.mar"; print "Starting $mar_file\n"; my $mar_file_path = "$releases_dir/$new_version/$mar_file"; + my $download_url = "$config->{download}{mars_url}/$new_version"; my $finished_file = sub { exit_error "Error creating $mar_file" unless $_[1] == 0; print "Finished $mar_file\n"; $config->{versions}{$new_version}{files}{$os}{$lang}{partial}{$from_version} = { type => 'partial', - URL => "$config->{versions}{$new_version}{download_url}/$mar_file", + URL => "$download_url/$mar_file", size => -s $mar_file_path, hashFunction => 'SHA512', hashValue => get_sha512_hex_of_file($mar_file_path), @@ -205,7 +208,7 @@ sub create_incremental_mars_for_version { $pm->run_on_finish(sub { $_[2]->(@_) }); my $v = $config->{versions}{$version}; foreach my $from_version (@{$v->{incremental_from}}) { - $config->{versions}{$from_version} //= { download_url => '' }; + $config->{versions}{$from_version} //= {}; get_version_files($config, $from_version); my $from_v = $config->{versions}{$from_version}; foreach my $os (keys %{$v->{files}}) {

1 0

[tor-browser-bundle/master] Bug 19854: fix URLs in downloads.json file
by gk＠torproject.org 11 Aug '16

11 Aug '16

commit 858cdf3e315b068277c9e6bda7ef6108498a85b5 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Aug 8 12:14:23 2016 +0200 Bug 19854: fix URLs in downloads.json file --- tools/update-responses/config.yml | 4 ++-- tools/update-responses/update_responses | 15 +++++++++------ 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/tools/update-responses/config.yml b/tools/update-responses/config.yml index 59b9179..050328e 100644 --- a/tools/update-responses/config.yml +++ b/tools/update-responses/config.yml @@ -3,6 +3,8 @@ releases_dir: ../../gitian download: archive_url: https://archive.torproject.org/tor-package-archive/torbrowser gpg_keyring: ../../gitian/gpg/torbrowser.gpg + bundles_url: https://dist.torproject.org/torbrowser + mars_url: https://cdn.torproject.org/aus1/torbrowser build_targets: linux32: Linux_x86-gcc3 linux64: Linux_x86_64-gcc3 @@ -19,7 +21,6 @@ versions: 6.0.1: platformVersion: 45.2.0 detailsURL: https://blog.torproject.org/blog/tor-browser-601-released - download_url: https://cdn.torproject.org/aus1/torbrowser/6.0.1 incremental_from: - 5.5.5 - 6.0 @@ -31,7 +32,6 @@ versions: 6.5a2: platformVersion: 45.3.0 detailsURL: https://blog.torproject.org/blog/tor-browser-65a2-released - download_url: https://cdn.torproject.org/aus1/torbrowser/6.5a2 incremental_from: - 6.5a1 migrate_archs: diff --git a/tools/update-responses/update_responses b/tools/update-responses/update_responses index cf307da..faa1037 100755 --- a/tools/update-responses/update_responses +++ b/tools/update-responses/update_responses @@ -84,6 +84,7 @@ sub get_version_files { return if $config->{versions}{$version}{files}; my $files = {}; my $vdir = "$releases_dir/$version"; + my $download_url = "$config->{download}{mars_url}/$version"; opendir(my $d, $vdir) or exit_error "Error opening directory $vdir"; foreach my $file (readdir $d) { next unless -f "$vdir/$file"; @@ -91,7 +92,7 @@ sub get_version_files { my ($os, $lang) = ($1, $2); $files->{$os}{$lang}{complete} = { type => 'complete', - URL => "$config->{versions}{$version}{download_url}/$file", + URL => "$download_url/$file", size => -s "$vdir/$file", hashFunction => 'SHA512', hashValue => get_sha512_hex_of_file("$vdir/$file"), @@ -102,7 +103,7 @@ sub get_version_files { my ($os, $from_version, $lang) = ($1, $2, $3); $files->{$os}{$lang}{partial}{$from_version} = { type => 'partial', - URL => "$config->{versions}{$version}{download_url}/$file", + URL => "$download_url/$file", size => -s "$vdir/$file", hashFunction => 'SHA512', hashValue => get_sha512_hex_of_file("$vdir/$file"), @@ -117,6 +118,7 @@ sub get_version_downloads { my ($config, $version) = @_; my $downloads = {}; my $vdir = "$releases_dir/$version"; + my $download_url = "$config->{download}{bundles_url}/$version"; opendir(my $d, $vdir) or exit_error "Error opening directory $vdir"; foreach my $file (readdir $d) { next unless -f "$vdir/$file"; @@ -131,8 +133,8 @@ sub get_version_downloads { next; } $downloads->{$os}{$lang} = { - binary => "$config->{versions}{$version}{download_url}/$file", - sig => "$config->{versions}{$version}{download_url}/$file.asc", + binary => "$download_url/$file", + sig => "$download_url/$file.asc", }; } closedir $d; @@ -175,12 +177,13 @@ sub create_incremental_mar { my $mar_file = "tor-browser-$os-${from_version}-${new_version}_$lang.incremental.mar"; print "Starting $mar_file\n"; my $mar_file_path = "$releases_dir/$new_version/$mar_file"; + my $download_url = "$config->{download}{mars_url}/$new_version"; my $finished_file = sub { exit_error "Error creating $mar_file" unless $_[1] == 0; print "Finished $mar_file\n"; $config->{versions}{$new_version}{files}{$os}{$lang}{partial}{$from_version} = { type => 'partial', - URL => "$config->{versions}{$new_version}{download_url}/$mar_file", + URL => "$download_url/$mar_file", size => -s $mar_file_path, hashFunction => 'SHA512', hashValue => get_sha512_hex_of_file($mar_file_path), @@ -205,7 +208,7 @@ sub create_incremental_mars_for_version { $pm->run_on_finish(sub { $_[2]->(@_) }); my $v = $config->{versions}{$version}; foreach my $from_version (@{$v->{incremental_from}}) { - $config->{versions}{$from_version} //= { download_url => '' }; + $config->{versions}{$from_version} //= {}; get_version_files($config, $from_version); my $from_v = $config->{versions}{$from_version}; foreach my $os (keys %{$v->{files}}) {

1 0

[tor-browser-bundle/hardened-builds] Cleaning up gk's key
by gk＠torproject.org 11 Aug '16

11 Aug '16

commit 857bd13779d9fccd0081f0cd69e7538d6531d616 Author: Georg Koppen <gk(a)torproject.org> Date: Thu Aug 11 11:30:42 2016 +0000 Cleaning up gk's key --- gitian/gpg/gk.gpg | Bin 113673 -> 41255 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/gitian/gpg/gk.gpg b/gitian/gpg/gk.gpg index 359ea69..9037e43 100644 Binary files a/gitian/gpg/gk.gpg and b/gitian/gpg/gk.gpg differ

1 0

[tor-browser-bundle/master] Cleaning up gk's key
by gk＠torproject.org 11 Aug '16

11 Aug '16

commit 2dcef4073f367121ba4a4e80a0a62892ffb011c7 Author: Georg Koppen <gk(a)torproject.org> Date: Thu Aug 11 11:20:27 2016 +0000 Cleaning up gk's key --- gitian/gpg/gk.gpg | Bin 72418 -> 41255 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/gitian/gpg/gk.gpg b/gitian/gpg/gk.gpg index 2f78b14..9037e43 100644 Binary files a/gitian/gpg/gk.gpg and b/gitian/gpg/gk.gpg differ

1 0

[tor-browser-bundle/maint-6.0] Cleaning up gk's key
by gk＠torproject.org 11 Aug '16

11 Aug '16

commit dc8a94bd5979fdbc3e4819e7948a8424cee72688 Author: Georg Koppen <gk(a)torproject.org> Date: Thu Aug 11 11:20:27 2016 +0000 Cleaning up gk's key --- gitian/gpg/gk.gpg | Bin 72418 -> 41255 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/gitian/gpg/gk.gpg b/gitian/gpg/gk.gpg index 2f78b14..9037e43 100644 Binary files a/gitian/gpg/gk.gpg and b/gitian/gpg/gk.gpg differ

1 0

[tor-browser/tor-browser-45.3.0esr-6.0-1] Bug 19890: Disable installation of system addons
by gk＠torproject.org 11 Aug '16

11 Aug '16

commit 61700466ddefff3005633a6e31f34133eb98197f Author: Georg Koppen <gk(a)torproject.org> Date: Thu Aug 11 11:00:24 2016 +0000 Bug 19890: Disable installation of system addons By setting the URL to "" we make sure that already installed system addons get deleted as well. --- browser/app/profile/firefox.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/browser/app/profile/firefox.js b/browser/app/profile/firefox.js index 1dd964b..6e424cb 100644 --- a/browser/app/profile/firefox.js +++ b/browser/app/profile/firefox.js @@ -69,8 +69,8 @@ pref("extensions.hotfix.cert.checkAttributes", true); pref("extensions.hotfix.certs.1.sha1Fingerprint", "91:53:98:0C:C1:86:DF:47:8F:35:22:9E:11:C9:A7:31:04:49:A1:AA"); pref("extensions.hotfix.certs.2.sha1Fingerprint", "39:E7:2B:7A:5B:CF:37:78:F9:5D:4A:E0:53:2D:2F:3D:68:53:C5:60"); -// Check AUS for system add-on updates. -pref("extensions.systemAddon.update.url", "https://aus5.mozilla.org/update/3/SystemAddons/%VERSION%/%BUILD_ID%/%BUILD_…"); +// No AUS check for system add-on updates for Tor Browser users. +pref("extensions.systemAddon.update.url", ""); // Disable add-ons that are not installed by the user in all scopes by default. // See the SCOPE constants in AddonManager.jsm for values to use here.

1 0