tbb-commits

tbb-commits@lists.torproject.org

1 participants
17674 discussions

[tor-browser-bundle/hardened-builds] Build gn in linux-webrtc.
by gk＠torproject.org 31 Mar '17

31 Mar '17

commit 45f6ab2698d447441095e8d388fba3a3aa855f57 Author: David Fifield <david(a)bamsoftware.com> Date: Fri Sep 23 13:14:07 2016 -0700 Build gn in linux-webrtc. There is a prebuilt gn binary for linux64, but none for linux32, and anyway we prefer to build from source for reproducibility reasons. --- gitian/descriptors/linux/gitian-webrtc.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/gitian/descriptors/linux/gitian-webrtc.yml b/gitian/descriptors/linux/gitian-webrtc.yml index f18da52..85da679 100644 --- a/gitian/descriptors/linux/gitian-webrtc.yml +++ b/gitian/descriptors/linux/gitian-webrtc.yml @@ -63,6 +63,8 @@ script: | ln -sf hardened-cc c++ cd ~/build export PATH=$INSTDIR/gcc/bin:$INSTDIR/binutils/bin:$PATH + # GN needs libatomic.so.1 here. + export LD_LIBRARY_PATH=$INSTDIR/gcc/lib ARCH="" if [ $GBUILD_BITS == "64" ]; then @@ -80,6 +82,15 @@ script: | # Building webrtc tar xaf webrtc.tar.gz cd webrtc/src + + # First, build a copy of GN, rather than use the prebuilt copy in buildtools/linux64. + cd tools/gn + # __STDC_FORMAT_MACROS is needed for a definition of PRIxPTR from inttypes.h. + CXXFLAGS=-D__STDC_FORMAT_MACROS=1 LDFLAGS=-latomic ./bootstrap/bootstrap.py --no-rebuild --no-clean + cd ../.. + # This is where bootstrap.py stashes the built gn. + GN="$PWD/out_bootstrap/gn" + export GYP_CROSSCOMPILE=1 export GYP_DEFINES="OS=linux target_arch=$(if [ $GBUILD_BITS = 32 ]; then echo ia32; else echo x64; fi)" # Do not use bundled utilities.

1 0

[tor-browser-bundle/master] Build gn in linux-webrtc.
by gk＠torproject.org 31 Mar '17

31 Mar '17

commit 38689af2a845dfa85765bf52e535c89476863378 Author: David Fifield <david(a)bamsoftware.com> Date: Fri Sep 23 13:14:07 2016 -0700 Build gn in linux-webrtc. There is a prebuilt gn binary for linux64, but none for linux32, and anyway we prefer to build from source for reproducibility reasons. --- gitian/descriptors/linux/gitian-webrtc.yml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/gitian/descriptors/linux/gitian-webrtc.yml b/gitian/descriptors/linux/gitian-webrtc.yml index ba654c5..cab2430 100644 --- a/gitian/descriptors/linux/gitian-webrtc.yml +++ b/gitian/descriptors/linux/gitian-webrtc.yml @@ -66,6 +66,8 @@ script: | ln -sf hardened-cc c++ cd ~/build export PATH=$INSTDIR/gcc/bin:$INSTDIR/binutils/bin:$PATH + # GN needs libatomic.so.1 here. + export LD_LIBRARY_PATH=$INSTDIR/gcc/lib ARCH="" if [ $GBUILD_BITS == "64" ]; then @@ -83,6 +85,15 @@ script: | # Building webrtc tar xaf webrtc.tar.gz cd webrtc/src + + # First, build a copy of GN, rather than use the prebuilt copy in buildtools/linux64. + cd tools/gn + # __STDC_FORMAT_MACROS is needed for a definition of PRIxPTR from inttypes.h. + CXXFLAGS=-D__STDC_FORMAT_MACROS=1 LDFLAGS=-latomic ./bootstrap/bootstrap.py --no-rebuild --no-clean + cd ../.. + # This is where bootstrap.py stashes the built gn. + GN="$PWD/out_bootstrap/gn" + export GYP_CROSSCOMPILE=1 export GYP_DEFINES="OS=linux target_arch=$(if [ $GBUILD_BITS = 32 ]; then echo ia32; else echo x64; fi)" # Do not use bundled utilities.

1 0

[tor-browser-bundle/master] Build webrtc-linux with GN instead of GYP.
by gk＠torproject.org 31 Mar '17

31 Mar '17

commit 121de1ca7cbee29f34f99753a7948c7447e38965 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Mar 20 16:20:37 2017 -0400 Build webrtc-linux with GN instead of GYP. * Bump webrtc to branch-heads/58 * Also, bumps go-webrtc and snowflake to the latest. * Removes the need to re-run ninja for FakeAudioCaptureModule since "pc_test_utils" includes it. * Update to the new default bridge --- Bundle-Data/PTConfigs/bridge_prefs.js | 2 +- .../linux/gitian-pluggable-transports.yml | 5 +- gitian/descriptors/linux/gitian-webrtc.yml | 54 +++++++++++----------- gitian/fetch-inputs.sh | 3 -- gitian/patches/webrtc-linux.patch | 25 ++++++++++ gitian/versions | 6 +-- gitian/versions.alpha | 6 +-- gitian/versions.beta | 6 +-- gitian/versions.nightly | 2 +- 9 files changed, 68 insertions(+), 41 deletions(-) diff --git a/Bundle-Data/PTConfigs/bridge_prefs.js b/Bundle-Data/PTConfigs/bridge_prefs.js index 61e2a9e..64b7f23 100644 --- a/Bundle-Data/PTConfigs/bridge_prefs.js +++ b/Bundle-Data/PTConfigs/bridge_prefs.js @@ -37,4 +37,4 @@ pref("extensions.torlauncher.default_bridge.obfs4.20", "obfs4 85.17.30.79:443 FC pref("extensions.torlauncher.default_bridge.meek-amazon.1", "meek 0.0.2.0:2 B9E7141C594AF25699E0079C1F0146F409495296 url=https://d2zfqthxsdq309.cloudfront.net/ front=a0.awsstatic.com"); pref("extensions.torlauncher.default_bridge.meek-azure.1", "meek 0.0.2.0:3 97700DFE9F483596DDA6264C4D7DF7641E1E39CE url=https://meek.azureedge.net/ front=ajax.aspnetcdn.com"); -pref("extensions.torlauncher.default_bridge.snowflake.1", "snowflake 0.0.3.0:1 B05EC36590AC0374F0E4D36DDA77908FD59D97F3"); +pref("extensions.torlauncher.default_bridge.snowflake.1", "snowflake 0.0.3.0:1 2B280B23E1107BB62ABFC40DDCC8824814F80A72"); diff --git a/gitian/descriptors/linux/gitian-pluggable-transports.yml b/gitian/descriptors/linux/gitian-pluggable-transports.yml index ecf49de..cd936d5 100644 --- a/gitian/descriptors/linux/gitian-pluggable-transports.yml +++ b/gitian/descriptors/linux/gitian-pluggable-transports.yml @@ -113,6 +113,8 @@ script: | ln -sf hardened-cc c++ cd ~/build export PATH=$INSTDIR/gcc/bin:$INSTDIR/binutils/bin:$PATH + # GN needs libatomic.so.1 here. + export LD_LIBRARY_PATH=$INSTDIR/gcc/lib ARCH="" if [ $GBUILD_BITS == "64" ]; then @@ -309,7 +311,8 @@ script: | # the prebuilt libwebrtc-magic.a libraries. Since we build our own # libwebrtc-magic.a with the C++11 ABI, we have to undo that setting here, # using CGO_CXXFLAGS. - CGO_CXXFLAGS=-D_GLIBCXX_USE_CXX11_ABI=1 go install github.com/keroserene/go-webrtc + # __STDC_FORMAT_MACROS is needed for a definition of PRIxPTR from inttypes.h. + CGO_CXXFLAGS="-D_GLIBCXX_USE_CXX11_ABI=1 -D__STDC_FORMAT_MACROS=1" CGO_LDFLAGS=-latomic go install github.com/keroserene/go-webrtc cd .. # Building snowflake diff --git a/gitian/descriptors/linux/gitian-webrtc.yml b/gitian/descriptors/linux/gitian-webrtc.yml index cab2430..0622fee 100644 --- a/gitian/descriptors/linux/gitian-webrtc.yml +++ b/gitian/descriptors/linux/gitian-webrtc.yml @@ -12,12 +12,13 @@ packages: - "hardening-wrapper" - "libasound2-dev" - "libexpat1-dev" +- "libglib2.0-dev" +- "libgtk2.0-dev" - "libudev-dev" - "libx11-dev" - "libxext-dev" - "libxrandr-dev" -- "default-jre-headless" -- "default-jdk" +- "pkg-config" reference_datetime: "2000-01-01 00:00:00" remotes: - "url": "https://chromium.googlesource.com/chromium/tools/depot_tools.git" @@ -29,6 +30,7 @@ files: - "gcc-linux32-utils.zip" - "gcc-linux64-utils.zip" - "webrtc.tar.gz" +- "webrtc-linux.patch" script: | INSTDIR="$HOME/install" export REFERENCE_DATETIME @@ -85,6 +87,7 @@ script: | # Building webrtc tar xaf webrtc.tar.gz cd webrtc/src + patch -p1 < ../../webrtc-linux.patch # First, build a copy of GN, rather than use the prebuilt copy in buildtools/linux64. cd tools/gn @@ -94,32 +97,31 @@ script: | # This is where bootstrap.py stashes the built gn. GN="$PWD/out_bootstrap/gn" - export GYP_CROSSCOMPILE=1 - export GYP_DEFINES="OS=linux target_arch=$(if [ $GBUILD_BITS = 32 ]; then echo ia32; else echo x64; fi)" + export GN_ARGS="" + # For a list of all possible GN args, do "gn gen out/Release; gn args --list out/Release". + # https://chromium.googlesource.com/chromium/src/+/master/tools/gn/docs/cross… + GN_ARGS+=" target_os=\"linux\" target_cpu=\"$(if [ $GBUILD_BITS = 32 ]; then echo x86; else echo x64; fi)\"" + # Not debug. + GN_ARGS+=" is_debug=false" + # There are warnings from unused returns. + GN_ARGS+=" treat_warnings_as_errors=false" + # Build static libraries. + GN_ARGS+=" is_component_build=false" # Do not use bundled utilities. - GYP_DEFINES+=" clang=0 host_clang=0 use_sysroot=0" - GYP_DEFINES+=" linux_use_bundled_gold=0 linux_use_gold_flags=0 linux_use_bundled_binutils=0" - # Avoid some dependencies. Some of these don't seem to work fully; for - # example even with use_alsa=0 we need libasound2-dev. - GYP_DEFINES+=" use_x11=0 use_gtk=0 use_gconf=0 use_alsa=0 use_pulseaudio=0 use_libpci=0 include_internal_audio_device=0 enabled_libjingle_device_manager=0" - # examples and tests also bring in dependencies. - GYP_DEFINES+=" include_examples=0 include_tests=0" - # embedded=1 is supposed to turn of various features; see - # https://bugs.chromium.org/p/chromium/issues/detail?id=318413. - GYP_DEFINES+=" embedded=1" - GYP_DEFINES+=" werror=" - JAVA_HOME=/usr/lib/jvm/default-java webrtc/build/gyp_webrtc.py - ninja -C out/Release - # Run ninja once more, without include_tests=0, in order to build just the - # FakeAudioCaptureModule that go-webrtc uses. - GYP_DEFINES="$(echo "$GYP_DEFINES" | sed -e 's/include_tests=0//g')" - JAVA_HOME=/usr/lib/jvm/default-java webrtc/build/gyp_webrtc.py - ninja -C out/Release obj/webrtc/api/test/peerconnection_unittests.fakeaudiocapturemodule.o + GN_ARGS+=" is_clang=false use_sysroot=false" + GN_ARGS+=" gold_path=\"$INSTDIR/binutils/bin\"" + # Make extra sure we don't use bundled binutils. + rm -rf third_party/binutils/Linux_* + # Avoid some dependencies. + GN_ARGS+=" use_ozone=true use_gconf=false" + GN_ARGS+=" rtc_include_opus=false rtc_include_ilbc=false rtc_include_internal_audio_device=false rtc_include_pulse_audio=false" + # Included for "field_trial" below. Maybe "fieldtrial_testing_like_official_build" could help here? + #GN_ARGS+=" rtc_include_tests=false" + rm -rf out/Release + "$GN" gen out/Release --args="$GN_ARGS" + ninja -C out/Release webrtc field_trial metrics_default pc_test_utils # https://github.com/keroserene/go-webrtc/issues/23#issuecomment-175312648 - # dump_syms_regtest.o is actually an executable, not an object file. If not - # excluded, it results in the error: - # libwebrtc-linux-386-magic.a(dump_syms_regtest.o): unsupported ELF file type 2 - ar crs libwebrtc-magic.a $(find . -name '*.o' -not -name '*.main.o' -not -name 'dump_syms_regtest.o' | sort) + ar crs libwebrtc-magic.a $(find . -name '*.o' -not -name '*.main.o' | sort) cd ../.. # Grabbing the result diff --git a/gitian/fetch-inputs.sh b/gitian/fetch-inputs.sh index b97a577..f141381 100755 --- a/gitian/fetch-inputs.sh +++ b/gitian/fetch-inputs.sh @@ -314,10 +314,7 @@ cd .. # depot_tools must have been downloaded before running this code. dir=webrtc PATH="$PATH:$PWD/depot_tools" -# GYP_CROSSCOMPILE=1 and GYP_DEFINES="use_x11=0" prevent probing for certain dependencies. # Use --no-history because the whole checkout with history is about 12 GB. -export GYP_CROSSCOMPILE=1 -export GYP_DEFINES="use_x11=0" # JAVA_HOME is needed in a hook for libjingle. The readlink line tries to find the current JRE. # default-java comes from the package default-jdk-headless. export JAVA_HOME=/usr/lib/jvm/default-java diff --git a/gitian/patches/webrtc-linux.patch b/gitian/patches/webrtc-linux.patch new file mode 100644 index 0000000..b1efb7a --- /dev/null +++ b/gitian/patches/webrtc-linux.patch @@ -0,0 +1,25 @@ +From ebc1773a92884bec7d93274b2d729ccf76050197 Mon Sep 17 00:00:00 2001 +From: David Fifield <fifield(a)eecs.berkeley.edu> +Date: Wed, 22 Mar 2017 09:53:07 -0400 +Subject: [PATCH] Disable some settings in build_overrides. + +--- + build_overrides/build.gni | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/build_overrides/build.gni b/build_overrides/build.gni +index af4924d4c..ffaafdebc 100644 +--- a/build_overrides/build.gni ++++ b/build_overrides/build.gni +@@ -14,7 +14,7 @@ mac_sdk_min_build_override = "10.11" + enable_java_templates = true + + # Some non-Chromium builds don't use Chromium's third_party/binutils. +-linux_use_bundled_binutils_override = true ++linux_use_bundled_binutils_override = false + + # Variable that can be used to support multiple build scenarios, like having + # Chromium specific targets in a client project's GN file etc. +-- +2.12.1 + diff --git a/gitian/versions b/gitian/versions index 91bea14..69abeb3 100755 --- a/gitian/versions +++ b/gitian/versions @@ -42,9 +42,9 @@ GO_X_NET_TAG=7dbad50ab5b31073856416cdcfeb2796d682f844 OBFS4_TAG=obfs4proxy-0.0.5 NOTOFONTS_TAG=720e34851382ee3c1ef024d8dffb68ffbfb234c2 DEPOT_TOOLS_TAG=28216cd14b44716db5c83634afbdc6e90492652b -WEBRTC_TAG=f33698296719f956497d2dbff81b5080864a8804 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/52 -GO_WEBRTC_TAG=69c5ca70533142494d145b43ef2f46ca94715557 -SNOWFLAKE_TAG=6cecd31fd896eb26e64ad8bab8a9ea510ec3b21d +WEBRTC_TAG=c279861207c5b15fc51069e96595782350e0ac12 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/58 +GO_WEBRTC_TAG=ab1b64862e0c4b4182010699911c2c5818f0a101 +SNOWFLAKE_TAG=9f2e9a6ecb696149708716ca06ce842df03cf492 UNIURI_TAG=8902c56451e9b58ff940bbe5fec35d5f9c04584a GITIAN_TAG=tor-browser-builder-4-1 diff --git a/gitian/versions.alpha b/gitian/versions.alpha index 6c06648..3539d59 100755 --- a/gitian/versions.alpha +++ b/gitian/versions.alpha @@ -45,9 +45,9 @@ ERRORS_TAG=248dadf4e9068a0b3e79f02ed0a610d935de5302 GB_TAG=06cc925cce6592e922dcc4839a8b44feb384e71e SANDBOX_TAG=sandboxed-tor-browser-0.0.3 DEPOT_TOOLS_TAG=28216cd14b44716db5c83634afbdc6e90492652b -WEBRTC_TAG=f33698296719f956497d2dbff81b5080864a8804 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/52 -GO_WEBRTC_TAG=69c5ca70533142494d145b43ef2f46ca94715557 -SNOWFLAKE_TAG=6cecd31fd896eb26e64ad8bab8a9ea510ec3b21d +WEBRTC_TAG=c279861207c5b15fc51069e96595782350e0ac12 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/58 +GO_WEBRTC_TAG=ab1b64862e0c4b4182010699911c2c5818f0a101 +SNOWFLAKE_TAG=9f2e9a6ecb696149708716ca06ce842df03cf492 UNIURI_TAG=8902c56451e9b58ff940bbe5fec35d5f9c04584a GITIAN_TAG=tor-browser-builder-4-2 diff --git a/gitian/versions.beta b/gitian/versions.beta index b1342b9..d7db1fd 100755 --- a/gitian/versions.beta +++ b/gitian/versions.beta @@ -30,9 +30,9 @@ GOPTLIB_TAG=0.2 MEEK_TAG=0.25 NOTOFONTS_TAG=720e34851382ee3c1ef024d8dffb68ffbfb234c2 DEPOT_TOOLS_TAG=28216cd14b44716db5c83634afbdc6e90492652b -WEBRTC_TAG=f33698296719f956497d2dbff81b5080864a8804 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/52 -GO_WEBRTC_TAG=69c5ca70533142494d145b43ef2f46ca94715557 -SNOWFLAKE_TAG=6cecd31fd896eb26e64ad8bab8a9ea510ec3b21d +WEBRTC_TAG=c279861207c5b15fc51069e96595782350e0ac12 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/58 +GO_WEBRTC_TAG=ab1b64862e0c4b4182010699911c2c5818f0a101 +SNOWFLAKE_TAG=9f2e9a6ecb696149708716ca06ce842df03cf492 UNIURI_TAG=8902c56451e9b58ff940bbe5fec35d5f9c04584a GITIAN_TAG=tor-browser-builder-3.x-6 diff --git a/gitian/versions.nightly b/gitian/versions.nightly index b9779e2..2045034 100755 --- a/gitian/versions.nightly +++ b/gitian/versions.nightly @@ -52,7 +52,7 @@ ERRORS_TAG=248dadf4e9068a0b3e79f02ed0a610d935de5302 GB_TAG=06cc925cce6592e922dcc4839a8b44feb384e71e SANDBOX_TAG=master DEPOT_TOOLS_TAG=master -WEBRTC_TAG=f33698296719f956497d2dbff81b5080864a8804 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/52 +WEBRTC_TAG=c279861207c5b15fc51069e96595782350e0ac12 # https://chromium.googlesource.com/external/webrtc.git/+/refs/branch-heads/58 GO_WEBRTC_TAG=master SNOWFLAKE_TAG=master UNIURI_TAG=master

1 0

[tor-browser-build/master] README: add missing dependencies
by boklm＠torproject.org 31 Mar '17

31 Mar '17

commit 041df309831d73f69b7f2fc47b15211dda7e5e31 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Fri Mar 31 12:57:04 2017 +0200 README: add missing dependencies --- README | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README b/README index 07725cf..08904fb 100644 --- a/README +++ b/README @@ -15,7 +15,7 @@ also be running. The sources of most components are downloaded using git, which needs to be installed. The sources of webrtc are downloaded using gclient, which -requires GTK+ 2.0 development files to be installed. +requires GTK+ 2.0 development files and curl to be installed. You also need a few perl modules installed: - YAML::XS @@ -27,6 +27,7 @@ You also need a few perl modules installed: - File::Temp - File::Slurp - File::Path +- File::Copy::Recursive - String::ShellQuote - Sort::Versions - Digest::SHA @@ -40,7 +41,7 @@ If you are running Debian or Ubuntu, you can install them with: libio-captureoutput-perl libfile-slurp-perl \ libstring-shellquote-perl libsort-versions-perl \ libdigest-sha-perl libdata-uuid-perl libdata-dump-perl \ - git libgtk2.0-dev + libfile-copy-recursive-perl git libgtk2.0-dev curl Starting a build

1 0

[tor-launcher/master] Bug 21546: Tor Launcher is multiprocessCompatible
by gk＠torproject.org 29 Mar '17

29 Mar '17

commit 336e7c8731296866dd67a090c010e41fcc26ef69 Author: Georg Koppen <gk(a)torproject.org> Date: Sun Mar 12 20:39:27 2017 +0000 Bug 21546: Tor Launcher is multiprocessCompatible --- src/install.rdf | 1 + 1 file changed, 1 insertion(+) diff --git a/src/install.rdf b/src/install.rdf index 493d976..881aa8a 100644 --- a/src/install.rdf +++ b/src/install.rdf @@ -8,6 +8,7 @@ <em:contributor>Pearl Crescent, LLC</em:contributor> <em:id>tor-launcher(a)torproject.org</em:id> <em:version>0.2.11.1</em:version> + <em:multiprocessCompatible>true</em:multiprocessCompatible> <em:homepageURL>https://www.torproject.org/projects/torbrowser.html</em:homepageURL> <em:updateURL>data:text/plain,</em:updateURL> <em:updateKey>-</em:updateKey>

1 0

[tor-browser-build/master] HTTPS Everywhere: Version bump - 5.2.13
by boklm＠torproject.org 26 Mar '17

26 Mar '17

commit bfc288d54a28f00e088ba9ce08bd718156d8ff2f Author: William Budington <bill(a)eff.org> Date: Sun Mar 26 07:12:49 2017 -0700 HTTPS Everywhere: Version bump - 5.2.13 --- projects/https-everywhere/config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/https-everywhere/config b/projects/https-everywhere/config index 17f9f82..7ef8568 100644 --- a/projects/https-everywhere/config +++ b/projects/https-everywhere/config @@ -1,5 +1,5 @@ # vim: filetype=yaml sw=2 -version: 5.2.11 +version: 5.2.13 git_url: https://git.torproject.org/https-everywhere.git git_hash: '[% c("version") %]' git_submodule: 1

1 0

[tor-browser-build/master] README: Add libgtk2.0-dev to the list of dependencies
by boklm＠torproject.org 26 Mar '17

26 Mar '17

commit 29f7cffa1199dd89da17a6d7792fe7f265d05a42 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Sun Mar 26 14:45:01 2017 +0200 README: Add libgtk2.0-dev to the list of dependencies --- README | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README b/README index f4a74e0..07725cf 100644 --- a/README +++ b/README @@ -13,7 +13,9 @@ Your user account should have access to the docker command without using sudo, so it should be in the docker group. The docker daemon should also be running. -The sources are downloaded using git which needs to be installed. +The sources of most components are downloaded using git, which needs to +be installed. The sources of webrtc are downloaded using gclient, which +requires GTK+ 2.0 development files to be installed. You also need a few perl modules installed: - YAML::XS @@ -38,7 +40,7 @@ If you are running Debian or Ubuntu, you can install them with: libio-captureoutput-perl libfile-slurp-perl \ libstring-shellquote-perl libsort-versions-perl \ libdigest-sha-perl libdata-uuid-perl libdata-dump-perl \ - git + git libgtk2.0-dev Starting a build

1 0

[tor-browser-build/master] Bug 21655: add a script to clean old built components
by boklm＠torproject.org 20 Mar '17

20 Mar '17

commit 5d046ea3c4af539e36a886ee3a7b03c07f5d7b48 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Mar 20 18:29:03 2017 +0100 Bug 21655: add a script to clean old built components --- Makefile | 5 ++- README | 10 ++++- rbm | 2 +- rbm.local.conf.example | 28 ++++++++++++ tools/clean-old | 114 +++++++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 155 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index 9481537..b82309c 100644 --- a/Makefile +++ b/Makefile @@ -86,6 +86,9 @@ submodule-update: fetch: submodule-update $(rbm) fetch -clean-old: submodule-update +clean: submodule-update ./tools/clean-old +clean-dry-run: submodule-update + ./tools/clean-old --dry-run + diff --git a/README b/README index a14fb42..f4a74e0 100644 --- a/README +++ b/README @@ -171,8 +171,14 @@ file. Cleaning obsolete files and containers images --------------------------------------------- -There will be a script to clean old build files and containers that are -no longer used, but it has not been added yet. +You can run 'make clean' to clean old build files and containers that +are no longer used in current builds. Before doing that, you need to +configure the branches and build targets you are using in the +rbm.local.conf file. The cleaning script will checkout all the configured +branches to create a list of used build files, and delete the files +from the 'out' directory that are not used. If you want to see the list +of files and containers that would be removed without doing it, you can +use 'make clean-dry-run'. Multiple build directories on the same host diff --git a/rbm b/rbm index 845bb90..3f3886e 160000 --- a/rbm +++ b/rbm @@ -1 +1 @@ -Subproject commit 845bb904fb87b6c56ab789f35cd02bce32453f52 +Subproject commit 3f3886e1f210ad2853209c5aecd0951350a6f758 diff --git a/rbm.local.conf.example b/rbm.local.conf.example index 2a86647..78de08f 100644 --- a/rbm.local.conf.example +++ b/rbm.local.conf.example @@ -39,6 +39,34 @@ var: ### file. #sign_build_gpg_opts: '--local-user XXXXXXXX' + ### The clean configuration is used by the cleaning script to find the + ### branches and build targets you are using, to compute the list of + ### files that should be kept. + ### + ### If you only do alpha builds for all platforms, you can use the + ### following configuration: + #clean: + # HEAD: + # - project: release + # target: + # - alpha + # - torbrowser-all + # + ### If you are doing 'release' builds in the maint-7.0 branch and + ### 'alpha' builds in the master branch, you can use the following + ### configuration: + #clean: + # master: + # - project: release + # target: + # - alpha + # - torbrowser-all + # maint-7.0: + # - project: release + # target: + # - release + # - torbrowser-all + targets: ### testbuild is based on alpha by default. Uncomment this if you want it diff --git a/tools/clean-old b/tools/clean-old new file mode 100755 index 0000000..4d603fc --- /dev/null +++ b/tools/clean-old @@ -0,0 +1,114 @@ +#!/usr/bin/perl -w +use strict; +use IO::CaptureOutput qw(capture_exec); +use FindBin; +use lib "$FindBin::Bin/../rbm/lib"; +use RBM; +use File::Slurp qw(read_dir write_file read_file); +use Getopt::Long; + +my %options; +my @options_list = qw(dry-run! list-used-files=s project=s target=s@); +Getopt::Long::GetOptionsFromArray(\@ARGV, \%options, @options_list) || exit 1; + +sub clean_file { + my ($file, $used_files) = @_; + return if $used_files->{$file}; + if (-d $file) { + my @l = read_dir($file); + foreach my $subfile (@l) { + clean_file("$file/$subfile", $used_files); + } + @l = read_dir($file); + rmdir $file unless @l; + } else { + print "Removing file $file\n"; + unlink $file unless $options{'dry-run'}; + } +} + +sub clean_docker_images { + my ($dockerdir, $used_files) = @_; + my $imgprefix = RBM::project_config('docker-image', 'docker_image_prefix'); + my @imgs = read_dir($dockerdir); + foreach my $dockerimage (@imgs) { + next if $used_files->{"$dockerdir/$dockerimage"}; + my $img = "$imgprefix:$dockerimage"; + print "Cleaning docker image $img\n"; + next if $options{'dry-run'}; + my ($out, $err, $success) = capture_exec('docker', 'rmi', '-f', $img); + if (!$success) { + print STDERR "Error removing docker image $img:\n$err\n"; + exit 1; + } + unlink "$dockerdir/$dockerimage"; + } +} + +sub get_project_input_files { + my ($project, @targets) = @_; + print "Getting input files for $project ", join(' ', @targets), "\n"; + $RBM::config->{run}{target} = \@targets; + my $res = RBM::project_config($project, 'input_files_paths', + {error_if_undef => 1}); + return ref $res eq 'ARRAY' ? @$res : (); +} + +sub get_input_files { + my ($c) = @_; + my @files; + foreach my $p (@$c) { + my $tmp = File::Temp->new(); + my @args = ("$FindBin::Bin/$FindBin::Script", '--list-used-files', $tmp, + '--project', $p->{project}); + foreach my $target (@{$p->{target}}) { + push @args, ('--target', $target); + } + RBM::exit_error 'Error getting used files' unless system(@args) == 0; + push @files, map { chomp; $_ } read_file($tmp); + } + return @files; +} + +RBM::load_config; +RBM::load_system_config; +RBM::load_local_config; +RBM::set_default_env; + +if ($options{'list-used-files'}) { + if (!$options{project} || !$options{target}) { + RBM::exit_error 'Missing option project or target'; + } + my @files = get_project_input_files($options{project}, @{$options{target}}); + write_file($options{'list-used-files'}, join("\n", @files)); + exit 0; +} + +my $clean = RBM::project_config('clean', 'var/clean'); +if (!$clean) { + print STDERR "Clean configuration is missing. ", + "You should add it to rbm.local.conf.\n"; + exit 1; +} + +my @files = get_input_files($clean->{HEAD}) if $clean->{HEAD}; + +foreach my $branch (keys %$clean) { + next if $branch eq 'HEAD'; + print "Checking out $branch branch\n"; + RBM::exit_error("Error checking out $branch branch") + unless system('git', 'checkout', $branch) == 0; + RBM::exit_error("Error running git submodule update --init") + unless system('git', 'submodule', 'update', '--init') == 0; + push @files, get_input_files($clean->{$branch}); + RBM::exit_error('Error checking out @{-1}') + unless system('git', 'checkout', '@{-1}') == 0; + RBM::exit_error("Error running git submodule update --init") + unless system('git', 'submodule', 'update', '--init') == 0; +} +my %used_files = map { $_ => 1 } @files; +my $outdir = $RBM::config->{basedir} . '/out'; +# Don't clean docker-image files yet +$used_files{"$outdir/docker-image"} = 1; +clean_file($outdir, \%used_files); +clean_docker_images("$outdir/docker-image", \%used_files);

1 0

[rbm/master] Add the input_files_paths option
by boklm＠torproject.org 15 Mar '17

15 Mar '17

commit 3f3886e1f210ad2853209c5aecd0951350a6f758 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Wed Mar 15 12:25:39 2017 +0100 Add the input_files_paths option --- NEWS | 2 ++ doc/rbm_config.asc | 6 ++++++ lib/RBM.pm | 23 ++++++++++++++++++++++- lib/RBM/DefaultConfig.pm | 1 + 4 files changed, 31 insertions(+), 1 deletion(-) diff --git a/NEWS b/NEWS index 5de36e0..4eac374 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,5 @@ +- Add the input_files_paths option + - Add the ENV option - Add the build_log option diff --git a/doc/rbm_config.asc b/doc/rbm_config.asc index 48e505c..2e9b8ab 100644 --- a/doc/rbm_config.asc +++ b/doc/rbm_config.asc @@ -220,6 +220,12 @@ input_files_id:: file of type +project+, and the +filename+ and the sha256sum of the file for any other type of input file. +input_files_paths:: + The value of this option is an array of all the paths of input + files that currently exist and are used in the build of the + current project and its dependencies. This is useful when + cleaning old build files, to find which ones are still used. + timestamp:: This is the UNIX timestamp, set as modification time on files created such as the sources tarball and rpm spec file. The diff --git a/lib/RBM.pm b/lib/RBM.pm index 632a8ba..79e0670 100644 --- a/lib/RBM.pm +++ b/lib/RBM.pm @@ -731,8 +731,9 @@ sub input_files { my ($action, $project, $options, $dest_dir) = @_; my @res_copy; my %res_getfnames; + my @res_getfpaths; my $getfnames_noname = 0; - my $need_dl = 1; + my $need_dl = $action ne 'getfpaths'; my $input_files_id = ''; $options = {$options ? %$options : ()}; my $input_files = project_config($project, 'input_files', $options,); @@ -840,6 +841,25 @@ sub input_files { $input_files_id .= "\n"; next; } + if ($action eq 'getfpaths') { + push @res_getfpaths, $fname if $fname; + if ($file_gpg_id && $fname) { + my $sig_ext = $t->('sig_ext'); + $sig_ext = ref $sig_ext eq 'ARRAY' ? $sig_ext : [ $sig_ext ]; + foreach my $s (@$sig_ext) { + if (-f "$fname.$s") { + push @res_getfpaths, "$fname.$s"; + last; + } + } + } + if ($input_file->{project} && $t->('project')) { + my $r = RBM::project_step_config($t->('project'), 'input_files_paths', + {%$options, step => $t->('pkg_type'), %$input_file}); + push @res_getfpaths, @$r if @$r; + } + next; + } exit_error "Missing file $name" unless $fname; if ($t->('sha256sum') && $t->('sha256sum') ne sha256_hex(read_file($fname))) { @@ -886,6 +906,7 @@ sub input_files { return sha256_hex($input_files_id) if $action eq 'input_files_id'; return @res_copy if $action eq 'copy'; return \%res_getfnames if $action eq 'getfnames'; + return \@res_getfpaths if $action eq 'getfpaths'; } sub system_log { diff --git a/lib/RBM/DefaultConfig.pm b/lib/RBM/DefaultConfig.pm index c0764bd..cf0ed74 100644 --- a/lib/RBM/DefaultConfig.pm +++ b/lib/RBM/DefaultConfig.pm @@ -583,6 +583,7 @@ ZIP_END arch => \&get_arch, input_files_by_name => sub { RBM::input_files('getfnames', @_); }, input_files_id => sub { RBM::input_files('input_files_id', @_); }, + input_files_paths => sub { RBM::input_files('getfpaths', @_); }, steps => { srpm => 'rpm', 'deb-src' => 'deb',

1 0

[tor-browser-build/master] Add snowflake
by boklm＠torproject.org 13 Mar '17

13 Mar '17

commit 832c2ce17672d5cd1f554354902df670e6ee928a Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Mar 13 19:39:02 2017 +0100 Add snowflake Only for linux-x86_64 for now, as it fails to build on linux-i686. --- .gitignore | 1 + projects/depot_tools/build | 8 +++ projects/depot_tools/config | 5 ++ projects/gcc/config | 2 +- projects/go-webrtc/config | 53 ++++++++++++++++++++ projects/go/config | 1 + projects/snowflake/build | 38 +++++++++++++++ projects/snowflake/config | 35 ++++++++++++++ projects/tor-browser/build | 3 ++ projects/tor-browser/config | 3 ++ projects/webrtc/build | 80 ++++++++++++++++++++++++++++++ projects/webrtc/config | 115 ++++++++++++++++++++++++++++++++++++++++++++ rbm.conf | 4 ++ 13 files changed, 347 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 9b6754f..0fe0aec 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ git_clones hg_clones +gclient out release alpha diff --git a/projects/depot_tools/build b/projects/depot_tools/build new file mode 100644 index 0000000..ac0f21b --- /dev/null +++ b/projects/depot_tools/build @@ -0,0 +1,8 @@ +#!/bin/bash +[% c("var/set_default_env") -%] +tar xf [% project %]-[% c("version") %].tar.gz +mv [% project %]-[% c("version") %] [% project %] +[% c('tar', { + tar_src => [ project ], + tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), + }) %] diff --git a/projects/depot_tools/config b/projects/depot_tools/config new file mode 100644 index 0000000..40a6d85 --- /dev/null +++ b/projects/depot_tools/config @@ -0,0 +1,5 @@ +# vim: filetype=yaml sw=2 +version: '[% c("abbrev") %]' +git_hash: 28216cd14b44716db5c83634afbdc6e90492652b +git_url: https://chromium.googlesource.com/chromium/tools/depot_tools.git +filename: '[% project %]-[% c("version") %].tar.gz' diff --git a/projects/gcc/config b/projects/gcc/config index bf73b7d..391e453 100644 --- a/projects/gcc/config +++ b/projects/gcc/config @@ -10,7 +10,7 @@ var: - libc6-dev-i386 setup: | mkdir -p /var/tmp/dist - tar -C /var/tmp/dist -xf [% c("compiler_tarfile") %] + tar -C /var/tmp/dist -xf $rootdir/[% c("compiler_tarfile") %] export PATH="/var/tmp/dist/gcc/bin:$PATH" [% IF c("var/osname") == "linux-i686" -%] export LD_LIBRARY_PATH=/var/tmp/dist/gcc/lib32 diff --git a/projects/go-webrtc/config b/projects/go-webrtc/config new file mode 100644 index 0000000..778d3db --- /dev/null +++ b/projects/go-webrtc/config @@ -0,0 +1,53 @@ +# vim: filetype=yaml sw=2 +version: '[% c("abbrev") %]' +git_url: https://github.com/keroserene/go-webrtc.git +git_hash: 69c5ca70533142494d145b43ef2f46ca94715557 +filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' +remote_docker: 1 + +build: '[% c("projects/go/var/build_go_lib") %]' + +var: + go_lib: github.com/keroserene/go-webrtc + build_go_lib_pre: | + [% pc(c('var/compiler'), 'var/setup', { compiler_tarfile => c('input_files_by_name/' _ c('var/compiler')) }) %] + [% IF c("var/linux") %] + tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/binutils') %] + export PATH="/var/tmp/dist/binutils/bin:$PATH" + [% END -%] + tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/webrtc') %] + # Replace the prebuilt webrtc library with our own one. + rm -rf include/ lib/ + cp -a /var/tmp/dist/webrtc/{include,lib} . + # The go-webrtc package sets _GLIBCXX_USE_CXX11_ABI=0 for compatibility with + # the prebuilt libwebrtc-magic.a libraries. Since we build our own + # libwebrtc-magic.a with the C++11 ABI, we have to undo that setting here, + # using CGO_CXXFLAGS. + export CGO_CXXFLAGS=-D_GLIBCXX_USE_CXX11_ABI=1 + +targets: + master: + git_hash: master + linux-x86_64: + var: + arch_deps: + - pkg-config + - libx11-dev + linux-i686: + var: + arch_deps: + - pkg-config:i386 + - libx11-dev:i386 + - lib32stdc++6 + +input_files: + - project: docker-image + - name: go + project: go + - name: webrtc + project: webrtc + - name: '[% c("var/compiler") %]' + project: '[% c("var/compiler") %]' + - name: binutils + project: binutils + enable: '[% c("var/linux") %]' diff --git a/projects/go/config b/projects/go/config index 5e55a09..870dc50 100644 --- a/projects/go/config +++ b/projects/go/config @@ -35,6 +35,7 @@ var: do patch -p1 < $p done + [% IF c("var/build_go_lib_pre"); GET c("var/build_go_lib_pre"); END; -%] [% IF c("var/go_lib_install") -%] [% FOREACH inst IN c("var/go_lib_install") %] go install [% inst %] diff --git a/projects/snowflake/build b/projects/snowflake/build new file mode 100644 index 0000000..6e08b4e --- /dev/null +++ b/projects/snowflake/build @@ -0,0 +1,38 @@ +#!/bin/bash +[% c("var/set_default_env") -%] +[% pc('go', 'var/setup', { go_tarfile => c('input_files_by_name/go') }) %] +[% pc(c('var/compiler'), 'var/setup', { compiler_tarfile => c('input_files_by_name/' _ c('var/compiler')) }) %] +[% IF c("var/linux") %] + tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/binutils') %] + export PATH="/var/tmp/dist/binutils/bin:$PATH" +[% END -%] +distdir=/var/tmp/dist/[% project %] +[% IF c("var/osx") %] + PTDIR="$distdir/Contents/MacOS/Tor/PluggableTransports" + DOCSDIR="$distdir/Contents/Resources/TorBrowser/Docs/snowflake" +[% ELSE %] + PTDIR="$distdir/TorBrowser/Tor/PluggableTransports" + DOCSDIR="$distdir/TorBrowser/Docs/snowflake" +[% END %] +mkdir -p $PTDIR $DOCSDIR + +tar -C /var/tmp/dist -xf [% c('input_files_by_name/go-webrtc') %] +tar -C /var/tmp/dist -xf [% c('input_files_by_name/uniuri') %] +tar -C /var/tmp/dist -xf [% c('input_files_by_name/goptlib') %] + +mkdir -p /var/tmp/build +tar -C /var/tmp/build -xf [% project %]-[% c('version') %].tar.gz +cd /var/tmp/build/[% project %]-[% c('version') %] + +cd client +go build -ldflags '-s' +cp -a client[% IF c("var/windows") %].exe[% END %] $PTDIR/snowflake-client[% IF c("var/windows") %].exe[% END %] + +cd .. +cp -a README.md LICENSE $DOCSDIR + +cd $distdir +[% c('tar', { + tar_src => [ '.' ], + tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), + }) %] diff --git a/projects/snowflake/config b/projects/snowflake/config new file mode 100644 index 0000000..5ba006b --- /dev/null +++ b/projects/snowflake/config @@ -0,0 +1,35 @@ +# vim: filetype=yaml sw=2 +version: '[% c("abbrev") %]' +git_url: https://git.torproject.org/pluggable-transports/snowflake.git +git_hash: 6cecd31fd896eb26e64ad8bab8a9ea510ec3b21d +filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' +remote_docker: 1 + +targets: + linux-i686: + var: + arch_deps: + - pkg-config + - libx11-dev:i386 + - lib32stdc++6 + linux-x86_64: + var: + arch_deps: + - pkg-config + - libx11-dev + +input_files: + - project: docker-image + - name: go + project: go + - name: '[% c("var/compiler") %]' + project: '[% c("var/compiler") %]' + - name: binutils + project: binutils + enable: '[% c("var/linux") %]' + - name: go-webrtc + project: go-webrtc + - name: uniuri + project: uniuri + - name: goptlib + project: goptlib diff --git a/projects/tor-browser/build b/projects/tor-browser/build index 904f1b6..814f7af 100644 --- a/projects/tor-browser/build +++ b/projects/tor-browser/build @@ -52,6 +52,9 @@ mv [% c('input_files_by_name/noscript') %] $TBDIR/$EXTSPATH/{73a6fe31-595d-460b- tar -C $TBDIR -xf [% c('input_files_by_name/obfs4') %] tar -C $TBDIR -xf [% c('input_files_by_name/meek') %] mv $TBDIR/meek-http-helper(a)bamsoftware.com.xpi $TBDIR/$MEEKPROFILEPATH/extensions/ +[% IF c("var/snowflake") %] + tar -C $TBDIR -xf [% c('input_files_by_name/snowflake') -%] +[% END -%] tar -C $TBDIR[% IF c("var/osx") %]/Contents/Resources[% END %] -xf [% c('input_files_by_name/fonts') %] diff --git a/projects/tor-browser/config b/projects/tor-browser/config index 95f444d..f2c1d72 100644 --- a/projects/tor-browser/config +++ b/projects/tor-browser/config @@ -48,6 +48,9 @@ input_files: name: meek - project: obfs4 name: obfs4 + - project: snowflake + name: snowflake + enable: '[% c("var/snowflake") %]' - filename: Bundle-Data - URL: https://addons.cdn.mozilla.net/user-media/addons/722/noscript_security_suit… name: noscript diff --git a/projects/webrtc/build b/projects/webrtc/build new file mode 100644 index 0000000..5a70fbe --- /dev/null +++ b/projects/webrtc/build @@ -0,0 +1,80 @@ +#!/bin/bash +[% c("var/set_default_env") -%] +distdir=/var/tmp/dist/[% project %] +[% pc(c('var/compiler'), 'var/setup', { compiler_tarfile => c('input_files_by_name/' _ c('var/compiler')) }) %] + +mkdir -p /var/tmp/dist + +[% IF c("var/linux") %] + tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/binutils') %] + export PATH="/var/tmp/dist/binutils/bin:$PATH" +[% END -%] + +# Setting up depot_tools +# https://dev.chromium.org/developers/how-tos/install-depot-tools +tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/depot_tools') %] +export PATH="$PATH:/var/tmp/dist/depot_tools" +# Disable automatic updating. +export DEPOT_TOOLS_UPDATE=0 + +mkdir -p /var/tmp/build +tar -C /var/tmp/build -xf webrtc-sources-[% c('version') %].tar.gz + +builddir=/var/tmp/build/[% project %]/src +cd $builddir +export GYP_CROSSCOMPILE=1 +export GYP_DEFINES="OS=[% c("var/webrtc/os") %] target_arch=[% c("var/webrtc/arch") %]" +# Do not use bundled utilities. +GYP_DEFINES+=" clang=0 host_clang=0 use_sysroot=0" +GYP_DEFINES+=" linux_use_bundled_gold=0 linux_use_gold_flags=0 linux_use_bundled_binutils=0" +# Avoid some dependencies. Some of these don't seem to work fully; for +# example even with use_alsa=0 we need libasound2-dev. +GYP_DEFINES+=" use_x11=0 use_gtk=0 use_gconf=0 use_alsa=0 use_pulseaudio=0 use_libpci=0 include_internal_audio_device=0 enabled_libjingle_device_manager=0" +# examples and tests also bring in dependencies. +GYP_DEFINES+=" include_examples=0 include_tests=0" +# embedded=1 is supposed to turn of various features; see +# https://bugs.chromium.org/p/chromium/issues/detail?id=318413. +GYP_DEFINES+=" embedded=1" +GYP_DEFINES+=" werror=" +[% IF c("var/linux") -%] + export CC=gcc + export CXX=g++ +[% END -%] +[% IF c("var/linux-i686") -%] + GYP_DEFINES+=" target_arch=ia32" + export CC='gcc -m32' + export CXX='g++ -m32' + export CFLAGS=-m32 + export CXXFLAGS=-m32 + export LDFLAGS=-m32 +[% END -%] +JAVA_HOME=/usr/lib/jvm/default-java webrtc/build/gyp_webrtc.py +ninja -C out/Release +# Run ninja once more, without include_tests=0, in order to build just the +# FakeAudioCaptureModule that go-webrtc uses. +GYP_DEFINES="$(echo "$GYP_DEFINES" | sed -e 's/include_tests=0//g')" +JAVA_HOME=/usr/lib/jvm/default-java webrtc/build/gyp_webrtc.py +ninja -C out/Release obj/webrtc/api/test/peerconnection_unittests.fakeaudiocapturemodule.o +# https://github.com/keroserene/go-webrtc/issues/23#issuecomment-175312648 +# dump_syms_regtest.o is actually an executable, not an object file. If not +# excluded, it results in the error: +# libwebrtc-linux-386-magic.a(dump_syms_regtest.o): unsupported ELF file type 2 +ar crs libwebrtc-magic.a $(find . -name '*.o' -not -name '*.main.o' -not -name 'dump_syms_regtest.o' | sort) + +mkdir -p $distdir +cd $distdir +mkdir -p include lib +cp -f $builddir/libwebrtc-magic.a [% c("var/webrtc/lib_path") %] +INCLUDE_DIR="$PWD/include" +cd $builddir +for h in $(find talk/ webrtc/ -type f -name '*.h'); +do + mkdir -p "$INCLUDE_DIR/$(dirname $h)" + cp -f "$h" "$INCLUDE_DIR/$h" +done + +cd /var/tmp/dist +[% c('tar', { + tar_src => [ project ], + tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), + }) %] diff --git a/projects/webrtc/config b/projects/webrtc/config new file mode 100644 index 0000000..52618c2 --- /dev/null +++ b/projects/webrtc/config @@ -0,0 +1,115 @@ +# vim: filetype=yaml sw=2 +version: '[% c("var/webrtc_tag") %]' +remote_docker: 1 +filename: 'webrtc-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' + +var: + webrtc_tag: f33698296719f956497d2dbff81b5080864a8804 + +input_files: + - project: docker-image + - project: webrtc + pkg_type: fetch_sources + - project: depot_tools + name: depot_tools + - name: '[% c("var/compiler") %]' + project: '[% c("var/compiler") %]' + - project: binutils + name: binutils + enable: '[% c("var/linux") %]' + +targets: + linux: + var: + webrtc: + os: linux + linux-i686: + var: + dockerbuild: "[% pc('docker-image', 'pre') %]" + sort_deps: 0 + arch_deps: + - default-jre-headless + - default-jdk + - lib32asound2-dev + - lib64expat1 + - libexpat1-dev:i386 + - libudev-dev:i386 + - libx11-dev:i386 + - libxext-dev:i386 + - libxrandr-dev:i386 + - lib32stdc++6 + - hardening-wrapper + webrtc: + arch: ia32 + lib_arch: 386 + lib_path: lib/libwebrtc-linux-386-magic.a + linux-x86_64: + var: + arch_deps: + - default-jre-headless + - default-jdk + - libasound2-dev + - libexpat1-dev + - libudev-dev + - libx11-dev + - libxext-dev + - libxrandr-dev + webrtc: + arch: x64 + lib_arch: amd64 + lib_path: lib/libwebrtc-linux-amd64-magic.a + +steps: + fetch_sources: + remote_docker: 0 + filename: 'webrtc-sources-[% c("var/webrtc_tag") %].tar.gz' + fetch_sources: | + #!/bin/bash + [% c("var/set_default_env") -%] + # WebRTC is special, having its own build system that brings in lots of Chromium dependencies. + # https://webrtc.org/native-code/development/ + tar xf [% c('input_files_by_name/depot_tools') %] + export PATH="$rootdir/depot_tools:$PATH" + # GYP_CROSSCOMPILE=1 and GYP_DEFINES="use_x11=0" prevent probing for certain dependencies. + # Use --no-history because the whole checkout with history is about 12 GB. + export GYP_CROSSCOMPILE=1 + export GYP_DEFINES="use_x11=0" + # JAVA_HOME is needed in a hook for libjingle. The readlink line tries to find the current JRE. + # default-java comes from the package default-jdk-headless. + export JAVA_HOME=/usr/lib/jvm/default-java + clone_dir='[% c("basedir") %]/gclient/webrtc' + mkdir -p "$clone_dir" + cd "$clone_dir" + if [ ! -d "src" ]; + then + # "fetch" is part of depot_tools. + #fetch --nohooks --no-history webrtc + # FIXME: To avoid the unconditional `gclient sync` in the call to fetch, + # we inline the result of a `fetch --dry-run` + gclient root + gclient config --spec 'solutions = [ + { + "managed": False, + "name": "src", + "url": "https://chromium.googlesource.com/external/webrtc.git", + "custom_deps": {}, + "deps_file": "DEPS", + "safesync_url": "", + }, + ] + ' + gclient sync --nohooks --no-history --with_branch_heads -r [% c("var/webrtc_tag") %] + cd src + git submodule foreach 'git config -f $toplevel/.git/config submodule.$name.ignore all' + git config --add remote.origin.fetch '+refs/tags/*:refs/tags/*' + git config diff.ignoreSubmodules all + cd .. + fi + # "gclient" is part of depot_tools. This download takes a long time the first time. + gclient sync --no-history --with_branch_heads -r [% c("var/webrtc_tag") %] + cd .. + tar --exclude .git -czf [% dest_dir _ '/' _ c('filename') %] webrtc + input_files: + - project: depot_tools + name: depot_tools + pkg_type: build diff --git a/rbm.conf b/rbm.conf index e5c69b0..e206193 100644 --- a/rbm.conf +++ b/rbm.conf @@ -105,6 +105,7 @@ targets: linux-x86_64: arch: x86_64 var: + linux-x86_64: 1 osname: linux-x86_64 deps: - build-essential @@ -115,9 +116,12 @@ targets: - libtool - zip - unzip + # We only build snowflake on linux-x86_64 for now + snowflake: 1 linux-i686: arch: i686 var: + linux-i686: 1 osname: linux-i686 configure_opt: '--host=i686-linux-gnu CFLAGS=-m32 CXXFLAGS=-m32 LDFLAGS=-m32' pre_pkginst: dpkg --add-architecture i386

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

tbb-commits