[tor-commits] [Git][tpo/applications/tor-browser-build][maint-12.0] 6 commits: Bug 28124: Switch to Mozilla's libdmg-hfsplus
Pier Angelo Vendrame (@pierov)
git at gitlab.torproject.org
Wed Feb 1 13:11:15 UTC 2023
Pier Angelo Vendrame pushed to branch maint-12.0 at The Tor Project / Applications / tor-browser-build
Commits:
9b54da11 by Pier Angelo Vendrame at 2023-02-01T10:53:27+01:00
Bug 28124: Switch to Mozilla's libdmg-hfsplus
To show the DMG icon it seems we need to create the DMG from a HFS
filesystem, rather than an ISO one.
So, to then do so, with this commit I am switching to Mozilla's fork of
libdmg-hfsplus, I am updating its build script and using it to build
also the hfsplus tool.
Also, add the hfsplus project, which is needed to create the HFS
filesystem in the first place.
- - - - -
3eb81812 by Pier Angelo Vendrame at 2023-02-01T10:53:34+01:00
Bug 28124: Switch from ISO to HFS and show the disk icon
Use the new tools from the previous commit to build the DMG from a HFS
filesystem, and configure it to show the custom volume icon.
- - - - -
6f5d0bed by Pier Angelo Vendrame at 2023-02-01T10:53:34+01:00
Bug 28124: Update the macOS volume icon
- - - - -
70ffd274 by Pier Angelo Vendrame at 2023-02-01T10:53:47+01:00
Bug 40744: Ensure reproducibility with HFS DMG
- - - - -
1dc2335c by Nicolas Vigier at 2023-02-01T10:55:27+01:00
Bug 40755: Use openssl-1.0.2 for building libdmg-hfsplus outside containers
libdmg-hfsplus fails to build with openssl1.1:
https://github.com/planetbeing/libdmg-hfsplus/issues/14
- - - - -
1ec878d6 by Nicolas Vigier at 2023-02-01T10:55:34+01:00
Bug 40755: Allow building hfsplus-tools without container
If clang is insalled, building hfsplus-tools should work without
container.
- - - - -
14 changed files:
- projects/browser/Bundle-Data/mac-applications.dmg/.VolumeIcon.icns
- − projects/browser/Bundle-Data/mac-applications.dmg/Applications
- projects/browser/build
- projects/browser/config
- projects/browser/ddmg.sh
- + projects/hfsplus-tools/build
- + projects/hfsplus-tools/config
- + projects/hfsplus-tools/newfs_hfs.diff
- projects/libdmg-hfsplus/build
- projects/libdmg-hfsplus/config
- + projects/openssl-1.0.2/build
- + projects/openssl-1.0.2/config
- tools/signing/ddmg.sh
- tools/signing/gatekeeper-bundling.sh
Changes:
=====================================
projects/browser/Bundle-Data/mac-applications.dmg/.VolumeIcon.icns
=====================================
Binary files a/projects/browser/Bundle-Data/mac-applications.dmg/.VolumeIcon.icns and b/projects/browser/Bundle-Data/mac-applications.dmg/.VolumeIcon.icns differ
=====================================
projects/browser/Bundle-Data/mac-applications.dmg/Applications deleted
=====================================
@@ -1 +0,0 @@
-/Applications
\ No newline at end of file
=====================================
projects/browser/build
=====================================
@@ -33,8 +33,9 @@ touch "$GENERATEDPREFSPATH"
TORBINPATH=Contents/MacOS/Tor
TORCONFIGPATH=Contents/Resources/TorBrowser/Tor
+ tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/hfsplus-tools') %]
tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/libdmg') %]
- export PATH=/var/tmp/dist/libdmg-hfsplus:$PATH
+ export PATH=/var/tmp/dist/hfsplus-tools:/var/tmp/dist/libdmg-hfsplus:$PATH
[% ELSE %]
TBDIR=$TB_STAGE_DIR/Browser
TBDIRS=("$TBDIR")
=====================================
projects/browser/config
=====================================
@@ -33,7 +33,6 @@ targets:
macos:
var:
arch_deps:
- - genisoimage
- faketime
- python3-dev
- python3-pip
@@ -106,6 +105,9 @@ input_files:
sha256sum: 14af6a3cbc269c045f2d950e1e4f7c29981b35a7abc61d2413f5bb8bd7311857
- filename: 'gtk3-settings.ini'
enable: '[% c("var/linux") %]'
+ - project: hfsplus-tools
+ name: hfsplus-tools
+ enable: '[% c("var/macos") %]'
- project: libdmg-hfsplus
name: libdmg
enable: '[% c("var/macos") %]'
=====================================
projects/browser/ddmg.sh
=====================================
@@ -1,3 +1,6 @@
+#!/bin/bash
+set -e
+
[% SET src = c('dmg_src', { error_if_undef => 1 }) -%]
find [% src %] -executable -exec chmod 0755 {} \;
find [% src %] ! -executable -exec chmod 0644 {} \;
@@ -5,17 +8,33 @@ find [% src %] ! -executable -exec chmod 0644 {} \;
find [% src %] -exec [% c("touch") %] {} \;
dmg_tmpdir=\$(mktemp -d)
-[% SET filelist = '"\$dmg_tmpdir/filelist.txt"' %]
-pushd [% src %]
-find . -type f | sed -e 's/^\.\///' | sort | xargs -i echo "{}={}" > [% filelist %]
-find . -type l | sed -e 's/^\.\///' | sort | xargs -i echo "{}={}" >> [% filelist %]
+hfsfile="\$dmg_tmpdir/tbb-uncompressed.dmg"
+# hfsplus sets all the times to time(NULL)
export LD_PRELOAD=[% c("var/faketime_path") %]
export FAKETIME="[% USE date; GET date.format(c('timestamp'), format = '%Y-%m-%d %H:%M:%S') %]"
-genisoimage -D -V "Tor Browser" -no-pad -R -apple -o "\$dmg_tmpdir/tbb-uncompressed.dmg" -path-list [% filelist %] -graft-points -gid 20 -dir-mode 0755 -new-dir-mode 0755
+# Use a similar strategy to Mozilla (they have 1.02, we have 1.1)
+size=\$(du -ms [% src %] | awk '{ print int( \$1 * 1.1 ) }')
+dd if=/dev/zero of="\$hfsfile" bs=1M count=\$size
+newfs_hfs -v "[% c("var/Project_Name") %]" "\$hfsfile"
+
+pushd [% src %]
+
+find -type d -mindepth 1 | sed -e 's/^\.\///' | sort | while read dirname; do
+ hfsplus "\$hfsfile" mkdir "/\$dirname"
+ hfsplus "\$hfsfile" chmod 0755 "/\$dirname"
+done
+find -type f | sed -e 's/^\.\///' | sort | while read filename; do
+ hfsplus "\$hfsfile" add "\$filename" "/\$filename"
+ hfsplus "\$hfsfile" chmod \$(stat --format '0%a' "\$filename") "/\$filename"
+done
+# hfsplus does not play well with dangling links
+hfsplus "\$hfsfile" symlink /Applications /Applications
+# Show the volume icon
+hfsplus "\$hfsfile" attr / C
-dmg dmg "\$dmg_tmpdir/tbb-uncompressed.dmg" [% c('dmg_out', { error_if_undef => 1 }) %]
+dmg dmg "\$hfsfile" [% c('dmg_out', { error_if_undef => 1 }) %]
popd
rm -Rf "\$dmg_tmpdir"
=====================================
projects/hfsplus-tools/build
=====================================
@@ -0,0 +1,24 @@
+#!/bin/bash
+[% c("var/set_default_env") -%]
+distdir=/var/tmp/dist/[% project %]
+mkdir /var/tmp/dist
+[% IF ! c("container/global_disable") -%]
+ tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/clang') %]
+ export PATH="/var/tmp/dist/clang/bin:$PATH"
+[% END -%]
+
+tar -xf diskdev_cmds-[% c("version") %].tar.gz
+cd diskdev_cmds-[% c("version") %]
+
+patch -p1 < $rootdir/newfs_hfs.diff
+
+make -j[% c("num_procs") %]
+
+mkdir -p "$distdir"
+cp newfs_hfs.tproj/newfs_hfs "$distdir/"
+
+cd /var/tmp/dist
+[% c('tar', {
+ tar_src => [ project ],
+ tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'),
+ }) %]
=====================================
projects/hfsplus-tools/config
=====================================
@@ -0,0 +1,23 @@
+# vim: filetype=yaml sw=2
+version: 540.1.linux3
+filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz'
+container:
+ use_container: 1
+var:
+ deps:
+ - build-essential
+ - libssl-dev
+ - uuid-dev
+input_files:
+ # See hfsplus-tools in taskcluster/ci/fetch/toolchains.yml
+ - URL: https://src.fedoraproject.org/repo/pkgs/hfsplus-tools/diskdev_cmds-540.1.linux3.tar.gz/0435afc389b919027b69616ad1b05709/diskdev_cmds-[% c("version") %].tar.gz
+ sha256: b01b203a97f9a3bf36a027c13ddfc59292730552e62722d690d33bd5c24f5497
+ - project: container-image
+ # The project uses a flag that is not supported by GCC
+ - name: clang
+ project: clang
+ enable: '[% ! c("container/global_disable") %]'
+ # Build only newfs (we do not care of fsck), remove a header that does not
+ # exist on Linux (at that path) and is not required on Linux either, and make
+ # the UUID deterministic.
+ - filename: newfs_hfs.diff
=====================================
projects/hfsplus-tools/newfs_hfs.diff
=====================================
@@ -0,0 +1,38 @@
+diff '--color=auto' -Naur diskdev_cmds-540.1.linux3_orig/Makefile diskdev_cmds-540.1.linux3/Makefile
+--- diskdev_cmds-540.1.linux3_orig/Makefile 2023-01-17 11:36:56.341279443 +0100
++++ diskdev_cmds-540.1.linux3/Makefile 2023-01-17 11:44:12.496479981 +0100
+@@ -3,7 +3,7 @@
+ CC := clang
+ CFLAGS := -g3 -Wall -fblocks -I$(PWD)/BlocksRunTime -I$(PWD)/include -DDEBUG_BUILD=0 -D_FILE_OFFSET_BITS=64 -D LINUX=1 -D BSD=1 -D VERSION=\"$(VERSION)\"
+ LDFLAGS := -Wl,--build-id -L$(PWD)/BlocksRunTime
+-SUBDIRS := BlocksRunTime newfs_hfs.tproj fsck_hfs.tproj
++SUBDIRS := newfs_hfs.tproj
+
+ all clean:
+ for d in $(SUBDIRS); do $(MAKE) -C $$d -f Makefile.lnx $@; done
+diff '--color=auto' -Naur diskdev_cmds-540.1.linux3_orig/newfs_hfs.tproj/makehfs.c diskdev_cmds-540.1.linux3/newfs_hfs.tproj/makehfs.c
+--- diskdev_cmds-540.1.linux3_orig/newfs_hfs.tproj/makehfs.c 2023-01-17 11:36:56.341279443 +0100
++++ diskdev_cmds-540.1.linux3/newfs_hfs.tproj/makehfs.c 2023-01-17 11:58:15.972059719 +0100
+@@ -38,8 +38,8 @@
+ #endif
+ #include <sys/errno.h>
+ #include <sys/stat.h>
+-#include <sys/sysctl.h>
+ #if !LINUX
++#include <sys/sysctl.h>
+ #include <sys/vmmeter.h>
+ #endif
+
+@@ -571,8 +571,10 @@
+ /* Adjust free blocks to reflect everything we have allocated. */
+ hp->freeBlocks -= blocksUsed;
+
+- /* Generate and write UUID for the HFS+ disk */
+- GenerateVolumeUUID(&newVolumeUUID);
++ /* Use a deterministic UUID for reproducibility */
++ memset(&newVolumeUUID, 0, sizeof(newVolumeUUID));
++ strncpy(&newVolumeUUID, defaults->volumeName, sizeof(newVolumeUUID));
++
+ finderInfoUUIDPtr = (VolumeUUID *)(&hp->finderInfo[24]);
+ finderInfoUUIDPtr->v.high = OSSwapHostToBigInt32(newVolumeUUID.v.high);
+ finderInfoUUIDPtr->v.low = OSSwapHostToBigInt32(newVolumeUUID.v.low);
=====================================
projects/libdmg-hfsplus/build
=====================================
@@ -1,16 +1,26 @@
#!/bin/bash
[% c("var/set_default_env") -%]
-distdir=$(pwd)/dist
-mkdir -p $distdir/[% project %]
-tar xf [% project %]-[% c('version') %].tar.gz
-cd [% project %]-[% c('version') %]
-patch -p1 < ../libdmg.patch
-cmake -DCMAKE_INSTALL_PREFIX:PATH=$distdir/[% project %] CMakeLists.txt
-cd dmg
-make -j[% c("num_procs") %]
-make install
-cd $distdir
+distdir=/var/tmp/dist/[% project %]
+mkdir -p /var/tmp/dist
+tar -C /var/tmp/dist -xf [% c('input_files_by_name/cmake') %]
+tar -C /var/tmp/dist -xf [% c('input_files_by_name/ninja') %]
+[% IF c("container/global_disable") -%]
+ tar -C /var/tmp/dist -xf [% c('input_files_by_name/openssl-1.0.2') %]
+[% END -%]
+export PATH="/var/tmp/dist/ninja:/var/tmp/dist/cmake/bin:$PATH"
+
+mkdir /var/tmp/build
+tar -C /var/tmp/build -xf [% project %]-[% c('version') %].tar.gz
+cd /var/tmp/build/[% project %]-[% c('version') %]
+patch -p1 < "$rootdir/libdmg.patch"
+cmake . -GNinja -DCMAKE_BUILD_TYPE=Release [% c("var/cmake_opts") %]
+ninja -j[% c("num_procs") %] -v
+
+mkdir $distdir
+# We take only dmg and hfsplus like Mozilla does
+cp dmg/dmg hfs/hfsplus $distdir/
+cd /var/tmp/dist
[% c('tar', {
tar_src => [ project ],
tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'),
- }) %]
+ }) %]
=====================================
projects/libdmg-hfsplus/config
=====================================
@@ -1,16 +1,28 @@
# vim: filetype=yaml sw=2
version: '[% c("abbrev") %]'
-git_url: https://github.com/vasi/libdmg-hfsplus
-git_hash: dfd5e5cc3dc1191e37d3c3a6118975afdd1d7014
+git_url: https://github.com/mozilla/libdmg-hfsplus
+git_hash: 2ee327795680101d36f9700bd0fb618362237718
filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz'
container:
use_container: 1
var:
deps:
- build-essential
- - cmake
- zlib1g-dev
- libbz2-dev
+targets:
+ no_containers:
+ var:
+ cmake_opts: |
+ -DOPENSSL_USE_STATIC_LIBS=1 \
+ -DOPENSSL_ROOT_DIR=/var/tmp/dist/openssl
input_files:
- project: container-image
+ - name: cmake
+ project: cmake
+ - name: ninja
+ project: ninja
- filename: libdmg.patch
+ - name: openssl-1.0.2
+ project: openssl-1.0.2
+ enable: '[% c("container/global_disable") %]'
=====================================
projects/openssl-1.0.2/build
=====================================
@@ -0,0 +1,15 @@
+#!/bin/bash
+[% c("var/set_default_env") -%]
+distdir=/var/tmp/dist/openssl
+mkdir -p /var/tmp/build
+tar -C /var/tmp/build -xf openssl-[% c('version') %].tar.gz
+cd /var/tmp/build/openssl-[% c('version') %]
+export SOURCE_DATE_EPOCH='[% c("timestamp") %]'
+./Configure --prefix="$distdir" -shared enable-ec_nistp_64_gcc_128 linux-x86_64
+make -j[% c("num_procs") %]
+make -j[% c("num_procs") %] install
+cd /var/tmp/dist
+[% c('tar', {
+ tar_src => [ 'openssl' ],
+ tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'),
+ }) %]
=====================================
projects/openssl-1.0.2/config
=====================================
@@ -0,0 +1,11 @@
+# vim: filetype=yaml sw=2
+#
+# We need openssl-1.0.2 for building libdmg-hfsplus:
+# https://github.com/planetbeing/libdmg-hfsplus/issues/14
+#
+version: 1.0.2u
+filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz'
+
+input_files:
+ - URL: 'https://www.openssl.org/source/openssl-[% c("version") %].tar.gz'
+ sha256sum: ecd0c6ffb493dd06707d38b14bb4d8c2288bb7033735606569d8f90f89669d16
=====================================
tools/signing/ddmg.sh
=====================================
@@ -21,20 +21,40 @@ find $src_dir ! -executable -exec chmod 0644 {} \; 2> /dev/null
find $src_dir -exec touch -m -t 200001010101 {} \; 2> /dev/null
set -e
+VOLUME_LABEL="${VOLUME_LABEL:-Tor Browser}"
+
dmg_tmpdir=$(mktemp -d)
-filelist="$dmg_tmpdir/filelist.txt"
-cd $src_dir
-find . -type f | sed -e 's/^\.\///' | sort | xargs -i echo "{}={}" > $filelist
-find . -type l | sed -e 's/^\.\///' | sort | xargs -i echo "{}={}" >> $filelist
+hfsfile="$dmg_tmpdir/tbb-uncompressed.dmg"
export LD_PRELOAD=$faketime_path
export FAKETIME="2000-01-01 01:01:01"
echo "Starting: " $(basename $dest_file)
-genisoimage -D -V "Tor Browser" -no-pad -R -apple -o "$dmg_tmpdir/tbb-uncompressed.dmg" -path-list $filelist -graft-points -gid 20 -dir-mode 0755 -new-dir-mode 0755
+# Use a similar strategy to Mozilla (they have 1.02, we have 1.1)
+size=$(du -ms "$src_dir" | awk '{ print int( $1 * 1.1 ) }')
+dd if=/dev/zero of="$hfsfile" bs=1M count=$size
+newfs_hfs -v "$VOLUME_LABEL" "$hfsfile"
+
+cd $src_dir
-dmg dmg "$dmg_tmpdir/tbb-uncompressed.dmg" "$dest_file"
+# hfsplus does not play well with dangling links, so remove /Applications, and
+# add it back again with the special command to do so.
+rm -f Applications
+
+find -type d -mindepth 1 | sed -e 's/^\.\///' | sort | while read dirname; do
+ hfsplus "$hfsfile" mkdir "/$dirname"
+ hfsplus "$hfsfile" chmod 0755 "/$dirname"
+done
+find -type f | sed -e 's/^\.\///' | sort | while read filename; do
+ hfsplus "$hfsfile" add "$filename" "/$filename"
+ hfsplus "$hfsfile" chmod $(stat --format '0%a' "$filename") "/$filename"
+done
+hfsplus "$hfsfile" symlink /Applications /Applications
+# Show the volume icon
+hfsplus "$hfsfile" attr / C
+
+dmg dmg "$hfsfile" "$dest_file"
echo "Finished: " $(basename $dest_file)
=====================================
tools/signing/gatekeeper-bundling.sh
=====================================
@@ -35,18 +35,22 @@ set -e
script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
source "$script_dir/functions"
-which genisoimage > /dev/null || \
- exit_error 'genisoimage is missing. You should install the genisoimage package.'
test -f $faketime_path || \
exit_error "$faketime_path is missing"
test -d $macos_stapled_dir || \
exit_error "The stapled macos zip files should be placed in directory $macos_stapled_dir"
-libdmg_file="$script_dir/../../out/libdmg-hfsplus/libdmg-hfsplus-dfd5e5cc3dc1-c9296e.tar.gz"
+libdmg_file="$script_dir/../../out/libdmg-hfsplus/libdmg-hfsplus-2ee327795680-555a7e.tar.gz"
test -f "$libdmg_file" || \
exit_error "$libdmg_file is missing." \
"You can build it with:" \
" ./rbm/rbm build --target no_containers libdmg-hfsplus" \
"See var/deps in projects/libdmg-hfsplus/config for the list of build dependencies"
+hfstools_file="$script_dir/../../out/hfsplus-tools/hfsplus-tools-540.1.linux3-66de66.tar.gz"
+test -f "$hfstools_file" || \
+ exit_error "$hfstools_file is missing." \
+ "You can build it with:" \
+ " ./rbm/rbm build --target no_containers hfsplus-tools" \
+ "You will need the clang and uuid-dev packages installed"
test -d "$macos_signed_dir" || mkdir "$macos_signed_dir"
tmpdir="$macos_stapled_dir/tmp"
@@ -55,7 +59,8 @@ mkdir "$tmpdir"
cp -rT "$script_dir/../../projects/browser/Bundle-Data/mac-applications.dmg" "$tmpdir/dmg"
tar -C "$tmpdir" -xf "$libdmg_file"
-export PATH="$PATH:$tmpdir/libdmg-hfsplus"
+tar -C "$tmpdir" -xf "$hfstools_file"
+export PATH="$PATH:$tmpdir/libdmg-hfsplus:$tmpdir/hfsplus-tools"
for lang in $bundle_locales
do
View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/855a1723d903e320d260e295b457a67f8ebfd2de...1ec878d63548323eb473e920ce62207d35049dac
--
View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/855a1723d903e320d260e295b457a67f8ebfd2de...1ec878d63548323eb473e920ce62207d35049dac
You're receiving this email because of your account on gitlab.torproject.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-commits/attachments/20230201/42e866e4/attachment-0001.htm>
More information about the tor-commits
mailing list