On Sun, 25 Jul 2021, David Fifield wrote:

I noticed that the "Resolving deltas" step of git clones was using a maximum of 3 threads. I thought about altering the command line in git_clone_fetch_chdir to include something like: '--config', 'pack.threads=' . config('buildconf/num_procs') (See https://git-scm.com/docs/git-config#Documentation/git-config.txt-packthreads and https://git-scm.com/docs/git-index-pack#Documentation/git-index-pack.txt---t....)

But it looks like nothing inside RBM itself uses 'buildconf/num_procs', or even just 'buildconf'. It's only used by tor-browser-build files. Is there a way to use this information inside RBM, or does what I'm trying to do not make sense?

Yes, rbm doesn't know about buildconf/num_procs or anything in buildconf/*, it's only used in tor-browser-build.

We usually use the following prefixes:

- var/ for options used by tor-browser-build only (but not by rbm) - buildconf/ for options used by tor-browser-build only, that the user can change in rbm.local.conf, and that we reset when computing `var/build_id`, so that changing this option does not change the build ids (which would cause everything to be rebuilt) - no prefix for the options used by rbm itself

I think we can add support for a `git_opt` option, that rbm would use in all git commands. And then we would be able to add this line to rbm.conf:

git_opt: '--config pack.thread=[% c("buildconf/num_procs") %]'

rbm.conf passes a buildconf.num_procs hash into the "c" function when computing the build_id, but I was not able to find where it might be used in the template expansion. build_id: '[% sha256(c("var/build_id_txt", { buildconf => { num_procs => 4 } })).substr(0, 6) %]' https://gitweb.torproject.org/builders/tor-browser-build.git/tree/rbm.conf?h...

var/build_id is used to compute a hash representing a build, based on its input files, git commit, container used, and build files. In some of the build files we use buildconf/num_procs, but we don't really want the hash to change when we change buildconf/num_procs, as it's not supposed to affect the output, so we override it in var/build_id.