[builders/rbm] branch master updated (d3bc044 -> ce18f12)
This is an automated email from the git hooks/post-receive script. gk pushed a change to branch master in repository builders/rbm. from d3bc044 Bug 40039: Report project name when version is missing new 313f08f Bug 40027: Add support for rbm.module.conf files new 278411f Bug 40028: Load projects config from modules new 4713309 Bug 40028: Add support for input_files and template files in modules new 03ab77d Bug 40028: Add support for keyring directories in modules new 3f930c1 Bug 40028: Remove p variable from templates new ce18f12 Bug 40028: Add documentation for modules The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: doc/Makefile | 1 + doc/documentation.asc | 1 + doc/options_tar.asc | 7 +- doc/rbm_config.asc | 36 ++++++-- doc/rbm_input_files.asc | 6 +- doc/rbm_layout.asc | 80 +++++++++++------ doc/rbm_modules.asc | 54 ++++++++++++ doc/rbm_templates.asc | 4 - lib/RBM.pm | 99 +++++++++++++++++++--- lib/RBM/DefaultConfig.pm | 21 ++++- rbm | 1 + test.pl | 63 +++++++++++++- test/modules/module_1/projects/a/config | 1 + test/modules/module_1/projects/common/common_2.txt | 1 + test/modules/module_1/projects/common/common_3.txt | 1 + test/modules/module_1/projects/common/common_4.txt | 1 + test/modules/module_1/projects/m1_a/config | 3 + test/modules/module_1/projects/m1_a/m1_a-i.txt | 1 + test/modules/module_1/rbm.module.conf | 3 + test/modules/module_2/projects/common/common_4.txt | 1 + test/modules/module_2/rbm.module.conf | 2 + test/modules/module_3/projects/m3_a/config | 7 ++ test/modules/module_3/projects/m3_a/m3.txt | 1 + test/modules/module_3/rbm.module.conf | 2 + test/projects/a/config | 3 + test/projects/common/common_1.txt | 1 + test/projects/common/common_3.txt | 1 + 27 files changed, 336 insertions(+), 66 deletions(-) create mode 100644 doc/rbm_modules.asc create mode 100644 test/modules/module_1/projects/a/config create mode 100644 test/modules/module_1/projects/common/common_2.txt create mode 100644 test/modules/module_1/projects/common/common_3.txt create mode 100644 test/modules/module_1/projects/common/common_4.txt create mode 100644 test/modules/module_1/projects/m1_a/config create mode 100644 test/modules/module_1/projects/m1_a/m1_a-i.txt create mode 100644 test/modules/module_1/rbm.module.conf create mode 100644 test/modules/module_2/projects/common/common_4.txt create mode 100644 test/modules/module_2/rbm.module.conf create mode 100644 test/modules/module_3/projects/m3_a/config create mode 100644 test/modules/module_3/projects/m3_a/m3.txt create mode 100644 test/modules/module_3/rbm.module.conf create mode 100644 test/projects/common/common_1.txt create mode 100644 test/projects/common/common_3.txt -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. gk pushed a commit to branch master in repository builders/rbm. commit 313f08fcc9d9bd4ef1ba4a98e92de2dd06e8c216 Author: Nicolas Vigier <boklm@torproject.org> AuthorDate: Mon Jun 27 14:49:17 2022 +0200 Bug 40027: Add support for rbm.module.conf files --- lib/RBM.pm | 19 ++++++++++++++++++- lib/RBM/DefaultConfig.pm | 3 ++- rbm | 1 + test.pl | 15 ++++++++++++++- test/modules/module_1/rbm.module.conf | 2 ++ test/modules/module_2/rbm.module.conf | 2 ++ test/modules/module_3/rbm.module.conf | 2 ++ 7 files changed, 41 insertions(+), 3 deletions(-) diff --git a/lib/RBM.pm b/lib/RBM.pm index 24c6d44..7e29d3e 100644 --- a/lib/RBM.pm +++ b/lib/RBM.pm @@ -79,6 +79,21 @@ sub load_local_config { $config->{local} = -f $cfile ? load_config_file($cfile) : {}; } +sub load_modules_config { + my ($project) = @_; + $config->{modules} = {}; + my $modules_dir = project_config($project ? $project : 'undef', 'modules_dir'); + for my $dir (reverse @{as_array($modules_dir)}) { + my $d = rbm_path($dir); + next unless -d $d; + for my $module (map { $_->basename } path($d)->children) { + my $cfile = "$d/$module/rbm.module.conf"; + $config->{modules}{$module} = load_config_file($cfile) + if -f $cfile; + } + } +} + sub find_config_file { for (my $dir = getcwd; $dir ne '/'; $dir = dirname($dir)) { return "$dir/rbm.conf" if -f "$dir/rbm.conf"; @@ -232,9 +247,11 @@ sub project_config { goto FINISH unless @$name; my $opt_save = $config->{opt}; $config->{opt} = { %{$config->{opt}}, %$options } if $options; + my @modules = map { [ 'modules', $_ ] } + sort keys %{ $config->{modules} }; $res = config($project, $name, $options, ['opt', 'norec'], ['opt'], ['run'], ['projects', $project], ['local'], [], - ['system'], ['default']); + @modules, ['system'], ['default']); if (!$options->{no_tmpl} && defined($res) && !ref $res && !notmpl(confkey_str($name), $project)) { $res = process_template($project, $res, diff --git a/lib/RBM/DefaultConfig.pm b/lib/RBM/DefaultConfig.pm index ce75c1a..4fbe079 100644 --- a/lib/RBM/DefaultConfig.pm +++ b/lib/RBM/DefaultConfig.pm @@ -104,6 +104,7 @@ our %default_config = ( tmp_dir => '[% GET ENV.TMPDIR ? ENV.TMPDIR : "/tmp"; %]', rbm_tmp_dir => \&rbm_tmp_dir, projects_dir => 'projects', + modules_dir => 'modules', output_dir => 'out', git_clone_dir => 'git_clones', hg_clone_dir => 'hg_clones', @@ -112,7 +113,7 @@ our %default_config = ( build => '[% INCLUDE build -%]', build_log => '-', build_log_append => '1', - notmpl => [ qw(projects_dir) ], + notmpl => [ qw(projects_dir modules_dir) ], abbrev_length => '12', abbrev => '[% IF c("git_url"); diff --git a/rbm b/rbm index 764141e..c99de28 100755 --- a/rbm +++ b/rbm @@ -80,6 +80,7 @@ sub set_options { } RBM::load_system_config(@_); RBM::load_local_config(@_); + RBM::load_modules_config(@_); if (!defined $val{step} && @_) { $RBM::config->{step} = RBM::project_config($_[0], 'pkg_type'); } diff --git a/test.pl b/test.pl index 6c99cf0..0700d9c 100755 --- a/test.pl +++ b/test.pl @@ -1,7 +1,7 @@ #!/usr/bin/perl -w use strict; use Path::Tiny; -use Test::More tests => 29; +use Test::More tests => 31; use lib 'lib/'; sub set_target { @@ -19,6 +19,7 @@ sub set_step { BEGIN { use_ok('RBM') }; chdir 'test'; RBM::load_config; +RBM::load_modules_config; RBM::set_default_env; ok($RBM::config, 'load config'); @@ -145,6 +146,18 @@ my @tests = ( config => [ 'c', 'option_deb' ], expected => '1', }, + { + name => 'Using option from rbm.module.conf', + target => [], + config => [ 'b', 'module_3'], + expected => '3', + }, + { + name => 'Using option defined in multiple rbm.module.conf', + target => [], + config => [ 'b', 'module_m'], + expected => '1', + }, { name => 'build + steps config - 1', target => [ 'version_1' ], diff --git a/test/modules/module_1/rbm.module.conf b/test/modules/module_1/rbm.module.conf new file mode 100644 index 0000000..243b870 --- /dev/null +++ b/test/modules/module_1/rbm.module.conf @@ -0,0 +1,2 @@ +module_m: 1 +module_1: 1 diff --git a/test/modules/module_2/rbm.module.conf b/test/modules/module_2/rbm.module.conf new file mode 100644 index 0000000..5a79ea6 --- /dev/null +++ b/test/modules/module_2/rbm.module.conf @@ -0,0 +1,2 @@ +module_m: 2 +module_2: 2 diff --git a/test/modules/module_3/rbm.module.conf b/test/modules/module_3/rbm.module.conf new file mode 100644 index 0000000..ed43fbd --- /dev/null +++ b/test/modules/module_3/rbm.module.conf @@ -0,0 +1,2 @@ +module_m: 3 +module_3: 3 -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. gk pushed a commit to branch master in repository builders/rbm. commit 278411fb12ef71b6f2973ce70669118748474124 Author: Nicolas Vigier <boklm@torproject.org> AuthorDate: Mon Jun 27 15:40:21 2022 +0200 Bug 40028: Load projects config from modules --- lib/RBM.pm | 31 ++++++++++++++++++++++++------ test.pl | 17 +++++++++++++++- test/modules/module_1/projects/a/config | 1 + test/modules/module_1/projects/m1_a/config | 2 ++ 4 files changed, 44 insertions(+), 7 deletions(-) diff --git a/lib/RBM.pm b/lib/RBM.pm index 7e29d3e..ee987c8 100644 --- a/lib/RBM.pm +++ b/lib/RBM.pm @@ -90,6 +90,13 @@ sub load_modules_config { my $cfile = "$d/$module/rbm.module.conf"; $config->{modules}{$module} = load_config_file($cfile) if -f $cfile; + next unless -d "$d/$module/projects"; + for my $project (map { $_->basename } path("$d/$module/projects")->children) { + next if $config->{projects}{$project}; + next unless -f "$d/$module/projects/$project/config"; + $config->{modules}{$module}{projects}{$project} = + load_config_file("$d/$module/projects/$project/config"); + } } } } @@ -249,8 +256,17 @@ sub project_config { $config->{opt} = { %{$config->{opt}}, %$options } if $options; my @modules = map { [ 'modules', $_ ] } sort keys %{ $config->{modules} }; + my $project_path = ['projects', $project]; + if (!$config->{projects}{$project}) { + for my $module (sort keys %{ $config->{modules} }) { + if ($config->{modules}{$module}{projects}{$project}) { + $project_path = [ 'modules', $module, 'projects', $project ]; + last; + } + } + } $res = config($project, $name, $options, ['opt', 'norec'], ['opt'], - ['run'], ['projects', $project], ['local'], [], + ['run'], $project_path, ['local'], [], @modules, ['system'], ['default']); if (!$options->{no_tmpl} && defined($res) && !ref $res && !notmpl(confkey_str($name), $project)) { @@ -381,8 +397,11 @@ sub valid_id { sub valid_project { my ($project) = @_; - exists $config->{projects}{$project} - || exit_error "Unknown project $project"; + return 1 if $config->{projects}{$project}; + for my $module (keys %{$config->{modules}}) { + return 1 if $config->{modules}{$module}{projects}{$project}; + } + exit_error "Unknown project $project"; } sub create_dir { @@ -395,7 +414,7 @@ sub create_dir { sub git_need_fetch { my ($project, $options) = @_; - return 0 if $config->{projects}{$project}{fetched}; + return 0 if $config->{_rbm}{fetched_projects}{$project}; my $fetch = project_config($project, 'fetch', $options); if ($fetch eq 'if_needed') { my $git_hash = project_config($project, 'git_hash', $options) @@ -440,13 +459,13 @@ sub git_clone_fetch_chdir { system('git', 'fetch', @fetch_submod, 'origin', '+refs/tags/*:refs/tags/*') == 0 || exit_error "Error fetching git repository"; - $config->{projects}{$project}{fetched} = 1; + $config->{_rbm}{fetched_projects}{$project} = 1; } } sub hg_need_fetch { my ($project, $options) = @_; - return 0 if $config->{projects}{$project}{fetched}; + return 0 if $config->{_rbm}{fetched_projects}{$project}; my $fetch = project_config($project, 'fetch', $options); if ($fetch eq 'if_needed') { my $hg_hash = project_config($project, 'hg_hash', $options) diff --git a/test.pl b/test.pl index 0700d9c..d71baaa 100755 --- a/test.pl +++ b/test.pl @@ -1,7 +1,7 @@ #!/usr/bin/perl -w use strict; use Path::Tiny; -use Test::More tests => 31; +use Test::More tests => 34; use lib 'lib/'; sub set_target { @@ -158,6 +158,21 @@ my @tests = ( config => [ 'b', 'module_m'], expected => '1', }, + { + name => 'Using option defined in a module project', + config => [ 'm1_a', 'm1_a' ], + expected => 'm1_a', + }, + { + name => 'Using option defined in a module project and rbm.module.conf', + config => [ 'm1_a', 'project_m' ], + expected => 'm1_a', + }, + { + name => 'Using option defined in main projects and a module project', + config => [ 'a', 'project_a' ], + expected => 'a', + }, { name => 'build + steps config - 1', target => [ 'version_1' ], diff --git a/test/modules/module_1/projects/a/config b/test/modules/module_1/projects/a/config new file mode 100644 index 0000000..2f2e097 --- /dev/null +++ b/test/modules/module_1/projects/a/config @@ -0,0 +1 @@ +project_a: from_module_1 diff --git a/test/modules/module_1/projects/m1_a/config b/test/modules/module_1/projects/m1_a/config new file mode 100644 index 0000000..9f8b6c6 --- /dev/null +++ b/test/modules/module_1/projects/m1_a/config @@ -0,0 +1,2 @@ +m1_a: m1_a +project_m: m1_a -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. gk pushed a commit to branch master in repository builders/rbm. commit 471330953fb61bab10bff35bb74c8dbb69094cfb Author: Nicolas Vigier <boklm@torproject.org> AuthorDate: Mon Jun 27 18:48:38 2022 +0200 Bug 40028: Add support for input_files and template files in modules --- lib/RBM.pm | 48 +++++++++++++++++++--- test.pl | 35 +++++++++++++++- test/modules/module_1/projects/common/common_2.txt | 1 + test/modules/module_1/projects/common/common_3.txt | 1 + test/modules/module_1/projects/common/common_4.txt | 1 + test/modules/module_1/projects/m1_a/config | 1 + test/modules/module_1/projects/m1_a/m1_a-i.txt | 1 + test/modules/module_1/rbm.module.conf | 1 + test/modules/module_2/projects/common/common_4.txt | 1 + test/modules/module_3/projects/m3_a/config | 7 ++++ test/modules/module_3/projects/m3_a/m3.txt | 1 + test/projects/a/config | 3 ++ test/projects/common/common_1.txt | 1 + test/projects/common/common_3.txt | 1 + 14 files changed, 96 insertions(+), 7 deletions(-) diff --git a/lib/RBM.pm b/lib/RBM.pm index ee987c8..afbe003 100644 --- a/lib/RBM.pm +++ b/lib/RBM.pm @@ -683,10 +683,11 @@ sub process_template { return $res; } $dest_dir //= rbm_path(project_config($project, 'output_dir')); - my $projects_dir = rbm_path(project_config($project, 'projects_dir')); + my $project_dir = modules_project_dir($project); + my $common_dirs = join(':', modules_common_dirs($project)); my $template = Template->new( ENCODING => 'utf8', - INCLUDE_PATH => "$projects_dir/$project:$projects_dir/common", + INCLUDE_PATH => "$project_dir:$common_dirs", ); my $vars = { config => $config, @@ -815,6 +816,39 @@ sub recursive_copy { return @copied; } +sub modules_project_dir { + my ($project, $options) = @_; + my $proj_dir = rbm_path(project_config($project, 'projects_dir', $options)); + return "$proj_dir/$project" if -f "$proj_dir/$project/config"; + my $modules_dir = project_config($project, 'modules_dir'); + for my $dir (@{as_array($modules_dir)}) { + my $d = rbm_path($dir); + next unless -d $d; + for my $module (sort map { $_->basename } path($d)->children) { + my $pdir = "$d/$module/projects/$project"; + return $pdir if -f "$pdir/config"; + } + } + return "$proj_dir/$project"; +} + +sub modules_common_dirs { + my ($project, $options) = @_; + #my $proj_dir = rbm_path(project_config($project, 'projects_dir', $options)); + my $proj_dir = rbm_path('projects'); + my @cdirs = ("$proj_dir/common"); + my $modules_dir = project_config($project, 'modules_dir'); + for my $dir (@{as_array($modules_dir)}) { + my $d = rbm_path($dir); + next unless -d $d; + for my $module (sort map { $_->basename } path($d)->children) { + push @cdirs, "$d/$module/projects/common" + if -d "$d/$module/projects/common"; + } + } + return @cdirs; +} + sub input_files { my ($action, $project, $options, $dest_dir) = @_; my @res_copy; @@ -827,8 +861,8 @@ sub input_files { my $input_files = project_config($project, 'input_files', $options); goto RETURN_RES unless $input_files; my $proj_dir = rbm_path(project_config($project, 'projects_dir', $options)); - my $src_dir = "$proj_dir/$project"; - my $common_dir = "$proj_dir/common"; + my $src_dir = modules_project_dir($project, $options); + my @modules_common_dirs = modules_common_dirs($project, $options); my $old_cwd = getcwd; chdir $src_dir || exit_error "cannot chdir to $src_dir"; foreach my $input_file_alias (@$input_files) { @@ -919,7 +953,8 @@ sub input_files { origin_project => $project, %$input_file}) if $input_file->{project}; exit_error("Missing filename:\n" . pp($input_file)) unless $name; - my ($fname) = file_in_dir($name, $src_dir, $proj_out_dir, $common_dir); + my ($fname) = file_in_dir($name, $src_dir, $proj_out_dir, + @modules_common_dirs); my $file_gpg_id = gpg_id($t->('file_gpg_id')); if (input_file_need_dl($input_file, $t, $fname, $action)) { if ($t->('content')) { @@ -947,7 +982,8 @@ sub input_files { dd $input_file; exit_error "Missing file $name"; } - ($fname) = file_in_dir($name, $src_dir, $proj_out_dir, $common_dir); + ($fname) = file_in_dir($name, $src_dir, $proj_out_dir, + @modules_common_dirs); exit_error "Error getting file $name" unless $fname; } if ($action eq 'input_files_id') { diff --git a/test.pl b/test.pl index d71baaa..8992ad7 100755 --- a/test.pl +++ b/test.pl @@ -1,7 +1,7 @@ #!/usr/bin/perl -w use strict; use Path::Tiny; -use Test::More tests => 34; +use Test::More tests => 40; use lib 'lib/'; sub set_target { @@ -173,6 +173,31 @@ my @tests = ( config => [ 'a', 'project_a' ], expected => 'a', }, + { + name => 'Using template file from common project', + config => [ 'a', 'c_1' ], + expected => "c1\n", + }, + { + name => 'Using template file from common project in a module', + config => [ 'a', 'c_2' ], + expected => "c2\n", + }, + { + name => 'Using template file in multiple common directories', + config => [ 'a', 'c_3' ], + expected => "c3_main\n", + }, + { + name => 'Using template file in multiple modules common directories', + config => [ 'a', 'c_4' ], + expected => "c4_module1\n", + }, + { + name => 'Using template file in project directories in a module', + config => [ 'm1_a', 'i' ], + expected => "i1\n", + }, { name => 'build + steps config - 1', target => [ 'version_1' ], @@ -195,6 +220,14 @@ my @tests = ( 'out/r3' => "1 - build\n2 - build\n3 - build\n", }, }, + { + name => 'build project in a module', + target => [], + build => [ 'm3_a', 'build', { pkg_type => 'build' } ], + files => { + 'out/m3-output' => "1 - build\n___m3\n" + }, + }, { name => 'mercurial repo', target => [], diff --git a/test/modules/module_1/projects/common/common_2.txt b/test/modules/module_1/projects/common/common_2.txt new file mode 100644 index 0000000..16f9ec0 --- /dev/null +++ b/test/modules/module_1/projects/common/common_2.txt @@ -0,0 +1 @@ +c2 diff --git a/test/modules/module_1/projects/common/common_3.txt b/test/modules/module_1/projects/common/common_3.txt new file mode 100644 index 0000000..7b6a471 --- /dev/null +++ b/test/modules/module_1/projects/common/common_3.txt @@ -0,0 +1 @@ +c3_module diff --git a/test/modules/module_1/projects/common/common_4.txt b/test/modules/module_1/projects/common/common_4.txt new file mode 100644 index 0000000..e0c56bd --- /dev/null +++ b/test/modules/module_1/projects/common/common_4.txt @@ -0,0 +1 @@ +c4_module1 diff --git a/test/modules/module_1/projects/m1_a/config b/test/modules/module_1/projects/m1_a/config index 9f8b6c6..9426965 100644 --- a/test/modules/module_1/projects/m1_a/config +++ b/test/modules/module_1/projects/m1_a/config @@ -1,2 +1,3 @@ m1_a: m1_a project_m: m1_a +i: '[% INCLUDE "m1_a-i.txt" -%]' diff --git a/test/modules/module_1/projects/m1_a/m1_a-i.txt b/test/modules/module_1/projects/m1_a/m1_a-i.txt new file mode 100644 index 0000000..ca49db8 --- /dev/null +++ b/test/modules/module_1/projects/m1_a/m1_a-i.txt @@ -0,0 +1 @@ +i1 diff --git a/test/modules/module_1/rbm.module.conf b/test/modules/module_1/rbm.module.conf index 243b870..1a3dd98 100644 --- a/test/modules/module_1/rbm.module.conf +++ b/test/modules/module_1/rbm.module.conf @@ -1,2 +1,3 @@ module_m: 1 module_1: 1 +c_2: '[% INCLUDE common_2.txt %]' diff --git a/test/modules/module_2/projects/common/common_4.txt b/test/modules/module_2/projects/common/common_4.txt new file mode 100644 index 0000000..2a9eb48 --- /dev/null +++ b/test/modules/module_2/projects/common/common_4.txt @@ -0,0 +1 @@ +c4_module2 diff --git a/test/modules/module_3/projects/m3_a/config b/test/modules/module_3/projects/m3_a/config new file mode 100644 index 0000000..3337d35 --- /dev/null +++ b/test/modules/module_3/projects/m3_a/config @@ -0,0 +1,7 @@ +filename: m3-output +build: | + #!/bin/sh + echo 1 - [% c('pkg_type') %] > [% dest_dir %]/[% c('filename') %] + cat m3.txt >> [% dest_dir %]/[% c('filename') %] +input_files: + - filename: m3.txt diff --git a/test/modules/module_3/projects/m3_a/m3.txt b/test/modules/module_3/projects/m3_a/m3.txt new file mode 100644 index 0000000..7f6edf0 --- /dev/null +++ b/test/modules/module_3/projects/m3_a/m3.txt @@ -0,0 +1 @@ +___m3 diff --git a/test/projects/a/config b/test/projects/a/config index c578032..28866fe 100644 --- a/test/projects/a/config +++ b/test/projects/a/config @@ -17,3 +17,6 @@ z_2: '[% c("z", { option_a => "Z" }) %]' Z_1: '[% c("z_1") _ c("z_2") _ c("z_1") %]' Z_2: '[% pc("a", "z_1", { step => "S"}) _ c("z_2") _ c("z_1") %]' Z: '[% c("Z_1") _ " " _ c("Z_2") _ " " _ c("Z_1") %]' +c_1: '[% INCLUDE common_1.txt -%]' +c_3: '[% INCLUDE common_3.txt -%]' +c_4: '[% INCLUDE common_4.txt -%]' diff --git a/test/projects/common/common_1.txt b/test/projects/common/common_1.txt new file mode 100644 index 0000000..ae93045 --- /dev/null +++ b/test/projects/common/common_1.txt @@ -0,0 +1 @@ +c1 diff --git a/test/projects/common/common_3.txt b/test/projects/common/common_3.txt new file mode 100644 index 0000000..4fb18ff --- /dev/null +++ b/test/projects/common/common_3.txt @@ -0,0 +1 @@ +c3_main -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. gk pushed a commit to branch master in repository builders/rbm. commit 03ab77d9190a1c700ca442160cae0e4d7b732858 Author: Nicolas Vigier <boklm@torproject.org> AuthorDate: Tue Jun 28 14:10:59 2022 +0200 Bug 40028: Add support for keyring directories in modules We also remove the `gpg_keyring_dir` option since this would make things more complex, and changing the `keyring` directory name is not very useful. --- doc/options_tar.asc | 7 +------ doc/rbm_input_files.asc | 6 +++--- doc/rbm_layout.asc | 3 +-- lib/RBM/DefaultConfig.pm | 18 +++++++++++++++--- 4 files changed, 20 insertions(+), 14 deletions(-) diff --git a/doc/options_tar.asc b/doc/options_tar.asc index e617c0b..b615dec 100644 --- a/doc/options_tar.asc +++ b/doc/options_tar.asc @@ -57,12 +57,7 @@ gpg_wrapper:: gpg_keyring:: The filename of the gpg keyring to use. Path is relative to the - +gpg_keyring_dir+ directory. This can also be an absolute path. - -gpg_keyring_dir:: - The directory containing gpg keyring files. The default is - +$basedir/keyring+ (with $basedir the directory where the main - config file is located). + keyring directory. This can also be an absolute path. gpg_bin:: The gpg command to be used. The default is +gpg+. diff --git a/doc/rbm_input_files.asc b/doc/rbm_input_files.asc index abe366b..0c450eb 100644 --- a/doc/rbm_input_files.asc +++ b/doc/rbm_input_files.asc @@ -112,9 +112,9 @@ file_gpg_id:: If this option is set to 1, the file is checked for a gpg signature. If it is set to an other non zero value, or an array of non zero values, those are expected to be the valid gpg key - IDs. The +gpg_wrapper+, +gpg_keyring+, +gpg_keyring_dir+, - +gpg_bin+, +gpg_args+ options can be used to configure the gpg - check. Those options can be overriden in the file descriptor. + IDs. The +gpg_wrapper+, +gpg_keyring+, +gpg_bin+, +gpg_args+ + options can be used to configure the gpg check. Those options + can be overriden in the file descriptor. sig_ext:: An array of file extensions for the gpg signature file. Those diff --git a/doc/rbm_layout.asc b/doc/rbm_layout.asc index ef700cb..c85658d 100644 --- a/doc/rbm_layout.asc +++ b/doc/rbm_layout.asc @@ -96,8 +96,7 @@ hg_clones:: keyring:: This is the directory where you can store gpg keyring files. Those keyrings contains the keys that are used to check signed - commits and tags. The path of this directory can be changed with - the 'gpg_keyring_dir' option. + commits and tags. projects:: This is the directory containing projects definitions. The path diff --git a/lib/RBM/DefaultConfig.pm b/lib/RBM/DefaultConfig.pm index 4fbe079..109ca8d 100644 --- a/lib/RBM/DefaultConfig.pm +++ b/lib/RBM/DefaultConfig.pm @@ -146,13 +146,25 @@ our %default_config = ( #### gpg_bin => 'gpg', gpg_args => '', - gpg_keyring_dir => '[% config.basedir %]/keyring', + gpg_keyring_path => sub { + my ($project, $options) = @_; + my $gpg_keyring = RBM::project_config($project, 'gpg_keyring', $options); + return undef unless $gpg_keyring; + return $gpg_keyring if $gpg_keyring =~ m|^/|; + my $rootpath = RBM::rbm_path("keyring/$gpg_keyring"); + return $rootpath if -f $rootpath; + for my $module (sort keys %{$RBM::config->{modules}}) { + my $modulepath = RBM::rbm_path("modules/$module/keyring/$gpg_keyring"); + return $modulepath if -f $modulepath; + } + RBM::exit_error("keyring file $gpg_keyring is missing") + }, gpg_wrapper => <<GPGEND, #!/bin/sh export LC_ALL=C [% - IF c('gpg_keyring'); - SET gpg_kr = '--keyring ' _ path(c('gpg_keyring'), path(c('gpg_keyring_dir'))) + IF c('gpg_keyring_path'); + SET gpg_kr = '--keyring ' _ c('gpg_keyring_path') _ ' --no-default-keyring --no-auto-check-trustdb --trust-model always'; END; -%] -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. gk pushed a commit to branch master in repository builders/rbm. commit 3f930c1dddee914f78817259f3eab113c698efc3 Author: Nicolas Vigier <boklm@torproject.org> AuthorDate: Wed Jun 29 14:06:29 2022 +0200 Bug 40028: Remove p variable from templates In templates, the `p` variable allowed to access the current project's rbm config. With the support for projects in modules, it would need to be adapted. However we never used this variable in tor-browser-build, and it is not very useful (it's usually better to use c() to access config), so we remove it. --- doc/rbm_templates.asc | 4 ---- lib/RBM.pm | 1 - 2 files changed, 5 deletions(-) diff --git a/doc/rbm_templates.asc b/doc/rbm_templates.asc index 200043f..41be18d 100644 --- a/doc/rbm_templates.asc +++ b/doc/rbm_templates.asc @@ -89,10 +89,6 @@ pc:: project:: The name of the project for which we are processing a template. -p:: - The project's configuration. This is a shortcut for the value - of `config.projects.$project`. - dest_dir:: The destination directory, where the resulting files will be stored at the end of the build. This is mainly useful in build diff --git a/lib/RBM.pm b/lib/RBM.pm index afbe003..88679f2 100644 --- a/lib/RBM.pm +++ b/lib/RBM.pm @@ -692,7 +692,6 @@ sub process_template { my $vars = { config => $config, project => $project, - p => $config->{projects}{$project}, c => sub { project_config($project, @_) }, pc => sub { my @args = @_; -- To stop receiving notification emails like this one, please contact the administrator of this repository.
This is an automated email from the git hooks/post-receive script. gk pushed a commit to branch master in repository builders/rbm. commit ce18f12cc5114ca82dc2964a62bce835dcc5a1f0 Author: Nicolas Vigier <boklm@torproject.org> AuthorDate: Wed Jun 29 16:26:41 2022 +0200 Bug 40028: Add documentation for modules --- doc/Makefile | 1 + doc/documentation.asc | 1 + doc/rbm_config.asc | 36 +++++++++++++++++++----- doc/rbm_layout.asc | 77 ++++++++++++++++++++++++++++++++++----------------- doc/rbm_modules.asc | 54 ++++++++++++++++++++++++++++++++++++ 5 files changed, 137 insertions(+), 32 deletions(-) diff --git a/doc/Makefile b/doc/Makefile index 8564a19..7aeb0f2 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -33,6 +33,7 @@ MAN1_ASC += rbm-build.asc MAN7_ASC += rbm_tutorial.asc MAN7_ASC += rbm_layout.asc MAN7_ASC += rbm_config.asc +MAN7_ASC += rbm_modules.asc MAN7_ASC += rbm_templates.asc MAN7_ASC += rbm_remote.asc MAN7_ASC += rbm_targets.asc diff --git a/doc/documentation.asc b/doc/documentation.asc index 4c5c9bc..eabf738 100644 --- a/doc/documentation.asc +++ b/doc/documentation.asc @@ -1,6 +1,7 @@ - link:rbm_tutorial.html[rbm_tutorial(7)] : A tutorial introduction to rbm - link:rbm_layout.html[rbm_layout(7)] : The rbm layout - link:rbm_config.html[rbm_config(7)] : The rbm configuration +- link:rbm_modules.html[rbm_modules(7)] : The rbm modules - link:rbm_cli.html[rbm_cli(7)] : The rbm command line interface - link:rbm_templates.html[rbm_templates(7)] : The rbm templates - link:rbm_remote.html[rbm_remote(7)] : The rbm remote builds diff --git a/doc/rbm_config.asc b/doc/rbm_config.asc index 0bfed04..46c1caf 100644 --- a/doc/rbm_config.asc +++ b/doc/rbm_config.asc @@ -8,10 +8,13 @@ rbm_config - The rbm configuration DESCRIPTION ----------- -All configuration options can be defined in 5 different places : +All configuration options can be defined in different places : - in the main configuration in your working directory (+rbm.conf+) +- in a module configuration at the root of a module directory + (+rbm.module.conf+) + - in the local configuration in your working directory (+rbm.local.conf+) - in the global system configuration (+/etc/rbm.conf+) @@ -48,6 +51,14 @@ The option values are used with the following priority order : - workspace config +- modules config for matching step and target + +- modules config for matching step + +- modules config for matching target + +- modules config + - system config for matching step and target - system config for matching step @@ -79,23 +90,33 @@ file is used to override some options from +rbm.conf+ relevent to your local setup, whithout modifying the +rbm.conf+ file which is usually maintained in a VCS. +The module configuration files are +rbm.module.conf+ located at the +root of each module directory: a module named +A+ will have its +configuration in +modules/A/rbm.module.conf+. Module configuration +is used in alphabetical order: configuration from a module named ++A+ will have priority over a module named +B+. + An example +rbm.conf+ file will look like this : ---- -projects_dir: projects compress_tar: xz +output_dir: "out/[% project %]" ---- -The +projects_dir+ option define the path to the directory containing -the projects definitions. - Adding a new project is done by creating a directory with the name of -the project inside the +projects_dir+ directory, and adding a +config+ +the project inside the +projects+ directory, and adding a +config+ file in this new directory. The +config+ file contains the configuration for the project. At the minimum it should contain the +git_url+ configuration, and any other configuration option you want to set for this project. +Projects can also be defined inside a module. In this case the module +directory will include a +projects+ directory working in a similar way +to the main +projects+ directory. If a project exists with the same +name both in a module and the main +projects+ directory, the module one +will be ignored. If a project is defined in multiple modules, only the +project from the first module in alphabetic order will be used. + OPTIONS ------- @@ -337,4 +358,5 @@ SEE ALSO -------- link:rbm.html[rbm(1)], link:rbm_targets.html[rbm_targets(7)], -link:rbm_templates.html[rbm_templates(7)] +link:rbm_templates.html[rbm_templates(7)], +link:rbm_modules.html[rbm_modules(7)] diff --git a/doc/rbm_layout.asc b/doc/rbm_layout.asc index c85658d..1261cf2 100644 --- a/doc/rbm_layout.asc +++ b/doc/rbm_layout.asc @@ -26,15 +26,17 @@ and 'project_2' will look like this : | | | |-- project_1 | | | - | | |-- project_1-0.1.tar.gz + | | |-- project_1-0.1-src.tar.gz | | | | | +-- project_1-0.1-1.x86_64.rpm + | | | + | | +-- project_1-0.1-1.x86_64.tar.gz | | | +-- project_2 | | - | |-- project_2-7.3.tar.bz2 + | |-- project_2-7.3.tar-src.bz2 | | - | +-- project_2-7.1-1.x86_64.rpm + | +-- project_2-7.1-1.x86_64.tar.gz | |-- git_clones | | @@ -46,29 +48,43 @@ and 'project_2' will look like this : | |-- keyring | | - | |-- keyring-1 + | |-- project_1.gpg | | - | +-- keyring-2 + | +-- project_2.gpg | +-- projects + | | + | |-- project_1 + | | | + | | |-- config + | | | + | | |-- project_1.spec + | | | + | | |-- bugfix.patch + | | + | +-- project_2 + | | + | |-- config + | | + | +-- build + | + +-- modules | - |-- project_1 - | | - | |-- config - | | - | |-- project_1.spec - | | - | |-- bugfix.patch - | | - | +-- debian_rules - | - +-- project_2 + +-- container | - |-- config + |-- rbm.module.conf | - |-- project_2.spec + |-- projects + | | + | +-- container-image + | | + | |-- config + | | + | +-- build | - +-- debian_rules + +-- keyring + | + +-- container-image.gpg ---- You will file the following files and directories : @@ -115,13 +131,24 @@ projects/project_1/project_1.spec:: projects/project_1/bugfix.patch:: This is an example patch file, used in the rpm package. -projects/project_1/debian_rules:: - This is an example path for a debian/rules file to be used in - the debian packaging for the project_1 project. All the debian - packaging can be defined entirely inside the 'config' file, but - it is usually convenient to split some of it in separate files. +projects/project_1/build:: + This is a build script for project_1, containing instructions + for building the project and generating a tar.gz file containing + the build output. + +modules/container/rbm.local.conf:: + In this example we use an external rbm module named 'container' + (usually included as a git-submodule). The 'rbm.local.conf' file + located at the root of the module contains the options related + to the projects included in the module. + +modules/container/projects:: + The projects included in the projects directory of a module + work in the same way as those in the main projects directory. + SEE ALSO -------- link:rbm.html[rbm(1)], -link:rbm_config.html[rbm_config(1)] +link:rbm_config.html[rbm_config(7)], +link:rbm_modules.html[rbm_modules(7)] diff --git a/doc/rbm_modules.asc b/doc/rbm_modules.asc new file mode 100644 index 0000000..9d0517a --- /dev/null +++ b/doc/rbm_modules.asc @@ -0,0 +1,54 @@ +rbm_modules(7) +============= + +NAME +---- +rbm_modules - rbm modules + +DESCRIPTION +----------- + +An rbm workspace can contain a number of different projects. In some +cases it can be useful to share some of those projects between multiple +workspaces. This can be done using modules. + +A module is a subdirectory in the +modules+ directory. If you are using +git, it is possible to store each module directory using a separate ++git-submodule+. + +A module directory structure is similar to the main workspace structure: +each module includes a +projects+ directory containing the projects, +a +keyring+ directory containing the gpg keyrings used by the projects, +and a +rbm.module.conf+ containing rbm configuration related to the +projects included in the module. + + +RBM CONFIGURATION +----------------- + +The rbm configuration from a module (the +rbm.module.conf+ and the +project +config+ files) can define any configuration option. +However, for clarity and to avoid name collisions with other modules or +with core rbm options, it is recommended to prefix all option names +with +var_${module_name}+ (where +${module_name}+ is the module name). + +The workspace configuration (+rbm.conf+) has priority over the modules +configuration (+rbm.module.conf+). Module configuration is used in +alphabetical order: configuration from a module named +A+ will have +priority over a module named +B+. See +link:rbm_config.html[rbm_config(1)] for more details. + +KEYRING +------- + +The keyring directory in a module contains gpg keyring files (see the ++gpg_keyring+ option). If a keyring file is available in multiple +modules, the one from the first module in alphabetical order will be +used. If a keyring file is also available in the main workspace's +keyring directory, then it will be used. + + +SEE ALSO +-------- +link:rbm.html[rbm(1)], +link:rbm_config.html[rbm_config(1)] -- To stop receiving notification emails like this one, please contact the administrator of this repository.
participants (1)
-
gitolite role