Using Runner as a sandbox during Compile runs?

ermo · January 23, 2021, 3:43pm

Not too long ago, we talked about the Freshen script, which was in days past used to automatically run through all installed packages and update the out-of-date ones.

However, Freshen was moved to legacy status because the state of dependency resolution in GoboLinux is currently such that package updates require a certain amount of nursing due to dependency-related build failures.

We have previously discussed perhaps using Runner to construct a clean build sandbox when using Compile.

Therefore I would like to ask how we might get started on using Runner to create a clean sandbox for each Compile run, initially as an option and later on (when core packages have had their dependencies cleaned up to the point where they are guaranteed to compile), we could potentially consider making it the default to the benefit of the health of GoboLinux and its community going forward?

As a corollary to this, once Compile is switched to using Runner sandbox functionality, we might also be able to re-assess whether to re-introduce an updated Freshen to the GoboLinux toolkit?

In addition, I have not been able to find a description of the intended difference between the BuildDependencies file and the Dependencies file in build recipes? Buildtime-only for the former and Build+Runtime for the latter?

I would tend to think that this question matters in relation to a more strict sandbox build?

EDIT: I should note that with the recent addition of no-interactive-requirement-for-wired-ethernet-via-DHCP functionality, I will now be able to trivially test proposed Compile/Runner changes. So don’t be shy.

lucasvr · January 24, 2021, 3:25am

It would be great to have a Compile sandbox that ensure packages are not compiled with unintended (or implicit) dependencies.

Runner has a --pure flag that forces the creation of a /System/Index hierarchy which only aggregates programs explicitly listed in the dependencies files provided as argument. The next steps as I see are:

To add a new --pure flag to Compile
To let Compile recursively identify all dependencies of the given program
To let Runner know about the dependencies identified in the previous step

Our CheckDependencies script should take care of (2).

I have just checked-in a new code change to let CheckDependencies ignore BuildDependencies files, as we don’t want to have them processed by default. BuildDependencies indicate programs that are needed at compile time only, such as Autoconf and Automake. We do want to process the BuildDependencies file shipped with the recipe, but not the ones from programs already installed (we could also just stop copying BuildDependencies to /Programs, as they really belong to the recipe tree)

I can look into adding the aforementioned changes to our tools in the coming days. I can anticipate some problems, though, such as CheckDependencies returning duplicated programs. We’ll have to fix some Dependencies files along the way, but this should be a good exercise.

ermo · January 24, 2021, 9:18am

Where is this/should this be documented?

For my local xfce-4.14 updates (yes, I know 4.16 is out now), I’ve been focusing on creating a shared BuildDependencies template, which currently looks like this:

Autoconf 2.60
Automake 1.11
Glibc 2.30
GTK-Doc 1.9 [doc]
Intltool 0.35.0
LibTool 2.4.0
# The omission of Perl is deliberate
Pkgconfig 0.20
Vala [vala]

Can you see anything obvious out of the ordinary?

Many other distributions that do dependency resolution typically define a system or system.base set, which contains a common set of minimal/implicit packages. Does Gobo have a similar concept? And if so, might it make sense to add this set implicitly to any Compile --pure invocations?

A really clever person might choose to make this implicit set an option somewhere, such that enterprising individuals would be able to create an alternative/improved system set without having to change the default system set?

I imagine this could be useful if one wanted to compile one or more packages with musl instead of glibc for instance, in which case being able to specify the implict system and cc sets as e.g. Compile --pure --system.set=Musl-1.22 --system.cc=Clang-10 SomePackage might be eminently practical?

Note that this might also provide a convenient way for Gobo developers to test updated system sets?

lucasvr · January 25, 2021, 2:25am

I believe that BuildDependencies should be documented here, but it looks like it’s not – there are only three references to BuildDependencies on our wiki. If you feel like writing some words about it, please be my guest

The BuildDependencies you shown looks good, by the way. Are you having any problems using it?

Regarding a “base” set, we do have it at /Programs/Compile/Current/Resources/Dependencies. All dependencies needed to launch Compile should be listed there. What I did in the past was to simply keep a copy of the original to come up with variations of the basic dependencies set. It’s a simple enough approach that doesn’t demand the modification of other tools.

ermo · January 25, 2021, 9:00am

Will give it a go.

EDIT: Looks like there’s a file from the old wiki missing? I see references to https://github.com/gobolinux/Documentation/wiki/Resources-files – but that file does not appear to exist in the GitHub wiki?

Filesystem virtualization with Runner · gobolinux/Documentation Wiki · GitHub references the Resources-files page.

EDIT2: Recipe Format Specification · gobolinux/Documentation Wiki · GitHub appears to adequately describe the purpose of the BuildDependencies file.

I’ll add an example and get to work on updating relevant links. EDIT3: Done.

No, it was more a question of “am I using BuildDependencies correctly?” since I wrote that file purely based on my assumptions of what the BuildDependencies file was supposed to contain (based on my experience with other package managers).

I get that there’s some trepidation involved when having to change code/tools that everything else relies on. How might I incentivise you to consider the basic idea a long term bonus (rather than a short term malus) in the larger scheme of things, particularly when considering Compile --pure functionality?

In any case, I think it makes sense to discuss policy in terms of where to specify the libc used for any given package, since you appear to like to specify that per package?

Having it specified somewhere in either the build artifact metadata or the recipe proper ensures that tools can parse it going forward for instrumentation/statistics/sorting purposes?

I do think the idea that you can specify different “named” base sets makes sense, particularly for when you want to test new GoboLinux versions in clean chroots, essentially creating “throwaway” base environments for e.g. VM smoke-testing scenarios?

For instance, what about leveraging USE flags for specifying base sets? I.e. have a base set named BaseGlibc and a base set named BaseMusl and then specify it like this in Dependencies?:

BaseGlibc 0.17.1 [glibc] # i.e. the specific set of binutils etc. that GoboLinux 0.17.1 was built with
BaseMusl 0.17.1 [musl]  # the equivalent for Musl

One problem I can see here is what to do if both are specified. If the default is [glibc], [musl] should probably override that, so essentially let the flag encountered last during parsing “win”?

That said, I completely understand if this ends up being a longer-term research project, so in the mean time I’ll familiarise myself with the current approach.

lucasvr · January 27, 2021, 1:42am

This suggestion of being able to use a different base set (or “build profile”) so one can build packages for different releases is interesting. I’m not sure if this belongs to USE flags, but at this point I’d say this is just a minor detail. We’ll first have to put the infrastructure in place before looking at the next step.

Thanks for bringing food for thought!

ermo · January 31, 2021, 5:38pm

@lucasvr

Are there any alternative Compile/Scripts branches with the --pure changes I can try out?

lucasvr · January 31, 2021, 7:54pm

Not yet, but I can look into developing + pushing the relevant code to a new branch tonight or tomorrow the latest. I’ll send you a message here to let you know when it’s available for cloning + testing.

lucasvr · February 1, 2021, 2:18pm

@ermo Done. Here are some notes:

Update your Compile and Scripts with git pull
Add LibCap >= 2.27 to `/Programs/CoreUtils/8.31/Resources/Dependencies
Add MPC >= 1.1.0 to /Programs/GCC/9.2.0/Resources/Dependencies
Add LibNL >= 3.5.0 to /Programs/GoboHide/Current/Resources/Dependencies
Optional: remove Glibc >= 2.30_bootstrap from /Programs/Glibc/Current/Resources/Dependencies. That one should belong to BuildDependencies.
Finally, run Compile --debug --pure <args>, as you’ll certainly want to keep an eye on what’s going on (especially to help you spot bogus dependencies)

[UPDATE: fixed the issue with the temporary file not being cleaned up]
[UPDATE 2: fixed the issue with the resolution of GoboHide dependencies]
[UPDATE 3: fixed the issue with the concatenation of the output of CheckDependencies]

Let me know how it works for you. I tested with a simple example (Compile --debug --pure Lazy, based on autoconf) so far. It’s going to be interesting to try other build systems such as CMake and Meson.

ermo · February 4, 2021, 3:57pm

Got my box up and running again, now with slightly more suitable (= less anemic) hardware.

After updating Compile and Scripts (and SymlinkProgram'ing them), I tried doing a sudo -H Compile --debug --pure Lazy run, but it looks like it fails to pull in NcursesW properly (see debug log below)?

Would it perhaps be possible/desirable to have Compile --debug --pure or CheckDependencies print out the name of the recipe it’s currently operating on at the top, possibly prefixed by a #?

That way it might become simpler/easier to troubleshoot wonky Dependencies or BuildDependencies per recipe, particularly if the tool then does a final “collapse” of the recursive list of deps with set operations and prints that too?

ermo@gobo /Programs]ls -l Compile 
total 8
lrwxrwxrwx 1 root root    12 Feb  4 16:01 Current -> git-20210204
drwxr-xr-x 7 root root  4096 Feb  4 16:01 git-20210204
drwxr-xr-x 3 ermo users 4096 Dec  2  2016 Settings
ermo@gobo /Programs]ls -l Scripts
total 8
lrwxrwxrwx 1 root root   12 Feb  4 16:04 Current -> git-20210204
drwxr-xr-x 9 root root 4096 Feb  4 16:03 git-20210204
drwxr-xr-x 4 root root 4096 May 17  2020 Settings
ermo@gobo /Programs]sudo nano -w CoreUtils/8.31/Resources/Dependencies 
ermo@gobo /Programs]sudo nano -w GCC/9.2.0/Resources/Dependencies 
ermo@gobo /Programs]sudo nano -w GoboHide/Current/Resources/Dependencies 
ermo@gobo /Programs]sudo nano -w GCC/9.2.0/Resources/Dependencies 
ermo@gobo /Programs]sudo nano -w GCC/9.2.0/Resources/BuildDependencies
ermo@gobo /Programs]cd /Data/Compile/Recipes 
ermo@gobo master+* /Data/Compile/Recipes]sudo -H Compile --debug --pure Lazy
warning: Pulling without specifying how to reconcile divergent branches is
discouraged. You can squelch this message by running one of the following
commands sometime before your next pull:

git config pull.rebase false  # merge (the default strategy)
git config pull.rebase true   # rebase
git config pull.ff only       # fast-forward only

You can replace "git config" with "git config --global" to set a default
preference for all repositories. You can also pass --rebase, --no-rebase,
or --ff-only on the command line to override the configured default per
invocation.

Already up to date.
Compile: Checking dependencies...
Compile: Compiling Lazy version 0.24e.
Compile: Runner is available, using it.
Compile: Gathering dependencies from Scripts and Compile
WARNING: The directory '/Users/root/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.                                                                                                                                 
WARNING: The directory '/Users/root/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.                                                                                                                                 
Compile: Gathering dependencies from Resources/Dependencies
Compile: Resolved dependencies:
ZLib >= 1.2.11                                                                                                                                                                                     
XZ-Utils                                                                                                                                                                                           
Util-Linux                                                                                                                                                                                         
Unzip                                                                                                                                                                                              
UnionFS-Fuse                                                                                                                                                                                       
Udev                                                                                                                                                                                               
Tcl >= 8.6.10                                                                                                                                                                                      
Tar                                                                                                                                                                                                
Sudo                                                                                                                                                                                               
Sed                                                                                                                                                                                                
Scripts > 2.9.8                                                                                                                                                                                    
Readline >= 8.0                                                                                                                                                                                    
Python >= 3.0, < 4.0                                                                                                                                                                               
Python >= 3.0.0                                                                                                                                                                                    
Procps-NG                                                                                                                                                                                          
Perl                                                                                                                                                                                               
PCRE >= 8.43                                                                                                                                                                                       
Patch                                                                                                                                                                                              
Ncurses >= 6.0                                                                                                                                                                                     
Mtail                                                                                                                                                                                              
MPFR >= 4.0.2                                                                                                                                                                                      
MPC >= 1.1.0                                                                                                                                                                                       
MkTemp                                                                                                                                                                                             
Make >= 4.3                                                                                                                                                                                        
Linux-PAM >= 1.3.1                                                                                                                                                                                 
Linux-Headers >= 5.4.15                                                                                                                                                                            
LibTIRPC >= 1.2.5                                                                                                                                                                                  
LibNL >= 3.5.0                                                                                                                                                                                     
LibCap >= 2.27                                                                                                                                                                                     
Kerberos >= 1.18                                                                                                                                                                                   
Gzip                                                                                                                                                                                               
Grep                                                                                                                                                                                               
GoboHide                                                                                                                                                                                           
GMP >= 6.2.0                                                                                                                                                                                       
Glibc >= 2.30_bootstrap                                                                                                                                                                            
Glibc >= 2.30                                                                                                                                                                                      
GCC >= 9.2.0                                                                                                                                                                                       
Gawk                                                                                                                                                                                               
Fuse >= 2.9.7, < 3.0.0                                                                                                                                                                             
Fuse                                                                                                                                                                                               
FindUtils >= 4.6.0                                                                                                                                                                                 
File                                                                                                                                                                                               
DiffUtils                                                                                                                                                                                          
DB >= 18.1.32                                                                                                                                                                                      
CoreUtils                                                                                                                                                                                          
Bzip2                                                                                                                                                                                              
BinUtils >= 2.33.1                                                                                                                                                                                 
Bash                                                                                                                                                                                               
ATTR >= 2.4.48                                                                                                                                                                                     
ACL >= 2.2.53                                                                                                                                                                                      
creating new namespace
mount(private) = -1
mount(bind) = 0
mount(private) = 0
WARNING: No packages matching requirements were found, skipping dependency Scripts
WARNING: The directory '/Users/root/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.                                                                                                                                 
WARNING: The directory '/Users/root/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.                                                                                                                                 
adding dependency /Programs/Bash/5.0
adding dependency /Programs/BinUtils/2.33.1
adding dependency /Programs/CoreUtils/8.31
adding dependency /Programs/GCC/9.2.0
adding dependency /Programs/Glibc/2.30
adding dependency /Programs/FindUtils/4.7.0
adding dependency /Programs/Gawk/5.0.1
adding dependency /Programs/GoboHide/1.3
adding dependency /Programs/Grep/3.4
adding dependency /Programs/Make/4.3
adding dependency /Programs/Patch/2.7.6
adding dependency /Programs/Perl/5.30.1
adding dependency /Programs/Python/3.8.6
adding dependency /Programs/Sed/4.8
adding dependency /Programs/Sudo/1.8.31p1
WARNING: /Programs/Udev: No such file or directory, ignoring dependency.
WARNING: No packages were found for dependency Udev
WARNING: No packages matching requirements were found, skipping dependency Scripts
WARNING: /Programs/Ncurses: No such file or directory, ignoring dependency.
WARNING: No packages were found for dependency Ncurses
WARNING: /Programs/MkTemp: No such file or directory, ignoring dependency.
WARNING: No packages were found for dependency MkTemp
adding dependency /Programs/ZLib/1.2.11
adding dependency /Programs/XZ-Utils/5.2.5
adding dependency /Programs/Util-Linux/2.35.1
adding dependency /Programs/Unzip/6.0
adding dependency /Programs/UnionFS-Fuse/2.1
adding dependency /Programs/Tcl/8.6.10
adding dependency /Programs/Tar/1.32
adding dependency /Programs/Readline/8.0
adding dependency /Programs/Procps-NG/3.3.15
adding dependency /Programs/PCRE/8.43
adding dependency /Programs/Mtail/1.2.0
adding dependency /Programs/MPFR/4.0.2
adding dependency /Programs/MPC/1.1.0
adding dependency /Programs/Linux-PAM/1.3.1
adding dependency /Programs/Linux-Headers/5.4.15
adding dependency /Programs/LibTIRPC/1.2.5
adding dependency /Programs/LibNL/3.5.0
adding dependency /Programs/LibCap/2.27
adding dependency /Programs/Kerberos/1.18
adding dependency /Programs/Gzip/1.10
adding dependency /Programs/GMP/6.2.0
adding dependency /Programs/Fuse/2.9.7
adding dependency /Programs/Fuse/3.9.0
adding dependency /Programs/File/5.38
adding dependency /Programs/DiffUtils/3.7
adding dependency /Programs/DB/18.1.32
adding dependency /Programs/Bzip2/1.0.8
adding dependency /Programs/ATTR/2.4.48
adding dependency /Programs/ACL/2.2.53
WARNING: /Programs/Ncurses: No such file or directory, ignoring dependency.
WARNING: No packages were found for dependency Ncurses
mount -t overlay none -o lowerdir=/Programs/XZ-Utils/5.2.5/bin:/Programs/Util-Linux/2.35.1/bin:/Programs/Unzip/6.0/bin:/Programs/UnionFS-Fuse/2.1/bin:/Programs/Tcl/8.6.10/bin:/Programs/Tar/1.32/bin:/Programs/Procps-NG/3.3.15/bin:/Programs/PCRE/8.43/bin:/Programs/Mtail/1.2.0/bin:/Programs/LibNL/3.5.0/bin:/Programs/Kerberos/1.18/bin:/Programs/Gzip/1.10/bin:/Programs/Fuse/2.9.7/bin:/Programs/Fuse/3.9.0/bin:/Programs/File/5.38/bin:/Programs/DiffUtils/3.7/bin:/Programs/DB/18.1.32/bin:/Programs/Bzip2/1.0.8/bin:/Programs/ATTR/2.4.48/bin:/Programs/ACL/2.2.53/bin:/Programs/Bash/5.0/bin:/Programs/BinUtils/2.33.1/bin:/Programs/CoreUtils/8.31/bin:/Programs/GCC/9.2.0/bin:/Programs/Glibc/2.30/bin:/Programs/FindUtils/4.7.0/bin:/Programs/Gawk/5.0.1/bin:/Programs/GoboHide/1.3/bin:/Programs/Grep/3.4/bin:/Programs/Make/4.3/bin:/Programs/Patch/2.7.6/bin:/Programs/Perl/5.30.1/bin:/Programs/Python/3.8.6/bin:/Programs/Sed/4.8/bin:/Programs/Sudo/1.8.31p1/bin:/Programs/Compile/git-20210204/bin:/Programs/UnionFS-Fuse/2.1/sbin:/Programs/Linux-PAM/1.3.1/sbin:/Programs/LibCap/2.27/sbin:/Programs/Fuse/2.9.7/sbin,upperdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/bin,workdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/bin /System/Index/bin
mount -t overlay none -o lowerdir=/Programs/ZLib/1.2.11/include:/Programs/XZ-Utils/5.2.5/include:/Programs/Util-Linux/2.35.1/include:/Programs/Tcl/8.6.10/include:/Programs/Readline/8.0/include:/Programs/Procps-NG/3.3.15/include:/Programs/PCRE/8.43/include:/Programs/MPFR/4.0.2/include:/Programs/MPC/1.1.0/include:/Programs/Linux-PAM/1.3.1/include:/Programs/Linux-Headers/5.4.15/include:/Programs/LibTIRPC/1.2.5/include:/Programs/LibNL/3.5.0/include:/Programs/LibCap/2.27/include:/Programs/Kerberos/1.18/include:/Programs/GMP/6.2.0/include:/Programs/Fuse/2.9.7/include:/Programs/Fuse/3.9.0/include:/Programs/File/5.38/include:/Programs/DB/18.1.32/include:/Programs/Bzip2/1.0.8/include:/Programs/ATTR/2.4.48/include:/Programs/ACL/2.2.53/include:/Programs/Bash/5.0/include:/Programs/BinUtils/2.33.1/include:/Programs/GCC/9.2.0/include:/Programs/Glibc/2.30/include:/Programs/Gawk/5.0.1/include:/Programs/Make/4.3/include:/Programs/Python/3.8.6/include:/Programs/Sudo/1.8.31p1/include,upperdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/include,workdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/include /System/Index/include
mount -t overlay none -o lowerdir=/Programs/ZLib/1.2.11/lib:/Programs/XZ-Utils/5.2.5/lib:/Programs/Util-Linux/2.35.1/lib:/Programs/Tcl/8.6.10/lib:/Programs/Tar/1.32/lib:/Programs/Readline/8.0/lib:/Programs/Procps-NG/3.3.15/lib:/Programs/PCRE/8.43/lib:/Programs/MPFR/4.0.2/lib:/Programs/MPC/1.1.0/lib:/Programs/Linux-PAM/1.3.1/lib:/Programs/LibTIRPC/1.2.5/lib:/Programs/LibNL/3.5.0/lib:/Programs/LibCap/2.27/lib:/Programs/Kerberos/1.18/lib:/Programs/GMP/6.2.0/lib:/Programs/Fuse/2.9.7/lib:/Programs/Fuse/3.9.0/lib:/Programs/File/5.38/lib:/Programs/DB/18.1.32/lib:/Programs/Bzip2/1.0.8/lib:/Programs/ATTR/2.4.48/lib:/Programs/ACL/2.2.53/lib:/Programs/Bash/5.0/lib:/Programs/BinUtils/2.33.1/lib:/Programs/CoreUtils/8.31/lib:/Programs/GCC/9.2.0/lib:/Programs/Glibc/2.30/lib:/Programs/FindUtils/4.7.0/lib:/Programs/Gawk/5.0.1/lib:/Programs/Perl/5.30.1/lib:/Programs/Python/3.8.6/lib:/Programs/Sudo/1.8.31p1/lib:/Programs/Compile/git-20210204/lib,upperdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/lib,workdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/lib /System/Index/lib
mount -t overlay none -o lowerdir=/Programs/ZLib/1.2.11/share:/Programs/XZ-Utils/5.2.5/share:/Programs/Util-Linux/2.35.1/share:/Programs/Unzip/6.0/share:/Programs/UnionFS-Fuse/2.1/share:/Programs/Tcl/8.6.10/share:/Programs/Tar/1.32/share:/Programs/Readline/8.0/share:/Programs/Procps-NG/3.3.15/share:/Programs/PCRE/8.43/share:/Programs/MPFR/4.0.2/share:/Programs/MPC/1.1.0/share:/Programs/Linux-PAM/1.3.1/share:/Programs/LibTIRPC/1.2.5/share:/Programs/LibNL/3.5.0/share:/Programs/LibCap/2.27/share:/Programs/Kerberos/1.18/share:/Programs/Gzip/1.10/share:/Programs/GMP/6.2.0/share:/Programs/Fuse/2.9.7/share:/Programs/Fuse/3.9.0/share:/Programs/File/5.38/share:/Programs/DiffUtils/3.7/share:/Programs/Bzip2/1.0.8/share:/Programs/ATTR/2.4.48/share:/Programs/ACL/2.2.53/share:/Programs/Bash/5.0/share:/Programs/BinUtils/2.33.1/share:/Programs/CoreUtils/8.31/share:/Programs/GCC/9.2.0/share:/Programs/Glibc/2.30/share:/Programs/FindUtils/4.7.0/share:/Programs/Gawk/5.0.1/share:/Programs/GoboHide/1.3/share:/Programs/Grep/3.4/share:/Programs/Make/4.3/share:/Programs/Patch/2.7.6/share:/Programs/Perl/5.30.1/share:/Programs/Python/3.8.6/share:/Programs/Sed/4.8/share:/Programs/Sudo/1.8.31p1/share:/Programs/Compile/git-20210204/share,upperdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/share,workdir=/Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/share /System/Index/share
Creating wrapper /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/wrapper
/bin/bash: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
upper layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/libexec
upper layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/bin
upper layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/include
upper layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/share
upper layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer/lib
upper layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer
upper layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/upper_layer
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/libexec
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/bin/index
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/bin/work
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/bin
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/include/index
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/include/work
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/include
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/share/index
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/share/work
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/share
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/lib/index
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/lib/work
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer/lib
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer
write layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/write_layer
working layer: deleting file /Users/root/.local/Runner/1612452238-Compile-cgwhAZ/wrapper
working layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ
working layer: deleting directory /Users/root/.local/Runner/1612452238-Compile-cgwhAZ
ermo@gobo master+* /Data/Compile/Recipes]ls -l /System/Index/lib/*tinfo*
lrwxrwxrwx 1 root root 38 May  8  2020 /System/Index/lib/libtinfo.so -> /Programs/NcursesW/6.2/lib/libtinfo.so
lrwxrwxrwx 1 root root 45 May  8  2020 /System/Index/lib/libtinfo.so.6 -> /Programs/NcursesW/6.2/lib/libncursesw.so.6.2
ermo@gobo master+* /Data/Compile/Recipes]

lucasvr · February 4, 2021, 5:54pm

That’s a good idea. I’ve just implemented this modification. Please update your copies of Scripts and Compile when you have a chance. With this modification you’ll have a chance to see where the reference to Ncurses is coming from and you’ll be able to rename that to NcursesW instead.

ermo · February 6, 2021, 1:18pm

I’m getting started with the list in Test plan for Compile --pure and am looking at Bullet as an example of a CMake recipe (since an old bug was filed against Bullet and I figured I might as well hit two birds with one stone).

As it turns out, BuildType_cmake ends up complaining that CMake isn’t installed? Might it make sense to plug in CMake-related deps automagically depending on the Recipe type or would you prefer that users add the necessary prerequisite Recipes manually to BuildDependencies?

The most straightforward argument in favour of manually adding necessary prerequisite Recipes to the BuildDependencies of the target Recipe is that it offers an opportunity to specify minimum versions of e.g. Autoconf, CMake, Meson etc.

Thoughts?

As an aside, it feels like this is an opportunity to make a nice RecipeLint script to be used with Compile --pure and have it check if Recipes of BuildType_foo has the corresponding necessary foo Recipes specified correctly in BuildDependencies? I’m thinking this might actually prove quite useful when updating the Recipe tree to be fully compatible with Compile --pure over time?

lucasvr · February 7, 2021, 12:17pm

This actually should belong to Compile/Resources/Dependencies. We already have GCC and Python. Lua and CMake should be there too. Also, the way that I implemented Compile --pure will allow any hand-written dependency file which states a specific version (or a range of versions) of a program gets prioritized over dependencies that simply list the program name. That is, if you add CMake to Compile/Resources/Dependencies and add CMake >= x.y.z to your recipe, the version requested by your recipe will be honored. Please feel free to send me a PR on the Compile repository to include those build_types dependencies.

If we have the dependencies added to Compile, then this becomes unnecessary.

ermo · February 7, 2021, 1:12pm

Done in Compile PR #49.

I’ll be sure to do it in a proper topic branch from now on (soz).

ermo · February 7, 2021, 5:29pm

@lucasvr Please also consider merging Compile PR #50.

Cheers. =)