HSF Packaging Group Meeting #28, October 3, 2018
Participants: Graeme Stewart, Pere Mato, Gerri Ganis, Liz Sexton-Kennedy,
Patrick Gartung, Guilherme Amadio, Lynn Garren, Chris Green, Jim
Amundson, Benedikt Hegner
- We can have another meeting in 2 weeks time, if people have
contributions to make.
- Guilherme could report on Portage stack.
- Pere can report on LCGCMake updates.
FNAL Spack Update
- MVP = Minimum Viable Product.
- This is for experiments to try and give feedback.
- SLC7, gcc7.3 with C++17, “prof” optimisation level.
- Top package is the art suite, should be good for interesting
- N.B. LArSoft can’t yet be used as the external requirements are
- Feedback on instructions and documentation as well as MVP itself
- When you install an MVP it’s self-contained - it downloads things to
your local system and builds them.
- There are a few pre-requisites (gzip, tar) needed from the
system and the bootstrapper checks that.
- As gcc gets built from source it all takes an hour on a powerful
srcs/ there is a top level CMakeLists.txt that orchestrates the build
of all of the local packages here.
- The CMakeLists.txt is constructed from the Spack recipe for each
of the packages.
- Captures any dependencies between these local packages.
- Spackdev requires spack concretisation, so it can get slow for
- Dependencies need to be fully built and installed before any
dependent package can proceed with its build.
- For Python packages?
- Only CMake supported (at the moment), so not possible to do
this, but this is a future feature.
- Extraction of the build recipe from Spack to push into
CMakeLists.txt makes things more complex than normal - not
using all the ‘native’ features of
- Pere: Bootstrap of CMakeLists from Spack seems the wrong way around.
Developer should start with CMake and then develop a Spack recipe
to share the package.
- Key feature is the ability to work on multiple developments and
have all dependencies resolved, so that requires Spack-level
information to bootstrap.
- Allows developments and dependencies to be handled more easily
than in ‘native’ Spack.
- This is a feature of the current FNAL development
environment and experiments wanted this to be reproduced.
- It’s hard to make Spack itself build from a checked out and
modified code base instead of from a git reference.
- Could that be an extension for Spack?
- Patrick - not a feature in Spack right now.
- Jim - Todd G is supportive of managing this external to
- SpackDev as a Spack extension would allow this all to look more
spack dev ...) and improve performance (only one
- Guilherme: Can Spack update itself?
- This essentially a
git pull; but then any updated recipe
changes hashes and would bring a lot of rebuilds along.
- Would maybe be useful to separate the recipes and the core Spack
code (Jim was supportive of this idea).
- Although you can override any recipes with your own (and
keep them the same).
- SpackChains and BuildCache demonstrate a good and healthy
interaction between HEP and Spack developers.
- Many other developments needed to move to a full product and working
- Patrick: a Spack environment command is being development and would
be able to record and lock versions, for making releases (writes a
- A packages.yaml file declares paths and versions only, so system
updates can happen (but update the version if something is
incompatible!). Full implications should be explored though,
we don’t really have experience of this.
- With Pere and Guilherme’s updates we plan to have the next
meeting in 2 weeks time.