1. INTRODUCTION
+Pmodules is the environment modules system at PSI. Since it uses AFS as storage backend it is an easy and convinient solution for software distribution.
+It is available for all PSI supported Linux systems including
+-
+
-
+
login.psi.ch (llc.psi.ch)
+
+ -
+
Merlin6
+
+ -
+
Ra
+
+
A list of available modules and changes is avail here.
+| + + | +
+
+
+Some parts of this documentation has been copied from the documentation +of Environment Modules and +Lmod. + |
+
2. PMODULES IN A NUTSHELL
+Before reading this section you should be familiar with an environment module system - either Environment Modules or Lmod.
+Pmodules organizes modules in groups like Tools, Programming, Libraries, System etc. for a clear arrangement. Groups of modules can easily be added to the available modules. Read section Module groups for more details about module groups.
One issue with Environment Modules is the flat naming scheme. Pmodules and Lmod solves the issue by organizing the modules hierarchically. Providing modules for a toolkit like parallel HDF5 ends up in dozen of modules. One user wants to use GCC 4.8.4 and OpenMPI 1.6.5 another Intel C 15 and MPICH 3.1.4, a third user requires GCC 4.9.2 with MPICH 3.1.4. After a couple of month, some users want new versions. In a flat naming scheme you see all installed versions, even if you just want to see what is available for particular compiler. The solution to this problem is a hierarchical organization of the modules. In section Module hierarchies we describe how to load and search for modules in a module hierarchy.
+Neither Lmod nor Environment Modules provides something like a life cycle management. A module is either available or not. There is no way to inform the users, whether a module is still unstable or already deprecated. Pmodules solves this problem by introducing releases like unstable, stable and deprecated. In section Module lifecycle we explain what this means to the user.
+
2.1. Motivation
+Environment module systems provide a solution for the dynamic +modification of a user’s environment by loading special files named +'modulefiles'.
+Each modulefile contains the information needed to configure the shell
+for an application or library. The environment can be modified on a
+per-module basis using the module command which interprets
+modulefiles. Typically modulefiles instruct the module command to
+alter or set shell environment variables such as PATH, MANPATH
+etc. modulefiles may be shared by many users on a system and users may
+have their own collection to supplement or replace the shared
+modulefiles.
Environment module systems are widely used on HPC systems to provide +multiple compilers and multiple versions of the same library or API +implementations to the users. The +Environment Modules system is the +most widely used implementation. Another - more recent - +implementation is +Lmod. Lmod +addresses some weaknesses of the Environment Modules, but solves only +some of them.
+One issue with the Environment Modules is the flat naming scheme. The +Lmod system and Pmodules solves the problem by organizing the modules +hierarchically. What is the problem with a flat naming scheme? Provide +different versions of the same library, like openmpi 1.6.5/1.8.4, +compiled with different (versions) of compilers, like gcc +4.7.4/4.8.4/4.9.2 and Intel 14.0.2/15.2, the number of modules is +growing quite fast. Providing dozens of libraries will end up in +hundreds of modules - not very clear to the users.
+Neither Environment Modules nor Lmod have build-in support for phasing +out modules or marking modules as unstable. Pmodules solves this +problem by introducing 'releases' like stable, unstable and +deprecated. A user will be warned while loading a non-stable module.
+With Environment Modules and Lmod everything must be coded in the
+modulefile - even things which are obvious. If a bin directory
+exists, why not adding this directory to the PATH variable
+automatically? The same can be done for MANPATH, LD_LIBRARY_PATH
+and other environment variables.
Usually modules are installed on a network file-systems. In some cases +it is convenient or even required to install modules locally. Pmodules +provides a tool to install all or a sub-set of the modules from any +location to another location.
+Pmodules has its own (simple) build environment. For a new module you +have to write a build script and a modulefile. Updating an existing +modules is very simple. In most cases it’s downloading the new version +and calling the build script. Anyway, nobody forces you to use this +build environment, but in most cases it is very handy.
+2.2. Module groups
+Pmodules organizes modules in groups. By default only the groups
+Tools and Programming are visible. To get a list of 'used' and
+'unused' groups, run the command module use:
$ module use +Used groups: + Tools + Programming + +Unused groups: + Libraries + System +...+
To make the modules in a given group available, run the command
+$ module use GROUP+
-
+
- Example +
$ module use System +$ module avail +--------------------------------------------- System --------------------------------------------- + +fsstress/1.0.0 nmap/6.46 + +--------------------------------------------- Tools --------------------------------------------- + +Eclipse/4.4.2 Firefox/31.0 Opera/23.0 Thunderbird/31.0 emacs/24.4 +global/6.3.1 gnuplot/4.6.3 + +------------------------------------------ Programming ------------------------------------------ + +Python/3.4.0 Python/3.4.3 autoconf/2.69 automake/1.14 cmake/2.8.12.2 gcc/4.7.4 +gcc/4.8.3 gcc/4.8.4 gcc/4.9.2 libtool/2.4.2 m4/1.4.17+
To make groups unavailable, run the command
+$ module unuse GROUP+
-
+
- Example +
-
+
Make modules in the groups
+SystemandProgrammingunavailable:
+
$ module unuse System Programming +$ module avail +--------------------------------------------- Tools --------------------------------------------- + +Eclipse/4.4.2 Firefox/31.0 Opera/23.0 Thunderbird/31.0 dialog/1.2.1(u) +emacs/24.4 emacs/24.5(u) git/2.3.3(u) global/6.3.1 gnuplot/4.6.3 gnuplot/5.0.0(u)+
+
2.3. Module hierarchies
+Unstructured, flat naming schemes are confusing. Why? Lets assume we have to
+provide modules for the compilers gcc-4.8.4, gcc-4.9.2, intel-15.2,
+the MPI implementations openmpi-1.6.5, openmpi-1.8.4, mpich-3.1.4
+and parallel versions of hdf5-1.8.10 and hdf5-1.8.14 compiled with
+all compiler/MPI combinations. So, we have to provide 30(!) modules:
-
+
-
+
3 compiler modules
+
+ -
+
9 MPI modules
+
+ -
+
18 hdf5 modules
+
+
Even with a reasonable naming convention this is quite confusing. In a flat naming scheme a listing of available modules might look like:
+$ module avail +gcc-4.8.4 +gcc-4.9.2 +hdf5-1.8.10-mpi-3.1.4-gcc-4.8.4 +hdf5-1.8.10-mpi-3.1.4-gcc-4.9.2 +hdf5-1.8.10-mpi-3.1.4-intel-15.2 +hdf5-1.8.10-openmpi-1.6.5-gcc-4.8.4 +hdf5-1.8.10-openmpi-1.6.5-gcc-4.9.2 +hdf5-1.8.10-openmpi-1.6.5-intel-15.2 +hdf5-1.8.10-openmpi-1.8.4-gcc-4.8.4 +hdf5-1.8.10-openmpi-1.8.4-gcc-4.9.2 +hdf5-1.8.10-openmpi-1.8.4-intel-15.2 +hdf5-1.8.14-mpi-3.1.4-gcc-4.8.4 +hdf5-1.8.14-mpi-3.1.4-gcc-4.9.2 +hdf5-1.8.14-mpi-3.1.4-intel-15.2 +hdf5-1.8.14-openmpi-1.6.5-gcc-4.8.4 +hdf5-1.8.14-openmpi-1.6.5-intel-15.2 +hdf5-1.8.14-openmpi-1.8.4-gcc-4.8.4 +hdf5-1.8.14-openmpi-1.8.4-gcc-4.9.2 +hdf5-1.8.14-openmpi-1.8.4-intel-15.2 +intel-15.3 +mpi-3.1.4-gcc-4.8.4 +mpi-3.1.4-gcc-4.9.2 +mpi-3.1.4-intel-15.2 +openmpi-1.6.5-gcc-4.8.4 +openmpi-1.6.5-gcc-4.9.2 +openmpi-1.6.5-intel-15.2 +openmpi-1.8.4-gcc-4.8.4 +openmpi-1.8.4-gcc-4.9.2 +openmpi-1.8.4-intel-15.2+
Got it? One combination is missing. But which one?
+Actually we have to provide more compilers than the three mentioned above: while writing this documentation the number is already seven(!): five GCC versions and two Intel C versions.
+Another issue with a flat naming scheme is, that all modules are listed independent of already loaded modules. But listing a module providing hdf5 1.8.14 compiled with Intel 15.2 and mpich 3.1.4 makes not much sense if modules for gcc 4.9.2 and openmpi 1.8.4 are already loaded. A solution to these issues is a to structure the modules hierarchically.
... _________________________________________|________________________________________... + / | \ + gcc gcc intel + 4.8.4 4.9.2 15.2 + _____________|_____________ _____________|_____________ _____________|_____________ + / | \ / | \ / | \ + openmpi openmpi mpich openmpi openmpi mpich openmpi openmpi mpich + 1.6.5 1.8.4 3.1.4 1.6.5 1.8.4 3.1.4 1.6.5 1.8.4 3.1.4 + __|__ __|__ __|__ __|__ __|__ __|__ __|__ __|__ __|__ + / \ / \ / \ / \ / \ / \ / \ / \ / \ + hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 hdf5 + 1.8.10 1.8.14 1.8.10 1.8.14 1.8.10 1.8.14 1.8.10 1.8.14 1.8.10 1.8.14 1.8.10 1.8.14 1.8.10 1.8.14 1.8.10 1.8.14 1.8.10 1.8.14+
| + + | +
+In a hierarchical structured module environment the command module
+avail does '''not''' reveal all modules. Modules like openmpi and
+hdf5 are not listed as long as the dependencies are not loaded.
+ |
+
-
+
- Example +
-
+
Before you can load a hdf5 module, you must first
+++-
+
-
+
load a particular compiler module,
+
+ -
+
than a particular MPI module
+
+ -
+
and then the desired hdf5 module.
+
+
+ -
+
$ module list +No Modulefiles Currently Loaded. +$ module avail + ------------------------------------------ Programming ------------------------------------------ + +autoconf/2.69 automake/1.14 binutils/2.25(u) cmake/2.8.12.2 cmake/3.1.3 +gcc/4.7.4 gcc/4.8.3 gcc/4.8.4 gcc/4.9.2 gcc/5.1.0(u) intel/14.0.3 +intel/15.2(u) libtool/2.4.2 m4/1.4.17 psi-python27/2.1.0(u) +psi-python34/2.1.0(u) Python/3.4.3 Tcl/8.6.3(u) Tcl/8.6.4(u) Tk/8.6.4(u)+
After loading the module gcc/4.9.2 we see the available MPI implementations compiled with this compiler:
$ module load gcc/4.9.2 +$ module avail +------------------------------------------ Programming ------------------------------------------ + +autoconf/2.69 automake/1.14 binutils/2.25(u) cmake/2.8.12.2 cmake/3.1.3 +gcc/4.7.4 gcc/4.8.3 gcc/4.8.4 gcc/4.9.2 gcc/5.1.0(u) intel/14.0.3 +intel/15.2(u) libtool/2.4.2 m4/1.4.17 psi-python27/2.1.0(u) +psi-python34/2.1.0(u) Python/3.4.3 Tcl/8.6.3(u) Tcl/8.6.4(u) Tk/8.6.4(u) + +--------------------------------------- Compiler/gcc/4.9.2 --------------------------------------- + +boost/1.55.0 boost/1.57.0 gsl/1.15 hdf5_serial/1.8.12 hdf5_serial/1.8.13 +hdf5_serial/1.8.14 mpich/3.1.4(u) OpenBLAS/0.2.9 OpenBLAS_OMP/0.2.9 +openmpi/1.6.5 openmpi/1.8.2 openmpi/1.8.4 root/5.34.19 root/5.34.26 SuperLU/4.3 +UMFPACK/5.6.2 vtk/5.10.1+
Next step is loading a MPI module:
+$ module load openmpi/1.8.4 +$ module avail +------------------------------------------ Programming ------------------------------------------ + +autoconf/2.69 automake/1.14 binutils/2.25(u) cmake/2.8.12.2 cmake/3.1.3 +gcc/4.7.4 gcc/4.8.3 gcc/4.8.4 gcc/4.9.2 gcc/5.1.0(u) intel/14.0.3 +intel/15.2(u) libtool/2.4.2 m4/1.4.17 psi-python27/2.1.0(u) +psi-python34/2.1.0(u) Python/3.4.3 Tcl/8.6.3(u) Tcl/8.6.4(u) Tk/8.6.4(u) + +--------------------------------------- Compiler/gcc/4.9.2 --------------------------------------- + +boost/1.55.0 boost/1.57.0 gsl/1.15 hdf5_serial/1.8.12 hdf5_serial/1.8.13 +hdf5_serial/1.8.14 mpich/3.1.4(u) OpenBLAS/0.2.9 OpenBLAS_OMP/0.2.9 +openmpi/1.6.5 openmpi/1.8.2 openmpi/1.8.4 root/5.34.19 root/5.34.26 SuperLU/4.3 +UMFPACK/5.6.2 vtk/5.10.1 + +---------------------------------- MPI/gcc/4.9.2/openmpi/1.8.4 ---------------------------------- + +BoxLib/2014-02-28 hdf5/1.8.12 hdf5/1.8.13 hdf5/1.8.14 ippl/1.1.4 +parmetis/3.2.0 SuperLU_DIST/3.3 trilinos/11.10.2 trilinos/11.12.1 +trilinos/11.14.1+
Now you can load the HDF5 module:
+$ module load hdf5/1.8.14+
Instead of
+$ module load gcc/4.9.2 +$ module load openmpi/1.8.4 +$ module load hdf5/1.8.14+
you can also execute
+$ module load gcc/4.9.2 openmpi/1.8.4 hdf5/1.8.14+
| + + | +
+Modules marked with (u) are unstable.
+---
+ |
+
2.4. Searching the module hierarchy
+Now you may ask: How do I know which hdf5 (or Trilinos, boost…) modules are really available? To get an answer to this question you can run the module search command.
-
+
- Example +
-
+
To get a list of all installed hdf5 modules run
+
+
$ module search hdf5 --all-releases +Module Release Group Requires +------------------------------------------------------------ +hdf5/1.8.12 unstable MPI gcc/4.7.4 mpich/3.1.4 +hdf5/1.8.12 stable MPI gcc/4.7.4 openmpi/1.6.5 +hdf5/1.8.12 stable MPI gcc/4.7.4 openmpi/1.8.2 +hdf5/1.8.12 stable MPI gcc/4.7.4 openmpi/1.8.4 +hdf5/1.8.12 stable MPI gcc/4.8.3 openmpi/1.6.5 +hdf5/1.8.12 stable MPI gcc/4.8.3 openmpi/1.8.2 +hdf5/1.8.12 stable MPI gcc/4.8.3 openmpi/1.8.4 +hdf5/1.8.12 unstable MPI gcc/4.8.4 mpich/3.1.4 +hdf5/1.8.12 stable MPI gcc/4.8.4 openmpi/1.6.5 +hdf5/1.8.12 stable MPI gcc/4.8.4 openmpi/1.8.2 +hdf5/1.8.12 stable MPI gcc/4.8.4 openmpi/1.8.4 +hdf5/1.8.12 unstable MPI gcc/4.9.2 mpich/3.1.4 +hdf5/1.8.12 stable MPI gcc/4.9.2 openmpi/1.6.5 +hdf5/1.8.12 stable MPI gcc/4.9.2 openmpi/1.8.2 +hdf5/1.8.12 stable MPI gcc/4.9.2 openmpi/1.8.4 +hdf5/1.8.12 unstable MPI gcc/5.1.0 mpich/3.1.4 +hdf5/1.8.12 unstable MPI gcc/5.1.0 openmpi/1.6.5 +hdf5/1.8.12 unstable MPI gcc/5.1.0 openmpi/1.8.4 +hdf5/1.8.12 unstable MPI intel/15.2 mpich/3.1.4 +hdf5/1.8.12 unstable MPI intel/15.2 openmpi/1.6.5 +hdf5/1.8.12 unstable MPI intel/15.2 openmpi/1.8.4 +hdf5/1.8.13 stable MPI gcc/4.7.4 openmpi/1.6.5 +hdf5/1.8.13 stable MPI gcc/4.7.4 openmpi/1.8.2 +hdf5/1.8.13 stable MPI gcc/4.7.4 openmpi/1.8.4 +hdf5/1.8.13 stable MPI gcc/4.8.3 openmpi/1.6.5 +hdf5/1.8.13 stable MPI gcc/4.8.3 openmpi/1.8.2 +hdf5/1.8.13 stable MPI gcc/4.8.3 openmpi/1.8.4 +hdf5/1.8.13 stable MPI gcc/4.8.4 openmpi/1.6.5 +hdf5/1.8.13 stable MPI gcc/4.8.4 openmpi/1.8.2 +hdf5/1.8.13 stable MPI gcc/4.8.4 openmpi/1.8.4 +hdf5/1.8.13 stable MPI gcc/4.9.2 openmpi/1.6.5 +hdf5/1.8.13 stable MPI gcc/4.9.2 openmpi/1.8.2 +hdf5/1.8.13 stable MPI gcc/4.9.2 openmpi/1.8.4 +hdf5/1.8.14 unstable MPI gcc/4.7.4 mpich/3.1.4 +hdf5/1.8.14 stable MPI gcc/4.7.4 openmpi/1.6.5 +hdf5/1.8.14 stable MPI gcc/4.7.4 openmpi/1.8.2 +hdf5/1.8.14 stable MPI gcc/4.7.4 openmpi/1.8.4 +hdf5/1.8.14 stable MPI gcc/4.8.3 openmpi/1.6.5 +hdf5/1.8.14 stable MPI gcc/4.8.3 openmpi/1.8.2 +hdf5/1.8.14 stable MPI gcc/4.8.3 openmpi/1.8.4 +hdf5/1.8.14 unstable MPI gcc/4.8.4 mpich/3.1.4 +hdf5/1.8.14 stable MPI gcc/4.8.4 openmpi/1.6.5 +hdf5/1.8.14 stable MPI gcc/4.8.4 openmpi/1.8.2 +hdf5/1.8.14 stable MPI gcc/4.8.4 openmpi/1.8.4 +hdf5/1.8.14 unstable MPI gcc/4.9.2 mpich/3.1.4 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.6.5 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.8.2 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.8.4 +hdf5/1.8.14 unstable MPI gcc/5.1.0 mpich/3.1.4 +hdf5/1.8.14 unstable MPI gcc/5.1.0 openmpi/1.6.5 +hdf5/1.8.14 unstable MPI gcc/5.1.0 openmpi/1.8.4 +hdf5/1.8.14 unstable MPI intel/15.2 mpich/3.1.4 +hdf5/1.8.14 unstable MPI intel/15.2 openmpi/1.6.5 +hdf5/1.8.14 unstable MPI intel/15.2 openmpi/1.8.4 +hdf5_serial/1.8.12 stable Compiler gcc/4.7.4 +hdf5_serial/1.8.12 stable Compiler gcc/4.8.3 +hdf5_serial/1.8.12 stable Compiler gcc/4.8.4 +hdf5_serial/1.8.12 stable Compiler gcc/4.9.2 +hdf5_serial/1.8.12 unstable Compiler intel/15.2 +hdf5_serial/1.8.14 stable Compiler gcc/4.7.4 +hdf5_serial/1.8.14 stable Compiler gcc/4.8.3 +hdf5_serial/1.8.14 stable Compiler gcc/4.8.4 +hdf5_serial/1.8.14 stable Compiler gcc/4.9.2 +hdf5_serial/1.8.14 unstable Compiler intel/15.2+
-
+
- Example +
-
+
To get a list of all installed hdf5 modules compiled with GCC 4.9.2, run
+
+
$ module search hdf5 --with=gcc/4.9.2 --all-releases +Module Release Group Requires +------------------------------------------------------------ +hdf5/1.8.12 unstable MPI gcc/4.9.2 mpich/3.1.4 +hdf5/1.8.12 stable MPI gcc/4.9.2 openmpi/1.6.5 +hdf5/1.8.12 stable MPI gcc/4.9.2 openmpi/1.8.2 +hdf5/1.8.12 stable MPI gcc/4.9.2 openmpi/1.8.4 +hdf5/1.8.13 stable MPI gcc/4.9.2 openmpi/1.6.5 +hdf5/1.8.13 stable MPI gcc/4.9.2 openmpi/1.8.2 +hdf5/1.8.13 stable MPI gcc/4.9.2 openmpi/1.8.4 +hdf5/1.8.14 unstable MPI gcc/4.9.2 mpich/3.1.4 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.6.5 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.8.2 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.8.4 +hdf5_serial/1.8.12 stable Compiler gcc/4.9.2 +hdf5_serial/1.8.13 stable Compiler gcc/4.9.2 +hdf5_serial/1.8.14 stable Compiler gcc/4.9.2+
-
+
- Example +
-
+
To get a list of all installed modules providing (parallel) HDF5 1.8.14, execute
+
+
$ module search hdf5/1.8.14 --with=gcc/4.9.2 --all-releases +Module Release Group Requires +------------------------------------------------------------ +hdf5/1.8.14 unstable MPI gcc/4.9.2 mpich/3.1.4 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.6.5 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.8.2 +hdf5/1.8.14 stable MPI gcc/4.9.2 openmpi/1.8.4+
| + + | +
+Without the option --all-releases, only modules of "used" releases are listed.
+ |
+
+
2.5. Module lifecycle
+Pmodules supports a simple lifecycle management. This solves the
+problem of introducing new modules, which are still in development or
+considered to be unstable and it supports phasing out old and
+deprecated modules. The lifecycle of a module is unstable →
+stable → deprecated. By default only stable modules are
+visible. If you want to use unstable or deprecated modules, you have
+to run
$ module use unstable+
or
+$ module use deprecated+
first.
+All modules start as unstable. This implies something like "use on
+your on risk". Unstable modules may be changed without warning or even
+be removed. If you load an unstable module, a warning will be printed.
A stable module is, well, stable. It will not be changed during the
+remaining life time and the provided software is considered to be
+stable.
A deprecated module should not be used any more. Deprecated modules
+might be removed without further warning. While loading a deprecated
+module, a warning will be printed. Kind module developers might add an
+end of life message with more detailed information.
2.6. Overlays
+TBW
+3. THE module COMMAND
+Table of Content
+The module-command: module
+List available modules: module avail
+Clear list of loaded modules: module clear
+Display information about chances in environment when loaded: module display
+Print sub-command or module-specific help: module help
+Add or remove modules from shell’s initialization file: module initcmds
+Search for keywords in 'whatis': [module-keyword]
+List loaded modules: module list
+Load and unload modules: module load|unload
+Purge all loaded modules: module purge
+Refresh loaded modules: module refresh
+Search installed modules: module search
+Replace a loaded module by another: module swap
+Manipulate module path, used releases, groups and overlays: module use
+Search for keywords in 'whatis': module whatis|keyword
3.1. module - using Pmodules
+NAME
+module - command line interface to the Pmodules package
SYNOPSIS
+module [ switches ] [ sub-command ] [ sub-command-args ]
DESCRIPTION
+Environment Modules provide a convenient way to dynamically change the
+users' environment through modulefiles. This includes easily adding or
+removing directories to the PATH environment variable.
+A modulefile contains the necessary information to allow a user to run
+a particular application or provide access to a particular
+library. All of this can be done dynamically without logging out and
+back in. Modulefiles for applications modify the user’s shell
+environment to make access easy. Modulefiles for Library packages
+provide environment variables that specify where the library and
+header files can be found.
+Packages can be loaded and unloaded cleanly through the module command.
Available sub-commands are:
+List available modules: module avail
Clear list of loaded modules: module clear
Display information about chances in environment when loaded: module display
Print sub-command or module-specific help: module help
Add or remove modules from shell’s initialization file: module initcmds
Search for keywords in 'whatis': [module-keyword]
+List loaded modules: module list
Load and unload modules: module load|unload
Purge all loaded modules: module purge
Refresh loaded modules: module refresh
Search installed modules: module search
Replace a loaded module by another: module swap
Manipulate module path, used releases, groups and overlays: module use
Search for keywords in 'whatis': module whatis|keyword
| + + | +
+
+
+For the time being Pmodules supports bash, tcsh and zsh only. + |
+
3.2. Sub-commands
+3.2.1. List available/loadable modules
+NAME
+module avail - list loadable modules
SYNOPSIS
+module avail [OPTIONS] [string…]
DESCRIPTION
+List all available module in the current MODULEPATH. If an argument is given, then each directory in the MODULEPATH is searched for modules whose pathname match the argument.
This command does not display all installed modules on the system. Only loadable modules are listed. To get a list of all installed modules use the command module search.
The list of available modules may change either by loading other modules, e.g. a compiler, or by using/accepting unstable or deprecated modules. For the later read the description of module use.
With the switches you can change the output format. The default output format is 'human' (readable). For the time being terse and long output are identical.
+OPTIONS
+-
+
-a
+-
+
List loadable modules in all release-stages. Unstable modules are marked with
+(u), +deprecated modules with(d).
+ -t | --terse
+-
+
Terse output.
+
+ -h | --human
+-
+
Human readable output. This is the default.
+
+ -l | --long
+-
+
Long output.
+
+
EXAMPLES
+$ module avail git +----------------------------------------- Tools ----------------------------------------- + +ygit/2.3.3 git/2.8.1 git/2.21.0 git/2.22.0 git/2.30.0 git/2.33.1 + +$ module avail -a gcc +-------------------------------------- Programming -------------------------------------- + +gcc/4.7.4 gcc/4.8.2 gcc/4.8.3 gcc/4.8.4 gcc/4.8.5 gcc/4.9.2 +gcc/4.9.3 gcc/4.9.4 gcc/5.1.0(d) gcc/5.2.0(d) gcc/5.3.0 gcc/5.4.0 +gcc/5.5.0 gcc/6.1.0 gcc/6.2.0 gcc/6.3.0 gcc/6.4.0 gcc/6.5.0 +gcc/7.1.0 gcc/7.2.0 gcc/7.3.0 gcc/7.4.0 gcc/7.5.0 gcc/8.1.0 +gcc/8.2.0 gcc/8.3.0 gcc/8.4.0 gcc/8.5.0 gcc/9.1.0 gcc/9.2.0 +gcc/9.3.0 gcc/9.5.0 gcc/10.1.0 gcc/10.2.0 gcc/10.3.0 gcc/11.2.0 +gcc/11.3.0 gcc/12.1.0+
SEE ALSO
+ +3.2.2. Clear list of loaded modules
+NAME
+module clear - clear list of loaded modules.
SYNOPSIS
+module clear
DESCRIPTION
+Force the Pmodules package to believe that no modules are currently loaded. The sub-command clear does not unload the loaded modules! In other words: changes made to the shell’s environment by loaded modules are not reverted.
+EXAMPLES
+After loading the module for gcc/4.9.2 the following environment variables are set:
$ env | grep ^GCC +GCC_HOME=/opt/psi/Programming/gcc/4.9.2 +GCC_DIR=/opt/psi/Programming/gcc/4.9.2 +GCC_PREFIX=/opt/psi/Programming/gcc/4.9.2 +GCC_VERSION=4.9.2 +GCC_INCLUDE_DIR=/opt/psi/Programming/gcc/4.9.2/include +GCC_LIBRARY_DIR=/opt/psi/Programming/gcc/4.9.2/lib+
After clearing the list of loaded modules the environment variables set by the GCC module are still set
+$ module clear +$ module list +No Modulefiles Currently Loaded. +$ env | grep ^GCC +GCC_HOME=/opt/psi/Programming/gcc/4.9.2 +GCC_DIR=/opt/psi/Programming/gcc/4.9.2 +GCC_PREFIX=/opt/psi/Programming/gcc/4.9.2 +GCC_VERSION=4.9.2 +GCC_INCLUDE_DIR=/opt/psi/Programming/gcc/4.9.2/include +GCC_LIBRARY_DIR=/opt/psi/Programming/gcc/4.9.2/lib+
SEE ALSO
+ +3.2.3. Display information about a module
+NAME
+module display\|show - display information about chances in environment
SYNOPSIS
+module display modulefile…
+module show modulefile…
DESCRIPTION
+Display information about modulefile(s). The display sub-command will list the full path of the modulefile(s) and all of the environment changes the modulefile(s) will make if loaded. (It will not display any environment changes found within conditional statements.)
+EXAMPLES
+Display chances the module gcc/5.4.0 will make to the environment if loaded:
$ module display gcc/5.4.0
+-------------------------------------------------------------------
+/opt/psi/Programming/modulefiles/gcc/5.4.0:
+
+module-whatis GNU Compiler Collection
+conflict gcc
+setenv GCC_VERSION 5.4.0
+setenv GCC_PREFIX /opt/psi/Programming/gcc/5.4.0
+setenv GCC_DIR /opt/psi/Programming/gcc/5.4.0
+setenv GCC_HOME /opt/psi/Programming/gcc/5.4.0
+prepend-path PATH /opt/psi/Programming/gcc/5.4.0/bin
+prepend-path MANPATH /opt/psi/Programming/gcc/5.4.0/share/man
+prepend-path C_INCLUDE_PATH /opt/psi/Programming/gcc/5.4.0/include
+prepend-path CPLUS_INCLUDE_PATH /opt/psi/Programming/gcc/5.4.0/include
+setenv GCC_INCLUDE_DIR /opt/psi/Programming/gcc/5.4.0/include
+prepend-path LIBRARY_PATH /opt/psi/Programming/gcc/5.4.0/lib
+prepend-path LD_LIBRARY_PATH /opt/psi/Programming/gcc/5.4.0/lib
+setenv GCC_LIBRARY_DIR /opt/psi/Programming/gcc/5.4.0/lib
+prepend-path LIBRARY_PATH /opt/psi/Programming/gcc/5.4.0/lib64
+prepend-path LD_LIBRARY_PATH /opt/psi/Programming/gcc/5.4.0/lib64
+setenv GCC_LIBRARY_DIR /opt/psi/Programming/gcc/5.4.0/lib64
+append-path PMODULES_LOADED_PROGRAMMING gcc/5.4.0
+remove-path PMODULES_LOADED_PROGRAMMING --APPMARKER--
+setenv COMPILER gcc
+setenv COMPILER_VERSION 5.4.0
+setenv CC /opt/psi/Programming/gcc/5.4.0/bin/gcc
+setenv CXX /opt/psi/Programming/gcc/5.4.0/bin/g++
+setenv F77 /opt/psi/Programming/gcc/5.4.0/bin/gfortran
+setenv F90 /opt/psi/Programming/gcc/5.4.0/bin/gfortran
+setenv FC /opt/psi/Programming/gcc/5.4.0/bin/gfortran
+setenv FORTRAN /opt/psi/Programming/gcc/5.4.0/bin/gfortran
+-------------------------------------------------------------------SEE ALSO
+3.2.4. Print sub-command or module-specific help
+NAME
+module help - print sub-command or module-specific help
SYNOPSIS
+module help [module|sub-command…]
DESCRIPTION
+Print help for specific modules, sub-commands or the module command itself.
+EXAMPLES
+Get help for the module git/2.3.3:
+$ module help git/2.3.3
+
+----------- Module Specific Help for 'git/2.3.3' ------------------
+
+distributed version control system
+Version: 2.3.3
+Homepage: http://git-scm.com/
+License: GNU GPL v2
+Maintainer: Achim Gsell <achim.gsell@psi.ch>
+
+Git is a free and open source distributed version control system
+designed to handle everything from small to very large projects
+with speed and efficiency.
+
+Git is easy to learn and has a tiny footprint with lightning fast
+performance. It outclasses SCM tools like Subversion, CVS, Perforce,
+and ClearCase with features like cheap local branching, convenient
+staging areas, and multiple workflows.Print help for the sub-command load:
+$ module help load
+
+USAGE:
+ module add modulefile...
+ module load modulefile...
+ Load modulefile(s) into the shell environment. Loading a
+ 'group-head' will extend the MODULEPATH. E.g.: loading a
+ compiler makes additional modules like openmpi and libraries
+ compiled with this compiler available.SEE ALSO
+ +3.2.5. Manipulate shell’s initialization file
+NAME
+module initadd|initprepend|initrm|initswitch|initrm|initlist|initclear - add or remove modules from shell’s initialization file
SYNOPSIS
+module initadd module…
+module initprepend module…
+module initrm module…
+module initswitch module1 module2
+module initlist
+module initclear
DESCRIPTION
+Add modulefile(s) to, list modulefile(s) in or remove modulefile(s) from the shell’s initialization file in the user’s home directory. The startup files checked (in order) are:
+-
+
- csh +
-
+
+.modules,.cshrc(.ext),.csh_variables, and.login(.ext)
+ - tcsh +
-
+
+.modules,.tcshrc,.cshrc(.ext),.csh_variables, and.login(.ext)
+ - bash +
-
+
+.modules,.bash_profile,.bash_login,.profile(.ext), and.bashrc(.ext)
+ - zsh +
-
+
+.modules,.zcshrc(.ext),.zshenv(.ext), and.zlogin(.ext)
+ module initadd module…
+-
+
If a module load line is found in any of these files, the modulefile(s) is(are) appended to any existing list of modulefiles. The module load line must be located in at least one of the files listed above for any of the init sub-commands to work properly. If the module load line is found in multiple shell initialization files, all of the lines are changed.
+
+ module initprepend module…
+-
+
Does the same as initadd but prepends the given modules to the beginning of the list.
+
+ module initrm module…
+-
+
Remove modulefile(s) from the shell’s initialization files.
+
+ module initswitch module1 module2
+-
+
Switch
+module1withmodule2in the shell’s initialization files.
+ module initlist
+-
+
List all of the module(s) loaded from the shell’s initialization file.
+
+ module initclear
+-
+
Clear all of the module(s) from the shell’s initialization files.
+
+
SEE ALSO
+ +3.2.6. List loaded modules
+NAME
+module list - list loaded modules
SYNOPSIS
+module list [OPTIONS]
DESCRIPTION
+List loaded modules.
+OPTIONS
+-
+
-t | --terse
+-
+
Terse output.
+
+ -h | --human
+-
+
Human readable output. This is the default.
+
+ -l | --long
+-
+
Long output.
+
+
EXAMPLES
+Assuming the modules gcc/4.8.3, openmpi/1.8.2 and hdf5/1.8.12. List with default (human readable) output:
$ module list
+Currently Loaded Modulefiles:
+ 1) gcc/4.8.3 2) openmpi/1.8.2 3) hdf5/1.8.12List in terse output:
+$ module list -t
+Currently Loaded Modulefiles:
+gcc/4.8.3
+openmpi/1.8.2
+hdf5/1.8.12Long listing:
+$ module list -l
+- Package -----------------------------+- Versions -+- Last mod. ------
+Currently Loaded Modulefiles:
+gcc/4.8.3 2014/12/05 17:39:24
+openmpi/1.8.2 2014/11/25 8:15:40
+hdf5/1.8.12 2014/11/25 8:15:40SEE ALSO
+ +3.2.7. Load and unloading a module
+NAME
+module add|load - load modules
+module rm|unload - unload modules
SYNOPSIS
+module add [OPTIONS] modulefile…
+module load [OPTIONS] modulefile…
+module rm modulefile…
+module unload modulefile…
DESCRIPTION
+Load modulefile(s) into shell environment. Loading a group-head will extend the MODULEPATH. E.g.: loading a compiler makes additional modules like openmpi and other libraries/packages compiled with this compiler available.
If you try to load a module which is not available in the current MODULEPATH but installed in the module hierarchy, a message will be printed showing prerequisite modules.
If you load a deprecated or unstable module, a warning will be printed.
+You can change the verbosity of the load sub-command by setting the environment variable PMODULES_VERBOSITY to silent, warn or verbose.
-
+
- silent +
-
+
Print error messages only.
+
+ - warn +
-
+
Print a warning message, when loading a deprecated or unstable module.
+
+ - verbose +
-
+
Print warning messages and print prerequisite modules if the module to load is not available, but installed in hierarchy.
+
+
OPTIONS / FLAGS
+-
+
-f | --force
+-
+
Force active dependency resolution. This will result in modules found on a prereq command inside a module file being load automatically. Unloading module files using this switch will result in all required modules which have been loaded automatically using the -f switch being unload. This switch is experimental at the moment.
+
+ -v | --verbose
+-
+
Set verbosity level to verbose.
+
+ -w | --warn
+-
+
Set verbosity level to warn.
+
+ -s | --silent
+-
+
Set verbosity level to silent.
+
+
EXAMPLES (add|load)
+Loading the modules gcc/4.9.2, openmpi/1.8.4 and hdf5/1.8.14:
$ module load gcc/4.9.2 +$ module load openmpi/1.8.4 +$ module load hdf5/1.8.14 +$ module list +Currently Loaded Modulefiles: + 1) gcc/4.9.2 2) openmpi/1.8.4 3) hdf5/1.8.14+
or
+$ module load gcc/4.9.2 openmpi/1.8.4 hdf5/1.8.14+
If you try to load hdf5/1.8.14 without the prerequisite modules, some "hints" will be printed, if the default verbosity level is set:
$ module purge +$ module load hdf5/1.8.14 +The module 'hdf5/1.8.14' cannot be loaded! +Try with one of the following command(s): + +module load gcc/4.7.4 openmpi/1.6.5 hdf5/1.8.14 +module load gcc/4.7.4 openmpi/1.8.2 hdf5/1.8.14 +module load gcc/4.7.4 openmpi/1.8.4 hdf5/1.8.14 +module load gcc/4.8.3 openmpi/1.6.5 hdf5/1.8.14 +module use deprecated; module load gcc/4.8.3 openmpi/1.8.2 hdf5/1.8.14 +module use deprecated; module load gcc/4.8.3 openmpi/1.8.4 hdf5/1.8.14 +module load gcc/4.8.4 openmpi/1.6.5 hdf5/1.8.14 +module use deprecated; module load gcc/4.8.4 openmpi/1.8.2 hdf5/1.8.14 +module use deprecated; module load gcc/4.8.4 openmpi/1.8.4 hdf5/1.8.14 +module use deprecated; module load gcc/4.8.4 openmpi/1.8.8 hdf5/1.8.14 +module use deprecated; module load gcc/4.9.2 openmpi/1.6.5 hdf5/1.8.14 +module use deprecated; module load gcc/4.9.2 openmpi/1.8.2 hdf5/1.8.14 +module use deprecated; module load gcc/4.9.2 openmpi/1.8.4 hdf5/1.8.14+
Use verbosity level warn to suppress the hints:
$ module load hdf5/1.8.14 --warn +module load: module unavailable -- hdf5/1.8.14+
Loading an unstable module:
+$ module use unstable +$ module load cmake/3.1.3 +Warning: the unstable module 'cmake/3.1.3' has been loaded.+
To supress the warning, use the --silent option.
EXAMPLES (rm|unload)
+$ module list +Currently Loaded Modulefiles: + 1) gcc/4.9.2 2) openmpi/1.8.4 3) hdf5/1.8.14 +$ module rm openmpi/1.8.4 +$ module list +Currently Loaded Modulefiles: + 1) gcc/4.9.2+
The module hdf5/1.8.14 is unloaded as a dependency of openmpi/1.8.4.
SEE ALSO
+ +3.2.8. Purge all loaded modules
+NAME
+module purge - purge all loaded modules
SYNOPSIS
+module purge
DESCRIPTION
+Unload all loaded module and reset everything to original state.
+EXAMPLES
+List loaded modules:
+$ module list +Currently Loaded Modulefiles: + 1) gcc/4.8.3 2) openmpi/1.8.2 3) gnuplot/4.6.3+
Now we purge everything and list the loaded modules again:
+$ module purge +$ module list +No Modulefiles Currently Loaded.+
SEE ALSO
+ +3.2.9. Refresh loaded modules
+NAME
+module refresh - refresh loaded modules
SYNOPSIS
+module refresh
DESCRIPTION
+Force a refresh of all non-persistent components of currently loaded modules. This should be used on derived shells where aliases need to be reinitialized but the environment variables have already been set by the currently loaded modules.
+SEE ALSO
+ +3.2.10. Search modules
+NAME
+module search - search installed modules
SYNOPSIS
+module search [switches] string…
DESCRIPTION
+List all modules in the current MODULEPATH and the module hierarchy. If an argument is given, search for modules whose name match the argument.
+Options
OPTIONS
+-
+
--no-header
+-
+
Suppress output of a header.
+
+ --release=RELEASE
+-
+
Search for modules within this release. You can specify this switch multiple times. Without this switch, the used releases will be searched.
+
+ -a|--all-releases
+-
+
Search within all releases.
+
+ --with=STRING
+-
+
Search for modules compiled with modules matching string. The command. See example below.
+
+ --src=dir
+-
+
Search module hierarchy in dir. +Eine Textbeschreibung der Funktionsweise des Befehls oder der Funktion. (Üblicherweise jedoch nicht der Benutzung, siehe unten.)
+
+
EXAMPLES
+Get list of all installed GCC:
+$ module search --all-releases gcc
+Module Release Group Requires
+------------------------------------------------------------
+gcc/4.7.4 stable Programming
+gcc/4.8.2 stable Programming
+gcc/4.8.3 stable Programming
+gcc/4.8.4 stable Programming
+gcc/4.8.5 stable Programming
+gcc/4.9.2 stable Programming
+gcc/4.9.3 stable Programming
+gcc/4.9.4 stable Programming
+gcc/5.1.0 deprecated Programming
+gcc/5.2.0 deprecated Programming
+gcc/5.3.0 stable Programming
+gcc/5.4.0 stable Programming
+gcc/6.1.0 stable Programming
+gcc/6.2.0 stable Programming
+gcc/6.3.0 stable Programming
+gcc/6.4.0 stable Programming
+gcc/7.1.0 stable Programming
+gcc/7.2.0 stable ProgrammingGet list of all open-mpi versions compiled with GCC 5.4.0:
+$ module search --all-releases openmpi --with=gcc/5.4.0
+Module Release Group Requires
+------------------------------------------------------------
+openmpi/1.10.2 stable Compiler gcc/5.4.0
+openmpi/1.10.4 stable Compiler gcc/5.4.0
+openmpi/2.0.1 stable Compiler gcc/5.4.0SEE ALSO
+3.2.11. Swapping modules
+NAME
+module swap|switch - replace a loaded module by another
SYNOPSIS
+module swap [modulefile1] modulefile2
+module switch [modulefile1] modulefile2
DESCRIPTION
+Switch loaded modulefile1 with modulefile2. If modulefile1 is not specified, then it is assumed to be the currently loaded module with the same root name as modulefile2.
OPTIONS / FLAGS
+None
+EXAMPLES
+$ module load gcc/4.8.4 +$ module swap gcc/4.9.2 +$ module list +Currently Loaded Modulefiles: + 1) gcc/4.9.2+
BUGS
+You can only swap between different version. The following commands are working (assuming that gcc/4.8.2 is loaded):
$ module swap gcc/4.9.2 +$ module swap gcc/4.9.2 gcc/5.4.0+
The following command does not working as expected:
+$ module swap gcc/4.8.2 intel/15.0+
This command does not work with the Tcl implementation (environment variable PMOUDLE_PURETCL set).
SEE ALSO
+ +3.2.12. Manipulate module path, used releases, groups or overlays
+NAME
+module use|unuse - manipulate module path, used releases or groups
SYNOPSIS
+module use [OPTIONS] [string…]
module unuse string…
DESCRIPTION
+If called without arguments, print a summary about used groups, releases and additional directories in MODULEPATH.
If string is a directory, append, prepend or remove this directory to/from MODULEPATH.
If string is a group name, append, prepend or remove the corresponding directory to/from MODULEPATH.
If string is a releases name, make all modules with this release available/unavailable. Known releases are;
+-
+
- stable +
-
+
Modules released as stable are considered to be production ready. The files of a stable module will never change.
+
+ - unstable +
-
+
These modules are still unstable. They may not be ready for production use. The files of unstable modules may change to fix bugs or add functionality. Use at your own risk.
+
+ - deprecated +
-
+
Deprecated modules should not be used any more and may be removed without further notice.
+
+
If string matches flag=[-_a-zA-Z0-9]* the RHS will be interpreted as use-flag.
OPTIONS / FLAGS
+-
+
-a | --append
+-
+
append directory or group to
+MODULEPATH.
+ -p | --prepend
+-
+
prepend directory or group to
+MODULEPATH.
+
EXAMPLES
+List used/unused groups, releases and additional directories in MODULEPATH:
$ module use +Used groups: + Tools + Programming + Compiler + +Unused groups: + Legacy + Libraries + System + +Used releases: + stable + unstable + +Unused releases: + deprecated + +Used flags: + omp + +Additonal directories in MODULEPATH: + none+
Adding the 'System' group
+$ module use System +$ module avail +-------------------------------------------- System: -------------------------------------------- + +filebench/1.4.9.1 fsstress/1.0.0 nmap/6.46 patchelf/0.8.1 + +-------------------------------------------- Tools: -------------------------------------------- + +ANSYS/18.2 emacs/24.4 git/2.3.3 global/6.3.1 gnuplot/4.6.3 gnuplot/5.0.0 +... + +----------------------------------------- Programming: ----------------------------------------- + +autoconf/2.69 automake/1.14 automake/1.15 binutils/2.25 cmake/2.8.12.2 cmake/3.1.3 +cmake/3.6.3 gcc/4.7.4 gcc/4.8.3 gcc/4.8.4 gcc/4.9.2 gcc/4.8.5 +...+
Adding a directory to MODULEPATH
$ module use /afs/psi.ch/project/amas/modulefiles +$ module avail +----------------------------------- /afs/psi.ch/project/amas: ----------------------------------- + +H5hut_parallel-toolchain/2.0 H5hut_serial-toolchain/2.0 OPAL/1.6 OPAL/1.6.0rc5 +opal-toolschain/1.6 opal-toolschain/2.0 + +-------------------------------------------- Tools: -------------------------------------------- + +ANSYS/18.2 emacs/24.4 git/2.3.3 global/6.3.1 gnuplot/4.6.3 gnuplot/5.0.0 +... +----------------------------------------- Programming: ----------------------------------------- + +autoconf/2.69 automake/1.14 automake/1.15 binutils/2.25 cmake/2.8.12.2 cmake/3.1.3 +cmake/3.6.3 gcc/4.7.4 gcc/4.8.3 gcc/4.8.4 gcc/4.9.2 gcc/4.8.5 +...+
$ module unuse /afs/psi.ch/project/amas/modulefiles +$ module unuse System +$ module unuse unstable+
SEE ALSO
+ +3.2.13. Show/search 'what is' information
+NAME
+module whatis - print one-line information about module
+module keyword|apropos - search for keywords in 'whatis'
SYNOPSIS
+module whatis [module…]
+module apropos string…
+module keyword string…
DESCRIPTION
+-
+
whatis
+-
+
Display the information set up by the
+module-whatiscommands inside +the specified modulefile(s). If no modulefile is specified, all whatis +lines will be shown.
+ keyword|apropos
+-
+
Search through the 'whatis' informations of all modulefiles for the +specified string. All whatis informations matching the string will be +displayed.
+
+
EXAMPLES
+Get whatis for all available Git modules
+$ module whatis git
+----------- /opt/psi/Tools/modulefiles -------------
+ git/2.3.3: distributed version control system
+ git/2.5.2: distributed version control system
+ git/2.8.1: distributed version control system
+ git/2.11.1: distributed version control system
+----------- /opt/psi/Programming/modulefiles -------------SEE ALSO
+4. BUILDING PMODULES
+4.1. Introduction
+In the simplest case adding a Pmodule can be done by adding a modulefile to a certain directory. Below is an example of a "do nothing" modulefile in the group Sandbox.
/opt/psi/Sandbox/modulefiles/null/1.0.0#%Pmodule
+
+module-whatis "does absolutely nothing"
+module-maintainer "Achim Gsell <achim.gsell@psi.ch>"
+module-license "MIT"
+
+module-help "
+This is a do nothing module.
+"In addition to the modulefile a corresponding configuration file - defining for example the release stage - should be installed in the same directory as the modulefile. See next section for a detailed documentation of configuration files.
+/opt/psi/Sandbox/modulefiles/null/.release-1.0.0 for the null/1.0.0 modulestableModulefiles are written in the Tool Command Language (Tcl). See section [modulefiles] for a detailed documentation. In Pmodules version 1.1.16 and newer modulefiles written in Lua are also supported but with some limitations.
+The above example doesn’t provide any software. In the following example we show a simple 'HelloWorld' module with a 'Hello, world!' program.
+/opt/psi/Sandbox/modulefiles/HelloWorld/1.0.0:#%Pmodule
+
+module-whatis "'Hello, world!' module"
+module-maintainer "Achim Gsell <achim.gsell@psi.ch>"
+module-license "MIT"
+module-url "http://pmodules.gitpages.psi.ch"
+
+module-help "
+The module provides a 'Hello, world!' program.
+"/opt/psi/Sandbox/modulefiles/HelloWorld/.release-1.0.0 for the HelloWorld/1.0.0 moduleunstable/opt/psi/Sandbox/HelloWorld/1.0.0/bin/hello:#!/usr/bin/env python3
+
+print("Hello, world!")In principal, it is possible to install modules 'by hand' by installing the software, module- and configuration files at the right place. For reproducibility, documentation and re-usability it is highly recommended to write build-recipes for each module or - at least - a README and host everything on a PSI Gitlab server.
+Overall the main cases for building modules are:
+-
+
-
+
Building a module from source. This is the most common case. Examples: git, +gnuplot, openmpi, hdf5, …
+
+In this case it is highly recommended to code each step - usually download, configure, compile and install - in a script, write a modulefile and a configuration file.
+ -
+
Building a module with a binary package. Examples: Intel compiler, Matlab, +ANSYS, Mathematica, …
+
+In this case best practice is to install the package by hand and write a "dummy" build script, a modulefile and a configuration file. If it is possible and simple to script the installation, code this in the build script. If you cannot easily script the installation, document each step in a README file.
+
| + + | +
+
+
+In any case: keep everything in a Git repository on either gitlab.psi.ch or git.psi.ch.
+The Git repository for the groups |
+
Many modules for the Pmodules environment have to be build from +source. The building plan of a module is coded in a so called +build-block. A build-block consists at least of a build-script with instruction how to download, compile and install a dedicated piece of +software, a modulefile and a configuration file.
+To avoid problems with dependencies to system libraries or installed software it is best practice to build modules on dedicated systems. At PSI:
+-
+
-
+
For modules which should be able to run on all RHEL7 and newer systems, use the system pmod7.psi.ch. If you need access to the system please contact achim.gsell@psi.ch.
+
+ -
+
Merlin6 specific modules - like
+openmpi,mpichand modules depending on them - must be compiled on a Merlin6 login node.
+ -
+
Modules specific for a beamline should be compiled on a Ra login node with RHEL8 or a dedicated RHEL8 beamline system.
+
+
| + + | +
+
+
+If you build a module on RHEL7, it might not run on RHEL8 due to newer versions of system libraries. In must cases this can be solved by installing these system libraries in the module itself and setting the |
+
In the next section we describe how to write build recipes.
+4.2. Predefined variables
+Since we use BASH for our build-scripts and Tcl for modulefiles, we use BASH/Tcl syntax for variables in this documentation. So, if VARIABLE is the name of a variable, $VARIABLE the value of it.
In modulefiles and build-scripts the following variables are predefined:
+-
+
P
+-
+
the name of the module
+
+ V
+-
+
the module version. The version number consists of a major- and a minor version number, a patch-level and a release number. All numbers but the major version number are optional. Major-, minor number and patch-level are separated by dots, the release number by a minus. Example:
+1.0.3-2
+ V_MAJOR
+-
+
the major version number. Example:
+1
+ V_MINOR
+-
+
the minor version number. Example:
+0
+ V_PATCHLVL
+-
+
the patch-level. Example:
+3
+ V_RELEASE
+-
+
the release number. Example:
+2
+ V_PKG
+-
+
version number without release.
+
+ GROUP
+-
+
group the module is in.
+
+ PREFIX
+-
+
installation prefix of the module.
+
+ PMODULES_ROOT
+-
+
root of Pmodules installation. At PSI this is
+/opt/psi.
+
In build-scripts the following variables are predefined too:
+-
+
TEMP_DIR
+-
+
directory for temporary files.
+
+ SRC_DIR
+-
+
directory of unpacked sources, set to
+$PMODULES_TMPDIR/$P-$V/src
+ BUILD_DIR
+-
+
build directory, set to
+$PMODULES_TMPDIR/$P-$V/build
+ BUILDBLOCK_DIR
+-
+
Directory where the build-script is in.
+
+ BUILD_SCRIPT
+-
+
Name of the build script.
+
+ SYSTEM,OS
+-
+
Can be set with the
+--systemoption. The value defaults to the string returned byuname -s. The use ofOSis obsolete.
+
In modulefiles the following variables are predefined too:
+-
+
name
+-
+
Same as
+P(obsolete, for historical reasons).
+ version
+-
+
Same as
+V(obsolete, for historical reasons).
+ group
+-
+
The group the module is member of.
+
+
4.3. Building a Pmodule
+To simplify the building of modules, Pmodules has a simple build system. The build system of Pmodules is far less powerful than the build system of Spack or NixOS, but fulfills its purpose for many modules.
+Perform the following steps for a new Pmodule:
+-
+
-
+
Create a directory with the name of the module.
+
+ -
+
Write the configuration file. See section Writing build configuration files.
+
+ -
+
Write the modulefile. See section Writing
+modulefiles.
+ -
+
Code all required steps to install a software in a script. If it is not possible to script everything, code as much as possible and document additional steps in a README file. See section Writing build scripts.
+
+
The first - and sometimes also to most challenging - step in building a Pmodule from source is to learn how to download, configure, compile and install a software package and dependencies the package has. The next steps are writing a configuration file, a modulefile and a script to download, configure, compile and install it.
+In general the following steps needs to be performed:
+-
+
- prepare +
-
+
Download, unpack, apply patches (optional). Downloading the software might not be scriptable due to password protection or other measurements.
+
+ - configure +
-
+
(only for software distributed as source code) This step heavily depends on the software itself. In many cases autotools or CMake is used, in some cases you have to deal with Makefiles or other tools.
+
+ - build +
-
+
(only for software distributed as source code) compile/build everything
+
+ - install +
-
+
Install everything in the target directory. In case of a binary package this might not be scriptable.
+
+
4.3.1. Writing a build configuration file
+| + + | +
+
+
+In Pmodules 1.1.22 and newer support for the old ("legacy") +configuration files has been dropped. In the appendix you find the +documentation of tje legacy configuration files for Pmodules 1.0 + |
+
4.3.2. The build configuration file
+The build configuration file is written in the YAML format. In the file +most information required to build a module is defined. This includes
+-
+
-
+
which versions to build
+
+ -
+
dependencies to build and run the software in the module
+
+ -
+
where to download the software
+
+ -
+
options/arguments how to configure the software for compilation
+
+ -
+
sub-packages which have to be build and installed in the module +(like git-lfs in the Git module)
+
+ -
+
the release stage
+
+ -
+
…
+
+
The default path to the configuration file is
+build-recipe-dir/files/config.yaml
This can be overwritten with the command line option --config-file.
4.3.3. Format
+The general format of the YAML configuration file is:
+format: 1 (1)
+name-1:
+ type: module | sub_package (2)
+ defaults: (3)
+ config-block
+ shasums: (4)
+ filename-1: sha256sum-1
+ ...
+ versions:
+ version-key-1:
+ config: (5)
+ config-block
+ variants: (6)
+ - config-block-1
+ - ...
+ ...
+name-N:
+ ...| 1 | +For now the format must be set to 1. |
+
| 2 | +Required if type is sub_package. |
+
| 3 | +Optional. | +
| 4 | +Optional, but highly recommended! | +
| 5 | +Optional, but then defaults have to be set. | +
| 6 | +Required if there is more than one variant. | +
The top-level format of the configuration file is pretty simple:
+format: 1
+name-1:
+ config for module 1
+name-2:
+ config for module 2
+...
+name-N:
+ config for module N| Field-Name | +Description | +
|---|---|
+
+
+ |
+
+ Format version of the YAML configuration file. For now it must be |
+
+
+
+ |
+
+ Configuration for the Pmodule |
+
On the second level of the YAML configuration file is the per module +configuration with the following keys:
+ type: module | sub_package
+ defaults:
+ config-block
+ shasums:
+ filename-1: sha256sum-1
+ ...
+ versions:
+ map with configurations for different versions| Field-Name | +Description | +
|---|---|
+
+
+ |
+
+ Type of configuration, either |
+
+
+
+ |
+
+ Default configuration. |
+
+
+
+ |
+
+ SHA256 hash sums of source files. Keys are filenames, the value the |
+
+
+
+ |
+
+ Configurations for different versions. See table Table 3 + documentation. + |
+
TBW
+| Field-Name | +Description | +
|---|---|
+
+
+ |
+
+ Configurations for this version keys
+ A version key is a semicolon separated string of version
+ numbers. Version numbers can be specified with shell brace
+ expansion. Example: |
+
+
+
+ |
+
+ Optional, version specific configuration block. Configurations
+ specified here are overriding the defaults. |
+
+
+
+ |
+
+ Variants can be used to compile the some software with different
+ options. Example: |
+
4.3.4. Configuration blocks
+| + + | +
+
+
+In configuration blocks everything is optional! + |
+
config-blockbuild_requires: [build-req-1, ...]
+compile_in_sourcetree: bool
+configure_with: auto|autotools|cmake
+default_variant: variant
+docfiles: [file-1, ...]
+group: group
+group_deps:
+ compilers:
+ cpmpiler-1: [version-1, ...]
+ mpi:
+ mpi-implemation-1: [version-1, ...]
+ hdf5:
+ hdf5: [version-1, ...]
+ hdf5_serial:
+ hdf5[_serial]: [_version-1, ...]
+overlay: overlay
+relstage: release-stage
+runtime_deps: [rt-dep-1, ...]
+suffix: suffix
+systems: [system-1, ...]
+urls:
+ - url: link
+ name: filename
+ - ...
+variant: [variant-1, ...]| Field-Name | +Type | +Description | +
|---|---|---|
|
+sequence of string |
+
+
+Modules required to build this module +
+ Example
+
+
+ |
+
|
+map |
+
+
+Functions to build the module. The functions must be defined either in the build script or available. The following keys are allowed:
+
+
+ Example
+
+
+ |
+
|
+sequence of strings |
++ |
|
+boolean |
+
+
+Compile in source tree or in a dedicate build directory. The default is
+ Example
+
+
+ |
+
|
+string |
+
+
+Choose software configuration system. Allowed values are:
+ Example
+
+
+ |
+
|
+sequence of strings |
++ |
|
+sequence of string |
+
+
+Array with documentation files to be installed in
+ Example
+
+
+ |
+
|
+string |
++ |
|
+string |
+
+ Group of the module + |
+
|
+group-deps-object |
+
+ Group/hierarchical dependencies like compiler, MPI, HDF5 modules + |
+
|
+string |
++ |
|
+string |
+
+ Overlay of the module + |
+
|
+sequence of strings |
++ |
|
+string |
+
+ Release stage of the module. Allowed values are: |
+
|
+sequence of strings |
+
+ Module to be loaded at runtime + |
+
|
+string |
+
+ This string will be added to the version of the module + |
+
|
+sequence of strings |
+
+ A 'system' is either a hostname or OS name like |
+
|
+map |
++ |
|
++ | + |
|
+sequence of links and optional file-names |
+
+ Software which must be downloaded to build a specific module. |
+
|
+sequence of strings |
++ |
|
+sequence of strings |
++ |
4.3.5. Examples
+format: 1
+gnuplot:
+ defaults: (1)
+ group: Tools (2)
+ overlay: base (3)
+ relstage: stable (4)
+ systems: [rhel8, rhel7, rhel6] (5)
+ docfiles: [Copyright, NEWS, README] (6)
+ urls: (7)
+ - url: https://sourceforge.net/projects/gnuplot/files/$P/$V/$P-${V_PKG}.tar.gz
+ shasums: (8)
+ gnuplot-5.4.10.tar.gz: 975d8c1cc2c41c7cedc4e323aff035d977feb9a97f0296dd2a8a66d197a5b27c
+ gnuplot-5.4.9.tar.gz: a328a021f53dc05459be6066020e9a71e8eab6255d3381e22696120d465c6a97
+ gnuplot-5.4.8.tar.gz: 931279c7caad1aff7d46cb4766f1ff41c26d9be9daf0bcf0c79deeee3d91f5cf
+ gnuplot-5.4.5.tar.gz: 66f679115dd30559e110498fc94d926949d4d370b4999a042e724b8e910ee478
+ gnuplot-5.4.4.tar.gz: 372300b7867f5b3538b25fc5d0ac7734af6e3fe0d202b6db926e4369913f0902
+ gnuplot-5.4.3.tar.gz: 51f89bbab90f96d3543f95235368d188eb1e26eda296912256abcd3535bd4d84
+ gnuplot-5.4.2.tar.gz: e57c75e1318133951d32a83bcdc4aff17fed28722c4e71f2305cfc2ae1cae7ba
+ gnuplot-5.4.1.tar.gz: 6b690485567eaeb938c26936e5e0681cf70c856d273cc2c45fabf64d8bc6590e
+ gnuplot-5.4.0.tar.gz: eb4082f03a399fd1e9e2b380cf7a4f785e77023d8dcc7e17570c1b5570a49c47
+ gnuplot-5.2.8.tar.gz: 60a6764ccf404a1668c140f11cc1f699290ab70daa1151bb58fed6139a28ac37
+ gnuplot-5.2.7.tar.gz: 97fe503ff3b2e356fe2ae32203fc7fd2cf9cef1f46b60fe46dc501a228b9f4ed
+ gnuplot-5.2.6.tar.gz: 35dd8f013139e31b3028fac280ee12d4b1346d9bb5c501586d1b5a04ae7a94ee
+ gnuplot-5.2.4.tar.gz: 1515f000bd373aaa53b16183f274189d4f5e0ae47d22f434857933d16a4770cb
+ gnuplot-5.0.0.tar.gz: 417d4bc5bc914a60409bb75cf18dd14f48b07f53c6ad3c4a4d3cd9a8d7370faf
+ gnuplot-4.6.3.tar.gz: df5ffafa25fb32b3ecc0206a520f6bca8680e6dcc961efd30df34c0a1b7ea7f5
+ versions:
+ 5.4.{0,1,2,3,4,5,8,9};5.2.{0,4,6,7,8};5.0.0;4.6.3: (9)
+ 5.4.10: (10)
+ config:
+ relstage: unstable
+| 1 | +Configuration is for gnuplot |
+
| 2 | +will be installed in the group Tools |
+
| 3 | +will be installed in the base overlay (/opt/psi) |
+
| 4 | +the default release stage is stable |
+
| 5 | +the module is available on these systems | +
| 6 | +install these files to $PREFIX/share/doc/gnuplot |
+
| 7 | +download via this link | +
| 8 | +SHA256 hash sums for all available versions | +
| 9 | +no specific configuration required for these versions | +
| 10 | +release stage of version 5.4.10 is still unstable, override
+default release stage.
+--- |
+
---
+# yamllint disable rule:line-length (1)
+format: 1
+hdf5: (2)
+ defaults:
+ group: MPI (3)
+ overlay: base (4)
+ relstage: stable (5)
+ systems: [rhel7, rhel8, rhel9] (6)
+ urls: (7)
+ - url: https://support.hdfgroup.org/ftp/HDF5/releases/$P-${V_MAJOR}.${V_MINOR}/$P-${V_PKG}/src/$P-${V_PKG}.tar.bz2
+ shasums: (8)
+ hdf5-1.8.10-patch1.tar.bz2: 292afb3615ad9e68f4d5d18ebb11e4a73f2aece39f2da3875a457ff1e109fc41
+ hdf5-1.8.12.tar.bz2: 10a369a4fc207bb09245f57c758e587420e06dfc0e445e337a58b0848b75a949
+ ...
+ hdf5-1.13.1.tar.bz2: e16973ec893e2d5aa9c8dc73e196db9b99a605578e7317b421c713936f8bf57d
+
+ versions:
+ 1.8.12:
+ config:
+ relstage: deprecated (9)
+ variants:
+ -
+ group_deps:
+ compiler: {gcc: [4.7.4, 4.8.3, 4.8.4, 4.9.2]}
+ mpi: {openmpi: [1.6.5, 1.8.2, 1.8.4]}
+ -
+ group_deps:
+ compiler: {gcc: [4.8.2]}
+ mpi: {openmpi: [1.6.5]}
+ ...
+ -
+ group_deps:
+ compiler: {gcc: [5.1.0], intel: [15.2, 15.3]}
+ mpi: {openmpi: [1.8.4]}
+ ...
+ 1.10.8_slurm:
+ variants: (10)
+ -
+ group_deps:
+ compiler: {gcc: [10.4.0]}
+ mpi: {openmpi: [4.1.4_slurm]}
+ -
+ relstage: unstable
+ group_deps:
+ compiler: {gcc: [9.5.0, 10.4.0, 11.4.0, 12.3.0, 13.1.0]}
+ mpi: {openmpi: [4.1.5_slurm]}
+ ...
+ 1.12.0:
+ variants:
+ -
+ group_deps:
+ compiler: {gcc: [7.5.0, 8.4.0, 9.3.0, 10.2.0]}
+ mpi: {openmpi: [4.0.5]}
+ -
+ relstage: unstable
+ group_deps:
+ compiler: {pgi: [21.5]}
+ mpi: {pgi-mpi: [21.5]}
+
+ 1.13.1:
+ variants:
+ -
+ suffix: _slurm (11)
+ variant: [_slurm]
+ relstage: unstable
+ group_deps:
+ compiler: {gcc: [11.2.0]}
+ mpi: {openmpi: [4.1.3_slurm]}
+| 1 | +disable rule to check line length in yamllint. Default is 80. | +
| 2 | +this configuration is for HDF5 | +
| 3 | +default group is MPI. | +
| 4 | +default overlay is base |
+
| 5 | +default release stage is stable |
+
| 6 | +this build-block can be used on RHEL7, RHEL8 and RHEL9 | +
| 7 | +download via this link | +
| 8 | +SHA256 hash sums for all available versions | +
| 9 | +all variants of version 1.8.12 are deprecated |
+
| 10 | +hdf5/1.10.8_slurm has two variants. The first variant compiled with GCC 10.4.0 and openmpi/4.1.4_slurm. The second variant is unstable and compiled with with GCC 9.5.0, 10.4.0, 11.4.0, 12.3.0, 13.1.0 and openmpi/4.1.5_slurm. | +
| 11 | +TBW | +
+
4.4. Writing modulefiles
+4.4.1. modulefiles
+4.4.1.1. NAME
+modulefile - files containing Tcl code for the Modules package
4.4.1.2. DESCRIPTION
+modulefiles are written in the Tool Command Language, Tcl(n) and are
+interpreted by the modulecmd program via the module(1) user
+interface. modulefiles can be loaded, unloaded, or switched on-the-fly
+while the user is working; and can be used to implement site policies
+regarding the access and use of applications.
A modulefile begins with the shebang, ‘#%Pmodule’ or
+‘#%Module’. The shebang ‘#%Pmodule’ should be used in modulefiles
+using the Pmodules extension. A version number may be placed after
+this string. The version number is useful as the modulefile format may
+change. If a version number doesn’t exist, then modulecmd will assume
+the modulefile is compatible with the latest version. The current
+modulefile version is 1.0. Files without the magic cookie will not be
+interpreted by modulecmd.
Each modulefile contains the changes to a user’s environment needed to +access an application. Tcl is a simple programming language which +permits modulefiles to be arbitrarily complex, depending upon the +application’s and the modulefile writer’s needs.
+A typical modulefiles is a simple bit of code that set or add entries
+to the PATH, MANPATH, or other environment variables. Tcl has
+conditional statements that are evaluated when the modulefile is
+loaded. This is very effective for managing path or environment
+changes due to different OS releases or architectures. The user
+environment information is encapsulated into a single modulefile kept
+in a central location. The same modulefile is used by every user on
+any machine. So, from the user’s perspective, starting an application
+is exactly the same irrespective of the machine or platform they are
+on.
modulefiles also hide the notion of different types of shells. From +the user’s perspective, changing the environment for one shell looks +exactly the same as changing the environment for another shell. This +is useful for new or novice users and eliminates the need for +statements such as "if you’re using the C Shell do this …, otherwise +if you’re using the Bourne shell do this …". Announcing and +accessing new software is uniform and independent of the user’s +shell. From the modulefile writer’s perspective, this means one set of +information will take care of every type of shell.
+Module specific commands
+The Pmodules Package uses commands which are extensions to the "standard" Tool Command Language Tcl(n) package.
| + + | +
+
+
+The following commands are Pmodules extensions: +
+
+
|
+
Unless otherwise specified, the Module commands return the empty string. Some commands behave differently when a modulefile is loaded or unloaded. The command descriptions assume the modulefile is being loaded.
+-
+
break
+-
+
This is not a Modules-specific command, it’s actually part of Tcl, which has been overloaded similar to the continue and exit commands to have the effect of causing the module not to be listed as loaded and not affect other modules being loaded concurrently. All non-environment commands within the module will be performed up to this point and processing will continue on to the next module on the command line. The break command will only have this effect if not used within a Tcl loop though.
+++An example: Suppose that a full selection of modulefiles are needed for various different architectures, but some of the modulefiles are not needed and the user should be alerted. Having the unnecessary modulefile be a link to the following
+notavailmodulefile will perform the task as required.++++#%Module1.0 +## notavail modulefile +## +proc ModulesHelp { } { + puts stderr "This module does nothing but alert the user" + puts stderr "that the [module-info name] module is not available" +} + +module-whatis "Notifies user that module is not available." +set curMod [module-info name] +if { [ module-info mode load ] } { + puts stderr "Note: '$curMod' is not available for [uname sysname]." +} +break+
+ chdir directory
+-
+
Set the current working directory to directory.
+
+ continue
+-
+
This is not a modules specific command but another overloaded Tcl command and is similar to the break or exit commands except the module will be listed as loaded as well as performing any environment or Tcl commands up to this point and then continuing on to the next module on the command line. The continue command will only have this effect if not used within a Tcl loop though.
+
+ exit [N]
+-
+
This is not a modules specific command but another overloaded Tcl command and is similar to the break or continue commands. However, this command will cause the immediate cessation of this module and any additional ones on the command line. This module and the subsequent modules will not be listed as loaded. No environment commands will be performed in the current module.
+
+ setenv variable value
+-
+
Set environment variable to value. The
+setenvcommand will also change the process' environment. A reference using Tcl’senvassociative array will reference changes made with thesetenvcommand. Changes made using Tcl’senvassociative array will NOT change the user’s environment variable like thesetenvcommand. An environment change made this way will only affect the module parsing process. Thesetenvcommand is also useful for changing the environment prior to theexecorsystemcommand. When a modulefile is unloaded,setenvbecomesunsetenv. If the environment variable had been defined it will be overwritten while loading the modulefile. A subsequent unload will unset the environment variable - the previous value cannot be restored! (Unless you handle it explicitly … see below.)
+ unsetenv variable [value]
+-
+
Unsets environment variable. However, if there is an optional value, then when unloading a module, it will set variable to value. The
+unsetenvcommand changes the process' environment likesetenv.
+ append-path|prepend-path [-d C|--delim C|--delim=C] variable value
+-
+
Append or prepend value to environment variable. The variable is a colon, or delimiter, separated list such as
+PATH=directory:directory:directory. The default delimiter is a colon ':', but an arbitrary one can be given by the--delimoption. For example a space can be used instead (which will need to be handled in the Tcl specially by enclosing it in " " or { }). A space, however, can not be specified by the--delim=Cform.++If the variable is not set, it is created. When a modulefile is unloaded,
+append-pathandprepend-pathbecomeremove-path.
+ remove-path [-d C|--delim C|--delim=C] variable value
+-
+
Remove value from the colon, or delimiter, separated list in variable. See
+prepend-pathorappend-pathfor further explanation of using an arbitrary delimiter. Every string between colons, or delimiters, in variable is compared to value. If the two match, value is removed from variable.
+ prereq|conflict modulefile…
+-
+
+prereqandconflictcontrol whether or not the modulefile will be loaded. Theprereqcommand lists modulefiles which must have been previously loaded before the current modulefile will be loaded. Similarly, the conflict command lists modulefiles which conflict with the current modulefile. If a list contains more than one modulefile, then each member of the list acts as a Boolean OR operation. Multipleprereqandconflictcommands may be used to create a Boolean AND operation. If one of the requirements have not been satisfied, an error is reported and the current modulefile makes no changes to the user’s environment.
+
If an argument for prereq is a directory and any modulefile from the directory has been loaded, then the prerequisite is met. For example, specifying X11 as a prereq means that any version of X11, X11/R4 or X11/R5, must be loaded before proceeding.
If an argument for conflict is a directory and any other modulefile from that directory has been loaded, then a conflict will occur. For example, specifying X11 as a conflict will stop X11/R4 and X11/R5 from being loaded at the same time.
-
+
is-loaded modulefile…
+-
+
The
+is-loadedcommand returns a true value if any of the listed modulefiles has been loaded. If a list contains more than one modulefile, then each member acts as a boolean OR operation. If an argument foris-loadedis a directory and any modulefile from the directory has been loadedis-loadedwould return a true value.
+ module [sub-command] [sub-command-args]
+-
+
Contains the same sub-commands as described in the
+module(1)man page in the Module Sub-Commands section. This command permits a modulefile to load or unload other modulefiles. No checks are made to ensure that the modulefile does not try to load itself. Often it is useful to have a single modulefile that performs a number of module load commands. For example, if every user on the system requires a basic set of applications loaded, then a core modulefile would contain the necessary +module load commands.
+ module-info mode [modetype]
+-
+
Returns the current modulecmd’s mode as a string if no modetype is given.
+++Returns 1 if modulecmd’s mode is
+modetype.modetypecan be:load,unload,remove,switch,display,help,testorwhatis.
+ module-info name
+-
+
Return the name of the modulefile. This is not the full pathname for modulefile. See the Modules Variables section for information on the full pathname.
+
+ module-info specified
+-
+
Return the name of the modulefile specified on the command line.
+
+ module-info shell [shellname]
+-
+
Return the current shell under which modulecmd.tcl was invoked if no shellname is given. The current shell is the first parameter of modulecmd.tcl, which is normally hidden by the module alias.
+++If a shellname is given, returns 1 if modulecmd.tcl’s current shell is shellname, returns 0 elsewhere. shellname can be: sh, bash, ksh, zsh, csh, tcsh, fish, tcl, perl, python, ruby, lisp, cmake, r.
+
+ module-info shelltype [shelltypename]
+-
+
Return the family of the shell under which modulefile was invoked if no shelltypename is given. As of module-info shell this depends on the first parameter of modulecmd.tcl. The output reflects a shell type determining the shell syntax of the commands produced by modulecmd.tcl.
+++If a shelltypename is given, returns 1 if modulecmd.tcl’s current shell type is shelltypename, returns 0 elsewhere. shelltypename can be: sh, csh, fish, tcl, perl, python, ruby, lisp, cmake, r.
+
+ module-info alias name
+-
+
Returns the full modulefile name to which the modulefile alias name is assigned
+
+ module-info version modulefile
+-
+
Returns the physical module name and version of the passed symbolic version modulefile. The parameter modulefile might either be a full qualified modulefile with name and version, another symbolic modulefile name or a modulefile alias.
+
+ module-info symbols modulefile
+-
+
Returns a list of all symbolic versions assigned to the passed modulefile. The parameter modulefile might either be a full qualified modulefile with name and version, another symbolic modulefile name or a modulefile alias.
+
+ module-version modulefile version-name…
+-
+
Assigns the symbolic version-name to the modulefile. This command should be placed in one of the modulecmd.tcl rc files in order to provide shorthand invocations of frequently used modulefile names.
+++The special version-name default specifies the default version to be used for module commands, if no specific version is given. This replaces the definitions made in the
+.versionfile in former modulecmd releases.++The parameter
+modulefilemay be either +* a fully or partially qualified modulefile with name / version. If name is '.' then the current directory name is assumed to be the module name. (Use this for deep modulefile directories.) +* a symbolic modulefile name +* another modulefile alias
+ module-alias name modulefile
+-
+
Assigns the modulefile to the alias name. This command should be placed in one of the modulecmd rc files in order to provide shorthand invocations of frequently used modulefile names.
+++The parameter modulefile may be either
+++-
+
-
+
a fully qualified modulefile with name and version
+
+ -
+
a symbolic modulefile name
+
+ -
+
another modulefile alias
+
+
+ -
+
module-whatis string
+-
+
Defines a string which is displayed in case of the invocation of the
+module whatiscommand. There may be more than onemodule-whatisline in a modulefile. This command takes no actions in case ofload,display, etc. invocations of modulecmd.++The string parameter has to be enclosed in double-quotes if there’s more than one word specified. Words are defined to be separated by white-space characters (space, tab, cr).
+
+ module-url url
+-
+
Sets an URL which is displayed in case of the invocation of the
+module helpcommand. The URL should point to the home-page of the software loaded by the module if any.
+ module-license string
+-
+
Defines a string which is displayed in case of the invocation of the
+module helpcommand. The string should either be the name of a well known license (like GPL V2) or a filename with the license text installed in the module.
+ module-maintainer email
+-
+
Defines a string which is displayed in case of the invocation of the
+module helpcommand. The +string should be the email address of the module maintainer.
+ module-addgroup group
+-
+
Add the hierarchical group
+grouptoMODULEPATH.
+ set-alias alias-name alias-string
+-
+
Sets an alias or function with the name
+alias-namein the user’s environment to the stringalias-string. For some shells, aliases are not possible and the command has no effect. When a modulefile is unloaded,set-aliasbecomesunset-alias.
+ unset-alias alias-name
+-
+
Unsets an alias with the name
+alias-namein the user’s environment.
+ system string
+-
+
Pass string to the Tcl built-in command exec(n). For the exec(n) call modulecmd redirects stdout to stderr since stdout would be parsed by the evaluating shell. The exit status of the executed command is returned.
+
+ uname field
+-
+
Provide lookup of system information. Most field information are retrieved from the tcl_platform array (see
+tclvars(n)man page).Unamewill return the string "unknown" if information is unavailable for the field.++
+unamewill invokeuname(1)command in order to get the operating system version anddomainname(1)to figure out the name of the domain.++field values are: +*
+sysname: the operating system name +*nodename: the hostname +*domain: the name of the domain +*release: the operating system release +*version: the operating system version +*machine: a standard name that identifies the system’s hardware
+ x-resource [resource-string|filename]
+-
+
Merge resources into the X11 resource database. The resources are used to control look and behavior of X11 applications. The command will attempt to read resources from filename. If the argument isn’t a valid file name, then string will be interpreted as a resource. Either filename or
+resource-stringis then passed down to bexrdb(1)command.++modulefiles that use this command, should in most cases contain one or more x-resource lines, each defining one X11 resource. The
+DISPLAYenvironment variable should be properly set and the X11 server should be accessible. If x-resource can’t manipulate the X11 resource database, the modulefile will exit with an error message.++Examples:
+
+
x-resource /u2/staff/leif/.xres/Ileaf+
The content of the Ileaf file is merged into the X11 resource database.
+x-resource [glob ~/.xres/ileaf]+
The Tcl glob function is used to have the modulefile read different resource files for different users.
+x-resource {Ileaf.popup.saveUnder: True}
+Merge the Ileaf resource into the X11 resource database.
+Modules Variables
+The ModulesCurrentModulefile variable contains the full pathname of the modulefile being interpreted.
Locating Modulefiles
+Every directory in MODULEPATH is searched to find the modulefile. A directory in MODULEPATH can have an arbitrary number of sub-directories. If the user names a modulefile to be loaded which is actually a directory, the directory is opened and a search begins for an actual modulefile. First, modulecmd looks for a file with the name .modulerc in the directory. If this file exists, its contents will be evaluated as if it was a modulefile to be loaded. You may place module-version and module-alias commands inside this file.
Additionally, before seeking for .modulerc files in the module directory, the global modulerc file is sourced, too. If a named version default now exists for the modulefile to be loaded, the assigned modulefile now will be sourced. Otherwise the file .version is looked up in the directory.
If the .version file exists, it is opened and interpreted as Tcl code and takes precedence over a .modulerc file in the same directory. If the Tcl variable ModulesVersion is set by the .version file, modulecmd will use the name as if it specifies a modulefile in the directory. This will become the default modulefile in this case.
If ModulesVersion is a directory, the search begins anew down that directory. If the name does not match any files located in the current directory, the search continues through the remaining directories in MODULEPATH.
Every .version and .modulerc file found is Tcl interpreted. The difference is that .version only applies to the current directory, and the .modulerc applies to the current directory and all subdirectories. Changes made in these files will affect the subsequently interpreted modulefile.
If no default version may be figured out, then the highest numerically sorted modulefile or module alias under the directory will be used. The dictionary comparison method of the lsort(n) Tcl command is used to achieve this sort. If highest numerically sorted element is an alias, search continues on its modulefile target.
For example, it is possible for a user to have a directory named X11 which simply contains a .version file specifying which version of X11 is to be loaded. Such a file would look like:
#%Module1.0 +## +## The desired version of X11 +## +set ModulesVersion "R4"+
The equivalent .modulerc would look like:
+#%Module1.0 +## +## The desired version of X11 +## +module-version "./R4" default+
If user names a modulefile that cannot be found in the first modulepath directory, modulefile will be searched in next modulepath directory and so on until a matching modulefile is found. If search goes through a module alias or a symbolic version, this alias or symbol is resolved by first looking at the modulefiles in the modulepath where this alias or symbol is defined. If not found, resolution looks at the other modulepaths in their definition order.
+When locating modulefiles, if a .modulerc, a .version, a directory or a modulefile cannot be read during the search it is simply ignored with no error message produced. Visibility of modulefiles can thus be adapted to the rights the user has been granted. Exception is made when trying to directly access a directory or a modulefile. In this case, the access issue is returned as an error message.
A modulefile whose name or element in its name starts with a '.' dot is considered hidden. Hidden modulefile is not displayed or taken into account except if it is explicitly named. By inheritance, a symbolic version-name assigned to a hidden modulefile is displayed or taken into account only if explicitly named. Module alias targeting a hidden modulefile appears like any other module alias.
+Modulefile Specific Help
+Users can request help about a specific modulefile through the module(1) command. The modulefile can print helpful information or start help oriented programs by defining a ModulesHelp subroutine. The subroutine will be called when the module help modulefile command is used.
Modulefile Specific Test
+Users can request test of a specific modulefile through the module(1) command. The modulefile can perform some sanity checks on its definition or on its underlying programs by defining a ModulesTest subroutine. The subroutine will be called when the module test modulefile command is used. The subroutine should return 1 in case of success. If no or any other value is returned, test is considered failed.
Modulefile Display
+The module display modulefile command will detail all changes that will be made to the environment. After displaying all of the environment changes modulecmd.tcl will call the ModulesDisplay subroutine. The ModulesDisplay subroutine is a good place to put additional descriptive information about the modulefile.
4.4.1.3. ENVIRONMENT
+-
+
MODULEPATH
+-
+
Path of directories containing modulefiles.
+
+
4.4.1.4. SEE ALSO
+module(1), Tcl(n), TclX(n), xrdb(1), exec(n), uname(1), domainname(1), tclvars(n), lsort(n)
4.4.1.5. NOTES
+Tcl was developed by John Ousterhout at the University of California at Berkeley.
+TclX was developed by Karl Lehenbauer and Mark Diekhans.
+Modules is covered by the GNU General Public License, version 2 and the GNU Lesser General Public License, version 2.1. Copyright © 1996-1999 John L. Furlani & Peter W. Osel, © 1998-2017 R.K.Owen, © 2002-2004 Mark Lakata, © 2004-2017 Kent Mein, © 2016-2017 Xavier Delaruelle. All rights reserved. Trademarks used are the property of their respective owners.
++
4.5. Writing build scripts
+Pmodules has it’s own 'interpreter' to run build-scripts written in Bash. The name of the interpreter is modbuild. Thus the first line (the so called 'shebang') of a build-script must be
#!/usr/bin/env modbuildAs already mentioned the four steps 'prepare', 'configure', 'build' and 'install' has to be performed to build a Pmodule. These steps maps to the functions:
+-
+
- prepare +
-
+
+pbuild::pre_prep()
+pbuild::prep()
+pbuild::post_prep()
+ - configure +
-
+
+pbuild::pre_configure()
+pbuild::configure()
+pbuild::post_configure()
+ - compile +
-
+
+pbuild::pre_compile()
+pbuild::compile()
+pbuild::post_compile()
+ - install +
-
+
+pbuild::pre_install()
+pbuild::install()
+pbuild::post_install()
+
In many cases you don’t have to implement the the functions pbuild::prep(), pbuild::configure(), pbuild::compile() and pbuild::install() or at least not all of them. The build-system provides default implementations for these functions. In the simplest case, the build script consists only of the shebang line (only for Pmodules >= 1.1). In this case all further information for building the module is in the configuration file. For software that uses autotools or CMake for configuration and creation of Makefiles, the standard functions can practically always be used. The functions pbuild::pre_STEP and pbuild::post_STEP can be used to hook into each step before and after the default function has been called. For example pbuild::pre_configure() can be used to set arguments for autotools or CMake as in the build-script for Gnuplot:
#!/usr/bin/env modbuild
+
+pbuild::pre_configure() {
+ pbuild::add_configure_args '--with-latex=no'
+ pbuild::add_configure_args '--with-qt=no'
+}4.5.1. Overloading the default implementations
+If the build-system of the software package isn’t autotools or CMake you have to overload the default implementation of pbuild::configure.
pbuild::configure(){
+ # add code to configure the software package
+}In case where no configuration is required, overload the default function with
+pbuild::configure(){
+ : # do nothing
+}You can do the same with the other default implementation.
+4.5.2. Modules from binary packages
+If you have a binary package like Matlab, Mathematica, ANSYS, it might not be possible to script the installation. In this case you should use a dummy build-script like below and document the installation in a README.md:
#!/usr/bin/env modbuild
+
+pbuild::prep() { :; }
+pbuild::configure() { :; }
+pbuild::compile() { :; }
+pbuild::install() { :; }If the installation can be scripted, code this in the function pbuild::install.
#!/usr/bin/env modbuild
+
+pbuild::prep() { :; }
+pbuild::configure() { :; }
+pbuild::compile() { :; }
+pbuild::install() {
+ # add code to call the installer here
+}4.5.3. Calling the build-script
+To call the build-script, cd into the directory of the script and run
+./build VERSION_TO_BE_BUILDpmod7:~/HelloWorld$ ./build 1.0.0
+Using YAML configuration file - /afs/psi.ch/user/g/gsell/HelloWorld/files/config.yaml
+HelloWorld/1.0.0: building ...
+HelloWorld/1.0.0:
+HelloWorld/1.0.0: start building ...
+HelloWorld/1.0.0: preparing sources ...
+HelloWorld/1.0.0: configuring ...
+HelloWorld/1.0.0: compiling ...
+HelloWorld/1.0.0: installing ...
+HelloWorld/1.0.0: running post-installation for Linux ...
+HelloWorld/1.0.0: Installing documentation to /opt/psi/Sandbox/HelloWorld/1.0.0/share/doc/HelloWorld
+HelloWorld/1.0.0: adding modulefile to overlay 'base' ...
+HelloWorld/1.0.0: Cleaning up '/var/tmp/gsell/HelloWorld-1.0.0/build'...
+HelloWorld/1.0.0: Cleaning up '/var/tmp/gsell/HelloWorld-1.0.0/src'...
+HelloWorld/1.0.0: Done ...
+* * * * *
+
+pmod7:~/HelloWorld$You can run
+./build --helpto get a list of all option with a description. The most common option are
+-
+
-f
+-
+
to force a rebuild.
+
+ --clean-install
+-
+
remove everything in the build directory and the module itself - if already exists - as first step.
+
+ --verbose
+-
+
to see more verbose output.
+
+ --debug
+-
+
to get a full debug trace.
+
+
4.5.4. Setup functions
+4.5.4.1. Adding patches to be applied: pbuild::add_patch
+Synopsis
+pbuild::add_patch patch_fname [strip_num_dirs]Description
+Apply a patch to the unpacked sources. The argument patch_fname must be a relative file-name to the build-block directory. The argument strip_num_dirs is optional and defaults to 1. Please read the patch(1) manual page for more details. The patches are applied inside the source directory with the command:
patch --strip=strip_num_dirs < "${BUILDBLOCK_DIR}/patch_fname"Example
+pbuild::pre_prep(){
+ pbuild::add_patch 'files/Makefile.pncf.sed.patch'
+}+
4.5.4.2. Set configuration options: pbuild::add_configure_args
+Synopsis
+pbuild::add_configure_args arg...Description
+Add arguments/options to the call of configure or cmake.
Example
+autotools:
+pbuild::pre_configure(){
+ pbuild::add_configure_args "--disable-shared"
+ pbuild::add_configure_args "CFLAGS=-fPIC"
+}CMake:
+pbuild::pre_configure(){
+ pbuild::add_configure_args "-DMETIS_PATH=${SRC_DIR}/metis"
+ pbuild::add_configure_args "-DGKLIB_PATH=${SRC_DIR}/metis/GKlib"
+}+
4.5.5. Functions to compare versions
+4.5.5.1. Compare two versions: pbuild::version_compare
+Synopsis
+pbuild::version_compare VERSION1 VERSION2
+pbuild::version_lt VERSION1 VERSION2
+pbuild::version_le VERSION1 VERSION2
+pbuild::version_gt VERSION1 VERSION2
+pbuild::version_ge VERSION1 VERSION2
+pbuild::version_eq VERSION1 VERSION2Description
+The function splits the version strings at the dots an compares each component. If a component is a number the comparison is numerical otherwise lexical.
+pbuild::version_compare VERSION1 VERSION2 returns
+ 0 if the versions are equal
+ 1 if VERSION1 is higher then VERSION2
+ 2 if VERSION1 is lower then VERSION2
pbuild::version_lt VERSION1 VERSION2 returns
+0 if VERSION1 is lower then VERSION2 otherwise a values greater than zero.
pbuild::version_le VERSION1 VERSION2 returns
+0 if VERSION1 is lower then or equal to VERSION2 otherwise a values greater than zero.
pbuild::version_gt VERSION1 VERSION2 returns
+0 if VERSION1 is higher then VERSION2 otherwise a values greater than zero.
pbuild::version_ge VERSION1 VERSION2 returns
+0 if VERSION1 is higher then or equal to VERSION2 otherwise a values greater than zero.
pbuild::version_eq VERSION1 VERSION2 returns
+0 if VERSION1 is equal to VERSION2 otherwise a values greater than zero.
Example
+pbuild::version_lt '1.1.0' '1.2.2' # result is 0
+pbuild::version_ge '9.1.2' '10.0.0' # result is greater zero4.5.6. Build functions
+4.5.6.1. Download and unpack: pbuild::prep
+Synopsis
+pbuild::pre_prep
+pbuild::prep
+pbuild::post_prepDescription
+Functions to prepare the sources. This includes
+-
+
-
+
downloading required files a verifying the checksums
+
+ -
+
unpacking
+
+ -
+
applying patches
+
+
Before the prep-functions are called, the build-system changes to the source
+directory (${SRC_DIR}).
In the most uses cases the default function pbuild::prep provided by
+the build-system can be used and nothing must be implemented in the
+build-script. In some rare cases pre- or post-hooks are required to
+make the default function feasible. One use case with autotools for a post-hook is to create the configure scripts, if only configure.ac is shipped with the software.
The default hooks provided by the build system are only stubs and do +nothing.
+If the default function cannot be used, it must be implemented in the build-script.
+Example
+The source distribution of the IOAPI library contains object files! These files must be removed after unpacking:
+pbuild::post_prep() {
+ find "${SRC_DIR}" -name "*.mod" -exec rm {} \;
+ find "${SRC_DIR}" -name "*.o" -exec rm {} \;
+}+
4.5.6.2. Configure software: pbuild::configure
+Synopsis
+pbuild::pre_configure
+pbuild::configure
+pbuild::post_configureDescription
+Configure the software for compilation.
+Before these functions are called, the build-system changes to the
+build directory (${BUILD_DIR}).
In the most uses cases the default function pbuild::configure provided by
+the build-system can be used and nothing must be implemented in the
+build-script.
The default function first searches whether the software can be
+configured with autotools. If yes, configuration with autotools will
+be performed. Otherwise the existence of a CMake script will be
+checked. If found, configuration will be done via CMake. If scripts
+for both configuration tools exist, the to be used tool can be
+selected with pbuild::use_autotools or
+pbuild::use_cmake. Arguments to configure or
+cmake can be set with
+pbuild::add_configure_args.
A common use case for the pre-configure hooks is to define arguments +passed to autotools or CMake by the default function.
+If the default function cannot be used, the pbuild::configure
+function must be implemented in the build-script.
Examples
+Excerpt from the parallel HDF5 build-script
+pbuild::pre_configure() {
+ pbuild::add_configure_args "CC=${MPICC}"
+ pbuild::add_configure_args "CXX=${MPICXX}"
+
+ pbuild::add_configure_args "--enable-shared"
+ pbuild::add_configure_args "--enable-parallel"
+ pbuild::add_configure_args "--enable-cxx"
+ pbuild::add_configure_args "--enable-unsupported"
+ #pbuild::add_configure_args "--enable-threadsafe"
+ pbuild::add_configure_args "--with-pic"
+
+ local enable_fortran='yes'
+
+ case "${COMPILER}" in
+ clang-macos )
+ enable_fortran='no'
+ # we do not have Fortran in Xcode
+ ;;
+ pgi )
+ # PGI uses GCC's include files, some object files and
+ # the STL implementation!
+ # The PGI C pre-processor is broken and doesn't work
+ # for HDF5. We use the pre-processor of the underlying
+ # GCC...
+ # This is a bit hackish!
+ #
+ # The following eval sets GCCDIR! Which is something
+ # like:
+ # /opt/psi/Programming/gcc/7.3.0/bin/../lib/gcc/x86_64-pc-linux-gnu/7.3.0
+ #
+ eval $(pgcc -show 2>/dev/null | \
+ awk '/^GCCDIR[[:space:]]*=/{gsub(/[[:space:]]/,""); print $0}')
+ pbuild::add_configure_args "CPP=${GCCDIR%%/..*}/cpp"
+ pbuild::add_configure_args "CFLAGS=-fPIC"
+ pbuild::add_configure_args "CXXFLAGS=-fPIC"
+ pbuild::add_configure_args "FCFLAGS=-fPIC"
+ ;;
+ esac
+
+ if [[ "${enable_fortran}" ===== 'yes' ]]; then
+ pbuild::add_configure_args "F77=${MPIF77}"
+ pbuild::add_configure_args "F90=${MPIF90}"
+ pbuild::add_configure_args "FC=${MPIFC}"
+ pbuild::add_configure_args "FORTRAN=${MPIFORTRAN}"
+ pbuild::add_configure_args "--enable-fortran"
+ fiSimplified excerpt from the OpenBLAS build-script
+pbuild::configure() {
+ case ${COMPILER} in
+ gcc )
+ CC='gcc'
+ ;;
+ intel )
+ CC='icc'
+ ;;
+ clang-macos )
+ CC='gcc'
+ ;;
+ * )
+ die 3 "Oops: unknown compiler: ${COMPILER}"
+ ;;
+ esac
+ cat <<EOF > "${SRC_DIR}/make.inc"
+SHELL = /bin/sh
+PLAT =
+DRVOPTS = \$(NOOPT)
+ARCHFLAGS= -ru
+EOF
+ echo "USE_SIMPLE_THREADED_LEVEL3 = 1" >> "${SRC_DIR}/Makefile.rule"
+ echo "NO_AVX = 1" >> "${SRC_DIR}/Makefile.rule"
+ echo "NO_AVX2 = 1" >> "${SRC_DIR}/Makefile.rule"
+ if pbuild::use_flag "omp"; then
+ echo "USE_THREAD = 1" >> "${SRC_DIR}/Makefile.rule"
+ else
+ echo "USE_THREAD = 0" >> "${SRC_DIR}/Makefile.rule"
+ fi
+}+
4.5.6.3. Compile software: pbuild::compile
+Synopsis
+pbuild::pre_compile
+pbuild::compile
+pbuild::post_compileDescription
+Compile the software.
+Before these functions are called, the build-system changes to the
+build directory (${BUILD_DIR}).
In the most uses cases the default function pbuild::compile provided by
+the build-system can be used and nothing must be implemented in the
+build-script. The pre- and post-hooks might be useful in cases where
-
+
-
+
to run other make targets than
+all
+ -
+
multiple packages must be compiled into one module
+
+ -
+
to compile a dependency required by the main package.
+
+
If the default function cannot be used, the pbuild::compile
+function must be implemented in the build-script.
Example
+Excerpt from perl build-script
+pbuild::post_compile() {
+ make test
+}+
4.5.6.4. Install software: pbuild::install
+Synopsis
+pbuild::pre_install
+pbuild::install
+pbuild::post_installDescription
+Compile the software.
+Before these functions are called, the build-system changes to the
+build directory (${BUILD_DIR}).
In the many uses cases the default function pbuild::install provided by
+the build-system can be used and nothing must be implemented in the
+build-script. A typical use case for the post-install hook is to install required libraries from other modules or the system to reduce or eliminate run-time dependencies
If the default function cannot be used, the pbuild::install
+function must be implemented in the build-script.
Example
+TBW4.6. Runtime configuration files
+Table of Contents
+Pmodules 1.0 configuration file .release-version
+Pmodules 1.1 and newer configuration file .config-version
| + + | +
+
+
+The runtime configuration files are evaluated only for modules inside
+the Pmodules hierarchy. If you use |
+
4.6.1. Pmodules 1.0: .release-version
+In .release-version the release stage of a module is
+defined. The file must be in the same directory as the modulefile. The
+content is a single line defining the release stage. Allowed text is
-
+
-
+
+unstable
+ -
+
+stable
+ -
+
+deprecated
+
If the file doesn’t exist and the module is inside the Pmodule
+hierarchy unstable. All modules outside the Pmodules hierarchy are
+considered as stable.
| + + | +
+
+
+This configuration file is deprecated in version 1.1 and newer. Please +see next section. + |
+
4.6.2. Pmodules 1.1 and newer: .config-version
+In .config-version the following properties of a module can be
+configured:
-
+
-
+
the release stage
+
+ -
+
the systems on which a module is available
+
+ -
+
the systems on which a module is unavailable
+
+
The format of the file is YAML.
+-
+
relstage
+-
+
The release stage of the module. Allowed values are +
+unstable,stableanddeprecated.
+ systems
+-
+
Sequence of systems on which the module is available. Systems +can be hostnames with shell style glob pattern or OS names like +(
+rhel7,rhel8).
+ blocklist
+-
+
Sequence of systems on which the module is no +available. Systems can be hostnames with shell style glob patterns or +OS names.
+
+
Example:
+relstage: unstable +systems: [merlin-*, ra-*] +blocklist: []+
Appendix A: "legacy" configuration files in Pmodules 1.0
+| + + | +
+
+
+These type of configuration file is deprecated in version 1.1 and +newer. Starting with version 1.2 the use of YAML configuration files +documented in the next section is recommended. + |
+
In the Pmodules 1.0 configuration files the following properties are +defined per version:
+-
+
-
+
module name and version
+
+ -
+
release stage
+
+ -
+
dependencies
+
+
Each definition must be on a single line.
+4.A.1. Configuration filenames and search order
+Configuration files are searched in the following order:
+-
+
-
+
+build-recipe-dir/files/variants.system
+ -
+
+build-recipe-dir/files/variants.OS
+ -
+
+build-recipe-dir/files/variants
+
Whereby
+-
+
system
+-
+
If the build script is called with the option +
+-system=sytem, this configuration file is used.
+ OS
+-
+
Is the operating system/kernel name returned by
+uname +-s. On Linux this isLinuxon macOS this isDarwin.
+
4.A.2. Format
+A line in a configuration file is either
+-
+
-
+
a comment line starting with
+#
+ -
+
a empty line or a with white-space only
+
+ -
+
a definition of a variant
+
+
4.A.3. Variant specifications
+The form of a variant specification is
+module-name release_stage dependencies
Whereby:
+-
+
module-name
+-
+
Is the name of the module including the version, an +optional release number and an optional suffix. The general form is
+++
+P/V[-V_RELEASE][_SUFFIX]
+ release_stage
+-
+
Defines the release stage of this module +variant. Allowed values are
+unstable,stable,deprecated, +removeandremoved. If the release stage isremoveorremoved, +this module variant will be removed (if not already removed).
+ dependencies
+-
+
Defines the module dependencies. Dependencies can +be either a hierarchical dependency, a runtime dependency or a build +dependency. Hierarchical and runtime dependencies are loaded before +a module itself. Build dependencies are only loaded during build-time +and must be prefixed with
+b:.++Dependencies are loaded in the specified order. It is recommended to +specify the dependencies of a the (hierarchical) group first, then +additional modules required at run-time and the modules required to +build it at the end.
+++If the module is in a hierarchical group like
+MPI, you must specify +the modules required for this group. For modules in the group++-
+
-
+
+Compilera compiler must be specified.
+ -
+
+MPIa compiler and a MPI implementation must be specified.
+ -
+
+HDF5a compiler, a MPI implementation and a parallel HDF5 module +must be specified.
+ -
+
+HDF5_seriala compiler and a serial HDF5 module +must be specified.
+
+ -
+
parmetis/4.0.3 stable gcc/7.3.0 openmpi/3.0.0 b:cmake/3.6.3+
+
Appendix B: Legacy build function (Pmodules version 1.0)
+| + + | +
+
+
+These functions are deprecated in Pmodules 1.1 and newer! + |
+
4.B.1. Define the group of a module: pbuild::add_to_group
+4.B.1.1. Synopsis
+pbuild::add_to_group GROUP4.B.1.2. Description
+Define the group the module will be installed in. Sometimes it is not obvious in which group a module should go. If in doubt, Tools might be the best.
A (incomplete) list of available groups:
+-
+
Tools
+-
+
Group for tools like Gnuplot, Git, vim, Emacs, openssl. This group is visible by default.
+
+ Programming
+-
+
Group for compilers, scripting languages and tools used for programming, like GCC, Python, CMake, autotools, Matlab. This group is visible by default.
+
+ Compiler
+-
+
Hierarchical group for modules compiled with a certain compiler module. Examples: open-mpi, MPICH, serial HDF5.
+
+ HDF5_serial
+-
+
Hierarchical group for modules compiled with a certain compiler and serial HDF5 module. Examples: netCDF, H5hut.
+
+ MPI
+-
+
Hierarchical group for modules compiled with a certain compiler and MPI module. Examples: parallel Boost, parallel HDF5, ParMETIS, Gromacs.
+
+ HDF5
+-
+
Hierarchical group for modules compiled with a certain compiler, MPI and parallel HDF5 module. Examples: netCDF, H5hut, Trilinos.
+
+ System
+-
+
Group for system tools. This group is not visible by default. Examples: filebench, fsstress, nmap, patchelf.
+
+ Libraries
+-
+
Group for libraries required to compile other modules but not providing (useful) tools. This group is not visible by default. Examples: GMP, MPC, MPFR
+
+ MX
+-
+
Group for special tools used at the MX beam-line. This group is visible only on dedicated systems.
+
+ EM
+-
+
Group for Ra specific software. This group is only visible and available on Ra.
+
+
4.B.1.3. Example
+pbuild::add_to_group 'Tools'+
4.B.2. Set download URL: pbuild::set_download_url
+4.B.2.1. Synopsis
+pbuild::set_download_url URL [fname]4.B.2.2. Description
+Tell the build system where to download the required (source-)files. If fname is passed, it will be used as the output file-name of the download.
In some cases software must be downloaded from a Git repository on Github, Bitbucket or Gitlab.
+To download a certain tag/version from Github, Bitbucket or Gitlab use:
+https://github.com/OWNER/PROJECT/archive/${V_PKG}/$P-${V_PKG}.tar.gz
+https://bitbucket.org/OWNER/PROJECT/get/${V_PKG}.tar.gz#/$P-%{V_PKG}.tar.gz
+https://gitlab.com/OWNER/PROJECT/-/archive/${V_PKG}/$P-${V_PKG}.tar.gz
+4.B.2.3. Examples
+Excerpt from Trilinos build-script:
+pbuild::set_download_url \
+ "https://github.com/$P/$P/tarball/$P-release-${V//./-}" \
+ "$P-$V.tar.gz"4.B.2.4. Compile in source tree: pbuild::compile_in_sourcetree (Pmodules 1.0)
+Synopsis
+pbuild::compile_in_sourcetreeDescription
+By default the build-system compiles software in a separate build-directory. This is not possible with all software. With the function pbuild::compile_in_sourcetree the build-directory is set to the source-directory.
Example
+pbuild::compile_in_sourcetree+
4.B.2.5. Set documentation files to be installed: pbuild::install_docfiles (Pmodules 1.0)
+Synopsis
+pbuild::install_docfiles fname...Description
+Install the passed files into $PREFIX/share/doc/$P. At least the file containing the license and copyright should be installed.
Example
+Excerpt from parallel-netcdf build-script:
+pbuild::add_docfiles 'AUTHORS' 'CREDITS'
+pbuild::add_docfiles 'COPYING' 'COPYRIGHT'
+pbuild::add_docfiles 'ChangeLog' 'NEWS'
+pbuild::add_docfiles 'RELEASE_NOTES'+
4.B.2.6. Pass SHA256 hash sum to build-system: pbuild::set_sha256sum (Pmodules 1.0)
+Synopsis
+pbuild::set_sha256sum SHA256_HASHDescription
+Tell the build-system which SHA256 hash sum a file must have. The format of the argument is file-name:hash-sum.
Examples
+Excerpt from Trilinos build-script:
+pbuild::set_sha256sum \
+ "trilinos-12.12.1.tar.gz:c8f2029fa36230b9f384c56139aaa33111227bcf653e73f7daf3c9efdecc1d2d"+
4.B.2.7. Supported compilers: pbuild::set_supported_compilers (Pmodules 1.0)
+Synopsis
+pbuild::supported_compilers STRING...Description
+Set list of compiler which can be used to compile the module.
+Example
+pbuild::supported_compilers gcc clang+
4.B.2.8. Set which OS are supported: pbuild::supported_systems (Pmodules 1.0)
+Synopsis
+pbuild::supported_system STRING...Description
+Some modules can be build only for dedicated operating systems. This is particularly the case for operating system specific tools like patchelf.
+
4.B.2.9. Forece use of autotools: pbuild::use_autotools (Pmodules 1.0)
+Synopsis
+pbuild::use_autotoolsDescription
+Force the use of autotools.
++
4.B.2.10. Force use of CMake: pbuild::use_cmake
+Synopsis
+pbuild::use_cmakeDescription
+Force the use of CMake.
+Appendix C: Links
+4.C.1. Other implementation of environment module systems
+-
+
-
+
Home page of Environment Modules
+
+ -
+
Home page of Lmod
+
+
4.C.2. Articles
+-
+
-
+
Environment Modules – A Great Tool for Clusters; Jeff Layton; Admin magazin
+
+ -
+
Lmod – Alternative Environment Modules; Jeff Layton; Admin magazin
+
+
+
4.C.3. Talks
+-
+
-
+
Lmod: A New Environment Modules System for HPC; Robert !McLay; Texas Advanced Computing Center
+
+ -
+
Pmodules: The PSI central SW distribution framework; Achim Gsell; Paul Scherrer Institute
+
+