Description

Environment Modules provide a convenient way to dynamically change the users’ environment through modulefiles. This includes easily adding or removing directories to the PATH environment variable.

List of commands

• ml av : list of all available modules
• ml ov : shorter list of all available modules
• ml list : list of all loaded modules
• ml $package ... : load module(s) given in argument $package …
• ml swap $package1 $package2 : quick exchange of version of a module installed in an environment
• ml spider $package1 : search for a list of modules that can be hidden
• ml help $package : display a help on a module
• ml whatis $package : display information about a module
• A more exhaustive list of commands can be found here.

Version

• The current version is 8.7.6
• Based on Lua

$ ml --version

Modules based on Lua: Version 8.7.6  2022-06-25 15:58 -05:00
    by Robert McLay mclay@tacc.utexas.edu

Tutorials

Useful Commands

A quick tour of modules’ command.

ml avail

• Shortcut : ml av .
• This command displays all available modules.
• You want to know all the modules of the compiler category

ml av compiler

• You want to know the modules julia

$ ml av julia

------------------------- /opt/ohpc/pub/modulefiles -------------------------------------
   compiler/julia/1.6.3    compiler/julia/1.7.2 (L)    compiler/julia/1.8.3 (D)

  Où:
   D:  Default Module
   L:  Le module est chargé

Utilisez "module spider" pour trouver tous les modules possibles.
Utilisez "module keyword key1 key2 ..." pour chercher tous les modules possibles qui correspondent à l'une des clés (key1, key2).

Another way to do it (with a pipe grep) :

$ ml av | grep julia
   compiler/julia/1.6.3                  libffi/3.4.4           (D)    paraver/4.10.4           utils/python/mamba
   compiler/julia/1.7.2           (L)    libs/tcl/8.6.13               pmix/4.2.1               valgrind/3.19.0
   compiler/julia/1.8.3           (D)    libs/tk/8.6.13                prun/2.2

Note the letters in brackets:

• (L) means that this module is loaded
• (D) the module loaded by default if no version is specified.

ml overview

• Shortcut : ml ov .
• This command displays all available modules in a shorter version.

$ ml ov julia
------------------------- /opt/ohpc/pub/modulefiles -------------------------------------
compiler/julia (3)

• The versions are not displayed.
• Nevertheless, the number of versions is displayed in brackets.
• To be compared with the command : ml av julia

ml list

• Shortcut : ml.
• Display of all loaded modules.

$ ml list

Currently Loaded Modules:
1) gnu12/12.2.0 2) hwloc/2.7.0 3) ucx/1.11.2 4) libfabric/1.13.0 5) openmpi4/4.1.4 6) compiler/julia/1.7.2

ml load

• Shortcut : ml $package .
• Loading an environment in order to have access to libraries, executables.
• Loading the software go

$ go version
-bash: go: command not found
$ ml compiler/go
$ go version
go version go1.20.1 linux/amd64

• The reverse command that unloads a package is ml unload .
• Unloading the software go

$ go version
go version go1.20.1 linux/amd64
$ ml unload compiler/go
$ go version
-bash: go: command not found

ml swap

• Quick version exchange of a module installed in an environment
• A user may want to switch from one compiler to another

$ gcc --version
gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10)
...
$ ml compiler/gnu9
$ gcc --version
gcc (GCC) 9.4.0
$ ml swap compiler/gnu9 gnu12
$ gcc --version
gcc (GCC) 12.2.0

• ml swap compiler/gnu9 gnu12 is the same as:

$ ml unload compiler/gnu9
$ ml gnu12

• Environment swapping can reactivate or deactivate environments.

$ ml list
Currently Loaded Modules:
  1) gnu12/12.2.0   2) hwloc/2.7.0   3) ucx/1.11.2   4) libfabric/1.13.0   5) openmpi4/4.1.4

$ ml swap openmpi4 mpich/3.4.3-ofi
$ ml list
Currently Loaded Modules:
  1) gnu12/12.2.0   2) libfabric/1.13.0   3) mpich/3.4.3-ofi
$ ml swap gnu12 compiler/gnu9
Modules inactifs:
  1) mpich/3.4.3-ofi

$ ml list
Currently Loaded Modules:
  1) compiler/gnu9/9.4.0   2) libfabric/1.13.0

Modules Inactifs:
  1) mpich/3.4.3-ofi

$ mpirun --version
-bash: mpirun: command not found
$ ml swap compiler/gnu9 gnu12

Chargement des modules :
  1) mpich/3.4.3-ofi

$ ml list
Currently Loaded Modules:
  1) libfabric/1.13.0   2) gnu12/12.2.0   3) mpich/3.4.3-ofi

$ mpirun --version
HYDRA build details:
    Version:                                 3.4.3
...

• This command is useful when it comes to large version or software changes.
• For example, for julia, no need to make module swap

$ ml compiler/julia/1.6.3
$ julia --version
julia version 1.6.3
$ ml compiler/julia/1.8.3

Les modules suivants ont été rechargés avec un changement de version :
  1) compiler/julia/1.6.3 => compiler/julia/1.8.3

$ julia --version
julia version 1.8.3

ml spider

• Search for a module that is not necessarily visible through the command : ml avail
• Example with python

$ ml spider python
-----------------------------------------------------------------------------------------
  compiler/python: compiler/python/3.11.0
-----------------------------------------------------------------------------------------
    Ce module peut être chargé directement : module load compiler/python/3.11.0
    Help:
      	modules - loads the modules software & application environment
      
      	This adds /opt/ohpc/pub/compiler/python/3.11.0/* to several of the
      	environment variables.
      
      	Version 3.11.0
      
-----------------------------------------------------------------------------------------
  utils/python: utils/python/mamba
-----------------------------------------------------------------------------------------

    Ce module peut être chargé directement : module load utils/python/mamba

    Help:
      	modules - loads the modules software & application environment
      
      	This adds /opt/ohpc/pub/utils/mamba/mamba/* to several of the
      	environment variables.
      
      	Version mamba
      
      	More information at :
      		https://mamba.readthedocs.io/en/latest/

• These modules are visible with the command ml av. But it is not the same with the R software.

$ ml av R

----------------------------- /opt/ohpc/pub/moduledeps/gnu12 ----------------------------
   R/4.2.1    superlu/5.2.1

----------------------------- /opt/ohpc/pub/modulefiles ---------------------------------
   charliecloud/0.15                     compiler/gnu9/9.4.0     compiler/julia/1.8.3   (L,D)    paraver/4.10.4       utils/mpfr/4.2.0
   compiler/aocc/4.0.0.deprecated        compiler/julia/1.6.3    compiler/python/3.11.0          prun/2.2             valgrind/3.19.0
   compiler/aocc/4.0.0            (D)    compiler/julia/1.7.2    libfabric/1.13.0       (L)      singularity/3.7.1

  Où:
   D:  Default Module
   L:  Le module est chargé

Utilisez "module spider" pour trouver tous les modules possibles.
Utilisez "module keyword key1 key2 ..." pour chercher tous les modules possibles qui correspondent à l'une des clés (key1, key2).

• With the ml spider command, we see that there is version 4.2.1.

$ ml spider R
-----------------------------------------------------------------------------------------
  R: R/4.2.1
-----------------------------------------------------------------------------------------
    Description:
      R is a language and environment for statistical computing and graphics (S-Plus like).

     Autres correspondances possibles :
        advisor, charliecloud, compiler, compiler-rt, compiler-rt32, compiler/aocc, compiler/gnu9, compiler/julia, compiler/python, compiler32, ...

    Vous devrez charger tous les modules de l'un des lignes suivantes avant de pouvoir charger le module "R/4.2.1".

      gnu12/12.2.0
 
    Help:
       
      This module loads the R package for statistical computing.
      Version 4.2.1  -----------------------------------------------------------------------------------------
  Pour trouver d'autres correspondances à votre recherche, exécutez :
      $ module -r spider '.*R.*'

• In addition, the ml spider R command shows us how to load this module.
• You have to load the gnu12 module

$ ml R
Lmod a détecté l'erreur suivante : Ce ou ces module(s) existent, mais ne peuvent pas être chargés tel que
demandé: "R"
   Utilisez: "module spider R" pour voir la façon de les charger.
$ ml gnu12
$ ml av R
---------------------- /opt/ohpc/pub/moduledeps/gnu12 -----------------------------------
   R/4.2.1    superlu/5.2.1
...
$ ml R
$ R --version
R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
...

• The command ml keyword also allows you to search for module
• But this command is more verbose.

$ ml keyword R
-----------------------------------------------------------------------------------------
The following modules match your search criteria: "R"
-----------------------------------------------------------------------------------------

  R: R/4.2.1
    R is a language and environment for statistical computing and graphics (S-Plus like).

  adios: adios/1.13.1
    The Adaptable IO System (ADIOS)

  advisor: advisor/latest
...

Display of information with the ml help and ml whatis commands

• The ml help command displays help about a module.
• The ml whatis command displays information about a module
• Display of the help on the gnu12 module

$ ml help gnu12
-------------- Module Specific Help for "gnu12/12.2.0" ----------------------------------
This module loads the GNU compiler collection"

See the man pages for gcc, g++, and gfortran for detailed information
on available compiler options and command-line syntax.

Version 12.2.0

• Display of information about the gnu12 module

$ ml whatis gnu12
gnu12/12.2.0        : Name: GNU Compiler Collection
gnu12/12.2.0        : Version: 12.2.0
gnu12/12.2.0        : Category: compiler, runtime support
gnu12/12.2.0        : Description: GNU Compiler Family (C/C++/Fortran for x86_64)
gnu12/12.2.0        : URL: http://gcc.gnu.org/

Loading a software

The loading of a module is done via one of these equivalent commands:

• module load $package
• ml load $package
• ml $package

For example, the loading of python3.

$ python3 --version
Python 3.6.8
$ python --version
Python 3.6.8
$ ml compiler/python/3.11.0 
$ python3 --version
Python 3.11.0
$ python --version
Python 3.6.8

• Note that the python3 command is overloaded in this example.
• However, the python command is not redefined.
• The python command is not in the binaries installed in compiler/python/3.11.0
• No module needed to load python 3.6.8
• Loading the compiler/python/3.11.0 module is optional if version 3.6.8 is sufficient
• Loading a module is often mandatory. For example, with julia

$ julia --version
-bash: julia: command not found
$ ml compiler/julia
$ julia --version
julia version 1.8.3

Search for a module

• This procedure assumes that you have an idea of the name of the module.
  • For example, openmpi
  • Neverthless, it is difficult to find a solution if you only have the name of the executable (for example mpirun)

# The mpirun command is available in various slurm modules.
# But it is difficult to find because it is hidden in a larger module.

# Here is a way to find the mpirun command assuming to use the openmpi library.

# This strategy can be seen on the MatriCS site:
# https://www.matrics.u-picardie.fr/derniere-nouvelle-de-2022-test-des-nouveaux-noeuds-de-calcul-possible/

# Trial 1

[login02 ~]$ module load openmpi
Lmod a détecté l'erreur suivante : Le ou les module(s) suivants sont
inconnus: "openmpi"

Veuillez vérifier l'orthographe ou le numéro de version. Vous pouvez aussi
essayer "module spider ..."
Il est aussi possible que votre cache soit désuète. Essayez :
  $ module --ignore_cache load "openmpi"

[login02 ~]$ module spider openmpi

----------------------------------------------------------------------------
  openmpi4:
----------------------------------------------------------------------------
    Description:
      A powerful implementation of MPI/SHMEM

     Versions:
        openmpi4/4.1.1
        openmpi4/4.1.4

----------------------------------------------------------------------------
  Pour de l'information détaillée à propos d'un module "openmpi4" spécifique (incluant comment charger ce module), utilisez le nom complet.
  Par exemple :

     $ module spider openmpi4/4.1.4
----------------------------------------------------------------------------


# Trial 2

[login02 ~]$ module load openmpi4/4.1.4
Lmod a détecté l'erreur suivante : Ce ou ces module(s) existent, mais ne
peuvent pas être chargés tel que demandé: "openmpi4/4.1.4"
   Utilisez: "module spider openmpi4/4.1.4" pour voir la façon de les charger.

[login02 ~]$ module spider openmpi4/4.1.4

----------------------------------------------------------------------------
  openmpi4: openmpi4/4.1.4
----------------------------------------------------------------------------
    Description:
      A powerful implementation of MPI/SHMEM


    Vous devrez charger tous les modules de l'un des lignes suivantes avant de pouvoir charger le module "openmpi4/4.1.4".

      gnu12/12.2.0
 
    Help:
       
      This module loads the openmpi4 library built with the gnu12 toolchain.
      
      Version 4.1.4

# Trial 3

[login02 ~]$ module load gnu12
[login02 ~]$ module load openmpi4
[login02 ~]$ mpirun --version
mpirun (Open MPI) 4.1.4

Report bugs to http://www.open-mpi.org/community/help/

• The most important thing to remember is to use the new module spider command (shortcut: ml spider $package)
• and to let yourself be guided by module which gives many indications and/or proposes solutions.
• To access the mpirun command (openmpi version), you have to do :

$ module load gnu12
$ module load openmpi4

Similarly, the mpirun command is available with the following modules:

• mvapich2
• mpich
• or impi (intel version)

Documentation

• Lmod