presentation

---

#Software installation demo
##Modules + EasyBuild

---

---

layout: false
.left-column[
## Modules
### Intro
]
.right-column[
#### Software on your laptop
- Single user, you are king
- Typically keep only latest version
- Versions follow the OS upgrades

#### Software on a shared HPC resource
- Many users, many different software needs
- We want to keep old versions when new ones arrive for **reproducibility**
- This means we need to support:
    - Different versions of the same software
    - Same software compiled with different compilers
]

---

layout: false
.left-column[
## Modules
### Intro
]
.right-column[
#### Need for an isolated environment
- (Scientific) software should not be installed in the global PATH
- Need some way of bringing software in and out of your environment
- Many different solutions to this:
    - Lmod (modules) + EasyBuild (installation)
    - Virtualenv + pip
    - Conda
    - Containers
    - Spack
- **We** use Lmod + EasyBuild for the scientific software stack on all our machines (Fram, Saga and Betzy)
- **Users** can use **any** of the above to manage their **own** software stack on our machines
]

---

layout: false
.left-column[
## Modules
### Intro
### Demo
]
.right-column[
#### Inspect existing module
```
$ ssh user@fram.sigma2.no
$ module list
$ module avail
$ module show Arm-PerfReports/20.0.3
$ ls /cluster/software/Arm-PerfReports/20.0.3/bin
$ which perf-report
$ module load Arm-PerfReports/20.0.3
$ which perf-report
$ perf-report --version
```

#### Make your own module
```
$ mkdir lmod-demo
$ cd lmod-demo
$ echo "setenv('FOO', 'foo')" >> SetFoo.lua
$ module use $PWD
$ echo $FOO
$ module load SetFoo
$ echo $FOO
$ module unload SetFoo
$ echo $FOO
```
]

---

#### Useful commands

```
$ module list                      # List currently loaded modules
$ module avail <name-of-module>    # Search for available modules
$ module show <name-of-module>     # Inspect module file

$ module load <name-of-module>     # Load module
$ module unload <name-of-module>   # Unload module
$ module purge                     # Unload all everything

$ module use </path/to/modules>    # Add path to module files
$ module unuse </path/to/modules>  # Remove path to module files

$ module save <name-of-env>        # Save current module environment
$ module restore <name-of-env>     # Resore saved module environment
$ module savelist                  # List saved module environments
```
]

---

layout: false
.left-column[
## Modules
### Intro
### Demo
]
.right-column[
#### Recommendations
- Keep your environment tidy, purge frequently and load from scratch
- Don't mix toolchains and compiler versions
- Don't load modules in your `.bashrc` etc.
- Make use of module save/restore for interactive work
- Be explicit in job scripts: purge first, then load all necessary modules
]

---

---

layout: false
.left-column[
## Modules
### Intro
### Demo
## EasyBuild
### Intro
]
.right-column[
#### Different ways of installing software

- Package managers in Linux: yum, apt, rpm, etc.
- Conda and pip for Python
- Packed binaries
- Containers
- Compiling from source: configure-make, cmake, etc.

EasyBuild can be used as wrapper around all the above

#### EasyBuilds focus
- Targeting HPC systems
- Automation: less prone for human errors
- Reproducibility: rebuild software stack, (usually) portable to other systems
- Performance: (usually) build from source for particular hardware
- Very pedantic with versioning and dependencies (blessing and curse)

EasyBuild is tightly connected to the module system and automatically generates
module files after installation
]

---

layout: false
.left-column[
## Modules
### Intro
### Demo
## EasyBuild
### Intro
### Demo
]
.right-column[
#### Inspect EasyBuild repo
```
$ ssh user@fram.sigma2.no
$ module load EasyBuild/4.3.3
$ eb --search mrchem 
$ eb --show-ec MRChem-1.0.0-foss-2020a-Python-3.8.2.eb
```

#### Install software in home
```
$ eb --show-config
$ eb --fetch MRChem-1.0.0-foss-2020a-Python-3.8.2.eb
$ ls .local/easybuild
$ eb --dry-run MRChem-1.0.0-foss-2020a-Python-3.8.2.eb
$ eb MRChem-1.0.0-foss-2020a-Python-3.8.2.eb            # (will fail)
```

#### Install software in project
```
$ export EASYBUILD_PREFIX="/cluster/projects/nn9989k/easybuild"
$ eb --show-config
$ eb --robot --parallel=1 MRChem-1.0.0-foss-2020a-Python-3.8.2.eb
$ ls /cluster/projects/nn9989k/easybuild
```
]

---

layout: false
.left-column[
## Modules
### Intro
### Demo
## EasyBuild
### Intro
### Demo
]
.right-column[
#### Inspecting the new module
```
$ ls /cluster/projects/nn9989k/easybuild
$ module use /cluster/projects/nn9989k/easybuild/modules/all
$ module avail
$ module show MRChem/1.0.0-foss-2020a-Python-3.8.2.eb
$ module load MRChem/1.0.0-foss-2020a-Python-3.8.2.eb
$ module list
$ which mrchem
$ mrchem --version
```

#### Using the new module
```
$ cd /cluster/projects/nn9989k/eb-demo
$ cat water.inp
$ cat water.run
$ sbatch water.run
$ cat water.out
```
]

---

layout: false
.left-column[
## Modules
### Intro
### Demo
## EasyBuild
### Intro
### Demo
]
.right-column[

#### Useful commands

```
$ module avail easybuild            # List available EB versions
$ module load EasyBuild/4.3.3       # Load the EB module
                                       
$ eb --show-config                  # Show the current configuration
$ eb --search <software>            # Search for easyconfigs
$ eb --show-ec <easyconfig>         # Print content of easyconfig
                                       
$ eb <easyconfig>                   # Install software 
$ eb --robot <easyconfig>           # Install incl. dependencies
$ eb --fetch <easyconfig>           # Only download sources
$ eb --dry-run <easyconfig>         # Perform "test" installation
$ eb --prefix=<path> <easyconfig>   # Install into given path
$ eb --parallel=N <easyconfig>      # Install using N processors
```
]

---

layout: false
.left-column[
## Modules
### Intro
### Demo
## EasyBuild
### Intro
### Demo
]
.right-column[

#### Recommendations
- Beware of disk quota in `$HOME`!

- Sometimes require manual download of source

- Software not available? Try the central EB repo, it might have been
  included there recently but haven't made it into our current EB release

- Same software, different toolchain:
```
$ eb --try-toolchain=<toolchain>,<version> <name-of-config>
```

- Same software, different version:
    - Copy and modify existing easyconfig
    - Beware of cascading dependencies!
]

---

layout: false
.left-column[
## Modules
### Intro
### Demo
## EasyBuild
### Intro
### Demo
## Links
]
.right-column[

#### Sigma2 documentation
- Modules: https://documentation.sigma2.no/software/modulescheme.html
- Installation: https://documentation.sigma2.no/software/userinstallsw.html

#### Lmod
- Home: https://www.tacc.utexas.edu/research-development/tacc-projects/lmod
- Documentation: https://lmod.readthedocs.io/en/latest/
- GitHub repo: https://github.com/TACC/Lmod

#### EasyBuild
- Home: https://easybuild.io/
- Documentation: https://docs.easybuild.io/
- Tutorial: https://easybuilders.github.io/easybuild-tutorial/
- GitHub repo: https://github.com/easybuilders/
]

---

Slideshow created using [remark] and served using [cicero]

[remark]: https://github.com/gnab/remark
[cicero]: https://github.com/bast/cicero