Installing and compiling

BUILD & RUN THOR (detailed instructions)

Main instructions to compile and run THOR. This version uses only a single GPU. Users of Anaconda Python distributions should take note of the warning at the bottom of this page.

Install

Tested on

• UBUNTU 17.04
• Debian unstable and Debian strech

0- Clone the repository

   /<myfolder>/ $ git clone https://github.com/exoclime/THOR.git

1- First, ensure that you have git, make, gcc, and g++ installed. If you would like to use cmake to build THOR instead of make, ensure that that is installed as well. On Ubuntu, these can be installed like so

   $ sudo apt-get install git make
   $ sudo apt-get install gcc g++
   $ sudo apt-get install cmake

2- Install CUDA. In Ubuntu, this can be done from the command line:

   $ sudo apt-get install nvidia-cuda-toolkit

Alternatively, you can find and download from the web: https://developer.nvidia.com/cuda-downloads

Your system's CUDA Toolkit and compiler must be able to work together. For nvidia-cuda-toolkit (version 9.x or 8.x):

• cuda 9.x use gcc > 5

• cuda 8.x use gcc = 5

• Debian unstable: Install nvidia-cuda-toolkit with the package manager.

• Debian stretch: NVidia toolkit on stretch and gcc versions are not compatible. Install NVidia Toolkit from NVidia.

Note that you may have to manually add the paths to the nvidia compiler and libraries to your environment file. See "Chapter 7: Post-Installation Actions" in the installation guide here: https://developer.download.nvidia.com/compute/cuda/10.0/Prod/docs/sidebar/CUDA_Installation_Guide_Linux.pdf

3- Install HDF5, from your package manager if possible or by hand (see below)

   $ sudo apt-get install libhdf5-dev libhdf5-10x  libhdf5-serial-dev libhdf5-cpp-10x

Replace the '10x' above with whatever version of HDF5 that is available for your version of Linux. For Ubuntu 18.04, this was '100' (1.10.0), but for Ubuntu 19.04, this was '103' (1.10.3). For the python plotting scripts, you will need h5py. You can install it with your OS package manager or pip:

   $ pip3 install h5py

4- Use git to clone this repository to a location you can find 6 months from now.

COMPILE THOR

Find your SM number

This depends on the GPU you are using. SM stands for Streaming Multiprocessor and the number indicates the features supported by the architecture. See https://developer.nvidia.com/cuda-gpus. Example: Tesla K20 -> 35. To get information on your GPU, type in the terminal:

   $ nvidia-smi

You can search online to find the "compute capability" of your device (nvidia-smi does not return this information). An easier way is to go to the tools directory of the repository and use the check_cuda.cu code:

   $ nvcc check_cuda.cu -o check_cuda
   $ ./check_cuda

which will print the SM value of your device.

(cmake will try to guess that for you, if you compile with Makefile, you need to set this).

Depending on how you installed the CUDA-toolkit, you may also have to install some additional utilities to use nvidia-smi:

   $ sudo apt-get install nvidia-utils-390

Using makefile

Define a local configuration Makefile.

Copy Makefile.conf.template to Makefile.conf. This defines a local makefile configuration that wont be added to git. You can define the SM number in there, so that you don't need to modify the main makefile (that can be overwritten when pulling from git) or add it to the command line each time.

Set your SM number in Makefile.conf

SM:=30 # Streaming Multiprocessor version

Or as argument on the command line.

Additionally, if you want to use the included physics modules (necessary for some of the cases in the ifile directory), set

MODULES_SRC := src/physics/managers/multi/

If you develop your own physics modules (see below), you will couple them by modifying this line to point to the directory containing those modules.

Compile, for release build (-j8 uses 8 parallel threads, to speed up compilation):

   $ make release -j8

for debug build:

   $ make debug -j8

TRICK

You can specify the SM number on the command line:

   $ make release -j8 SM=35

default build builds release and SM=30

   $ make -j8

To show commands echoed

   $ make VERBOSE=1

If it fails, check the makefile variables output at the beginning of the compilation, it shows the variables and the path detected for h5, which is a common cause of issue during compilation.

To get it to find the number of cores automatically:

   $ make -j $(nproc) release 

Physics modules

You can use your own physics modules by setting the path to the physics module in the local makefile configuration file Makefile.conf, see How to add your own physics modules.

Using the Álfröðull (C++ rendering of the HELIOS wavelength dependent radiative transfer) model

When the Alfrodull folder is empty, THOR will compile without the wavelength dependent RT module (based on HELIOS). A few extra steps are needed to fetch and activate this code. See Álfröðull: THOR's HELIOS Two Streams Radiative Transfer Physics Module

Alternative: Using CMake

Thor also provides a CMake script to try to configure the SM and find the libraries for CUDA and HDF5 automatically.

CMake generates a makefile with the configuration options given and looks by itself for the installed libraries. Then, you can compile the makefile.

Create a build directory and move into it. You can build a debug and release directory to build both version:

   $ mkdir build
   $ mkdir build/release
   $ cd build/release

Generate the makefile (don't forget the two points ../../ at the end of the command, it tells cmake to look into the base folder for it's configuration file CMakelist.txt). This configures a built type and the SM number:

   $ cmake -DCMAKE_BUILD_TYPE=Release -DSM=61 ../../

Use Debug to build in debug.

Compile:

   $ make -j8

To recompile, run the make command again, no need to rerun cmake as long as you don't add files or change the build process.

TRICKS

To specify the SM architecture:

   $ cmake -DSM=30 ..

For more verbosity to debug makefile by showing commands:

   $ make VERBOSE=1

INSTALL HDF5 from source

1- Install hdf5 (https://support.hdfgroup.org/HDF5/release/obtainsrc.html). Download the source code. Follow all the steps from the instructions, for example:

   $ cd <install_directory>

   $ mkdir build

   $ cd build

   $ sudo /path_to_HDF5_source/configure

   $ sudo make

   $ sudo make check

   $ sudo make install

   $ sudo apt install hdf5-helpers

2- Create a config file in "/etc/ld.so.config.d" called for example "mylib.conf" with the following line:

/path_to_hdf5_libs

Run:

   $ sudo ldconfig

Note to Anaconda (Python) users!

There is a potential conflict between THOR and the popular Python distribution, Anaconda. Above, we have suggested several ways of installing the HDF5 libraries that are used by THOR for saving model output. When THOR is compiled, the make file auto-detects the location of the necessary HDF5 files. Unfortunately, Anaconda seems to install some HDF5 components of its own, and if the Anaconda environment is loaded at time of compile, the compiler will tend to find the Anaconda versions. Also, unfortunately, it does not seem to install all the necessary components for THOR, so when the user attempts to run the model (annoyingly, it usually compiles just fine), they will see an error message like this:

bin/esp: error while loading shared libraries: libhdf5.so.103: cannot open shared object file: No such file or directory

There are a few ways around the issue:

1. Do not allow Anaconda to initialize automatically. Auto-initialization can be disabled in your shell file. Alternatively, one can disable auto initialization during install. You'll have to then load and unload the Anaconda environment on your system manually when you want to use it.
2. Use a different Python distribution. On Linux, you can install Python with apt-get, for example.
3. Manually set the correct HDF5 paths in Makefile.