monolith
Here you find information about the system https://www.nsc.liu.se/systems/retiredsystems/monolith/.
The Monolith machine is a cluster of 2.4Ghz Xeon processors with 2GB of memory. The ScaMPI implementation of MPI has a problem, but we can use MPICH.
Add these two line to the .modules
file:
python/2.3.3
mkl/9.0p18
The Numeric Python module on the system is way too old, so we build
our own version with this customize.py
file:
use_system_lapack = 1
mkl = '/usr/local/intel/ict/l_ict_p_3.0.023/cmkl/9.0'
lapack_library_dirs = [mkl + '/lib/32']
lapack_libraries = ['mkl', 'mkl_lapack', 'g2c']
use_dotblas = 1
dotblas_include_dirs = [mkl + '/include']
dotblas_cblas_header = '<mkl_cblas.h>'
Set these environment variables in the .bashrc
file:
export PYTHONPATH=$HOME/campos-ase-2.3.4:$HOME/gpaw:$HOME/lib/python/Numeric
export GPAW_SETUP_PATH=$HOME/setups
export LD_LIBRARY_PATH=$MKL_ROOT
and build GPAW (python setup.py build_ext
) with this
customize.py
file:
extra_compile_args += ['-w1']
mpicompiler = 'icc -Nmpich'
custom_interpreter = True
compiler = 'icc'
Jobs can be submitted like this:
qsub -l nodes=2:ppn=2 -A <account> -l walltime=2:00:00 \
-m abe run.sh
where run.sh
looks like this:
cd $PBS_O_WORKDIR
mpirun $HOME/gpaw/build/bin.linux-i686-2.3/gpaw-python gpaw-script.py