Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
public:install_lofar_centos6.5 [2016-12-22 14:22] – [3. Installation the LOFAR software] Added -DCXX11=NO and changed cmake options to YES/NO Frank Breitling | public:install_lofar_centos6.5 [2017-03-08 15:27] (current) – external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Installing LOFAR Software on a CentOS 6.5 (Final) | + | ====== Installing LOFAR Software on a CentOS 6.5 (Final) |
+ | This document is evolving and was created by | ||
--- // | --- // | ||
- | I was mainly following the previous installation notes provided by others: \\ | + | A useful starting point were previous installation notes such as: \\ |
http:// | http:// | ||
http:// | http:// | ||
- | However | + | However |
Ideas and support by **Arno Schoenmakers**, | Ideas and support by **Arno Schoenmakers**, | ||
- | ===== 1. Setup of the environment ===== | + | |
+ | |||
+ | ===== 1. Setup of build environment ===== | ||
The following packages are already provided on the cluster: | The following packages are already provided on the cluster: | ||
Line 17: | Line 20: | ||
* BLAS | * BLAS | ||
* LAPACK | * LAPACK | ||
- | * Boost 1.41.0 | + | * Boost 1.41.0 |
* cfitsio 3.240 | * cfitsio 3.240 | ||
* HDF5 | * HDF5 | ||
* OpenMP | * OpenMP | ||
+ | The cluster also provides a //module system// to load newer versions of the installed software. \\ | ||
+ | Its used below to replace the default version of gcc 4.4.7 with gcc 4.8.3 for C++11 support. | ||
+ | |||
+ | |||
+ | === 1.1 Paths === | ||
+ | |||
+ | The following changes are required: | ||
+ | |||
+ | * A more recent version of GCC with C++11 required by Casacore is activated through the module system. This also requires adding / | ||
+ | < | ||
+ | File "/ | ||
+ | from ._tables import Table | ||
+ | ImportError: | ||
+ | </ | ||
+ | |||
+ | * The packages will be installed in $HOME/ | ||
+ | * Pyrap (python-casacore) needs a writable Python path. \\ | ||
- | === 1.1 Pathes === | + | This adds the following lines to $HOME/ |
- | This instruction will install all packages in $HOME/ | + | |
- | And for the installation of Pyrap (python-casacore) we need a Python path with write access. \\ | + | |
- | So I added the following | + | |
< | < | ||
+ | source / | ||
+ | module load gcc/ | ||
+ | export LD_LIBRARY_PATH=/ | ||
export PYTHONPATH=$HOME/ | export PYTHONPATH=$HOME/ | ||
- | export LD_LIBRARY_PATH=$HOME/ | ||
export PATH=$HOME/ | export PATH=$HOME/ | ||
</ | </ | ||
Line 35: | Line 54: | ||
=== 1.2 CMake === | === 1.2 CMake === | ||
+ | |||
The 4 Newton head nodes provide CMake 2.6 by default, but the LOFAR Software requires at least 2.6.6. A sufficiently recent version of CMake 3.0.0 is provided via [[https:// | The 4 Newton head nodes provide CMake 2.6 by default, but the LOFAR Software requires at least 2.6.6. A sufficiently recent version of CMake 3.0.0 is provided via [[https:// | ||
- | The environment modules also provide more recent HDF5 libraries. Both could be loaded by adding | + | The environment modules also provide more recent HDF5 libraries. Both could be loaded by adding '' |
- | '' | + | |
But unfortunately the CMake 3.0.0 loaded on the head node newl1 is broken. Furthermore loading the HDF5 libs has no effect on the libs found by CMake, which are still the system' | But unfortunately the CMake 3.0.0 loaded on the head node newl1 is broken. Furthermore loading the HDF5 libs has no effect on the libs found by CMake, which are still the system' | ||
+ | |||
< | < | ||
mkdir -p $HOME/ | mkdir -p $HOME/ | ||
Line 49: | Line 69: | ||
make install | make install | ||
</ | </ | ||
- | ===== 2. Installation | + | |
+ | |||
+ | === 1.3 Python | ||
+ | To use matplotlib 2 one also needs Python | ||
+ | |||
+ | < | ||
+ | V_PY=2.7.13 | ||
+ | cd ~/ | ||
+ | wget https:// | ||
+ | tar xf Python-$V_PY.tgz | ||
+ | cd Python-$V_PY | ||
+ | ./configure --prefix=$HOME/ | ||
+ | time make -j | ||
+ | make install | ||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | === 1.4 Boost === | ||
+ | Also a newer version | ||
+ | |||
+ | < | ||
+ | cd $HOME/ | ||
+ | V_BOOST=1_63_0 | ||
+ | wget wget https:// | ||
+ | tar xf boost_$V_BOOST.tar.bz2 | ||
+ | cd boost_$V_BOOST | ||
+ | ./ | ||
+ | time ./b2 install --prefix=${HOME}/ | ||
+ | </ | ||
+ | |||
+ | |||
+ | ===== 2. Installation of dependencies | ||
Next we need to build and install these other packages as follows: | Next we need to build and install these other packages as follows: | ||
+ | |||
+ | === 2.1 WCSLIB === | ||
+ | The [[http:// | ||
+ | |||
< | < | ||
cd $HOME/ | cd $HOME/ | ||
wget ftp:// | wget ftp:// | ||
tar xf packages/ | tar xf packages/ | ||
- | cd wcslib-5.6/ | + | cd wcslib-5.15 # (-5.6) |
./configure --prefix=$HOME/ | ./configure --prefix=$HOME/ | ||
- | make install | + | time make -j install |
+ | </ | ||
+ | === 2.2 FFTW === | ||
+ | The [[http:// | ||
+ | |||
+ | < | ||
+ | V_FFTW=3.3.5 | ||
cd $HOME/ | cd $HOME/ | ||
- | wget http:// | + | wget http:// |
- | cd fftw-3.3.4 | + | cd fftw-$V_FFTW |
- | configure --prefix=$HOME/ | + | ./configure --prefix=$HOME/ |
- | make -j12 install | + | time make -j install |
</ | </ | ||
+ | |||
===== 3. Installation the LOFAR software ===== | ===== 3. Installation the LOFAR software ===== | ||
- | Now we can continue building | + | Now we can build the LOFAR software. \\ |
+ | We will need the following CMake switches: | ||
- | === 3.1 Essential CMake switches === | + | 1. -DCMAKE_INSTALL_PREFIX: |
- | We will need these essential CMake switches: | + | 2. -DBOOST_ROOT=$HOME/ |
+ | 3. -DCMAKE_CXX_FLAGS=" | ||
+ | # 4. -DBoost_NO_BOOST_CMAKE=YES (not necessary with new installed Boost) | ||
- | 1. -DUSE_OPENMP=YES | + | 1.) Sets the installation path to $HOME/ |
- | 2. -DCMAKE_INSTALL_PREFIX: | + | |
- | 3. -DBoost_NO_BOOST_CMAKE=YES | + | |
- | 4. -DENABLE_TABLELOCKING=NO (only for Casacore) | + | |
- | 5. -DUSE_LOG4CPLUS=NO (only for the LOFAR packages) | + | |
- | 6. -DUSE_BACKTRACE=NO | + | |
- | 1.) OpenMP makes use of multi-core CPUs. We want to use this since it speeds up our processing. | + | 2.) Necessary |
- | 2.) Sets the installation | + | 3.) Necessary to link against |
- | 3.) Is necessary when using some older versions of boost with cmake-2.8.6-rc2 | + | # 4.) Was required by some older versions of boost e.g. 1.41 with cmake-2.8.6-rc2, |
- | (See https:// | + | https:// |
- | Otherwise make will show this error later: | + | |
< | < | ||
make[2]: *** No rule to make target `/ | make[2]: *** No rule to make target `/ | ||
Line 94: | Line 155: | ||
make: *** [all] Error 2 | make: *** [all] Error 2 | ||
</ | </ | ||
+ | It is not necessary any more with the new boost installation. | ||
- | 4.) On the AIP Lustre file system | + | |
+ | === 3.2 Casacore === | ||
+ | [[https:// | ||
+ | This is for example provided with [[http:// | ||
+ | Here we assume CASA is already installed and can be found in the $CASAPATH given below. For convenience this is a link to the CASA folder. | ||
+ | Then the data is in $CASAPATH/ | ||
+ | We will specify this via -DDATA_DIR, otherwise we need a ~/.casarc file with the line: '' | ||
+ | |||
+ | '' | ||
< | < | ||
msoverview: Version 20110407GvD | msoverview: Version 20110407GvD | ||
Line 103: | Line 173: | ||
/ | / | ||
</ | </ | ||
- | With table locking switched off these errors are avoided. | + | For practical work there is no disadvantage in disabling table locking. \\ |
- | This switch is only needed for Casacore! | + | |
- | + | ||
- | 5.) Is necessary for the LOFAR packages if the Log4cplus libs are not installed. | + | |
- | + | ||
- | 6.) Is necessary if building shared libraries (the default with LOFAR), but libiberty.so is not installed on your system, only libiberty.a.\\ | + | |
- | (See https:// | + | |
- | + | ||
- | So we can build the rest of the packages as follows. | + | |
- | + | ||
- | === 3.2 Casacore === | + | |
- | Casacore also requires the CASA data. This is for example provided with [[http:// | + | |
- | Here we assume CASA is already installed and $HOME/ | + | |
- | We will specify this via -DDATA_DIR, otherwise we need a ~/.casarc file with the line: '' | + | |
- | Casacore v2.2 or later uses c++11 by default which is not supported by the installed gcc 4.4.7 compiler. | + | |
- | So we need to disable c++11 with cmake -DCXX11=NO. | + | |
< | < | ||
- | cd ~local/src/ | + | # 2.1, 2.2 |
+ | cd ~/local/src/ | ||
git clone https:// | git clone https:// | ||
mkdir build/ | mkdir build/ | ||
cd build/ | cd build/ | ||
- | time cmake -DBUILD_PYTHON=YES -DCMAKE_INSTALL_PREFIX=~/ | + | CASAPATH=$HOME/ |
- | -DUSE_OPENMP=YES -DDATA_DIR=~/ | + | time cmake ../ |
- | time make -j16 | + | -DUSE_OPENMP=YES -DDATA_DIR=$CASAPATH/data -DUSE_HDF5=YES -DCMAKE_PREFIX_PATH=$HOME/ |
+ | time make -j | ||
make install | make install | ||
</ | </ | ||
Line 143: | Line 200: | ||
== 3.3.1 Alternative Pyrap installation == | == 3.3.1 Alternative Pyrap installation == | ||
- | Alternatively we can install Pyrap with //pip// if we install pip before: | + | Alternatively we can install |
< | < | ||
wget https:// | wget https:// | ||
Line 150: | Line 207: | ||
| | ||
</ | </ | ||
+ | |||
=== 3.4 Casarest === | === 3.4 Casarest === | ||
Now we continue with the rest: | Now we continue with the rest: | ||
< | < | ||
- | cd ~local/src/ | + | cd ~/local/src/ |
svn co https:// | svn co https:// | ||
mkdir build/ | mkdir build/ | ||
cd build/ | cd build/ | ||
- | cmake -DCASACORE_ROOT_DIR=$HOME/ | + | cmake ../ |
- | | + | |
- | time make -j16 | + | # not necessary any more -DBoost_NO_BOOST_CMAKE=YES |
+ | time make -j | ||
make install | make install | ||
</ | </ | ||
Line 166: | Line 225: | ||
=== 3.5 AOFlagger === | === 3.5 AOFlagger === | ||
- | Since LOFAR Offline release 2.15.0 | + | The [[https:// |
- | A download | + | [[http:// |
- | and http://www.lofar.org/ | + | For further details see the LOFAR release |
+ | [[https:// | ||
+ | The AOFlagger v2.8.0 | ||
< | < | ||
- | cd ~local/ | + | V_AOF=2.9.0 |
- | tar xf aoflagger-2.7.1.tar.bz2 | + | cd ~/local/src/ |
- | mkdir aoflagger-2.7.1/build/ | + | wget http:// |
- | cd aoflagger-2.7.1/build/ | + | tar xf aoflagger-$V_AOF.tar.bz2 |
- | cmake -DCMAKE_INSTALL_PREFIX: | + | mkdir aoflagger-$V_AOF/build/ |
- | time make -j 16 | + | cd aoflagger-$V_AOF/build/ |
+ | time cmake ../ -DCMAKE_INSTALL_PREFIX: | ||
+ | -DCMAKE_CXX_FLAGS=" | ||
+ | time make -j | ||
make install | make install | ||
</ | </ | ||
+ | |||
=== 3.6 The LOFAR packages === | === 3.6 The LOFAR packages === | ||
- | Finally we can build the LOFAR packages. We can chose between a release or the latest version in trunk/ by commenting in and out the corresponding RELEASE lines. \\ | + | Finally we can build the LOFAR packages. We can chose between |
- | An overview of available releases | + | [[http:// |
+ | or the latest version in [[https:// | ||
+ | |||
+ | '' | ||
+ | (See https://support.astron.nl/lofar_issuetracker/issues/ | ||
< | < | ||
- | cd ~local/ | + | cd ~/local/ |
- | RELEASE=LOFAR-Release-2_16 && svn checkout --ignore-externals -N https:// | + | RELEASE=LOFAR-Release-2_19 && svn checkout --ignore-externals -N https:// |
+ | # 2_16_4, 2_16, 2_12 | ||
# | # | ||
svn update $RELEASE/ | svn update $RELEASE/ | ||
mkdir -p build/ | mkdir -p build/ | ||
cd build/ | cd build/ | ||
- | time cmake -DCASACORE_ROOT_DIR=$HOME/ | + | time cmake ../ |
- | | + | # -DUSE_LOG4CPLUS=NO |
- | time make -j16 | + | time make -j |
make install | make install | ||
</ | </ |