This post is addressed to a new ParFlow user or to an experienced user setting up a new computer. If you don't fit in that category, this will still be useful to you but you'll have to exercise some creative license to adapt the instructions to your needs.
This post will describe a “typical” ParFlow installation, that is one that uses the most common options and configurations on macOS using the default bash shell. We’ll primarily use the command line for this so go ahead and open a terminal window (if you don’t know where terminal is use spotlight search).
Some Preliminaries
For commands entered into the terminal window I’ll use Courier font and the following convention where the command is preceded with “>>” for example:
>> make
OK, first thing’s first. We need to define a few environment variables for the installation. This may seem abstract right now but it’ll come in handy later and we’ll explain everything as it get used in the process. Navigate to your home folder and create (or edit) your bash_profile:
>> cd ~
>> vim .bash_profile
In vim, hit “i” to enter insert mode and add the following lines, preferably at the bottom:
export MPI_DIR=~/ParF/ OMPI-4.0.0
export MPI_RUN_DIR=~/ParF/OMPI-4.0.0/bin
export PARFLOW_DIR=~/ParF/parflow
export HYPRE_DIR=~/ParF/hypre
export SILO_DIR=~/ParF/silo
export CC=clang
export CXX=clang++
export FC=gfortran
export F77=gfortran
After that, this next line needs to be at the very bottom of this file:
export PATH=$MPI_RUN_DIR:$PATH
then press “escape” followed by “:” then “x” to save the changes and exit. To apply the changes:
>> source .bash_profile
So why did that do? We’ve just defined some “environment variables” and shortcuts we’ll take advantage of later on that will be loaded automatically when you start your computer or open a new terminal window. You may need to update some of these depending on how you decide to configure installation and I’ll refer to these environment variables as we come to them and point out where changes might be needed.
Next, we’ll create a directory to hold all the various bits that comprise a typical installation. From your home directory, make a folder called “ParF” and enter it:
>> cd ~
>> mkdir ParF
>> cd ParF
There are 6 “parts” to these instructions and we’re going to handle each separately: 1) install gcc/gfortran, 2) install cmake, 3) install OpenMPI, 4) Install Silo, 5) Install Hypre, 6) install ParFlow. All except step 6 are supporting libraries or programs that ParFlow needs for our typical installation.
Step 1) Installing gfortran and gcc
Truth be told we only really need gfortran but gcc is nice to have, and since they’re often bundled it makes sense to knock both out. You could try building them from source code, but the easier way is to use a binary that is pre-built for MacOS. Those can be downloaded here: http://hpc.sourceforge.net
The option you want is usually the first entry right below the word “Binaries” after the “big” paragraph. Today, for me that is “gcc-8.1-bin.tar.gz” and clicking on that link put it in my “Downloads” folder. You don’t need to bother with the “gfortran” link because it’s already included in this one. In your terminal window, we first need to enable compiling programs on your mac. This is necessary after every major MacOS upgrade before you can compile new programs (old one will be fine though).
>> sudo xcode-select --install
A confirmation window will likely open and after you OK through all of that, we’re done with that. You might also see a message saying command line tools are already installed, which is also fine.
Next, navigate to wherever you put that download and extract it to /usr/local with:
>> cd ~/Downloads
>> sudo tar -xvf gcc-8.1-bin.tar -C /
A lot of text will fly by then you’ll get a command prompt. To make sure this has copied correctly, at the command prompt type:
>> gfortran -v
And you should get something like this:
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/x86_64-apple-darwin17.5.0/8.1.0/lto-wrapper
Target: x86_64-apple-darwin17.5.0
Configured with: ../gcc-8.1.0/configure --enable-languages=c++,fortran
Thread model: posix
gcc version 8.1.0 (GCC)
A potentially really important note:
There is one last critically important step that you may need to take that was not necessary in the past. Depending on your system, as you work through the installation steps below you might encounter an error that says "error: cannot run C compiled program" which causes the installation to fail and stop. Usually this happens when using GCC instead of clang (which can happen accidentally sometimes) but fortunately there is a fix. You'll need to install the package in MacOS that has some of the libraries that are needed in order to compile programs. These are located in “/Library/Developer/CommandLineTools/Packages/” and the easiest way to get there is to open a finder window, press command+shift+g, then paste in that path. There is only one package in this folder (macOS_SDK_headers_for_macOS_10.14.pkg), though its name might change, and to install it just double click on it and follow the instructions. If you get errors, you may also need run:
>> sudo --xcode-select --s /Library/Developer/CommandLineTools
to make sure the system is pointing to the right version. These headers used to be included in macOS but they're being phased out. Further, Apple may be depreciating the package in future macOS releases, but we’ll update you if/when that happens.
OK, time to take a quick break while you restart your system right meow. Seriously, go ahead and do it; these instructions will still be here when you’re done with that, and this could save you from a lot of headaches later on.
All that being said, if you use "clang" and "clang++" with the instructions given here you probably won't see that error and can skip this little step, but it's here in case you need it. Or you can do it now anyway since installing the headers doesn't break anything, even if they aren't used.
OK, time to take a quick break while you restart your system right meow. Seriously, go ahead and do it; these instructions will still be here when you’re done with that, and this could save you from a lot of headaches later on.
All that being said, if you use "clang" and "clang++" with the instructions given here you probably won't see that error and can skip this little step, but it's here in case you need it. Or you can do it now anyway since installing the headers doesn't break anything, even if they aren't used.
Step 2) Installing cmake
Past installation instructions have used the GNU autoconfig system.
We’re going to download the latest version of the CMake source. There is also a GUI for CMake but we’re not going to use it because it's tougher to explain. Before proceeding, check here:https://cmake.org/download/ for the latest version. On the download page, go ahead and click on the link for the archive in the Unix/Linux source box and the download should complete shortly. Assuming the archive is in your ~/Downloads/ folder, we’ll copy it into ParF, the expand the archive: As of this writing that file is “cmake-3.14.0-rc1.tar.gz”
To move that download directly into the ParF folder, in your terminal window, be sure you’re in the ParF directory, then type:
>> cd ~/ParF/
>> cp ~/Downloads/cmake-3.14.0-rc1.tar .
>> tar -xvf cmake-3.14.0-rc1.tar
Note the period at the end of the second line. You will need to update the file name as newer versions of CMake are released, or if you used a different download or ParF path.
>> cd cmake-3.14.0-rc1
>> ./bootstrap
>> make
>> sudo make install
Those middle two steps will take a long time. At the end of the make process you’ll see percentage signs on the left edge of your screen and if it makes it to 100 without errors, you’re all set.
Important: Remember that sudo is the “nuclear option” on unix-esque computers and should only be used when absolutely necessary. It is here so that cmake will be available across your entire system, but this and installing gcc are the only places we need it.
Step 3) Installing OpenMPI
OpenMPI is the message passing interface we use for ParFlow (note that this is not the same as OpenMP), and this is what makes it parallel. Note that this section is a slight departure from past installation instructions. Even if you have an older version of OpenMPI, I recommend following these instructions to add a newer version to your system because there have been some changes and the newer versions tend to be more reliable. The following instructions will not overwrite or replace any existing MPI installations; to return to compiling/running with your existing OpenMPI version (if you have one), simply comment out the “export PATH” line we added to your .bash_profile, then source the file and you’re good to go. However, our goal is to install a new version, which is 4.0.0 as of this writing.
That aside, navigate back to your ParF directory and create a run folder for OpenMPI to run from (this folder should be the same as the environment variable we defined called MPI_DIR):
>> cd ~/ParF
>> mkdir OMPI-4.0.0
Next we’re going to download openmpi, build the package, then install it to our MPI_RUN directory.
The simplest way to do that is from the command line using the “curl” command, which is like “wget” for a mac. However, before doing so you might want to check to make sure you’re downloading the most recent version at: https://www.open-mpi.org/software/ompi/You’ll need to modify the version numbers if using a more recent release but otherwise it’ll be just like this:
>> curl "https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.0.tar.gz" -o "openmpi-4.0.0.tar.gz"
(Note: When copying and pasting, sometimes quotes get messed up. If you get an error from that command about libcurl, or some other strange result, just delete each quotation mark then add them back manually once you’re in the terminal window.)
Next, we’ll decompress and extract the archive in one step:
>> gunzip -c openmpi-4.0.0.tar.gz | tar xf -
Now we’ll configure the installation package for our system. Navigate into the newly created openmpi-4.0.0 folder and type:
>> ./configure --prefix=$MPI_DIR CC=/usr/bin/clang CXX=/usr/bin/clang++
Those CC and CXX arguments force the computer to use a specific compiler, which is clang. On most systems this should install fine without those flags, but it doesn’t hurt to have them. After you hit return, lots of stuff will fly by your screen for a long time. Once that’s done, type:
>> make
>> make install
Step 4) Installing Silo
Silo is a file format that is used as the default for viewing ParFlow outputs, but we also now have other options available. Many of the test cases use it so we always recommend installing it. The latest version of silo can be found at: https://wci.llnl.gov/simulation/computer-codes/silo/downloads
You could download the archive in a web browser and move it but instead I’m going to use curl again. Navigate to the ParF folder and the commands are:
>> cd ~/ParF
>> curl "https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/silo-4.10.2/silo-4.10.2.tar.gz" -o "silo-4.10.2.tar.gz"
Mimicking the process we used for OpenMPI, to expand and extract the archive type:
>> gunzip -c silo-4.10.2.tar.gz | tar xf -
Now we’ll rename the newly expanded directory to match our environment variable SILO_DIR:
>> mv silo-4.10.2 silo
Now we’ll navigate into the new directory, configure, and install:
>> cd silo
>> ./configure --disable-silex
>> make install
>> cd ..
And that completes the installation.
Note that if your version of Silo differs, you’ll need to update the appropriate file/directory names environment variable we set to reflect your installation path.
Step 5) Installing the HYPRE library
HYPRE is a library of numerical solvers for high-performance computing applications and ParFlow hooks into it for some of its functionality.
As before the simplest way to get this is curl. There were some issues with hypre version 10 and ParFlow but the current release (2.11.2) or newer should work just fine. You can check for those here:
Installing this is nothing more than a minor variation of what we’ve already done:
>> cd ~/ParF
>> curl "https://computation.llnl.gov/projects/hypre-scalable-linear-solvers-multigrid-methods/download/hypre-2.11.2.tar.gz" -o "hypre-2.11.2.tar.gz"
>> gunzip -c hypre-2.11.2.tar.gz | tar xf –
>> mv hypre-2.11.2 hypre
(That would vary if the version number changes, and your HYPRE_DIR variable would need to reflect your version/directory names). Now onto the setup.
>> cd ~/ParF/ hypre/src
>> ./configure --prefix=$HYPRE_DIR
>> make install
This installation also takes a while to install, but it’s usually not as long as OpenMPI. You’re likely to see a lot of warnings about “could not find any symbols” but as long as there are no errorsyou should be able to move on to ParFlow.
Step 6) Installing ParFlow
This step is where we depart significantly from the previous installation instructions, but in a good way. The ParFlow build process is now more streamlined because it uses CMake. Instead of separate builds for pfsimulator (the model itself) and pftools (the controlling interface), there is one unified build process. There are a lot of different setup options for ParFlow but for our “typical” setup we’re assuming you want to run in parallel, you might want to run CLM, you have Silo and Hypre, and a few other little things.
First, we need to download the code. ParFlow is hosted on GitHub (https://github.com/parflow/parflow) , but to obtain the code take this approach:
>> cd ~/ParF
>> git clone https://github.com/parflow/parflow.git --branch master --single-branch
which will create a folder called parflow in ParF and now you’ve got the source code. You may want to rename the folder (I like to use the date I downloaded it so /parflow.20190315 for example) but you’ll need to update your PARFLOW_DIR too if you change that. I’m going to proceed assuming you have not renamed it.
Next, we’ll create a directory for the incremental files of the build process called “build”
>> mkdir build
>> cd build
The next step will tell CMake how to configure ParFlow using the typical options (copy and paste will be your friend on this):
>> cmake ../parflow -DCMAKE_INSTALL_PREFIX=$PARFLOW_DIR -DHYPRE_ROOT=$HYPRE_DIR -DPARFLOW_AMPS_LAYER=mpi1 -DPARFLOW_AMPS_SEQUENTIAL_IO=true -DPARFLOW_ENABLE_TIMING=true -DSILO_ROOT=$SILO_DIR -DPARFLOW_HAVE_CLM=ON
Note that if your parflow source tree is NOT in a folder called parflow, you’ll need to change that right after “cmake” in the above command or things will go badly for you. The manual provides a few extra details on how you can customize an installation using the “ccmake” semi-GUI interface but we don’t need that now. Next type:
>> make
and lots of stuff will happen. If the last two lines resemble:
[100%] Linking C shared library libpftools.dylib
[100%] Built target pftools
Then you’re ready for the last command:
>> make install
The difference is that “make” builds everything in our temporary structure (the reason we created that build directory in the first), and “make install” copies the final build into our installation location. Think of it this way: cmake fully compiles the package in an isolated compartment, then, after that works, we copy it to the final destination.
On a few computers we’ve found that after the “make” command you might see something like this:
collect2: error: ld returned 1 exit status
make[2]: *** [pftools/libpftools.dylib] Error 1
make[1]: *** [pftools/CMakeFiles/pftools.dir/all] Error 2
make: *** [all] Error 2
If that’s the case, just back up and add “-DCMAKE_SHARED_LINKER_FLAGS=-ltcl8.5” to the end of that long list for the “cmake” command, then “make” again and you should be all sorted out.
And that’s it for the installation. The last step is to test the installation. Navigate to:
>> cd ~/ParF/parflow/test
Then type:
>> make check
A lot of things will fly by as ParFlow verifies that it is working correctly. Most should pass, but some will likely fail if you have fewer processors than are being tested; for example, some of the tests need nine processors and these will fail on a system with only 4. You’ll see a report at the end and as long as most pass, you’re all done and ParFlow has been installed successfully.
Step 7) Some troubleshooting tips - What do I do if:
- ...things aren't being found by my computer? The single most common reason steps of the installation don’t go well is not being in the correct path. You can always verify your location in the directory tree using the “pwd” command and each step tells you where you should be before you begin entering commands.
- ...things still aren't being found by the computer? Your mac is also case-sensitive so pay attention to capitalization. That also tends to be a wrench in the gears of the setup process. If you’re getting a bunch of strange behaviors, it’s a good idea to pause and restart your system before trying too many things.
- ...my system uses a different shell? If you prefer to use the c-shell instead of the bash-shell most of the steps are the same but your environment variables are defined differently. Fortunately, the setup example in the ParFlow manual outlines the c-shell variations of all the steps we covered here.
- ...my thesis is due tomorrow? Um, now is probably not the time to be re-installing ParFlow.
