Yes, we know…it's been quite a while since an update has been posted on how to install ParFlow on Mac OS X. In our defense, for the most part, not much has changed in the last two updates (Lion and Mountain Lion) but the latest update, OS X 10.9 Mavericks, could give you some problems, particularly if you're updating from an older OS X installation. The quick explanation is that Apple has moved a few things around behind the scenes in Mavericks and we need to make sure the compiler we're intending to use is actually the one that gets used.
This post is structured similarly to the last posted instructions with a few changes. Included this time are also notes on how to update your OpenMPI installation and a few other tips for working with a Mac that has been updated to Mavericks as opposed to one that came with Mavericks pre-installed. From here on, I'm assuming you are already running Mavericks.
The basic outline of the process is:
- Install GCC
- Install OpenMPI
- Install SILO
- Install HYPRE
- Install ParFlow
Most of these sections will also include notes about how to recompile or update if you just upgraded from an older OS X version. After all the installation instructions are a few "what ifs" we've seen when building ParFlow on various systems. Alrightythen, here we go.
0. Install GCC
If you don't already have them installed, you'll need the command line tools from Apple before you can proceed. There are two ways to get them: 1) download Xcode (which now includes them starting with OSX Mavericks), or 2) just install command line tools. Xcode is a complete development environment but it takes up 4-5Gb of space that might not be worth it if you don't plan on writing/modifying much code; just the command line tools take up slightly more than 300Mb.Whichever you decide on, both are available from the Apple developers website (https://developer.apple.com/downloads/index.action). If you're not a registered developer, it's free to register but make sure you download the latest version for your OS. After downloading and running the package installer you chose, download the GNU compiler collection (GCC) from http://hpc.sourceforge.net using this link:
gcc-4.9-bin.tar.gz
This is actually a pre-compiled version of GCC specifically for OS X Mavericks that is very easy to install compared to building your own GCC from the source code. Open a terminal window and navigate to your downloads folder (or wherever the file is saved):
> cd ~/Downloads
Your browser may have automatically unzipped the archive for you, creating a file called "gcc-4.9-bin.tar" (but if not type gunzip gcc-4.9-bin.tar.gz to unzip the archive). In your terminal window, execute the following:
> sudo tar -xvf gcc-4.9-bin.tar -C /
which installs GCC to "/usr/local/". Check the version by typing "gfortran -v" and you'll see something like this among the output lines:
gcc version 4.9.0 20140309 (experimental) (GCC)
telling us the version we just installed is actually recognized. Now type "gcc -v" and see what comes back. If you see the same thing as before, you should be fine and can move on to the next step
...but if you see something like this in the output:
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Apple's internal C-compiler "clang" is still being used instead of the GCC we just installed; this is one of those "behind the scenes" changes Apple made in Mavericks. There are a few solutions to this problem but the simplest is to change your PATH variable. Navigate to your home area and edit your .bash_profile
> cd ~
> vim .bash_profile
Depending on your setup, there might be a lot in this file or it might be empty. Key down to the bottom of the file, press "i" to enter insert mode, move to the end of this line and hit "return" twice to add some blank lines. Next add the following line to the BOTTOM of the file:
export PATH=/usr/local/bin/:$PATH
This forces your Mac to look in /usr/local/bin/ first while preserving the rest of your PATH variable and any changes made to it in the rest of your .bash_profile. Hit "esc" to exit insert mode, and type ":x" followed by "return" to save the changes and exit. Back at the command prompt, type:
> source .bash_profile
To activate the change. Verify gcc is now being used by typing "gcc -v" and you should now see:
gcc version 4.9.0 20140309 (experimental) (GCC)
Now we can move on to the next step.
1. Installing or updating OpenMPI
To start, let's assume you've never installed OpenMPI; we'll cover updating/recompiling in a minute. If you're recompiling a previously installed version of ParFlow, you probably followed the last posted instructions and placed all of your required code in a folder called "ParF". If not, go ahead and create one now. In a terminal window, type the following:
> cd ~
> mkdir ParF
> cd ParF
We're now going to download the latest version (as of this writing) of OpenMPI directly in to this folder then install it:
> curl "http://www.open-mpi.org/software/ompi/v1.8/downloads/openmpi-1.8.1.tar.gz" -o "openmpi-1.8.1.tar.gz"
> tar -xvf openmpi-1.8.1.tar.gz
> cd openmpi-1.8.1
> ./configure --prefix=/usr/local/
> sudo make all install
Doing all that can take anywhere from "a short while" to "a very long time" in order to finish but MPI should now be installed. If you want to remove the "openmpi-1.8.1.tar.bz" you can. Note: If you're not familiar with "curl" it's the OS X equivalent of "wget"
These next two sub-sections are special installation cases; if you're installing ParFlow for the first time, just skip to Step 3.
> curl "http://www.open-mpi.org/software/ompi/v1.8/downloads/openmpi-1.8.1.tar.gz" -o "openmpi-1.8.1.tar.gz"
> tar -xvf openmpi-1.8.1.tar.gz
> cd openmpi-1.8.1
> ./configure --prefix=/usr/local/
> sudo make all install
Doing all that can take anywhere from "a short while" to "a very long time" in order to finish but MPI should now be installed. If you want to remove the "openmpi-1.8.1.tar.bz" you can. Note: If you're not familiar with "curl" it's the OS X equivalent of "wget"
These next two sub-sections are special installation cases; if you're installing ParFlow for the first time, just skip to Step 3.
a. Recompiling a previous OpenMPI installation
Let's say you upgraded your Mac to Mavericks but already had ParFlow installed. It is very likely you'll need to recompile OpenMPI in order to run ParFlow but be sure to verify that the correct compiler is being used first! The installation might seem to go fine, but you'll probably see strange errors once you try to run a model if you don't rebuild.Navigate to your OpenMPI installation. My old installation is in "~/ParF/openmpi-1.6.5"
> cd ~/ParF/openmpi-1.6.5
> sudo make uninstall
> ./configure --prefix=/usr/local/
> sudo make all install
The "make uninstall" completely removes all of the files associated with an installation and the rest rebuilds everything using the compilers we set up in step 1.
b. Updating your OpenMPI version
This is just a combination of the last two items, but we're going to remove the old install completely, then install the new version. Again, be sure to verify that the correct compiler is being used first!Navigate to your OpenMPI installation. Again, my old installation is in "~/ParF/openmpi-1.6.5"
> cd ~ParF/openmpi-1.6.5
> sudo make uninstall
> cd ..
> sudo rm -rf openmpi-1.6.5
The old version of OpenMPI is now completely wiped from your system. Installing the new version is exactly the same as above:
> curl "http://www.open-mpi.org/software/ompi/v1.8/downloads/openmpi-1.8.1.tar.gz" -o "openmpi-1.8.1.tar.gz"
> tar -xvf openmpi-1.8.1.tar.gz
> ./configure --prefix=/usr/local/
> sudo make all install
Version 1.8.1 (or whatever version you picked) of OpenMPI is now installed.
3. Setup environment variables
In the terminal, navigate to your home folder and edit your ".bash_profile" for example "vim .bash_profile". We want to make sure the following lines are included in this file:export SILO_DIR=~/ParF/silo/
export HYPRE_DIR=~/ParF/hypre/
export PARFLOW_DIR=~/ParF/parflow/
Assuming those are the paths you intend to use, if not modify as necessary. Modify or add the following lines:
export CC=gcc
export CXX=g++
export FC=gfortran
export F77=gfortran
Save the file and type "source .bash_profile" to apply the changes to your active terminal.
4. Download and install SILO
Navigate to "ParF", download SILO and extract the archive with:> cd ~/ParF
> curl "https://wci.llnl.gov/codes/silo/silo-4.9.1/silo-4.9.1.tar.gz" -o "silo-4.9.1.tar.gz"
> tar -xvf silo-4.9.1.tar.gz
If you have an existing folder named silo, remove it (i.e. "rm -rf silo") or rename it (i.e. "mv silo silo_old"), then complete the setup with:
> mv silo-4.9.1 silo
> cd silo
> ./configure --disable-silex
> make install
> cd ..
If the setup was successful, near the end of the output you'll see a "NOTICE:" about the test cases not being compiled by default.
5. Download and install HYPRE
Assuming you're back in the folder "ParF", download, extract, and install HYPRE with:> curl "http://computation.llnl.gov/casc/hypre/download/hypre-2.9.0b.tar.gz" -o "hypre-2.9.0b.tar.gz"
> tar -xvf hypre-2.9.0b.tar.gz
> mv hypre-2.9.0b hypre
> cd hypre/src
> ./configure --prefix=$HYPRE_DIR
> make install
It might take a little time to install, but much less than OpenMPI took. HYPRE should now be installed.
Note: If you are changing versions of HYPRE, the method for removing the old version is very similar to what's outlined for OpenMPI. Just switch out the OpenMPI paths for the HYPRE path.
6. Install ParFlow
Note: If you are reinstalling a perviously downloaded/installed version of ParFlow, navigate to "$PARFLOW_DIR/pfsimulator" and type "make veryclean" before proceeding, or remove the "parflow" folder entirely from ~/ParFDownload the source code and move it to a folder called "parflow" within the "ParF" directory (if you already have ParFlow, navigate to "$PARFLOW_DIR/pfsimulator" and reconfigure). One way of getting ParfFow is to execute the following "curl" command, just make sure to check Reed Maxwell's download page and substitute the "r670" part of the "tar.gz" for whatever the most recent version is. To do so, execute the following:
> cd ~/ParF
> curl "http://inside.mines.edu/%7Ermaxwell/parflow.r670.tar.gz" -o "parflow.tar.gz"
> tar -xvf parflow.tar.gz
> cd parflow/pfsimulator
> ./configure --prefix=$PARFLOW_DIR --with-clm --enable-timing --with-silo=$SILO_DIR --with-hypre=$HYPRE_DIR --with-amps=mpi1 --with-amps-sequential-io
and pause for a moment. First, past users might not recognize the "sequential-io" flag. This is optional but it tells ParFlow to use a single file for distributed PFB outputs instead of 1 file for each processor. More importantly, before continuing we need to modify the Makefile to fix a few errors or else the installation will fail.
Within the "config" folder, there exists a file called "Makefile.config"; locate it and open it in the editor of your choice. The changes we need to make happen in the LDLIBS specification near the bottom of the file. There are three changes to make: 1) add a "-l" in front of "gfortran", 2) remove the stray "-l", and 3) remove the space between a "-l" and "gfortran". Each change is highlighted in red below:
BEFORE:
LDLIBS = $(LDLIBS_EXTRA) -lHYPRE -lsilo -lmpi gfortran -L/usr/local/lib/gcc/x86_64-apple-darwin13.1.0/4.9.0/../../.. -L/usr/local/lib/gcc/x86_64-apple-darwin13.1.0/4.9.0 -l -lgfortran -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lm -lm -lgfortran -lquadmath -lm -lgfortran -lquadmath -lm -l gfortran -lgfortran -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lm
AFTER:
LDLIBS = $(LDLIBS_EXTRA) -lHYPRE -lsilo -lmpi -lgfortran -L/usr/local/lib/gcc/x86_64-apple-darwin13.1.0/4.9.0/../../.. -L/usr/local/lib/gcc/x86_64-apple-darwin13.1.0/4.9.0 -lgfortran -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lm -lm -lgfortran -lquadmath -lm -lgfortran -lquadmath -lm -lgfortran -lgfortran -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lm
> make install
You'll see lots of things flash by, including lots of warnings, but at the end you should see:
Installing parflow to /Users/nengdahl/ParF/parflow
7. Install pftools
Note: If you are reinstalling a perviously installed version of pftools, navigate to "~/ParF/parflow/pftools" and type "make veryclean" before proceedingWe also need to install PFtools so from "~/ParF/parflow/pfsimulator" do the following:
> cd ../pftools
> ./configure --prefix=$PARFLOW_DIR --with-clm --enable-timing --with-silo=$SILO_DIR --with-hypre=$HYPRE_DIR --with-amps=mpi1 --with-amps-sequential-io
You need to make the exact same change to the "Makefile.config" in the "pftools/config" folder that you did for pfsimulator. After making that change, type "make install" to finish the installation.
8. Test the installation
It would be very bad to assume a seemingly error free installation is valid, so a suite of test cases are provided with ParFlow to verify its behavior. To run them all, navigate to the "test" folder:> cd $PARFLOW_DIR/test
> make check
Some of the tests will indicate failures but that has to do with specific processor topologies that don't exist on your computer. As long as most are passing, you're in good shape.
If you just want to run one test, try seeing if default_richards.tcl passes:
> tclsh default_richards.tcl 1 1 1
Hopefully that returns:
default_richards : PASSED
and all is now well in the world of ParFlow. Within "parflow/test", type "make clean" to remove the outputs of the tests and that's all there is to the installation.
9. What if TCL isn't found?
Some folks have reported that TCL isn't found sometimes during the installation of pftools. TCL is installed on your Mac by default so you just need to point the configuration utility to the correct path for TCL. To do this, add the following line to the end of the ./configure line above for pftools:--with-tcl=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/
The path on your system might be slightly different so you may have to find the path yourself. Regardless, once you know the path to TCL, add it to the ./configure and you should be fine.
10. What if I'm using different compilers?
The main change that needs to be made in order to use a different compiler is in the setup of your .bash_profile since this is where ParFlow, HYPRE, and SILO look to determine which compiler to use. So, if we had access to and wanted to use the MPI wrapped Intel compilers our .bash_profile would change to something like:export CC=mpiicc
export CXX=mpiicpc
export FC=mpif90
export F77=mpif90
Everything else should be about the same in the installation but there or may not be changes that need to be made to your LDLIBS line in the Makefile.config for pfsimulator and pftools. We've found that the configure script sometimes tries to link to a library as -lrt" (yes, it has a quotation mark hanging off the back) when using the Intel compilers and this will usually cause the installation to fail. If you see one or more flags in your LDLIBS line that includes a single quotation mark, just delete the entire flag.
