ParFlow: Single File Outputs for CLM

by Jen Jefferson & Nick Engdahl

Keywords: Output files, CLM

Manual Location

Manual version: July, 2014 v.693

Single file outputs – Chapter 6.3

CLM variables – Page 137

On many super computers there is often a file storage limit (limiting the total file size or file count). Or, in a more practical sense, you might be seeking a way to reduce the number of files you have to deal with to analyze model output. Using the single file output option within PF-CLM you can reduce the number of .pfb output files. A “single file” contains variables output from CLM as well as the upper soil layer temperatures. If you do not use the single file output option you will see an individual file for each CLM variable for each time step. For example, if you run your model for 1 year (8.760 hours) and output data (i.e., 13 CLM variables) every hour you will end up with a total of 113,880 files (not even counting the other PF output files)! If you choose to use the single file output option you would have (only!) 8,760 files that would contain the same information as all of the individual files. Combined with some of the write suppressing options, like Solver.CLM.WriteLogs, this can be hugely beneficial for output file management.

You must add the following key to your TCL script to activate the single file output option:

pfset Solver.CLM.SingleFile True

Single files will appear with a “C” in the .pfb file name and are output for each time step (e.g., runname.out.clm_output.00001.C.pfb). Structurally, these files are formatted like any other PFB but the contents of each layer corresponds to a 2-D output parameter from CLM. Below is an example of single file output from a single column model simulation (1m x 1m x 10 m, dz = 0.1). Note: Because the model is a single column, there is only one value per row; when you have larger grids each row will contain nx by ny values.

1 1 23 nx=1, ny=1, nlayer=23

(# of variables contained in single file output)

4.805467e+01 eflx_lh_tot

3.818246e+02 eflx_lwrad_out

-6.762173e+00 eflx_sh_tot b

1.199176e+01 elfx_soil_grnd

1.914223e-05 qflx_evap_tot

1.914223e-05 qflx_evap_grnd

1.914223e-05 qflx_evap_soi

0.000000e+00 qflx_evap_veg

0.000000e+00 qflx_tran_veg

-1.270223e-05 qflx_infl

0.000000e+00 swe_out

2.870696e+02 t_grnd

0.000000e+00 qflux_qirr

2.819986e+02 Temperature of bottom soil layer; recall that the initial ground temperature in CLM is set to 300K, so if this value is 300K the model hasn’t been “heated” enough for the deeper layers to change temperature.

2.820123e+02 Temperature of soil layer

2.820283e+02 Temperature of soil layer

2.820340e+02 Temperature of soil layer

2.820463e+02 Temperature of soil layer

2.821715e+02 Temperature of soil layer

2.826579e+02 Temperature of soil layer

2.838075e+02 Temperature of soil layer

2.855920e+02 Temperature of soil layer

2.870696e+02 Temperature of top soil layer;

This temperature should match t_grnd

There are a several approaches to access or see data from the C.pfb file (pseudocode shown below):

1. Read the C.pfb files using a .pfb reader (readpfb) in R.

runname = "sfo_example"

tottime=4

data_clm=matrix(0,ncol=3,nrow=tottime)

for(k in 1:tottime){

fin1=sprintf("%s.out.clm_output.%05d.C.pfb", runname,k)

data1=readpfb(fin1,F)

data_clm[k,1]=data1[1,1,5] #qflx_evap_tot (mm/s)

data_clm[k,2]=data1[1,1,12] #t_grnd (K)

data_clm[k,3]=data1[1,1,23] #temperature of top soil layer

}

data_clm

[,1] [,2] [,3]

[1,] 1.914223e-05 287.0696 287.0696

[2,] 1.595737e-05 284.1664 284.1664

[3,] 1.116288e-05 282.7117 282.7117

[4,] 1.170877e-05 281.7334 281.7334

2. Import the C.pfb into Matlab. In the ../parflow/pftools/prepostproc folder is a Matlab.m file that is setup for .pfb files. All you need to do in order for it to work is insert the contents of the file into an m-file and replace ‘PARFLOW.out.FILE.pfb’ with the file name of your output.

3. Convert the C.pfb to a silo output file and view in VisIT.

4. Convert the C.pfb to .txt file using TCL script.

lappend auto_path $env(PARFLOW_DIR)/bin

package require parflow

namespace import Parflow::*

pfset FileVersion 4

set runname [lindex $argv 0]

puts "Convert PFB to TXT"

set Tend 4

#loop through variables and write text files

for {set j 1} {$j <=$Tend} {incr j} {

puts "File: $j"

set fin [format $runname.out.clm_output.%05d.C.pfb $j]

puts $fin

set input [pfload $fin]

set fout [format $runname.out.clm_output.%05d.C.txt $j]

pfsave $input -sa $fout

pfdelete $input

}

puts "...Done with year $runname pressure file conversions"

Text files (.txt) will be created and contents will look similar to the example data displayed above.

5. Use pfload and pfgetelt to query specific locations within the single file outputs. The location of the first entry in the single file (e.g., 23.729, from above) has a location of 0,0,0 (not 1,1,1) so you will need to subtract one from the layer number. Below is an example of a TCL script to obtain evapotranspiration data (i.e., qflx_evap_tot, which is in layer 5 (layer index of 4) in the single file output) for 4 time steps. If you want to see a different variable/layer just replace $layer for the layer of the CLM variable you want (minus one, of course).

lappend auto_path $env(PARFLOW_DIR)/bin

package require parflow

namespace import Parflow::*

pfset FileVersion 4

set runname sfo_example

# should be index - 1

set layer 4

set t_min 1

set t_max 4

set t_inc 1

# ------------------------------------------------------------

puts "----- Obtain Value from Single File Output -----"