# README file for program ICT2GFDLCSgrid
## Gus Correa <gus AT ldeo.columbia.edu> and Arlene Fiore <amfiore AT ldeo.columbia.edu>
## LDEO - Columbia University, 2021-05-26
### This README file and the commented.input.nml file fully document this program. Keep them!

1. Caveat

> This program is provided with no guarantee.
> Use it at your own risk.
> Neither the authors or their employer is liable for 
> any use or misuse of this program.

2. License

- This program is provided under the Gnu Public License Version 3:
  https://www.gnu.org/licenses/gpl-3.0.txt
- The file gpl-3.0.txt is included.

3. Purpose

The main purpose of this program is to insert along aircraft track samples
read from files in NASA ICARTT format into initial condition / restart files
in netCDF format in the GFDL Cubed Sphere (CS) grid, 
in the grid cell closest to the input location.

4. Compilation

- You need a Fortran-90 or later compiler. 
- You also need the netCDF C and Fortran 90 libraries, 
  along whith the netcdf.mod Fortran 90 module. 
  The netCDF libraries musts have been compiled 
  with the same cmpiler you will use for this program.
- Go to the *code* directory.
- The Makefile was written for the Intel ifort compiler.
  Hence, if you are using ifort,
  along with netCDF libraries compiled with ifort,
  edit the Makefile so that the INC and LIBS variables
  point to the location of those include files and libraries
  in your computer. 
- If you are using another compiler, you also need to change
  the FC variable, to point to your compiler, 
  and the FFLAGS variable, adjusting it to your compiler flags. 
- Then type "make".
  This should produce the program executable named **ICT2GFDLCSgrid**.
- Once the program executable is created, you can type
  "make clean" to remove the object (.o) and module (.mod) files 
  that are no longer needed, and keep only the source code files (.F90),
  the Makefile, and the program executable.
- You should then create a softlink from the top directory 
  to the program executable with these commands:
  cd .. ; ln -s code/ICT2GFDLCSgrid .

5. Input 

5.1. ICARTT file with the samples along the aircraft track.

This is the file that contains the variables and parcels/samples
collected by the aircract, which will be inserted in 
restart/initial condition file referred in section 4.4 below.
The file ATomSim_UCI_20160816_R1.ict, from the ATom campaing, 
is included as an example.
Please examine it and become familiar to its structure.

The first 60 lines are a long header with informative metadata,
which includes the name and units of each variable, 
missing value convention, and more.

Line 60 is an ordered list of the variables in the file 
as they apear in the file's data lines.

Lines 61 to the end of the file are data.
A few variables are in integer format 
(in this particular example file only the first three, namely: 
Start_UTC, Stop_UTC, ATom_parcel), 
while most variables are in floating point format,
and are indentified just by the presence of the decimal point.

For details about the ICARTT file format, please see the web links below:
[ICARTT file format version 1.](https://www-air.larc.nasa.gov/missions/etc/IcarttDataFormat.htm)
[ICARTT file format version 2.](https://earthdata.nasa.gov/esdis/eso/standards-and-references/icartt-file-format)

5.2. Horizontal grid files - GFDL Cubed Sphere (CS):

The GFDL CS grid is split in 6 square tiles which correspond 
to the six faces of the cubed sphere projection of the Earth's globe.
For each tile the CS grid information is codified 
in a separate file that contains information such as the two-dimensional
variables holding the latitude and longitude of each grid cell, 
grid cell area, etc.

The included directory  input/grid/c180/ contains 
the GFDL c180 grid files of the six CS tiles , named C180_grid.tile[1-6].nc,
along with additional files with other grid information.
Although these additional files are not used by this program, 
they may be help those that may want to adapt this code to other
models to understand the structure of the GFDL CS grid.

For details about the GFDL Cubed Sphere (CS) grid, please see the web links below:
[GFDL Cubed Sphere (CS) grid defined.](https://www.gfdl.noaa.gov/fv3/fv3-grids/)
[GFDL Cubed Sphere (CS) grid visualized.](https://extranet.gfdl.noaa.gov/~atw/ferret/cubed_sphere/)

Please, examine the GFDL CS horizontal grid files to understand their 
information content and structure. 
Tools such as ncdump, ncview, and Panoply may help inspect those files.

5.3. Vertical grid files:

The aircraft samples in the ICARTT input file typically contain the 
pressure level at each sampling location.
By contrast, the GFDL atmospheric model vertical grid is a hybrid-sigma grid.
Therefore, to insert the aircraft samples in the GFDL files one needs to be
able to convert from pressure level to hybrid sigma level.

The GFDL atmospheric model can store the information about the vertical grid
in different files at the user's choice.
The included directory input/restart/20130816 contains the files with the
information required to convert from pressure to hybrid-sigma and vice-versa.

First, the file fv_core.res.nc stores the two fixed coefficient sequences, 
named *ak* and *bk*, which are used to relate the hybrid-sigma level to pressure.
Second, the six CS tile files fv_core.res.tile[1-6].nc,
store the time-dependent variable named *delp*, with the vertical pressure increment 
in each grid cell, which can be used along with the *ak* and *bk* coefficient
sequences to convert to pressure.

In the current implementation of this program, the variable *delp* has a single
value in time, and since this time comes from a restart file, it is the time
that corresponds to the last time step of the model integration. 
Therefore, all conversions are done as if the all aircraft samples
were collected at the same time, and synchronously with, the model time of the *delp* variable.
This limitation is not considered severe for short-lived aicraft flights and for models
focused on climate rather than instantaneous weather response.

Please, examine the GFDL CS vertical grid files to understand their 
information content and structure. 
Tools such as ncdump, ncview, and Panoply may help inspect those files.

5.4. Files containing the variables that will be replaced by the aircraft samples

These are the files that contain the variables that will be replaced by
the aircraft samples. 
Some of these variables are exactly the same quantity that was measured
by the aircraft instrument.
Other variables can be a linear combination of a number of variables 
collected by the aircraft instrument.
The program handles both possibilities.
The aircraft samples will be placed in the grid cell closest to the aircraft
location.

In the current implementation of this program, these files are restart/initial
condition files corresponding to the last time step of the model integration.
Therefore, as explained above, all aircraft samples will be inserted synchronously
with the time of this file.

The included directory input/restart/20130816 contains these restart/initial condition
files, which are 6 CS tiled files named fv_tracer.res.tile[1-6].nc

Please, examine these restart/initial condition files to understand their 
information content and structure. 
Tools such as ncdump, ncview, and Panoply may help inspect those files.

5.5. Namelist file and input parameters:

The program accepts a number of input parameters 
via a namelist file called *input.nml*.
Fortran-90 namelists are lists of *key=value* pairs in a text file, 
one pair per line, where the "key" is the name of the 
input parameter in the program, 
and "value" is obviously the chosen value.

Namelist files start with the namelist name header, 
which is *&input_nml* for this program,
and end with a backslash */*.
Some compilers accept Fortran-90 style comments
inside namelist files, i.e., 
lines starting with an exclamation symbol *!*.
This is handy documentation, to clarify the meaning 
of input parameters, etc, 
but not all compilers have this capability.

A *commented.input.nml* file is provided with comments 
explaining in detail the meaning of each namelist parameter,
and how to use and modify them.
Please, read that file to understand its 
information content and structure.

If your compiler doesn't accept Fortran-90 comments in the
namelist file, you can generate an input.nml file,
identical to commented.input.nml, 
but with all comments removed,
with this Linux shell command:

sed '/^!/d' commented.input.nml > input.nml

6. Output:

6.1. Main output:

The main output consists of copies of the input
CS files referred in item 4.4 above, 
which contain the variables meant to be replaced by the aircraft samples,
equally named (fv_tracer.res.tile[1-6].nc in this example), 
but with the data in the grid cells closest to the ICARTT parcels replaced
by the corresponding ICARTT parcels.

6.2. Quality Control (QC) ouptut:

Three types of QC files are produced, to help visualize where the 
ICARTT parcels were located, and identify and quantify errors and location tolerance.
The file names below assume you choose the QC filename template "ATomQC".

- CS files *ATomQC.CS.tile[1-6].nc*, with the 2D variable *parcCS2Dloc*,
and the 3D variable *parcCS3Dloc*, showing the location of the inserted
ICARTT parcels in the CS 2D and 3D grids, respectively.

- A Lat-Lon-Pressure_level grid file *ATomQC.LL.nc*, with the 2D variable *parc2Dloc*,
and the 3D variable *parc3Dloc*, showing the olcation of the inserted
ICARTT parces in the Lat-Lon-Pressure_level 2D and 3D grids, respectively.

- A text file *ATomQC.LOCATION.txt*, with a list of all ICARTT parcels,
with their number, ICARTT lat-lon and pressure location,
along with their CS grid indices (i.e. tile number tt, 
horizontal grid indices ii and jj, and vertical grid index kk) 
of the grid cell where they were inserted, plus
the CS grid cell center latitude, longitude and pressure,
and also the geodetic distance in meters between 
the actual ICARTT parcel location and its location in the CS grid.

- The program also writes a substantial amount of information to STDOUT and STDERR,
which can be capured to a log file by running the program 
in Linux (sh or bash) with a command like this:

./ICT2GFDLCSgrid |& tee ICT2GFDLCSgrid.`date +"%Y%m%d-%H%M%S"`.log

7. Algorithm:

The algorithm description below is a bird's eye view.
It is not meant to follow strictly each step of the program,
but to give the overall logical description of what the program does.

After reading the namelist file, the program reads the CS grid files
and creates a direct and reverse lookup tables relating the CS
grid indices (tile index tt, horizontal grid indices ii, jj, vertical grid index kk)
and the 2D geographical coordinates (lat, lon) of the CD grid cell center.
The direct lookup table answers the question: What is the latitude and longitude of
a cell with indices (tt,ii,jj,kk)?
The reverse lookup table answers the questsion: What are the (tt,ii,jj,kk) indices of
a CS grid cell whose center has geographic coordinates (lat,lon)?

Then the program reads the ICARTT parcel file, and for each parcel it searches
for the closest (in the geodetic sense) CS grid cell center.
The net result of this search is the a set of indices (tt,ii,jj), 
but the kk vertical index is still missing.
Next it takes care of the vertical location, by converting the whole 
column above the parcel location in the CS grid from hybrid-sigma level 
to pressure level, and finding the parcel vertical location by interpolation,
or if needed, by extrapolation, and finally chosing the appropriate vertical index kk.
The final result is the CS location of the parcel via CS indices (tt,ii,jj,kk).
In addition, the geodetic distance between the ICARTT actual parcel location and its
location in the CS grid is also calculated.

The program then writes the main output, which is a copy of the input files, with the
ICARTT parcels inserted in the appropriate CS grid locations.
Besides, the program also writes the QC files.

8. Code modifications and adaption to other atmospheric models:

This program was written and meant to work with ICARTT formatted parcel files,
and the GFDL AM3/AM4/FMS CS grid, along with specific variables of both ICARTT
and the GFDL model.

However, the main structure of the program and of its source files 
can be adapted to work with other atmospheric models and 
other choices of variables, although this may require 
substantial changes to the code.

9. Example:

After you compile the code, and create a softlink to the program executable, 
as per item 2 above, you can run the example 
using the input files provided in this distribution.

First create a soft link to input.nml with this command:
ln -s commented.input.nml input.nml

Then, to run the program, and generate a log file tagged with the current date-time, 
use this command:

./ICT2GFDLCSgrid |& tee ICT2GFDLCSgrid.$(date +"%Y%m%d-%H%M%S").log