Fabian Wachsmann
Scientific programmer in the joint research project DICAD at DKRZ Developer of Climate Data Operators (CDO)
Blue beam: Explanation slides
Red beam: Contains instructions and executable code
EXAMPLE VALUES, NOT REPRESENTATIVE!
Introduction
CMIP6
Recent era of Coupled Model Inter-comparison Project
Convention about climate data accepted in CMIP
Collection of operators to process climate data
Definitions and motivation
"Systematic analysis across models only easy to do if model output is written in
Detailled requirements for coordinates, output variables and metadata can be found in the document
CMIP6_output_metadata_requirements_5Jan2018 from Taylor et. al (2018).
More information about global attributes, controlled vocabulary and filenames can be found in the document
CMIP6_global_attributes_filenames_CVs from Taylor et. al (2017)
# Get the strucutre of header of a example raw model output:
ncdump -h example_interface.nc
# results:
netcdf example_interface {
dimensions:
time = UNLIMITED ; // (12 currently)
lon = 384 ;
lat = 192 ;
height_2 = 1 ;
height = 1 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:units = "day as %Y%m%d.%f" ;
time:calendar = "proleptic_gregorian" ;
time:axis = "T" ;
float lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
float lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
double height_2(height_2) ;
height_2:standard_name = "height" ;
height_2:long_name = "height" ;
height_2:units = "m" ;
height_2:positive = "up" ;
height_2:axis = "Z" ;
double height(height) ;
height:standard_name = "height" ;
height:long_name = "height" ;
height:units = "m" ;
height:positive = "up" ;
height:axis = "Z" ;
float uas(time, height_2, lat, lon) ;
uas:units = "m s-1" ;
uas:code = 1 ;
uas:table = 255 ;
float tas(time, height, lat, lon) ;
tas:units = "K" ;
tas:code = 1 ;
tas:table = 255 ;
// global attributes:
:CDI = "Climate Data Interface version 1.9.0 (http://mpimet.mpg.de/cdi)" ;
:Conventions = "CF-1.6" ;
:history = "Tue Sep 12 10:30:24 2017: cdo_recent_cmor2 merge testuas.nc testtas.nc test.nc\n",
"Tue Sep 12 10:30:01 2017: cdo_recent_cmor2 -setunit,m s-1 -selname,uas example_interface.nc testuas.nc\n",
"Tue Sep 12 10:03:02 2017: cdo_recent_cmor2 merge test.nc test2.nc test3.nc\n",
"Tue Sep 12 10:02:45 2017: cdo_recent_cmor2 -f nc setname,tas test.grb test.nc" ;
:institution = "Max-Planck-Institute for Meteorology" ;
:CDO = "Climate Data Operators version 1.9.1rc1 (http://mpimet.mpg.de/cdo)" ;
}
# Get the strucutre of a header of a CMIP6 compliant file:
ncdump -h .//CMIP6/CMIP/MPI-M/MPI-ESM-1-2-HR/historical/r1i1p1f1/Amon/tas/gn/v20180109/tas_Amon_MPI-ESM-1-2-HR_historical_r1i1p1f1_gn_185001-185012.nc
# results:
netcdf tas_Amon_MPI-ESM-1-2-HR_historical_r1i1p1f1_gn_185001-185012 {
dimensions:
time = UNLIMITED ; // (12 currently)
lat = 192 ;
lon = 384 ;
bnds = 2 ;
variables:
double time(time) ;
time:bounds = "time_bnds" ;
time:units = "days since 1850-1-1 00:00:00" ;
time:calendar = "proleptic_gregorian" ;
time:axis = "T" ;
time:long_name = "time" ;
time:standard_name = "time" ;
double time_bnds(time, bnds) ;
double lat(lat) ;
lat:bounds = "lat_bnds" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:bounds = "lon_bnds" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
double lon_bnds(lon, bnds) ;
double height ;
height:units = "m" ;
height:axis = "Z" ;
height:positive = "up" ;
height:long_name = "height" ;
height:standard_name = "height" ;
float tas(time, lat, lon) ;
tas:standard_name = "air_temperature" ;
tas:long_name = "Near-Surface Air Temperature" ;
tas:comment = "near-surface (usually, 2 meter) air temperature" ;
tas:units = "K" ;
tas:cell_methods = "area: time: mean" ;
tas:cell_measures = "area: areacella" ;
tas:history = "2018-01-09T13:40:08Z altered by CMOR: Treated scalar dimension: \'height\'. 2018-01-09T13:40:08Z altered by CMOR: replaced missing value flag (-9e+33) with standard missing value (1e+20)." ;
tas:coordinates = "height" ;
tas:missing_value = 1.e+20f ;
tas:_FillValue = 1.e+20f ;
// global attributes:
:Conventions = "CF-1.7 CMIP-6.2" ;
:activity_id = "CMIP" ;
:branch_method = "standard" ;
:branch_time_in_child = 365. ;
:branch_time_in_parent = 365. ;
:contact = "cmip6-mpi-esm@dkrz.de" ;
:creation_date = "2018-01-09T13:40:08Z" ;
:data_specs_version = "01.00.16" ;
:experiment = "all-forcing simulation of the recent past" ;
:experiment_id = "historical" ;
:external_variables = "areacella" ;
:forcing = "GHG Oz SD Sl Vl LU" ;
:forcing_index = 1 ;
:frequency = "mon" ;
:further_info_url = "https://furtherinfo.es-doc.org/coolness" ;
:grid = "gn" ;
:grid_label = "gn" ;
:history = "2018-01-09T13:40:08Z ; CMOR rewrote data to be consistent with CMIP6, CF-1.7 CMIP-6.2 and CF standards.;\n",
"Model raw output postprocessed with modelling environment (IMDI) at DKRZ: URL: :https://svn.dkrz.de/mad/Model/IMDI/trunk" ;
:initialization_index = 1 ;
:institute_id = "MPI-M" ;
:institution = "Max Planck Institute for Meteorology, Hamburg 20146, Germany" ;
:institution_id = "MPI-M" ;
:mip_era = "CMIP6" ;
:model_id = "MPI-ESM-1-2-HR" ;
:nominal_resolution = "100 km" ;
:parent_activity_id = "CMIP" ;
:parent_experiment_id = "piControl" ;
:parent_experiment_rip = "N/A" ;
:parent_mip_era = "CMIP6" ;
:parent_source_id = "MPI-ESM-1-2-HR" ;
:parent_time_units = "days since 1850-1-1 00:00:00" ;
:parent_variant_label = "r1i1p1f1" ;
:physics_index = 1 ;
:product = "output" ;
:project_id = "CMIP6" ;
:realization_index = 1 ;
:realm = "atmos" ;
:references = "ECHAM6: Stevens, B., et al. (2013), Atmospheric component of the MPI-M Earth system model: ECHAM6, J. Adv. Model. Earth Syst., 5, 146–172, doi:10.1002/jame.20015. JSBACH: Reick, C. H., T. Raddatz, V. Brovkin, and V. Gayler (2013), The representation of natural and anthropogenic land cover change in MPIESM, J. Adv. Model. Earth Syst., 5, 1–24, doi:10.1002/jame.20022" ;
:source = "MPI-ESM1.2-HR (2017): \n",
"aerosol: none, prescribed MACv2-SP\n",
"atmos: ECHAM6.3 (spectral T127; 384 x 192 longitude/latitude; 95 levels; top level 0.01 hPa)\n",
"atmosChem: none\n",
"land: JSBACH3.20\n",
"landIce: none/prescribed\n",
"ocean: MPIOM1.63 (tripolar TP04, approximately 0.4deg; 802 x 404 longitude/latitude; 40 levels; top grid cell 0-12 m)\n",
"ocnBgchem: HAMOCC\n",
"seaIce: unnamed (thermodynamic (Semtner zero-layer) dynamic (Hibler 79) sea ice model)" ;
:source_id = "MPI-ESM-1-2-HR" ;
:source_type = "AOGCM" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "Amon" ;
:table_info = "Creation Date:(08 November 2017) MD5:0e73e044458fa79166d97d5cf546d0be" ;
:title = "MPI-ESM-1-2-HR output prepared for CMIP6" ;
:variable_id = "tas" ;
:variant_label = "r1i1p1f1" ;
:license = "CMIP6 model data produced by DKRZ is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
:cmor_version = "3.2.8" ;
:tracking_id = "hdl:21.14100/3b2ffff9-68d2-47e8-8c2c-c05711cde5da" ;
}
CMIP6 standard violations in example raw model output:
Differences between CMIP5 and CMIP6:
Filenames and directory structures templates have been modified (see next slides)
A few CMIP5 global attributes have been renamed.
A number of additional global attributes are now required.
The controlled vocabularies have been modified.
#CMIP5
institute_id="MPI-M"
institution="Max Planck Institute for Meteorology"
#member=r1i1p1
#CMIP6
institution_id="MPI-M"
institution="Max Planck Institute for Meteorology, Hamburg 20146, Germany"
variant_label=r1i1p1f1
?
No user side preparation of CMIP format description
Use the power of CDOs...
?
... to simplify CMOR usage:
?
cmor_setup();
cmor_dataset_json();
cmor_load_table();
cmor_set_table();
cmor_axis();
cmor_grid();
cmor_set_grid_mapping();
cmor_time_varying_grid_coordinate();
cmor_zfactor();
cmor_variable();
cmor_set_deflate();
cmor_set_variable_attribute();
cmor_create_output_path();
cmor_write();
cmor_close();
are included by one cdo cmor operator
cdo cmor,Amon,\
i=info_table.txt,\
gi=grid_info.nc,\
mt=mapping_table.txt\
infile
"variable_entry":{
"ccb": {
"frequency": "mon",
"modeling_realm":"atmos"
}
}
CMIP6_Amon.json
grid_
info.nc
mapping_
table.txt
ncdump -h mrsofc_fx_MPIESM-1-2-HR_historical_r1i1p1f1_gn.nc
#stdout:
netcdf test {
dimensions:
lat = 192 ;
lon = 384 ;
bnds = 2 ;
variables:
double lat(lat);
double lat_bnds(lat,bnds);
double lon(lon);
double lon_bnds(lon,bnds);
¶meter cmor_name=tasmax code=201 units="K" cell_methods="m" project_mip_table=Amon /
¶meter cmor_name=tasmax code=201 units="K" cell_methods="m" project_mip_table=day /
¶meter cmor_name=tasmin code=202 units="K" cell_methods="m" project_mip_table=day /
¶meter cmor_name=tasmin code=202 units="K" cell_methods="m" project_mip_table=Amon /
¶meter cmor_name=rsds units="W m-2" cell_methods="m" positive="d" /
¶meter cmor_name=rlds units="W m-2" cell_methods="m" positive="d" /
info_
table.txt
activity_id="CMIP"
experiment="AMIP"
experiment_id="amip"
forcing_index="1"
grid="All grid attributes are set for the native grid and based on information from attribute source."
grid_label="gn"
initialization_index="1"
institution="Max Planck Institute for Meteorology, Hamburg 20146, Germany"
institution_id="MPI-M"
license="CMIP6 model data produced by REQUIRED is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law."
mip_era="CMIP6"
nominal_resolution="100 km"
physics_index="1"
realization_index="1"
source="MPIESM1.2-HR (2017): \naerosol: none, prescribed MACv2-SP\natmos: ECHAM6.3 (spectral T127; 384 x 192 longitude/latitude; 95 levels; top level 0.01 hPa)\natmosChem: none\nland: JSBACH3.2\nlandIce: none/prescribed\nocean: MPIOM1.63 (tripolar TP04, approximately 0.4deg; 900 x 450 longitude/latitude; 40 levels; top grid cell 0-12 m)\nocnBgchem: HAMOCC\nseaIce: unnamed (thermodynamic (Semtner zero-layer) dynamic (Hibler 79) sea ice model)"
source_id="MPIESM-1-2-HR"
source_type="AGCM"
sub_experiment="none"
sub_experiment_id="none"
parent_activity_id="no parent"
parent_experiment_id="no parent"
calendar="proleptic_gregorian"
required_time_units="days since REQUIRED"
Installation
ls -1 /work/bm0021/cdo_incl_cmor/
..
cdo_2018-08-03_cmor2_gcc
cdo_2019-09-01_cmor3_gcc
cdo_recent_cmor2 -> cdo_2018-08-03_cmor2_gcc/src/cdo
cdo_recent_cmor3 -> /work/bm0021/cdo_incl_cmor/cdo_2019-09-01_cmor3_gcc/src/cdo
sudo apt-get install cdo
Installation of CDO with CMOR
CMOR2
creates standard of CMIP5 and CORDEX
CMOR3
creates standard of CMIP6 and CORDEX2
Installation of CDO with CMOR
CMOR3
creates standard of CMIP6 and CORDEX2
Installation of CDO with CMOR
CMOR2
creates standard of CMIP5 and CORDEX
Technical implementation leads to up- and downward incompatibility of CMOR input data.
The interface of cdo cmor and the format of user input does not change with the installed CMOR versions. Scripts and files used for one project can be the starting point for the next project.
CMOR3
which is able to create standard of CMIP6 and CORDEX2
Installation of CDO with CMOR
#update conda
conda update conda
#name the environment for cdo with cmor
cdoenv=cdocmor
#install develop-cdo which contains CMOR "conda-forge/label/dev::cdo"
#use conda-forge channel with "-c conda-forge"
#set $cdoenv as environment with --name
conda create --name ${cdoenv} conda-forge/label/dev::cdo -c conda-forge
#then activate it
source activate ${cdoenv}
#update cdo cmor:
conda install --name ${cdoenv} conda-forge/label/dev::cdo -c conda-forge
Using conda and install CDO with
#!/bin/sh
# INSTALL SCRIPT FOR CDO WITH SUPPORT FOR NETCDF, GRIB2, HDF5 AND CMOR on a LINUX platform
# based on the instructions by Mithil Shah (http://www.studytrails.com/blog/install-climate-data-operator-cdo-with-netcdf-grib2-and-hdf5-support/)
# Author: Fabian Wachsmann
#
#You might need to install a fortran compiler, C compiler and a macro environment:
#sudo apt-get install gfortran
#sudo apt-get install gcc
#sudo apt-get install m4
#
cd ../libs/
INSTALLDIR=$(pwd)
#Install CDO with support:
#ZLIB, HDF5, EXPAT, UDUNITS, UUID, NETCDF, JASPER, GRIB_API, CMOR
zlib=N
szlib=N
hdf5=N
expat=N
udunits=N
uuid=N
netcdf=N
jasper=N
gribapi=N
eccodes=N
unixuuid=Y
jsonc=Y
cmor=N
cdo=N
#ZLIB 1.2.8:
#for unzip and zipping:
zlib_link="http://www.zlib.net/fossils/zlib-1.2.8.tar.gz"
#szlib 0.3.4:
szlib_link="https://gitlab.dkrz.de/k202009/libaec/uploads/e3a951204308ba1f623b529cf9e5f574/libaec-0.3.4.tar.gz"
#HDF5 1.8.12
#data model used by netCDF allowing infinity file sizes
hdf5_link="https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8/hdf5-1.8.13/src/hdf5-1.8.13.tar.gz"
#EXPAT 2.2.2
#xml parser (used by cmor2)
expat_link="https://github.com/libexpat/libexpat/releases/download/R_2_2_2/expat-2.2.2.tar.bz2"
#UDUNITS 2.2.25
#changes units of variables
udunits_link="ftp://ftp.unidata.ucar.edu/pub/udunits/udunits-2.2.25.tar.gz"
#UUID 1.6.2
#creates universal unique identifier
uuid_link="https://launchpad.net/ubuntu/+archive/primary/+files/ossp-uuid_1.6.2.orig.tar.gz"
#NETCDF 4.4.1
#data format of climate data
netcdf_link="https://github.com/Unidata/netcdf-c/archive/v4.4.1.tar.gz"
#JASPER 1.9.1
#jpeg compressor
jasper_link="http://www.ece.uvic.ca/~mdadams/jasper/software/jasper-1.900.1.zip"
#GRIB_API 1.14.4
#data format of climate data
gribapi_link="https://software.ecmwf.int/wiki/download/attachments/3473437/grib_api-1.14.4-Source.tar.gz?api=v2"
#ECCODES 2.3.0
#recent version of GRIB_API
eccodes_link="https://confluence.ecmwf.int/download/attachments/45757960/eccodes-2.3.0-Source.tar.gz?api=v2"
#JSON-C 0.13.1
jsonc_link="https://github.com/json-c/json-c/archive/json-c-0.13.1-20180305.tar.gz"
if [ "$zlib" = "Y" ]; then
#ZLIB 1.2.8
#wget ${zlib_link} &&
tar xf zlib* || exit 1
cd zlib* || exit 1
./configure --prefix=${INSTALLDIR} || exit 1
make && make check && make install || exit 1
cd ..
fi
if [ "$szlib" = "Y" ]; then
#SZLIB 0.3.4
wget ${szlib_link} &&
tar -xzf libaec-0.3.4.tar.gz || exit 1
cd libaec* || exit 1
./configure --prefix=${INSTALLDIR} || exit 1
make && make check && make install || exit 1
cd ..
fi
if [ "$hdf5" = "Y" ]; then
#HDF5 1.8.12
#wget --no-check-certificate ${hdf5_link} &&
tar xf hdf5* || exit 1
cd hdf5* || exit 1
./configure --with-zlib=${INSTALLDIR} --prefix=${INSTALLDIR} --enable-hl CFLAGS=-fPIC || exit 1
make && make check || exit 1
make install || exit 1
cd ..
fi
if [ "$expat" = "Y" ]; then
#EXPAT 2.2.2
#wget --no-check-certificate ${expat_link} &&
tar xf expat* || exit 1
cd expat* || exit 1
./configure --prefix=${INSTALLDIR} CFLAGS=-fPIC || exit 1
make && make check && make install || exit 1
cd ..
fi
if [ "$udunits" = "Y" ]; then
#UDUNITS 2.2.25
#wget ${udunits_link} &&
tar xf udunits* || exit 1
cd udunits* || exit 1
CPPFLAGS=-I${INSTALLDIR}/include LDFLAGS=-L${INSTALLDIR}/lib ./configure --prefix=${INSTALLDIR} CFLAGS=-fPIC || exit 1
make && make check && make install || exit 1
cd ..
fi
if [ "$uuid" = "Y" ]; then
#UUID 1.6.2
#wget --no-check-certificate ${uuid_link} &&
tar xf ossp-uuid* || exit 1
cd uuid*
./configure --prefix=${INSTALLDIR} CFLAGS=-fPIC || exit 1
make && make check && make install || exit 1
cd ..
fi
if [ "$netcdf" = "Y" ]; then
#NETCDF 4.4.1
#wget --no-check-certificate ${netcdf_link} &&
tar xf v4* || exit 1
cd net*
CPPFLAGS=-I${INSTALLDIR}/include LDFLAGS=-L${INSTALLDIR}/lib ./configure --prefix=${INSTALLDIR} --enable-netcdf-4 CFLAGS=-fPIC || exit 1
make && make check && make install || exit 1
cd ..
fi
if [ "$jasper" = "Y" ]; then
#JASPER 1.9.1
#wget ${jasper_link} &&
unzip jasper* || exit 1
cd jasper*
./configure --prefix=${INSTALLDIR} CFLAGS=-fPIC || exit 1
make && make check && make install || exit 1
cd ..
fi
if [ "$gribapi" = "Y" ]; then
#GRIB_API 1.14.4
#wget --no-check-certificate ${gribapi_link} &&
tar xf grib* || exit 1
cd grib*
./configure --prefix=${INSTALLDIR} CFLAGS=-fPIC --with-netcdf=${INSTALLDIR} --with-jasper=${INSTALLDIR} || exit 1
make && make install || exit 1
cd ..
fi
if [ "$eccodes" = "Y" ]; then
#ECCODES 2.3.0
#wget --no-check-certificate ${eccodes_link} &&
tar -xzf eccodes-2.3.0-Source.tar.gz* || exit 1
mkdir eccodes-2.3.0-build
cd eccodes-2.3.0-build
cmake -DCMAKE_INSTALL_PREFIX=${INSTALLDIR} ../eccodes-2.3.0-Source
make
ctest
make install || exit 1
cd ..
fi
if [ "$unixuuid" = "Y" ]; then
#sudo apt-get install uuid-dev
mkdir -p ${INSTALLDIR}/libs4cmor/include/
cp /usr/include/uuid/uuid.h ${INSTALLDIR}/libs4cmor/include/ || exit 1
fi
if [ "$jsonc" = "Y" ]; then
#JSON-C 0.13.1
#wget ${jsonc_link} &&
#tar -xzf json-c-0.13.1*.tar.gz || exit 1
cd json-c-json-c*
./configure --prefix=${INSTALLDIR}/libs4cmor/ || exit 1
make && make install || exit 1
cd ..
fi
#if [ "$cmor" = "Y" ]; then
#CMOR 3.3.1.
#cd cmor3_v331/ || exit 1
#CFLAGS=-fPIC CPPFLAGS=-I${INSTALLDIR}/include LDFLAGS=-L${INSTALLDIR}/lib ./configure --prefix=${INSTALLDIR} --with-udunits2=${INSTALLDIR} --with-uuid=${INSTALLDIR} --with-netcdf=${INSTALLDIR} || exit 1
#make && make install || exit 1
#cd ${INSTALLDIR}/../cdo/
#fi
if [ "$cmor" = "Y" ]; then
#CMOR 2.9.2
cd cmor2-lib/cmor2_v292/ || exit 1
F77=gcc CC=gcc CFLAGS=-fPIC CPPFLAGS=-I${INSTALLDIR}/include LDFLAGS=-L${INSTALLDIR}/lib ./configure --prefix=${INSTALLDIR} --with-uuid=${INSTALLDIR} --with-udunits2=${INSTALLDIR} --with-netcdf=${INSTALLDIR}
make install
cd ../../
fi
if [ "$cdo" = "Y" ]; then
cd ../cdo
cmor_support/branches/cdo_incl_cmor/cdo_2019-08-15/
echo "CDO 1.9.5rc1";
#CDO 1.9.5rc1
tar xf cdo-1.9.5rc1.tar.gz || exit 1
cd cdo-1.9.5rc1|| exit 1
make distclean
CPPFLAGS="-I${INSTALLDIR}/include -I${INSTALLDIR}/local/include -I${INSTALLDIR}/local/include/json-c -I${INSTALLDIR}/local/include/cdTime" OPENMP=0 ./configure --prefix=$(pwd)/ --with-cmor=${INSTALLDIR} --with-hdf5=${INSTALLDIR} --with-netcdf=${INSTALLDIR} --with-udunits2=${INSTALLDIR} --with-grib_api=${INSTALLDIR} --with-ossp-uuid=${INSTALLDIR} || exit 1
make -j8 && make install
cd ..
fi
You can install a local cdo version on a unix platform with full support for different libraries using this script with a CDO version greater equal 1.9.4
1. Option:
Start a jupyter server, copy /work/bm0021/cdo_incl_cmor/examples/cdocmor-handson-cmip6.ipynb to your working directory and run it.
on mistral
Preparation
#connect to mistral
ssh -X <usernr>@mistral.dkrz.de
#create workdir
mkdir cdocmor_handson
cd cdocmor_handson
#copy examples directory
cdoDir=/work/bm0021/cdo_incl_cmor/
cp -r ${cdoDir}examples ./
cd examples
#link cdo cmor installation
alias cdo=${cdoDir}cdo_recent_cmor3
Preparation
#get examples directory from cloud
wget -r -H -N --cut-dirs=2 --content-disposition -I "/v1/" "https://swiftbrowser.dkrz.de/tcl_objects/2019-10-22T11:30:05Z/r_a86d66aead1ab64da2897925934f7b0aac981e74/w_/dkrz_5c667751-d212-48fd-96a1-6c02f7c7a919/examples/0/?show_all"
mv swift.dkrz.de/examples ./
cd examples
#Use the local installation for cdo:
alias cdo=${PathToLocalCdoCmor}
on mistral
local
2. Option:
#**General usage:
#cdo cmor,MIP-table[,keyvaluelist] infile
#**The keyvaluelist is of format
#key=value[,value]
#**Some keywords only allow exactly one value.
#**Others allow a comma separated list (CSL) of values.
#**If the input file is of netCDF format and contains all required meta data,
#**it can be sufficient to call:
#cdo cmor,MIP-table infile
#**example:
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json example_interface.nc
#**Parameter one MUST be the MIP table
#**It can be specified with a relative or absoulte path
Usage
#example:
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json example_interface.nc
Usage
The outfile is generated by CMOR and its name must not be specified.
Name and path of the outfile are generated according to templates defined by the corresponding project:
Filename | Filepath | |
---|---|---|
CMOR2 |
|
|
CMOR3 |
|
<variable_id>_<table_id>_
<source_id>_<experiment_id>_
<variant_label>_<grid_label>
<variable_id>_<table_id>_
<model_id>_<experiment_id>_
<member>
<activity_id>/<product>/<institute_id>/
<model_id>/<experiment_id>/<frequency>/
<realm>/<variable_id>/<member>
<mip_er>/<activity_id>/<institution_id>/<source_id>/
<experiment_id>/<variant_label>/<table_id>/
<variable_id>/<grid_label>/<version>
First steps
#example:
#select tas from infile
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json,\
cn=tas example_interface.nc
#select tas and uas from infile
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json,\
cn=tas,uas example_interface.nc
Keyword cmor_name selects a subset of variables to be processed
#example:
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json example_interface.nc
#example:
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json example_interface.nc
First steps
~/cdo-git/src/cdo cmor,Amon example_interface.nc
cdo cmor: MIP table file = 'cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json'.
cdo cmor: Attribute 'time_units' = 'days since 1850-1-1 00:00:00'
cdo cmor: Since you set attribute 'parent_experiment_id'='piControl', further attributes are checked.
cdo cmor: Function 'all axes registration':
You have not requested a particular variable via 'cmor_name'.
There are several in infile and all will be processed.
Notice that attributes specified in the cmdline will be used for all infile variables.
cdo cmor: Vertical axis is either default and scalar or not available.
cdo cmor: Vertical axis is either default and scalar or not available.
cdo cmor: Output mode: (R)eplace.
cdo cmor: Output mode: (R)eplace.
cdo cmor: File stored in: './/CMIP6/CMIP/MPI-M/MPI-ESM1-2-HR/historical/r1i1p1f1/Amon/uas/gn/v20191008/uas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_185001-185012.nc' with cmor!
cdo cmor: File stored in: './/CMIP6/CMIP/MPI-M/MPI-ESM1-2-HR/historical/r1i1p1f1/Amon/tas/gn/v20191008/tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_185001-185012.nc' with cmor!
cdo cmor: Processed 10 variables over 12 timesteps [4.65s 452MB].
#The usual CDO options are valid for this operator as well:
#cdo cmor can be called in silent (-s) or verbose (-v) mode.
#The verbose mode is recommended if the processing is unclear:
cdo -v cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json \
example_interface.nc
cdo -s cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json \
example_interface.nc
#In many cases, operator chaining can be helpful:
cdo -cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json \
-addc,10 example_interface.nc
First steps
Configuration and data streams
Infile
CDO
CMOR
Outfile
replace
append
variables (attributes)
coordinates (attributes)
global
(attributes)
mapping
table
grid info
file
info
table
MIP-
table
CV
predefinitons
user setting
data stream
cdo
cmor
mapping
table
grid info
file
info
table
MIP-
table
infile
md
data
new file
md
data
old file
md
data
command
replace
append
line
metadata stream
data stream
Project standard
variables (attributes)
coordinates (attributes)
global
(attributes)
MIP-
table
CV
predefinitons
user setting
data stream
MIP-
table
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json example_interface.nc
CMIP6_3hr.json
CMIP6_6hrLev.json
CMIP6_6hrPlev.json
CMIP6_6hrPlevPt.json
CMIP6_AERday.json
CMIP6_AERhr.json
CMIP6_AERmon.json
CMIP6_AERmonZ.json
CMIP6_Amon.json
CMIP6_CF3hr.json
CMIP6_CFday.json
CMIP6_CFmon.json
CMIP6_CFsubhr.json
CMIP6_coordinate.json
CMIP6_CV.json
CMIP6_CV_test.json
CMIP6_day.json
CMIP6_E1hrClimMon.json
CMIP6_E1hr.json
CMIP6_E3hr.json
CMIP6_E3hrPt.json
CMIP6_E6hrZ.json
CMIP6_Eday.json
CMIP6_EdayZ.json
CMIP6_Efx.json
CMIP6_Emon.json
CMIP6_EmonZ.json
CMIP6_Esubhr.json
CMIP6_Eyr.json
CMIP6_formula_terms.json
CMIP6_fx.json
CMIP6_grids.json
CMIP6_IfxAnt.json
CMIP6_IfxGre.json
CMIP6_ImonAnt.json
CMIP6_ImonGre.json
CMIP6_IyrAnt.json
CMIP6_IyrGre.json
CMIP6_LImon.json
CMIP6_Lmon.json
CMIP6_Oclim.json
CMIP6_Oday.json
CMIP6_Odec.json
CMIP6_Ofx.json
CMIP6_Omon.json
CMIP6_Oyr.json
CMIP6_SIday.json
CMIP6_SImon.json
{
"Header": {
"data_specs_version": "01.00.13",
"table_id": "Table Amon",
"realm": "atmos atmosChem",
"cmor_version": "3.2",
"table_date": "12 July 2017",
"missing_value": "1e20",
"product": "model-output",
"approx_interval": "30.00000",
"generic_levels": "alevel alevhalf",
"mip_era": "CMIP6",
"Conventions": "CF-1.7 CMIP-6.0"
},
"variable_entry": {
"ccb": {
"frequency": "mon",
"modeling_realm": "atmos",
"standard_name": "air_pressure_at_convective_cloud_base",
"units": "Pa",
"cell_methods": "area: time: mean",
"cell_measures": "area: areacella",
"long_name": "Air Pressure at Convective Cloud Base",
"comment": "Where convective cloud is present in the grid cell, the instantaneous cloud base altitude should be that of the bottom of the lowest level containing convective cloud. Missing data should be reported in the absence of convective cloud. The time mean should be calculated from these quantities averaging over occasions when convective cloud is present only, and should contain missing data for occasions when no convective cloud is present during the meaning period.",
"dimensions": "longitude latitude time",
"out_name": "ccb",
"type": "real",
"positive": "",
"valid_min": "",
"valid_max": "",
"ok_min_mean_abs": "",
"ok_max_mean_abs": ""
},
"cct": {
"frequency": "mon",
"modeling_realm": "atmos",
"standard_name": "air_pressure_at_convective_cloud_top",
"units": "Pa",
"cell_methods": "area: time: mean",
"cell_measures": "area: areacella",
"long_name": "Air Pressure at Convective Cloud Top",
"comment": "Where convective cloud is present in the grid cell, the instantaneous cloud top altitude should be that of the top of the highest level containing convective cloud. Missing data should be reported in the absence of convective cloud. The time mean should be calculated from these quantities averaging over occasions when convective cloud is present only, and should contain missing data for occasions when no convective cloud is present during the meaning period.",
"dimensions": "longitude latitude time",
"out_name": "cct",
"type": "real",
"positive": "",
"valid_min": "",
"valid_max": "",
"ok_min_mean_abs": "",
"ok_max_mean_abs": ""
},
"cfc113global": {
"frequency": "mon",
"modeling_realm": "atmos atmosChem",
"standard_name": "mole_fraction_of_cfc113_in_air",
"units": "1e-12",
"cell_methods": "area: time: mean",
"cell_measures": "",
"long_name": "Global Mean Mole Fraction of CFC113",
"comment": "",
"dimensions": "time",
"out_name": "cfc113global",
"type": "real",
"positive": "",
"valid_min": "",
"valid_max": "",
"ok_min_mean_abs": "",
"ok_max_mean_abs": ""
},
"cfc11global": {
"frequency": "mon",
"modeling_realm": "atmos atmosChem",
"standard_name": "mole_fraction_of_cfc11_in_air",
"units": "1e-12",
"cell_methods": "area: time: mean",
"cell_measures": "",
"long_name": "Global Mean Mole Fraction of CFC11",
"comment": "",
"dimensions": "time",
"out_name": "cfc11global",
"type": "real",
"positive": "",
"valid_min": "",
"valid_max": "",
"ok_min_mean_abs": "",
"ok_max_mean_abs": ""
}
}
}
Prefix | Frequency | Suffix | Qualifier |
---|---|---|---|
A, AER, CF, E, I, O, L, LI | fx, hr, 1hr, 3hr, 6hr, day, mon, yr, clim | Lev, Plev, Ant, Gre | Pt, Z, Off |
e.g.: 6hrPlevPt
Note that
MIP-
table
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json example_interface.nc
MIP-
table
{
"Header": {
"data_specs_version": "01.00.13",
The CMIP6 infrastructure components depend on each other (Taylor and Balaji). Therefore, it is important to make sure to use compatible versions of components. For cdo cmor, the workflow is:
Data request (dreq)
MIP-tables and CV
CMOR
CDO
If the dreq is updated, new MIP-tables are created. The MIP-table processing in CMOR is adapted to the most recent MIP-tables. Subsequently, CDO is adapted to CMOR.
The data request version the MIP-table is based on
MIP-
table
{
"Header": {
"data_specs_version": "01.00.13",
Data request (dreq)
MIP-tables and CV
CMOR
CDO
There are incompatabilities:
--> Which versions of CDO and CMOR one has to use depends on the dreq version the user used to configure the model simulation
The data request version the MIP-table is based on
A CMOR-variable is the unique combination of the cmor_name and the corresponding MIP-table which includes the cmor_name
MIP-
table
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json,cn=tas example_interface.nc
Exercise: Examine the MIP-tables, especially CMIP6_Amon.json
MIP-
table
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json example_interface.nc
info table and global attributes
Infile
CDO
CMOR
Outfile
replace
append
global
(attributes)
info
table
CV
CV
The Controlled Vocabulary (CV) is available as a MIP-table and defines
{
"CV":{
"required_global_attributes":[
"Conventions",
"activity_id",
"creation_date",
"data_specs_version",
"experiment",
"experiment_id",
"forcing_index",
"frequency",
"further_info_url",
"grid",
"grid_label",
"initialization_index",
"institution",
"institution_id",
"license",
"mip_era",
"nominal_resolution",
"physics_index",
"product",
"realization_index",
"realm",
"source",
"source_id",
"source_type",
"sub_experiment",
"sub_experiment_id",
"table_id",
"tracking_id",
"variable_id",
"variant_label"
],
"version_metadata":{
"author":"Paul J. Durack <durack1@llnl.gov>",
"creation_date":"Wed Jul 5 10:30:00 2017 -0700",
"institution_id":"PCMDI",
"latest_tag_point":"3.2.4 (56; gcc9356e)",
"note":"Revise source_id CNRM-CM6-1",
"previous_commit":"f042b9339fd1cb3f5ee39bb2b5ccf958812bbd21"
},
"activity_id":[
"AerChemMIP",
"C4MIP",
"CFMIP",
"CMIP",
"CORDEX",
"DAMIP",
"DCPP",
"DynVarMIP",
"FAFMIP",
"GMMIP",
"GeoMIP",
"HighResMIP",
"ISMIP6",
"LS3MIP",
"LUMIP",
"OMIP",
"PMIP",
"RFMIP",
"SIMIP",
"ScenarioMIP",
"VIACSAB",
"VolMIP"
],
"institution_id":{
"AWI":"Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research, Am Handelshafen 12, 27570 Bremerhaven, Germany",
"BNU":"Beijing Normal University, Beijing 100875, China",
"CAMS":"Chinese Academy of Meteorological Sciences, Beijing 100081, China",
"CCCR-IITM":"Centre for Climate Change Research, Indian Institute of Tropical Meteorology Pune, Maharashtra 411 008, India",
"CCCma":"Canadian Centre for Climate Modelling and Analysis, Victoria, BC V8P 5C2, Canada",
"CMCC":"Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy",
"CNRM-CERFACS":"CNRM (Centre National de Recherches Meteorologiques, Toulouse 31057, France), CERFACS (Centre Europeen de Recherche et de Formation Avancee en Calcul Scientifique, Toulouse 31057, France)",
"CSIR-CSIRO":"CSIR (Council for Scientific and Industrial Research - Natural Resources and the Environment, Pretoria, 0001, South Africa), CSIRO (Commonwealth Scientific and Industrial Research Organisation and Bureau of Meteorology, Melbourne, Victoria 3208, Australia)",
"CSIRO-BOM":"Commonwealth Scientific and Industrial Research Organisation and Bureau of Meteorology, Melbourne, Victoria 3208, Australia",
"EC-Earth-Consortium":"KNMI, The Netherlands; SMHI, Sweden; DMI, Denmark; AEMET, Spain; Met Eireann, Ireland; CNR-ISAC, Italy; Instituto de Meteorologia, Portugal; FMI, Finland; BSC, Spain; Centro de Geofisica, University of Lisbon, Portugal; ENEA, Italy; Geomar, Germany; Geophysical Institute, University of Bergen, Norway; ICHEC, Ireland; ICTP, Italy; IMAU, The Netherlands; IRV, Sweden; Lund University, Sweden; Meteorologiska Institutionen, Stockholms University, Sweden; Niels Bohr Institute, University of Copenhagen, Denmark; NTNU, Norway; SARA, The Netherlands; Unite ASTR, Belgium; Universiteit Utrecht, The Netherlands; Universiteit Wageningen, The Netherlands; University College Dublin, Ireland; Vrije Universiteit Amsterdam, the Netherlands; University of Helsinki, Finland; KIT, Karlsruhe, Germany; USC, University of Santiago de Compostela, Spain; Uppsala Universitet, Sweden; NLeSC, Netherlands eScience Center, The Netherlands",
"FIO-RONM":"FIO (First Institute of Oceanography, State Oceanic Administration, Qingdao 266061, China), RONM (Laboratory for Regional Oceanography and Numerical Modeling, Qingdao National Laboratory for Marine Science and Technology, Qingdao 266237, China)",
"INM":"Institute for Numerical Mathematics, Russian Academy of Science, Moscow 119991, Russia",
"INPE":"National Institute for Space Research, Cachoeira Paulista, SP 12630-000, Brazil",
"IPSL":"Institut Pierre Simon Laplace, Paris 75252, France",
"MESSy-Consortium":"The Modular Earth Submodel System (MESSy) Consortium, represented by the Institute for Physics of the Atmosphere, Deutsches Zentrum fur Luft- und Raumfahrt (DLR), Wessling, Bavaria 82234, Germany",
"MIROC":"JAMSTEC (Japan Agency for Marine-Earth Science and Technology, Kanagawa 236-0001, Japan), AORI (Atmosphere and Ocean Research Institute, The University of Tokyo, Chiba 277-8564, Japan), NIES (National Institute for Environmental Studies, Ibaraki 305-8506, Japan), and AICS (RIKEN Advanced Institute for Computational Science, Hyogo 650-0047, Japan)",
"MOHC":"Met Office Hadley Centre, Fitzroy Road, Exeter, Devon, EX1 3PB, UK",
"MPI-M":"Max Planck Institute for Meteorology, Hamburg 20146, Germany",
"MRI":"Meteorological Research Institute, Tsukuba, Ibaraki 305-0052, Japan",
"NASA-GISS":"Goddard Institute for Space Studies, New York, NY 10025, USA",
"NCAR":"National Center for Atmospheric Research, Boulder, CO 80301, USA",
"NCC":"NorESM Climate modeling Consortium consisting of CICERO (Center for International Climate and Environmental Research, Oslo 0349), MET-Norway (Norwegian Meteorological Institute, Oslo 0313), NERSC (Nansen Environmental and Remote Sensing Center, Bergen 5006), NILU (Norwegian Institute for Air Research, Kjeller 2027), UiB (University of Bergen, Bergen 5007), UiO (University of Oslo, Oslo 0313) and UNI (Uni Research, Bergen 5008), Norway",
"NERC":"Natural Environment Research Council, STFC-RAL, Harwell, Oxford, OX11 0QX, UK",
"NIMS-KMA":"National Institute of Meteorological Sciences/Korea Meteorological Administration, Climate Research Division, Seoho-bukro 33, Seogwipo-si, Jejudo 63568, Republic of Korea",
"NOAA-GFDL":"National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, Princeton, NJ 08540, USA",
"NUIST":"Nanjing University of Information Science and Technology, Nanjing, 210044, China",
"PCMDI":"Program for Climate Model Diagnosis and Intercomparison, Lawrence Livermore National Laboratory, Livermore, CA 94550, USA",
"THU":"Department of Earth System Science, Tsinghua University, Beijing 100084, China"
},
"source_id":{
"ACCESS-1-0":{
"activity_participation":[
"CMIP"
],
"cohort":[
"CMIP5"
],
"institution_id":[
"CSIRO-BOM"
],
"source_id":"ACCESS-1-0",
"source":"ACCESS 1.0 (2011): \naerosol: CLASSIC (v1.0)\natmos: HadGAM2 (r1.1; N96, 192 x 145 longitude/latitude; 38 levels; top level 39255 m)\natmosChem: none\nland: MOSES2.2\nlandIce: none\nocean: ACCESS-OM (MOM4p1; tripolar primarily 1deg, 360 x 300 longitude/latitude; 50 levels; top grid cell 0-10 m)\nocnBgchem: none\nseaIce: CICE4.1"
},
"AWI-CM-1-0-HR":{
"activity_participation":[
"CORDEX",
"HighResMIP",
"OMIP",
"SIMIP",
"VIACSAB"
],
"cohort":[
"Registered"
],
"institution_id":[
"AWI"
],
"source_id":"AWI-CM-1-0-HR",
"source":"AWI-CM 1.0 HR (2017): \naerosol: none\natmos: ECHAM6.3.02p4 (T127L95 native atmosphere T127 gaussian grid; 384 x 192 longitude/latitude; 95 levels; top level 80 km)\natmosChem: none\nland: JSBACH 3.10\nlandIce: none\nocean: FESOM 1.4 (unstructured grid in the horizontal with 1306775 wet nodes; 46 levels; top grid cell 0-5 m)\nocnBgchem: none\nseaIce: FESOM 1.4"
},
"AWI-CM-1-0-LR":{
"activity_participation":[
"CMIP",
"CORDEX",
"HighResMIP",
"OMIP",
"PMIP",
"SIMIP",
"ScenarioMIP",
"VIACSAB"
],
"cohort":[
"Registered"
],
"institution_id":[
"AWI"
],
"source_id":"AWI-CM-1-0-LR",
"source":"AWI-CM 1.0 LR (2017): \naerosol: none\natmos: ECHAM6.3.02p4 (T63L47 native atmosphere T63 gaussian grid; 192 x 96 longitude/latitude; 47 levels; top level 80 km)\natmosChem: none\nland: JSBACH 3.10\nlandIce: none\nocean: FESOM 1.4 (unstructured grid in the horizontal with 126859 wet nodes; 46 levels; top grid cell 0-5 m)\nocnBgchem: none\nseaIce: FESOM 1.4"
},
global
(attributes)
Info table format convention:
info_table.txt
#control keywords:
mip_table_dir="cmip6_mip_tables/"
#required global attributes:activity_id="CMIP"
experiment="AMIP"
experiment_id="amip"
forcing_index="1"
grid="All grid attributes are set for the native grid and based on information from attribute source."
grid_label="gn"
initialization_index="1"
institution="Max Planck Institute for Meteorology, Hamburg 20146, Germany"
institution_id="MPI-M"
license="CMIP6 model data produced by REQUIRED is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law."
mip_era="CMIP6"
nominal_resolution="100 km"
physics_index="1"
realization_index="1"
source="MPIESM1.2-HR (2017): \naerosol: none, prescribed MACv2-SP\natmos: ECHAM6.3 (spectral T127; 384 x 192 longitude/latitude; 95 levels; top level 0.01 hPa)\natmosChem: none\nland: JSBACH3.2\nlandIce: none/prescribed\nocean: MPIOM1.63 (tripolar TP04, approximately 0.4deg; 900 x 450 longitude/latitude; 40 levels; top grid cell 0-12 m)\nocnBgchem: HAMOCC\nseaIce: unnamed (thermodynamic (Semtner zero-layer) dynamic (Hibler 79) sea ice model)"
source_id="MPIESM-1-2-HR"
source_type="AGCM"
sub_experiment="none"
sub_experiment_id="none"
parent_activity_id="no parent"
parent_experiment_id="no parent"
calendar="proleptic_gregorian"
required_time_units="days since REQUIRED"
info
table
Info tables
global
(attributes)
info
table
Most important attributes: | project_id | experiment_id | source_id | institution_id |
---|---|---|---|---|
Example values: | CMIP6, CMIP5, CORDEX | historical, piControl, amip | MPI-ESM1-2-HR, MPI-ESM1-2-LR | MPI-M, AWI |
Connected required attributes: | experiment, member, required_time_ units |
source, calendar, grid |
It is recommended to combine required attributes which are linked by a higher level topic in one info table
E.g., by using info tables source.txt and experiment.txt, one can easily detect all required attributes which have to be changed when performing another experiment or using another source.
global
(attributes)
Attributes\experiment_id | 1pctCO2 | amip | ssp585 |
---|---|---|---|
activity_id | CMIP | CMIP | ScenarioMIP |
experiment | 1 percent per year increase in CO2 | AMIP | update of RCP8.5 based on SSP5 |
sub_experiment_id | none | none | none |
parent_activity_id | CMIP | no parent | CMIP |
parent_experiment_id | piControl | no parent | historical |
Experiments are registered in the CV with attached predefined attributes:
Attribute\project | CMIP6 | CMIP5 |
---|---|---|
MIP: | activity_id | project_id |
Model: | source_id | model_id |
Institute: | institution_id | institute_id |
Ensemble member: | variant_label | member |
Grid resolution: | nominal_resolution |
Project dependence of attribute nomenclature:
info
table
Ifile=example_interface.nc
Mtable=cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json
#since a default file ".cdocmorinfo" exist,
#it is sufficient to call
cdo cmor,${Mtable} ${Ifile}
# which is equivalent to:
cdo cmor,${Mtable},info=.cdocmorinfo ${Ifile}
global
(attributes)
The default info table is
# The operator can create the path to the MIP-table
# with the help of keywords mip_table_dir and project_id
# so that it is sufficient to call:
cdo cmor,Amon ${Ifile}
# which is equivalent to:
cdo cmor,cmip6_tables/MIP_tables/Tables/CMIP6_Amon.json ${Ifile}
We can specify "control" keywords like mip_table_dir in info tables to simplify cdo cmor calls.
info
table
global
(attributes)
A tool to create an info table is provided at
info
table
global
(attributes)
#Exercises:
#1. Create your own cdocmorinfo on
# c6dreq.dkrz.de/cdocmorinfo
#2. Copy that file to CWD/.cdocmorinfo
#3. Repeat
cdo cmor,Amon example_interface.nc
#4. Adapt the infofile so that it finally works
info
table
Infile
CDO
CMOR
Outfile
replace
append
variables (attributes)
predefinitons
user setting
data stream
variable mapping
mapping
table
Infile
CDO
CMOR
Outfile
replace
append
temp2
(x,y,z,t)
tas
(time,lat,lon)
variable mapping
Note: cdo cmor cannot do diagnostics except for 2 special cases.
Use 'cdo expr' for diagnostics.
Infile
CDO
CMOR
Outfile
replace
append
variable mapping
temp2
(x,y,z,t)
tas
(time,lat,lon)
Infile
variables (attributes)
mapping
table
Flexible configuration:
Variable attributes can be provided via
Variable attributes specifications can be combined. The priority order is:
(A command line attribute specification overwrites all prior specifications)
Renaming within the command line is only allowed for one variable per operator call. The mapping table can include mapping for all target variables.
Infile
variables (attributes)
cmor_name, name and code are infile variable selectors.
mapping
table
Keyword | Short name | Value format |
---|---|---|
cmor_name | cn | CMOR variable name |
name | n | Model variable name |
code | c | GRIB code (Integer from {0,255}) |
cmor_name
name
code
cmor_name
Infile
variables (attributes)
# ways to configure the
# infile variable selector (ivs)
# and mapping table line selector (mtls)
#
# 1. ivs=cmor_name
cdo cmor,${MIP-table},cmor_name=${cn} ${infile}
# 2. ivs=name
cdo cmor,${MIP-table},cmor_name=${cn},name=${name} ${infile}
#
# with mapping table:
# 3. mtls=cmor_name, ivs=what is in corresponding mtl
cdo cmor,${MIP-table},mapping_table=${mt},cmor_name=${cn} ${infile}
# 4. mtls=cmor_name, ivs=name (overwrites mtl name)
cdo cmor,${MIP-table},mapping_table=${mt},cmor_name=${cn},name=${name} ${infile}
# 5. mtls=name/code ivs=ALL
cdo cmor,${MIP-table},mapping_table=${mt} ${infile}
mapping
table
Keyword | Short name | Value format |
---|---|---|
cmor_name | cn | CMOR variable name |
name | n | Model variable name |
code | c | GRIB code (Integer from {0,255}) |
Infile
variables (attributes)
Keyword | Short name | Value format | Default |
---|---|---|---|
cmor_name | cn | Variable name included in MIP-table | |
name | n | Input variable name | |
code | c | Three digits integer. GRIB code. | |
units | u | String. Must be readable by udunits. | |
cell_methods | cm | Character (see below) | m |
positive | p | u=upward, d=downward | |
variable_comment | vc | String |
Five valid time cell methods can be specified. It is used to choose the correct time axis.
mapping
table
Infile
variables (attributes)
Keyword | Short name | Value format | Default |
---|---|---|---|
cmor_name | cn | Variable name included in MIP-table | |
name | n | Input variable name | |
code | c | Three digits integer. GRIB code. | |
units | u | String. Must be readable by udunits. | |
cell_methods | cm | Character | m |
positive | p | u=upward, d=downward | |
variable_comment | vc | String |
A positive flux direction must be specified if the request demands it. This is the case for fluxes, e.g. radiation variables. Otherwise, cmor aborts.
mapping
table
Infile
variables (attributes)
mapping_
table.txt
¶meter cmor_name="expcalc" project_mip_table="Emon" units="mol m-2 s-1" cell_methods="m" positive="d" z_axis="olevel" /
¶meter cmor_name="expsi" project_mip_table="Emon" units="mol m-2 s-1" cell_methods="m" positive="d" z_axis="olevel" /
¶meter cmor_name="ppdiaz" name="phosy_cya" project_mip_table="Emon" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="ppmisc" name="phosy" project_mip_table="Emon" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bsi" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bsios" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="calc" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="chl" project_mip_table="Omon" units="kg m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="chldiaz" project_mip_table="Omon" units="kg m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="chlos" project_mip_table="Omon" units="kg m-3" cell_methods="m" /
¶meter cmor_name="co3" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="co3satcalc" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="detoc" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dfe" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dfeos" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="dissic" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dissicnat" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dissoc" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dmso" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dmsos" name="surf_dms" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="dpco2" project_mip_table="Omon" units="Pa" cell_methods="m" z_axis="depth0m" /
¶meter cmor_name="dpo2" project_mip_table="Omon" units="Pa" cell_methods="m" z_axis="depth0m" /
¶meter cmor_name="epc100" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="depth100m" /
¶meter cmor_name="epcalc100" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="depth100m" /
¶meter cmor_name="epsi100" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="depth100m" /
¶meter cmor_name="expc" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" positive="d" z_axis="olevel" /
¶meter cmor_name="fbddtalk" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fbddtdic" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fbddtdife" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fbddtdin" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fbddtdip" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fbddtdisi" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fddtalk" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fddtdic" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fddtdife" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fddtdin" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fddtdip" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fddtdisi" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="olayer100m" /
¶meter cmor_name="fgco2" project_mip_table="Omon" units="kg m-2 s-1" cell_methods="m" positive="d" z_axis="depth0m" /
¶meter cmor_name="fgco2nat" project_mip_table="Omon" units="kg m-2 s-1" cell_methods="m" positive="d" z_axis="depth0m" /
¶meter cmor_name="fgdms" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" positive="u" z_axis="depth0m" /
¶meter cmor_name="fgo2" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" positive="d" z_axis="depth0m" /
¶meter cmor_name="fric" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="frn" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="froc" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="fsfe" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="depth0m" /
¶meter cmor_name="graz" project_mip_table="Omon" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="icfriver" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="depth0m" /
¶meter cmor_name="intdic" project_mip_table="Omon" units="kg m-2" cell_methods="m" /
¶meter cmor_name="intdoc" project_mip_table="Omon" units="kg m-2" cell_methods="m" /
¶meter cmor_name="intpbfe" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="intpbn" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="intpbp" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="intpbsi" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="intpcalcite" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="intpn2" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="intpoc" project_mip_table="Omon" units="kg m-2" cell_methods="m" /
¶meter cmor_name="intpp" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="intppdiaz" name="intpdiaz" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" /
¶meter cmor_name="no3" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="no3os" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="o2" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="o2min" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="o2os" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="o2sat" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="o2satos" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="ocfriver" project_mip_table="Omon" units="mol m-2 s-1" cell_methods="m" z_axis="depth0m" /
¶meter cmor_name="ph" project_mip_table="Omon" units="1.0" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyc" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phycos" project_mip_table="Omon" units="mol m-3" cell_methods="m" /
¶meter cmor_name="phydiaz" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyfe" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phymisc" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyn" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyp" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="po4" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pon" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pop" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pp" project_mip_table="Omon" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="si" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="spco2" project_mip_table="Omon" units="Pa" cell_methods="m" z_axis="depth0m" /
¶meter cmor_name="talk" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="zo2min" project_mip_table="Omon" units="m" cell_methods="m" /
¶meter cmor_name="zooc" project_mip_table="Omon" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="zsatcalc" project_mip_table="Omon" units="m" cell_methods="m" /
¶meter cmor_name="bddtalk" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bddtdic" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bddtdife" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bddtdin" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bddtdip" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bddtdisi" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="bsi" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="chl" project_mip_table="Oyr" units="kg m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="chldiaz" project_mip_table="Oyr" units="kg m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dcalc" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dfe" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dissicnat" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="dmso" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="expc" project_mip_table="Oyr" units="mol m-2 s-1" cell_methods="m" positive="d" z_axis="olevel" /
¶meter cmor_name="expcalc" project_mip_table="Oyr" units="mol m-2 s-1" cell_methods="m" positive="d" z_axis="olevel" /
¶meter cmor_name="expsi" project_mip_table="Oyr" units="mol m-2 s-1" cell_methods="m" positive="d" z_axis="olevel" /
¶meter cmor_name="fgco2" project_mip_table="Oyr" units="kg m-2 s-1" cell_methods="m" positive="d" /
¶meter cmor_name="fgco2nat" project_mip_table="Oyr" units="kg m-2 s-1" cell_methods="m" positive="d" /
¶meter cmor_name="graz" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="no3" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="o2" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="o2sat" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pbfe" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pbsi" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pcalc" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyc" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyfe" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyn" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="phyp" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pon" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pop" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="pp" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="ppdiaz" name="phosy_cya" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="ppmisc" name="phosy" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="remoc" project_mip_table="Oyr" units="mol m-3 s-1" cell_methods="m" z_axis="olevel" /
¶meter cmor_name="si" project_mip_table="Oyr" units="mol m-3" cell_methods="m" z_axis="olevel"
A Mapping table
Each line should contain
mapping
table
Keyword | Short name | Value format | Default |
---|---|---|---|
project_mip_table | pmt | Substring of MIP-table name following the underscore '_' |
infile=example_mapping.grb
#1. Commandline mapping:
cdo cmor,Amon,cn=tas,c=167,u=K,cm=m ${infile}
# Variable with code=167 from ${infile}.
# is used. Its model units is Kelvin (u=K)
# and its time cell method is mean (cm=m).
#2. Mapping table application:
# create mapping table:
echo '¶meter cn=tas c=167 u=K cm=M' >>mt.txt
# apply cdo cmor with mapping table:
cdo cmor,Amon,mapping_table=mt.txt ${infile}
# Without additional information,
# all variables from ${infile}
# are taken and converted
cdo cmor,Amon,cmor_name=tas,mt=mt.txt ${infile}
# If a cmor_name is provided, it is used
# as a mapping table line selector.
# The variable selector is taken
# from the corresponding mapping table line.
Infile
variables (attributes)
mapping
table
Exercises:
Convert the variable included in example_T_2M.nc to CMIP standard. Use mt.txt from the last slide and overwrite the infile variable selector.
Convert the variable included in example_celsius.nc to CMIP standard. Use mt.txt from the last slideand adapt the correct model variable unit.
#1.:
cdo cmor,Amon,cn=tas,n=T_2M,mt=mt.txt example_T_2M.nc
#2.:
cdo cmor,Amon,cn=tas,n=T_2M,mt=mt.txt,u=„degC“ example_celsius.nc
#Convert many variables at once with a
#complete mapping table mtPERFECT.txt:
cdo cmor,Amon,mt=mtPERFECT.txt example_collect.grb
Infile
variables (attributes)
mapping
table
A tool to create an mappping table is provided at
Infile
CDO
CMOR
Outfile
replace
append
coordinates (attributes)
coordinates and grid info file
grid info
file
coordinates (attributes)
grid info
file
"latitude": {
"standard_name": "latitude",
"units": "degrees_north",
"axis": "Y",
"long_name": "latitude",
"climatology": "",
"formula": "",
"must_have_bounds": "yes",
"out_name": "lat",
"positive": "",
"requested": "",
"requested_bounds": "",
"stored_direction": "increasing",
"tolerance": "",
"type": "double",
"valid_max": "90.0",
"valid_min": "-90.0",
"value": "",
"z_bounds_factors": "",
"z_factors": "",
"bounds_values": ""
}
"tas": {
"frequency": "mon",
"modeling_realm": "atmos",
"standard_name": "air_temperature",
"units": "K",
"cell_methods": "area: time: mean",
"cell_measures": "area: areacella",
"long_name": "Near-Surface Air Temperature",
"comment": "near-surface (usually, 2 meter) air temperature",
"dimensions": "longitude latitude time height2m",
"out_name": "tas",
"type": "real",
"positive": "",
"valid_min": "",
"valid_max": "",
"ok_min_mean_abs": "",
"ok_max_mean_abs": ""
}
coordinates (attributes)
grid info
file
bool is_lat_axis(const char *units, const char *stdname)
{
bool status = false;
char lc_units[16];
memcpy(lc_units, units, 15);
lc_units[15] = 0;
str_tolower(lc_units);
if ( (str_is_equal(lc_units, "degree") || str_is_equal(lc_units, "radian")) &&
(str_is_equal(stdname, "grid_latitude") || str_is_equal(stdname, "latitude")) )
{
status = true;
}
else if ( str_is_equal(lc_units, "degree")
&& !str_is_equal(stdname, "grid_longitude")
&& !str_is_equal(stdname, "longitude") )
{
int ioff = 6;
if ( lc_units[ioff] == 's' ) ioff++;
if ( lc_units[ioff] == '_' ) ioff++;
if ( lc_units[ioff] == 'n' || lc_units[ioff] == 's' ) status = true;
}
return status;
}
coordinates (attributes)
grid info
file
grid_
info.nc
ncdump -h example_gridinfo_CCLM4-8-17.nc
netcdf example_gridinfo_CCLM4-8-17 {
dimensions:
rlon = 424 ;
rlat = 412 ;
vertices = 4 ;
time = UNLIMITED ; // (1 currently)
bnds = 2 ;
variables:
double lon(rlat, rlon) ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
lon:_CoordinateAxisType = "Lon" ;
lon:bounds = "lon_bnds" ;
double lon_bnds(rlat, rlon, vertices) ;
double lat(rlat, rlon) ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
lat:_CoordinateAxisType = "Lat" ;
lat:bounds = "lat_bnds" ;
double lat_bnds(rlat, rlon, vertices) ;
double rlon(rlon) ;
rlon:standard_name = "grid_longitude" ;
rlon:long_name = "longitude in rotated pole grid" ;
rlon:units = "degrees" ;
rlon:axis = "X" ;
double rlat(rlat) ;
rlat:standard_name = "grid_latitude" ;
rlat:long_name = "latitude in rotated pole grid" ;
rlat:units = "degrees" ;
rlat:axis = "Y" ;
int rotated_pole ;
rotated_pole:grid_north_pole_latitude = 39.25 ;
rotated_pole:grid_north_pole_longitude = -162. ;
rotated_pole:grid_mapping_name = "rotated_latitude_longitude" ;
double height ;
height:standard_name = "height" ;
height:long_name = "height" ;
height:units = "m" ;
height:positive = "up" ;
height:axis = "Z" ;
double time(time) ;
time:standard_name = "time" ;
time:long_name = "time" ;
time:bounds = "time_bnds" ;
time:units = "days since 1949-12-01 00:00:00" ;
time:calendar = "365_day" ;
time:axis = "T" ;
double time_bnds(time, bnds) ;
float tas(time, rlat, rlon) ;
tas:standard_name = "air_temperature" ;
tas:long_name = "Near-Surface Air Temperature" ;
tas:units = "K" ;
tas:grid_mapping = "rotated_pole" ;
tas:coordinates = "height lat lon" ;
tas:_FillValue = 1.e+20f ;
tas:missing_value = 1.e+20f ;
tas:cell_methods = "time: mean" ;
}
coordinates (attributes)
grid info
file
time axis
time values
time bounds
1970-01-02
1970-01-03
1970-01-04
1970-01-05
1970-01-06
1
2
3
4
5
Absolute:
Relative:
days since 1970-01-01 00:00:00
coordinates (attributes)
grid info
file
# MIP-Table request:
"cell_methods": "time: mean"
"dimensions": "time"
# Mapping attribute:
cm=m
# MIP-Table request:
"cell_methods": "time: point"
"dimensions": "time1"
# Mapping attribute:
cm=p
# MIP-Table request:
# Climate:
"cell_methods": "time: mean within years time: mean over years"
"dimensions": "time2"
# Mapping attribute:
cm=c
# Diurnal:
"cell_methods": "time: mean within days time: mean over days"
"dimensions": "time3"
# Mapping attribute:
cm=d
time values
time bounds
time axis
coordinates (attributes)
grid info
file
# MIP-Table request:
"cell_methods": "time: mean"
"dimensions": "time"
# Mapping attribute:
cm=m
Based on the time values, time bounds are created frequency dependent. For subdaily frequencies, the time bounds are adjusted to the CMIP requested ones. The following time bounds are generated:
"2018-01-01 00:00:00"
"2018-02-01 00:00:00"
"2018-01-01 00:00:00"
"2018-01-01 06:00:00"
Time bounds are provided
time values
time bounds
time axis
time_axis=cmip ||
!taxisHasBounds(taxisID) ||
cell_methods=c ||
frequency=decadal
For monthly frequency:
For 6-hourly frequency:
If the time value is in between the interval:
"2018-01-01 00:00:00"
"2018-01-31 23:59:59"
If the time value is in between the interval:
"2018-01-01 00:00:00"
"2018-01-01 05:59:59"
coordinates (attributes)
grid info
file
# MIP-Table request:
"cell_methods": "time: point"
"dimensions": "time1"
# Mapping attribute:
cm=p
Exact time values are provided.
time values
time bounds
time axis
if ( ta == "cmip" )
In both cases, time bounds are irrelevant and not available in the output
For subdaily frequencies, the time bounds are adjusted to the CMIP requested ones
coordinates (attributes)
grid info
file
# MIP-Table request:
# Climate:
"cell_methods": "time: mean within years time: mean over years"
"dimensions": "time2"
# Mapping attribute:
cm=c
# Diurnal:
"cell_methods": "time: mean within days time: mean over days"
"dimensions": "time3"
# Mapping attribute:
cm=d
time values
time bounds
The time bounds requested for 'climate' and 'diurnal cycle' axes differ from 'mean' axis requests because of double averaging. The bounds generation is exemplified for both axis types:
"1971-01-01 00:00:00"
"2000-02-01 00:00:00"
"1971-01-01 00:00:00"
"1971-02-01 01:00:00"
time axis
coordinates (attributes)
grid info
file
# MIP-Table request:
# Climate:
"cell_methods": "time: mean within years time: mean over years"
"dimensions": "time2"
# Mapping attribute:
cm=c
# Diurnal:
"cell_methods": "time: mean within days time: mean over days"
"dimensions": "time3"
# Mapping attribute:
cm=d
time values
time bounds
The bounds variable for a 'climate' axis is named "climatology_bnds" in outfile. For both axis types, these bounds can be provided as time_bnds variable similar to the procedure for 'mean' types. If not applicable, you can do this:
time axis
coordinates (attributes)
grid info
file
time values
time bounds
time axis
Name | Short name | Format | Default |
---|---|---|---|
required_ time_units |
rtu | ’<Frequency> since <Year>-<Month>-<Day> <Hours>:<Minutes>:<Seconds>’, example: ’days since 1979-1-1 00:00:00’. |
No default |
calendar | Value is one of ’standard’ (’gre- gorian’), ’proleptic gregorian’, ’360 day’, ’noleap’ and ’all leap’. The calendar depends on the model configuration |
A calendar is usually available in infile |
|
climatology _interval |
Two comma separated years as integers. E.g.: 2001,2010 |
No default | |
t_axis | ’cmip’ for creating time values and bounds as requested by CMIP. | 'n' |
coordinates (attributes)
grid info
file
scalar axes
character axes
# Get the strucutre of header of a example raw model output:
ncdump -h example_interface.nc
# results:
netcdf example_interface {
dimensions:
time = UNLIMITED ; // (12 currently)
lon = 384 ;
lat = 192 ;
height = 1 ;
A dimension with size 1 is named 'scalar'. The CMIP standard provides that this type should not occur as a dimension in the header. Only a one valued coordinate variable (without a dimension) associated with attribute 'coordinates' of the requested variable is used to represent this axis:
variables:
double height ;
height:units = "m" ;
height:axis = "Z" ;
height:positive = "up" ;
height:long_name = "height" ;
height:standard_name = "height" ;
float tas(time, lat, lon) ;
tas:coordinates = "height" ;
coordinates (attributes)
grid info
file
scalar axes
character axes
"height2m": {
"valid_max": "10.0",
"valid_min": "1.0",
"value": "2.",
If the model level deviates from the requested level,
mapping table
¶meter cn=tas c=201 u=K cm=m pmt=Amon z_axis=height2m /
coordinates (attributes)
grid info
file
scalar axes
character axes
"basin": {
"standard_name": "region",
"requested": [
"atlantic_arctic_ocean",
"indian_pacific_ocean",
"global_ocean"
],
"type": "character",
mapping table
¶meter cn=hfbasinpadv c=004 pmt=Omon character_axis=basin /
"hfbasinpadv": {
"dimensions": "latitude basin time",
If only an index dimension is available in infile, the procedure similar to the scalar axes method can be an option:
coordinates (attributes)
grid info
file
vertical parametric axis
Right now, only this ’alternate hybrid sigma’ axis is enabled and test data for other axes built by a formula is required to guarantee a correct processing
ps must also be available in infile or in a substitution grid info netCDF file. The operator identifies the ps via variable name ’ps’. If a mapping table is specified, the name which is in the line with cmor name=ps is used to find the correct infile variable.
coordinates (attributes)
grid info
file
vertical parametric axis
Right now, only this ’alternate hybrid sigma’ axis is enabled and test data for other axes built by a formula is required to guarantee a correct processing
coordinates (attributes)
grid info
file
vertical axis
Type | Default units |
---|---|
ZAXIS_PRESSURE | "Pa" |
ZAXIS_HYBRID | "Pa" for all parameters |
ZAXIS_DEPTH_BELOW_SEA | "m" |
ZAXIS_DEPTH_BELOW_LAND | "cm" |
name="rho" | "kg m^-3" |
Scalar | "m" |
If no z-axis units are provided in infile via the units attribute of the coordinate variable, the operator uses the following default values
special configuration keywords
global
(attributes)
info
table
global
(attributes)
info
table
Keyword | Associated global attributes in CMIP6 | Associated global attributes in CMIP5 | Format |
---|---|---|---|
branch_dates | branch_time_in_parent, branch_time_in_child | branch_time | Two comma separated integers: <parentBranchDate>*,<childBranchDate> * |
member | realization_index, initialization_index, physics_index, forcing_index | realization, initialization_method, physcis_version | r%di%dp%df%d (CMIP6) or r%di%dp%d (CMIP5) where %d is an integer for the respective index |
keep_all_ attributes |
All infile attributes | All infile attributes | y or n (default) |
*<date> has the format <year><month><day>, e.g.: 18500101
The operator offers keywords to specifiy several attributes at once. Specify them in the info table as well.
Keyword | Usage | Format |
---|---|---|
drs | Defines whether the output_path_template should be built | y (default) or n |
drs_root | Defines where the output file should be saved | Default: "./" |
output_path_template | Hard codes the entire DRS for the output path in CMIP6 | String. Default: CMIP CV |
output_file_template | Hard codes the entire DRS for the output file name in CMIP6 | String. Default: CMIP CV |
2. Neglect the ESGF restrictions
Keyword | Usage | Format |
---|---|---|
tracking_prefix | Defines whether a tracking prefix should be set for the tracking_id attribute | y (default) or n |
version_date | Hard codes the <version> directory in the DRS in CMIP6 | *dataContructionDate |
*<date> has the format <year><month><day>, e.g.: 18500101
2. Neglect the ESGF restrictions
The default of <version> is the date when CMOR is called and it is constructed by CMOR. In operational mode, more than one version directories may be created when simulations last for several days.
The tracking_id attribute in CMIP6 is a PID (persistent identifier). PIDs become active when they are registered in the ESGF. For test simulations, it is recommended to avoid building PIDs by setting the tracking_prefix to n.
Infile
CDO
CMOR
Outfile
replace
append
predefinitons
user setting
data stream
output control
1851
1852
1850
CMOR
1850_post
1851_post
1852_post
tas_
185001_185012.nc
tas_
185001_185112.nc
tas_
185001_185212.nc
model simulation year |
Postprocessing skript |
CMIP6 compliant example variable |
Workflow
Name | Short name | Format | Default |
---|---|---|---|
output_mode | om | Character | a |
last_chunk | lc | Strings | Read from chunk description file |
max_size | ms | Integer | 2 |
CMOR
Outfile
replace
append
Name | Short name | Format | Default |
---|---|---|---|
output_mode | om | Character | a |
last_chunk | lc | Strings | Read from chunk description file |
max_size | ms | Integer | 2 |
CMOR
Outfile
replace
append
Name | Short name | Format | Default |
---|---|---|---|
deflate_level | dl | Integer | 1 |
save_chunk | sc | Character | n |
CMOR
Outfile
replace
append
CMOR
Outfile
replace
append
#From earlier examples, you should have created a tas-file
#for year 2001. Now you want to append year 2002:
#
#Save the last chunk in a variable:
creationDate=v20171010
DRS=CMIP6/CMIP/MPI-M/MPIESM-1-2-HR/historical/ri1ip1f1/Amon/tas/gn/${creationDate}/
lc=${DRS}tas_Amon_MPIESM-1-2-HR_historical_r1i1p1f1_gn_200101-200112.nc
cdo cmor,Amon,mt=mapping_table.txt,last_chunk=${lc} example_append2002.grb
#The outfile path and filename is saved in
#„CHUNK_FILE_tas_Amon_MPI-ESM_amip_r1i1p1.txt“.
#This is the description for the default chunk in append mode.
#Now you can append the next year without defining lc:
cdo cmor,Amon,mt=mapping_table.txt example_append2003.grb
Name | Function | Short name | command line | mapping table | grid info | config tables | "E:Experiment | |||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
M:Model | ||||||||||||
U:User V:Variable" | MIP era | additional CORDEX attributes | "Man(datory), | |||||||||
If req(uired ), | ||||||||||||
Def(ault), Opt(ional)" | "CV: Controlled Vocabulary | |||||||||||
CV restricted: Format of value is restricted by Controlled Vocabulary" | Default | |||||||||||
cmor_name | Variable selector keyword | cn | y | y | man | CSV. Value must be in MIP-table. | ||||||
code | Variable selector keyword | c | y | y | opt | Three digits Intege. GRIB-cod. | ||||||
name | Variable selector keyword | n | y | y | opt | Word | ||||||
drs | Output control keyword | d | y | y | def | y' for building a DRS and 'n' for not building a DRS | y | |||||
drs_root | Output control keyword | dr | y | y | def | Filepath to where DRS will be built | ./ | |||||
last_chunk | Output control keyword | lc | y | y | opt | CSV of filenames | ||||||
max_size | Output control keyword | ms | y | y | def | Unit: Gb | 2 | |||||
output_mode | Output control keyword | om | y | y | def | r for replace or a for append | a | |||||
project_mip_table | MIP table of CMOR variable | pmt | y | y | 5,6 | man | CV: day, Omon, 6hrLev, ... | |||||
cell_methods | Mapping keyword | cm | y | y | def | CV: 'm', 'p', 'c', 'n' | m | |||||
character_axis | Mapping keyword | ca | y | y | ifReq | Cmor axis label | ||||||
positive | Mapping keyword | p | y | y | ifReq | d for downward, u for upward or blank for undirected | ||||||
units | Mapping keyword | u | y | y | man | String, must be readable by udunits | ||||||
variable_comment | Mapping keyword | vc | y | y | opt | String | ||||||
z_axis | Mapping keyword | za | y | y | ifReq | Cmor axis label | ||||||
activity_id | Global attribute | y | E | 6 | man | CV | ||||||
branch_method | Global attribute | y | E | 6 | ifReq | |||||||
branch_time | Global attribute | y | E | 5 | def | Double | 0.0 in CMIP5 | |||||
branch_time_in_child | Global attribute | y | E | 6 | ifReq | CV restricted | ||||||
branch_time_in_parent | Global attribute | y | E | 6 | ifReq | CV restricted | ||||||
calendar | Global attribute | y | y | M | def | CV: standard, proleptic, ... | standard | |||||
comment | Global attribute | y | E | opt | ||||||||
contact | Global attribute | y | U | 5,6 | opt | |||||||
cordex_domain | Global attribute | y | E | y | man | CV | ||||||
driving_experiment_name | Global attribute | y | E | y | man | CV | ||||||
driving_model_id | Global attribute | y | E | y | man | CV | ||||||
experiment_id | Global attribute | y | E | 5,6 | man | CV | ||||||
forcing | Global attribute | y | E | 5 | man | |||||||
forcing_index | Global attribute | y | E | 6 | man | CV restricted | ||||||
grid | Global attribute | y | E | 5,6 | man | |||||||
grid_label | Global attribute | y | E | 6 | man | CV | ||||||
history | Global attribute | y | E | opt | ||||||||
initialization | Global attribute | y | E | 5 | man | CV | ||||||
initialization_index | Global attribute | y | E | 6 | man | CV restricted | ||||||
institute_id | Global attribute | y | U | 5 | man | CV | ||||||
institution | Global attribute | y | U | 5,6 | man | CV | ||||||
institution_id | Global attribute | y | U | 6 | man | CV | ||||||
license | Global attribute | y | E | 6 | man | CSV restricted | ||||||
mip_era | Global attribute | y | U | 6 | man | CV | ||||||
model_id | Global attribute | y | M | 5 | man | CV | ||||||
nominal_resolution | Global attribute | y | M | 6 | man | CV | ||||||
parent_activity_id | Global attribute | y | E | 6 | ifReq | CV | ||||||
parent_experiment_id | Global attribute | y | E | 5,6 | ifReq | CV | ||||||
parent_experiment_rip | Global attribute | y | E | 5 | ifReq | CV restricted | ||||||
parent_source_id | Global attribute | y | E | 6 | ifReq | CV | ||||||
parent_time_units | Global attribute | y | E | 6 | ifReq | CV restricted | ||||||
parent_variant_label | Global attribute | y | E | 6 | ifReq | CV restricted | ||||||
physics_index | Global attribute | y | E | 6 | man | CV restricted | ||||||
physics_version | Global attribute | y | E | 5 | man | CV restricted | ||||||
product | Global attribute | y | E | 5,6 | man | CV | ||||||
project_id | Global attribute | y | E | 5 | man | CV | ||||||
rcm_version_id | Global attribute | y | E | y | man | CV restricted | ||||||
realization | Global attribute | y | E | 5 | man | CV restricted | ||||||
realization_index | Global attribute | y | E | 6 | man | CV restricted | ||||||
references | Global attribute | y | M | def | No references available for $model_id | |||||||
source | Global attribute | y | M | 5,6 | man | CV | ||||||
source_id | Global attribute | y | M | 6 | man | CV | ||||||
source_type | Global attribute | y | M | 6 | ifReq | CV restricted | ||||||
sub_experiment | Global attribute | y | E | 6 | ifReq | CV | ||||||
sub_experiment_id | Global attribute | y | E | 6 | ifReq | CV | ||||||
variant_info | Global attribute | y | E | opt | ||||||||
keep_all_atts | Global attribute specifier | y | E | 5,6 | def | Either y for keeping all attributes from infile or n for not | n | |||||
member | Global attribute specifier | y | E | 5,6 | man | "r%di%dp%d(f%d)" where %d is an integer | ||||||
parent_dates | Global attribute specifier | y | E | 5,6 | ifReq | |||||||
tracking_prefix | Global attribute specifier | y | U | 5,6 | def | Either "y" for use the project dependend tracking prefix or n for not | y | |||||
grid_info | Filename keyword | gi | y | y | M | opt | Filename. Grid of a netCDF file must fit to infile variable dimensions | |||||
grid_info_dir | Filename keyword | y | M | opt | Filepath | |||||||
info | Filename keyword | i | y | def | CSV of filenames. | ".cdocmorinfo" in the current working directory | ||||||
mapping_table | Filename keyword | mt | y | y | M | opt | Filename | |||||
mapping_table_dir | Filename keyword | y | M | opt | Filepath | |||||||
mip_table_dir | Filename keyword | y | U | opt | Filepath | |||||||
output_file_template | Filename keyword | y | U | 6 | def | CV | CV | |||||
output_path_template | Filename keyword | y | U | 6 | def | CV | CV | |||||
version_date | Filename keyword | vd | y | y | U | 6 | def | Date $year$month$day | Current date (<version>) | |||
climatology_interval | Coordinate keyword | y | y | E | ifReq | |||||||
leap_month | Coordinate keyword | y | y | M | ifReq | |||||||
leap_year | Coordinate keyword | y | y | M | ifReq | |||||||
req_time_units | Coordinate keyword | y | y | E | man | "$units since $year-$month-$day $hours:$minutes:$seconds" | ||||||
switch_z | Coordinate keyword | y | y | M | def | Either y for substitute the z-axis with the grid info zaxis or n for not | y | |||||
time_bounds | Coordinate keyword | y | y | M | def | Either „y“ for create or „n" for infile | n |
By Fabian Wachsmann
Get to know the CMIP6 data standard. Explore the advantages of CDO, CMOR and cdo cmor. Learn how to convert variables into the CMIP6 format. Learn how to provide a complete metadata set. Integrate the app into the operational workflow.
Scientific programmer in the joint research project DICAD at DKRZ Developer of Climate Data Operators (CDO)