Climate extremes indices (CEI)

with CDO's

According to the Expert Team on Climate Change Detection and Indices (ETCCD)

This work is licensed under a Creative Commons Attribution 4.0 International License .

Motivation

Definitions

Examples

Climate index defintion

A climate index is a calculated measure for the state and/or variations of the climate system.
DWD: Klimakenntag: If a climatological parameter exceeds a specific threshold

Examples

The famous ones:
- ENSO and NAO for teleconnection
- Ice days for winter strength
Special ones:
- Palmer drought severity index

To gain a uniform perspective on observed changes in weather and climate extremes, ETCCDI has

defined a core set of descriptive indices of extremes. The indices describe particular characteristics of

extremes, including

frequency, amplitude and persistence.

Motivation

Absolut

Threshold

Duration

Percentiles

txx
txn
tnx
tnn

dtr
sdii

rx1day
rx5day
prcptot

fd
id
su
tr

r1mm
r10mm
r20mm

csdi
wsdi

cdd
cwd

gsl

tx10p
tx90p
tn10p
tn90
r95p
r99p

Climate extremes indices definitons of the Expert Team on Climate Change Detection and Indices (ETCCDI) can be subdivided approximately into four categories:

All of them are calculated based on daily time series of three variables:

#tx: daily maximum temperature
#tn: daily minimum temperature
#pr: daily precipation rate

Number of different Kenntagen for station Hamburg Fuhlsbüttel (DWD) averaged over 1971 bis 2000.

Absolut

Threshold

Duration

Percentiles

Climate extremes indices definitons of the Expert Team on Climate Change Detection and Indices (ETCCDI) include four categories:

“Values of absolute extremes can often be related to extreme events that affect human society and the natural environment”

“Indices based on the count of days crossing certain fixed thresholds (for example, the 0°C threshold as used in the frost days index FD) can also be related to observed impacts, in particular if the thresholds refer to values of physical, hydrological or biological significance.“

“The reason for choosing mostly percentile thresholds rather than fixed thresholds is that the number of days exceeding percentile thresholds is more evenly distributed in space and is meaningful in every region”

#txx:        Maximum of tx in a period
#txn:        Minimum of tx in a period
#tnx:        Maximum of tn in a period
#tnn:        Minimum of tn in a period

#dtr:        Diurnal temperature range. Periodical average of tx minus tn.
#sdii:       Simple daily intensity:
#            Average of pr on "wet days" (usually pr > 1mm) in a period. 

#rx1day:     Maximum of pr in a period
#rx5day:     Maximum 5-day sum of pr in a period
#prcptot:    Total wet day pr in a period.

#fd:         Number of days in a period where tn < 0 degC
#id:         Number of days in a period where tx < 0 degC
#su:         Number of days in a period where tx > 25 degC
#tr:         Number of days in a period where tn > 20 degC

#r1mm:       Number of wet days in a period where pr > 1mm
#r10mm:      Number of very wet days in a period where pr > 10mm
#r20mm:      Number of extremely wet days in a period where pr > 20mm

Definitions of absolute indices:

Definitions of threshold indices:

#csdi: Cold spell duration index:
#      Let TNin10 be the calendar day 10th percentile
#      centered on a 5 day window for the base period 1961–1990.
#      Then the number of days per period is summed where,
#      in intervals of at least 6 consecutive days: tn < TNin10
#wsdi: Warm spell duration index:
#      Let TXin90 be the calendar day 90th percentile
#      centered on a 5 day window for the base period 1961–1990.
#      Then the number of days per period is summed where,
#      in intervals of at least 6 consecutive days: tx > TXin90

#cdd:  Consecutive dry days:
#      Count the largest number of consecutive days
#      in a period where pr < 1 mm
#cwd   Consecutive wet days:
#      Count the largest number of consecutive days
#      in a period where pr > 1 mm

#gsl:  Growing season length:
#      Let T be the mean temperature ((TN + TX)/2) on day i in period j.
#      Count the number of days between the first occurrence of
#      at least 6 consecutive days with T > 5°C and the first occurrence 
#      after 1st July (NH) or 1st January (SH) of 
#      at least 6 consecutive days with Tij < 5°C

Definitions of duration indices:

Climdex and

eca operators

Reasons for new operators

For the European Climate Assessment (ECA), 75 climate indices have been calculated for all participating stations. These indices are implemented in the CDOs.
Scientists complain that the Climdex results differ from eca operators. In this column, all ETCCDI indices calculated with cdo ecas and climdex are compared.

Climate extremes indices calculation with ETCCDI program climdex:

input.files <- c("pr_day_MPI-ESM-LR_historical_r1i1p1_19600101-19991231_hamburg.nc",
"tasmin_day_MPI-ESM-LR_historical_r1i1p1_19600101-19991231_hamburg.nc",
"tasmax_day_MPI-ESM-LR_historical_r1i1p1_19600101-19991231_hamburg.nc")

author.data <- list(institution="Max Planck Institute for Meteorology", institution_id="MPI-M")

create.indices.from.files(input.files, "climdexOutput/", input.files[1],
                         author.data, base.range=c(1960,1989),
                         parallel=8, max.vals.millions=70)

ls climdexOutput/
altcddETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
altcsdiETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
altcwdETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
altwsdiETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cddETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
csdiETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cwdETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
dtrETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
dtrETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
fdETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
gslETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
idETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
prcptotETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
r10mmETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
r1mmETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
r20mmETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
r95pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
r99pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
rx1dayETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
rx1dayETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
rx5dayETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
rx5dayETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
sdiiETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
suETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
tn10pETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
tn10pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
tn90pETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
tn90pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
tnnETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
tnnETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
tnxETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
tnxETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
trETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
tx10pETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
tx10pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
tx90pETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
tx90pETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
txnETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
txnETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
txxETCCDI_mon_MPI-ESM-LR_historical_r1i1p1_196001-199912.nc
txxETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
wsdiETCCDI_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc

cdo subc,273.15 -yearmin $tnTimeseries $tnnOfile

cdo subc,273.15 -yearmin $txTimeseries $txnOfile

cdo subc,273.15 -yearmax $txTimeseries $txxOfile

cdo subc,273.15 -yearmax $tnTimeseries $tnxOfile

cdo yearmean -sub ${txTimeseries} ${tnTimeseries} $dtrOfile

tnn

txn

txx

tnx

dtr

Absolute indices with CDO's

ETCCDI climate index

#tn: daily minimum temperature
#tx: daily maximum temperature

Results of both calculations are equal for absolute indices.

rx5Input=$cdoOutput/runsum5_pr_day_MPI-ESM-LR_historical_r1i1p1_19600101-19991231.nc
CDO_TIMESTAT_DATE="last" cdo -runsum,5 -mulc,86400 $prInput $rx5Input
for year in {1960..1999}
do
    output1=$cdoOutput/Rx1day_MPI-ESM-LR_historical_r1i1p1_${year}.nc
    output2=$cdoOutput/Rx5day_MPI-ESM-LR_historical_r1i1p1_${year}.nc
    output3=$cdoOutput/Sdii_MPI-ESM-LR_historical_r1i1p1_$year.nc
#
    cdo eca_rx1day -mulc,86400 -selyear,${year} $prInput $output1
    cdo eca_rx5day -selyear,${year} $rx5Input $output2
    cdo eca_sdii -mulc,86400 -selyear,${year} $prInput $output3
#FIXME Why are there NANs in sdii?
done
cdoRx1day=$cdoOutput/Rx1day_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoRx5day=$cdoOutput/Rx5day_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoSdii=$cdoOutput/sdii_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
#
cdo mergetime $cdoOutput/Rx1day_MPI-ESM-LR_historical_r1i1p1_*.nc $cdoRx1day
cdo mergetime $cdoOutput/Rx5day_MPI-ESM-LR_historical_r1i1p1_*.nc $cdoRx5day
cdo mergetime $cdoOutput/Sdii_MPI-ESM-LR_historical_r1i1p1_*.nc $cdoSdii

rx1day

rx5day

sdii

rx5day

Absolute indices with CDO's

ETCCDI climate index

Results of both calculations are equal for absolute indices. However,

cdo requires a loop over years for yearly output frequency
cdo eca_rx5day does not calculate the 5 day precipitation sum; it requires it as input

prcptot

cdoPrcptot=$cdoOutput/prcptot_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdo mulc,86400 -yearsum -mul ${prInput} -gtc,1 -mulc,86400 ${prInput} $cdoPrcptot

Absolute indices with CDO's

ETCCDI climate index

Results of both calculations are equal for absolute indices. However,

there is no operator for dtr (rather easy) and prcptot (a bit complicated)
unit conversion has to be considered

for year in {1960..1999}
do
  cdo eca_fd -selyear,${year} $tasminInput tempFd$year.nc
  cdo eca_su -selyear,${year} $tasmaxInput tempSu$year.nc
  cdo eca_id -selyear,${year} $tasmaxInput tempId$year.nc
  cdo eca_tr -selyear,${year} $tasminInput tempTr$year.nc
#
  cdo eca_r10mm -mulc,86400 -selyear,${year} $prInput tempR10mm$year.nc
  cdo eca_pd,1  -mulc,86400 -selyear,${year} $prInput tempR1mm$year.nc
  cdo eca_r20mm -mulc,86400 -selyear,${year} $prInput tempR20mm$year.nc
done
cdoFd=$cdoOutput/fd_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoSu=$cdoOutput/su_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoId=$cdoOutput/id_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoTr=$cdoOutput/tr_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoR10mm=$cdoOutput/r10mm_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoR1mm=$cdoOutput/r1mm_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoR20mm=$cdoOutput/r20mm_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
#
cdo mergetime tempFd* $cdoFd
cdo mergetime tempSu* $cdoSu
cdo mergetime tempId* $cdoId
cdo mergetime tempTr* $cdoTr
cdo mergetime tempR10mm* $cdoR10mm
cdo mergetime tempR1mm* $cdoR1mm
cdo mergetime tempR20mm* $cdoR20mm

fd
su
id
tr

r10mm
r1mm
r20mm

Threshold indices with CDO

ETCCDI climate index

Results of both calculations are equal for threshold indices.

for y in {1960..1999}  do
  cdo eca_cdd -mulc,86400 -selyear,${y} $PR cdd${y}.nc
  cdo eca_cwd -mulc,86400 -selyear,${y} $PR cwd${y}.nc
done
cdoCdd=$cdoOutput/cdd_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdoCwd=$cdoOutput/cwd_yr_MPI-ESM-LR_historical_r1i1p1_1960-1999.nc
cdo mergetime cdd* $cdoCdd
cdo mergetime cwd* $cdoCwd

cdd
cwd

Duration indices with CDO

ETCCDI climate index

Results of both calculations differ for duration indices.

TimeSeries=tasmax_day_MPI-ESM-LR_historical_19600101-19991231.nc
bTimeSeries=tasmax_day_MPI-ESM-LR_historical_19600101-19891231.nc
p90=tx90p_MPI-ESM-LR_historical_19600101-19891231.nc
cdo ydrunpctl,90,5 ${bTimeSeries} -ydrunmin,5 ${bTimeSeries} -ydrunmax,5 ${bTimeSeries} ${p90}
#
for year in {1960..1999}
do
  output=eca-tx90p_MPI-ESM-LR_historical_${year}.nc
  cdo eca_tx90p -selyear,${year} ${TimeSeries} ${p90} $output
done
cdo mergetime eca-tx90p_MPI-ESM-LR_historical_*.nc eca-tx90p_MPI-ESM-LR_historical_1960-1999.nc

tx90p

Percentile indices with CDO

ETCCDI climate index

Results of both calculations differ for percentile indices.

Comparison of climdex and cdo shows that there are problematic differences in the processing of duration and percentiles indices:

Duration indices are calculated in yearly frequency and also counted over overlapping years in climdex. This is not possible with cdo
- However, for cdd, the climdex method leads to strange results having grid points with dry days for the entire time series
the dates 29th feb are ignored in climdex
Percentiles calculation method is different between climdex and cdo
Percentiles indices with a base period require a resampling (e.g. bootstrapping) for the exceedance estimation inside the base period

Green: climdex

Blue: CDO

For the reference period 1960-1989, the climdex values are higher than CDO values because of a bias that occurs when the exceedence rate is calculated with the same data that has been taken for the percentile estimation. This can lead to wrong trends.

Bootstrapping method for percentile estimation:

Treat the in-base year like an out-base year

Create a 30-yr block of data by using 29 yr base period and adding an additional year from the base period. Estimate the threshold.
Repeat the latter step 28 times by repeating each of the remaining 28 in-base years.
The final percentile is obtained by averaging the 29 estimates.

New

cdo etccdi

operators

# CDO Test version:
conda create --name ${cdoenv} conda-forge/label/dev::cdo -c conda-forge
# Scripts including new ETCCDI operators:
git clone https://gitlab.dkrz.de/k204210/cdo_cei.git

Absolut

Threshold

Duration

Percentiles

txx
txn
tnx
tnn

dtr
sdii

rx1day
rx5day
prcptot

fd
id
su
tr

r1mm
r10mm
r20mm

csdi
wsdi

cdd
cwd

#gsl

tx10p
tx90p
tn10p
tn90
r95p
r99p

New etccdi operators are implemented for the following CEIs

All of them are calculated based on daily time series of three variables:

#tx: daily maximum temperature
#tn: daily minimum temperature
#pr: daily precipation rate

Absolut

Threshold

txx
txn
tnx
tnn

dtr
sdii

rx1day
rx5day
prcptot

fd
id
su
tr

r1mm
r10mm
r20mm

New etccdi operators are implemented for the following CEIs

New operators allow to set the output frequency by keyvalue argument

cdo etccdi_fx,freq=month infile outfile

Duration

csdi
wsdi

cdd
cwd

#gsl

New etccdi operators are implemented for the following CEIs

A period of consecutive days over overlapping years is counted and accounted for the last contributing year

cdo etccdi_cdd,freq=month infile outfile

Percentiles

tx10p
tx90p
tn10p
tn90
r95p
r99p

New etccdi operators are implemented for the following CEIs

Percentile calculations take the edges of the time series into account
The R percentile calculation method "type 8" is applied if sufficient memory is available
Bootstrapping is applied for a reference period

window=5
startboot=1961
endboot=1990
freq=year
export CDO_PCTL_NBINS = 302
cdo etccdi_tx90p,$window,$startboot,$endboot,freq=$freq infile outfile

Metadata

Name, longname and units are set as Climdex does
Timestamps of the output of the new ETCCDI operators lie at the mid of the output frequency

There is no standard for climate indices metadata!