Meteorological data for energy systems modelling
Ed Sharp:
ed.sharp@ucl.ac.uk | www.esenergyvis.wordpress.com | @steadier_eddy
Overview
- Choosing data
-
Met mast data
- MIDAS
- Wunderground
- Offshore Buoys
- Data above 10 m
-
Reanalysis datasets
- NASA MERRA
- NCEP CFSR
- Alternatives and developments
-
Forecasted data
- Short term
- Long term
-
Maintained Database
- Access
- Contents
-
Usage case studies
- Population weighting
- Points
- Grids
Provide an overview of the different types of data, strengths and weakness, access and usage. For more detail see knowledge_base.doc which includes links, which are also provided at the end of this presentation
Choosing Historical Meteorological Data
Met Mast Data
For academic projects in GB the Met Office Integrated Data Archive System (MIDAS) Land and Marine Surface Stations Data (1853-current) database is the best choice
- Free for research
- 100's of stations spread out across GB
- Accessible through the Centre for Environmental Data Analysis (CEDA), registration necessary
- Multiple variables
- Hourly data available inc. wind speed and rainfall
- < 10 m elevation
- Biased to populated areas
- Not homogeneous w. some data quality issues
Weather Underground
Non academic projects and those outside GB may be able to use this alternative with a larger spatial scope, sites may also compliment those offered by MIDAS
- Global network, 250,000 + stations
- Amateur stations, usually high quality
- API download available
- No license restrictions
- You can also register your station here.
- Interactive map in iframe
Offshore Buoys
If offshore sites are of interest there are data from weather stations at sea
- The British Oceanographic Centre (BODC) National Oceanographic Database (NODB)
- Approx 20 offshore buoys and permanent "lightships"
- Data at 6 m above sea
- Wind speed, temp, wave height
- Timeseries available as well as current conditions
- API and bulk download
Offshore Masts
Temporary masts are often installed at planned wind farm locations
- Wind speed timeseries at hub equivalent heights e.g. 100m
- Data available from the Marine Data Exchange - not all reported data can be extracted - map shows accessible timeseries
- These sites will occur less outside of GB, excluding Denmark, due to less offshore wind capacity
- Several years often available
- Wind resource evaluation
- Model calibration
Reanalysis Datasets
Gridded global or regional data, huge number of weather variables for up to 100 years
- Derived from Numerical Weather Prediction Models (NWP)
- Tethered to observed data
-
Convention to provide 3 hourly "forecasts"
- High resolution data = hourly
- From 10 - 100 years of hindcasted data
- Homogeneous and clean
-
Resolution is compromised for scope in both space and time, four common combinations
- Long global coarse in space and time
- Long global coarse in space
- Long regional fine in space coarse in time
- Short regional fine in space time
- Quality varies regionally .....
NASA MERRA
37 years of global data at fine(ish) spatial and and fine temporal resolution
- Preferred dataset, db maintained to near present day
- 1979 - Present (1-2 month lag)
- MERRA 1 ends 2010, MERRA 2 = latest product
- 0.5° × 0.66° grid with 72 layers.
- Very large number of variables. Many Hourly.
- Wind, Temperature, Water, Humidity - see database slide for those maintained on NAS
- Wind speed at 2, 10, 50 m above surface as well as pressure levels - key for simulation ...
- Widely used in research and commercially - may be replaced by regional reanalyses or ERA5 in near future
- Files retrievable in NETCDF form or GRIB
- GRIB are meteorologist file of choice
- NETCdf contain better metadata - see late slides on access
- Interpolated Extrapolated data used in simulation (will describe in detail later)
- Can estimate the accuracy of the data and method by comparing to high met mast data
- Plots show performance .....
NCEP CFSR
37 years of global data at fine(ish) spatial and and fine temporal resolution
- Very similar to MERRA in many ways
- 37 years of hourly data, multiple layers
- CFSR V1- 2010, CFSR v2 from 2010
- Slightly improved spatial resolution on MERRA
-
Only 1 wind speed height above surface
- mutliple pressure levels can be used - see next presentation
- Used in PhD and HighRES
- Gives some novelty as MERRA is more widely used.
- GB database on NAS, not maintained
Alternative Reanalysis Datasets
Regional reanalyses provide enhanced spatial resolution, other satellite derived data are available;
- The choice of regional reanalysis depends on Geography
- COSMO REA - see figure
- Fewer variables, wind at 2 and 10 m
- pressure level data available
- 2 or 6 km resolution!
- 2 km only for central Europe
- ECMWF ERA 5
- currently under development
- global, hourly, 30 km
- improvement on MERRA according to some
- 1950 - present (long!)
- Python module for access
- CMSAF AVHRR
- satellite derived irradiance
- improved accuracy on MERRA
- Less stable
- Using Multiple products may improve individual accuracy, but detaches homogeneous meteorology
- Others exist - see region specific originators +
downscaling
Using reanalysis data - fundamentals
Reanalyis data represents a high quality record of past meteorology, this can be used to represent the future, with caution.
-
Long time series from reanalysis provide information on how climate has changed as well as how weather varies over multiple temporal resolutions (hours - decades) - plot shows minimum annual temperatures by country in Europe.
- These timeseries have therefore been used to examine how other things are effected by this change and variability, e.g. weather driven energy generation and demand
- The process is often called hindcasting - projecting a past timeseries into the future
-
However - Not all change and variability is captured
- Weather and climate will not be the same in the future as it has been in the past.
- Therefore when using the data it may be better to select extreme or "average" periods or introduce stochasticity through the use of multiple periods for single simulations etc. DISCUSS .....
Forecasts
Where hindcasting is not appropriate, forecasted data are available at multiple scales and resolutions. Don't just add 2 degrees to reanalysis data!
- Forecast modelling, as opposed to simulation modelling should use a different dataset - for example:
-
Short term prediction or forecasting, e.g. simulation of wind generation on grid to balance grid, predict price or reduce carbon content of activity (Baking messaging....)
-
Global Forecast System (GFS) - see gif, gives gridded 3 hourly via FTP
- up to ten days, 0.5 degrees
- multi decade historical data for training
- ECMWF also provide gridded forecasts to members
-
Wunderground (e.g.) give site specific hourly forecasts
- to ten days
- free via api (with limits)
- Esembles (multiple datasets) can be used to fill time, accuracy degrades with time
-
Global Forecast System (GFS) - see gif, gives gridded 3 hourly via FTP
-
Long terms projections of climate change are available from multiple sources
- UKCPO9 = weather generator, Coupled Model Intercomparison Project (CMIP5) to 2100
- Coordinated Regional Downscaling Experiment (CORDEX) 10 km, same framework as COSMO
Population weighting weather data
Population weighting a gridded weather dataset is a way to get a single value for each timestep that represents the weather experienced by a subset of people, for example in a country.
- Recreate MERRA grid in GIS, using centroid coordinates and Voronoi polygons
-
GRUMP population GPW v4 data
- Extract population by country
- Sum all pop grids within a MERRA grid cell, pop grids are smaller, approximate method, with some crossover.
- New pop grid by counrty with 0's outside borders
- Multiply timeslice weather by new grid, sum and divide by total country population
- temperature at 2m (K), wind speed at 2m (m/s), net downward solar radiation at the surface (W/m2), specific humidity at 2 m (kg/kg), Wet bulb temperature (K) and Dew Point temperature (K) for all countries, 1980 - present on NAS
Accessing and interpreting the maintained database
- On the UCL network (inc. VPN), a Network Attached Storage (NAS) can be accessed (details available from me)
- NASA MERRA
- 1979 - near present, global grid, 23 hourly variables (see next slide)
- Mix of MERRA 1 and MERRA 2
- MERRA grid shapefiles
- NCEP CFSR GB subset to approx 2010
- Population weighted weather timeseries for whole globe as detailed previously
- Access via browser:
- Python FTP access with Case study for querying data
MERRA data on the maintained database
Links and further material
MIDAS data @ CEDA http://catalogue.ceda.ac.uk/uuid/220a65615218d5c9cc9e4785a3234bd0
Weather Underground https://www.wunderground.com/wundermap
Buoys Map http://www.ndbc.noaa.gov/maps/United_Kingdom.shtml
Buoys data https://www.bodc.ac.uk/data/bodc_database/nodb/search/
Marine Data exchange http://www.marinedataexchange.co.uk/wind-data.asp
Reanalysis general info http:www.reanalysis.org
MERRA 2 download https://disc.gsfc.nasa.gov/daac-bin/FTPSubset2.pl
ECMWF ERA5 https://software.ecmwf.int/wiki/display/CKB/How+to+download+ERA5+data+via+the+ECMWF+Web+API
COSMO REA2 http://reanalysis.meteo.uni-bonn.de/?Download_Data___COSMO-REA2
CMSAF https://wui.cmsaf.eu/safira/action/viewDoiDetails?acronym=SARAH_V002
Global Forecast System https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs
NAS - see me