Webinar Workshop:

Presented by Patrick Labbett for NAEO

https://slides.com/calltheory/nws-weather-api

Follow the presentation live!

National Weather Service API

The Plan:

Get real-time weather data directly from the National Weather Service API and integrate temperature checks into Intelligent Series scripting.

 

We will also showcase using MergeComm to keep weather updated in the background. 

API Documentation

https://www.weather.gov/documentation/services-web-api

  • Pricing and usage information
  • API Authentication method
  • API request format and parameters
  • API response format for parsing
  • Usage Examples

Analyze and plan, then implement.

Pricing & Usage

Understand capability

"All of the information presented via the API is intended to be open data, free to use for any purpose."

"As a public service of the United States Government, we do not charge any fees for the usage of this service, although there are reasonable rate limits in place to prevent abuse and help ensure that everyone has access."

"The rate limit is not public information, but allows a generous amount for typical use. If the rate limit is exceed a request will return with an error, and may be retried after the limit clears (typically within 5 seconds). Proxies are more likely to reach the limit, whereas requests directly from clients are not likely."

API Authentication

Getting an API key?

"A User Agent is required to identify your application. This string can be anything, and the more unique to your application the less likely it will be affected by a security event. If you include contact information (website or email), we can contact you if your string is associated to a security event. This will be replaced with an API key in the future."

Key:Value

 

More from MDN:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers

HTTP headers use a key:value format.

 

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Date: Sat, 12 Feb 2022 15:23:13 GMT
Keep-Alive: timeout=5, max=1000
Last-Modified: Mon, 25 Jan 2022 04:32:39 GMT
Server: nginx/1.22
X-Powered-By: SugarAndSpiceAndEverythingNice

<!DOCTYPE html>
<html>
  <head>
  	<title>Call Theory Website</title>
  </head>
  <body>
  	<p>Content of my website</p>
  </body>
</html>

What are headers?

 Authentication

  • Send HTTP User-Agent header
  • The value should include contact info

 Recommended 


# Recommendation from NWS

User-Agent: (myweatherapp.com, contact@myweatherapp.com)

# Recommendation from MDN
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent

User-Agent: <product> / <product-version> <comment>

# The User-Agent we will use:
# Map `product` to `scripting`
# Map `version` to IS Account
# Map `comment` to `support@notifi.us`

User-Agent: scripting/1234 support@calltheory.com

Are HTTP header key names case-sensitive?

# User-Agent HTTP header
User-Agent: scripting/1234 support@calltheory.com

# Can we use lower case?
user-agent: scripting/1234 support@calltheory.com

# What about meme case?
UsEr-AgEnT: scripting/1234 support@calltheory.com

# What about removing spaces?
User-Agent:scripting/1234 support@calltheory.com

# How about these cases?
User-Agent : scripting/1234 support@calltheory.com
User-Agent :scripting/1234 support@calltheory.com

# https://www.rfc-editor.org/rfc/rfc7230#appendix-B
#
# OWS stands for Optional White Space
#
# header-field  = field-name ":" OWS field-value OWS
#

API Requests

Format and Parameters

Request Setup

  • User-Agent HTTP header
  • HTTP method (GET, POST, etc.)
  • API endpoints
  • Parameters for endpoints

https://www.weather.gov/documentation/services-web-api#/

API Response

Handling returned data

The response from

the API will be JSON. 

 

We can use the Intelligent Series ParseJSON scripting element along with JSONPATH syntax to pull information from the API response.

Usage Examples

Sample API calls and flow

  • API specification from weather.gov
  • Unofficial Postman Collection
  • OpenAPI / Swagger format
  • Github pages documentation
  1. https://www.weather.gov/documentation/services-web-api#/
  2. https://www.postman.com/api-evangelist/workspace/national-oceanic-and-atmospheric-administration-noaa/overview
  3. https://api.weather.gov/openapi.json
  4. https://weather-gov.github.io/api/

Plan of Attack

We have a basic understanding of the API usage, but we need to link the pieces together. 

Latest Observation Data

If we know the station identifier, we can get the latest data from this endpoint:

GET /stations/KTZR/observations/latest
{

    "properties": {
        "@id": "https://api.weather.gov/stations/KTZR/observations/2022-02-16T06:55:00+00:00",
        "station": "https://api.weather.gov/stations/KTZR",
        "timestamp": "2022-02-16T06:55:00+00:00",
        "rawMessage": "KTZR 160655Z AUTO 16006KT 10SM CLR 00/M06 A3026 RMK AO2 T00031056",
        "textDescription": "Clear",
        "icon": "https://api.weather.gov/icons/land/night/skc?size=medium",
        "presentWeather": [],
        "temperature": {
            "unitCode": "wmoUnit:degC",
            "value": 0.29999999999999999,
            "qualityControl": "V"
        },
        "dewpoint": {
            "unitCode": "wmoUnit:degC",
            "value": -5.5999999999999996,
            "qualityControl": "V"
        },
        "windDirection": {
            "unitCode": "wmoUnit:degree_(angle)",
            "value": 160,
            "qualityControl": "V"
        },
    }
}

Known Lat/Long

If we know the lat/long, we can pull general

details for the location from the API. 

GET /points/39.8819,-83.0939
 {
     /* ... */
        "cwa": "ILN",
        "forecastOffice": "https://api.weather.gov/offices/ILN",
        "gridId": "ILN",
        "gridX": 81,
        "gridY": 76,
        "forecast": "https://api.weather.gov/gridpoints/ILN/81,76/forecast",
        "forecastHourly": "https://api.weather.gov/gridpoints/ILN/81,76/forecast/hourly",
        "forecastGridData": "https://api.weather.gov/gridpoints/ILN/81,76",
        "observationStations": "https://api.weather.gov/gridpoints/ILN/81,76/stations",
        "relativeLocation": {
            "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [
                    -83.088183999999998,
                    39.901524000000002
                ]
            },
            "properties": {
                "city": "Urbancrest",
                "state": "OH",
                "distance": {
                    "unitCode": "wmoUnit:m",
                    "value": 2235.9207116644998
                },
                "bearing": {
                    "unitCode": "wmoUnit:degree_(angle)",
                    "value": 192
                }
            }
        },
        "forecastZone": "https://api.weather.gov/zones/forecast/OHZ055",
        "county": "https://api.weather.gov/zones/county/OHC049",
        "fireWeatherZone": "https://api.weather.gov/zones/fire/OHZ055",
        "timeZone": "America/New_York",
        "radarStation": "KILN"
    /* ... */
 }

Get a list of observation stations for the points

GET /gridpoints/ILN/81,76/stations
/* ... */
    "features": [
        {
            /* ... */
            "properties": {
                "@id": "https://api.weather.gov/stations/KTZR",
                /* ... */
                "stationIdentifier": "KTZR",
                "name": "Columbus, Bolton Field Airport",
                "timeZone": "America/New_York",
                "forecast": "https://api.weather.gov/zones/forecast/OHZ055",
                "county": "https://api.weather.gov/zones/county/OHC049",
                "fireWeatherZone": "https://api.weather.gov/zones/fire/OHZ055"
            }
        },
        {
            /* ... */
            "properties": {
                "@id": "https://api.weather.gov/stations/KLCK",
                "@type": "wx:ObservationStation",
               /* ... */
                "stationIdentifier": "KLCK",
                "name": "Rickenbacker Air National Guard Base",
                "timeZone": "America/New_York",
                "forecast": "https://api.weather.gov/zones/forecast/OHZ055",
                "county": "https://api.weather.gov/zones/county/OHC049",
                "fireWeatherZone": "https://api.weather.gov/zones/fire/OHZ055"
            }
        },
      ]
/* ... */

Get latest observations

We're back where we started: we know the observation station identifier

Option #1

 

  • Manually lookup station to reduce amount of API calls/complexity/scripting

Option #2

 

  • Follow API references to dynamically pull station for the points

Considerations

Observation data can be null

"properties": {
        "@id": "https://api.weather.gov/stations/KTZR/observations/2022-02-16T07:15:00+00:00",
        "@type": "wx:ObservationStation",
        "elevation": {
            "unitCode": "wmoUnit:m",
            "value": 280
        },
        "station": "https://api.weather.gov/stations/KTZR",
        "timestamp": "2022-02-16T07:15:00+00:00",
        "rawMessage": "KTZR 160715Z AUTO 16007KT 10SM CLR 01/M05 A3025 RMK AO2 T00141050",
        "textDescription": "Clear",
        "icon": "https://api.weather.gov/icons/land/night/skc?size=medium",
        "presentWeather": [],
        "temperature": {
            "unitCode": "wmoUnit:degC",
            "value": null,
            "qualityControl": "Z"
        },
        "dewpoint": {
            "unitCode": "wmoUnit:degC",
            "value": null,
            "qualityControl": "Z"
        },
        "windDirection": {
            "unitCode": "wmoUnit:degree_(angle)",
            "value": null,
            "qualityControl": "Z"
        },

Simple Cache

MergeComm can be used as a simple caching method for saving shared data (like weather) without having to make every request on-demand. A busy client could result in API rate limits and other failures.

Speed Boost

A well known method of improving user experience on web and software applications is to put potentially long running tasks into a background process (or thread) so a slow email server or API request does not manifest as lag for your agents. 

Serendipitous MergeComm Benefits

How can we use MergeComm?

  • Setup an IS Schedule to run MergeComm job
  • Pull the data from the API during each run
  • Update an IS Shared Field or database entry
  • Refer to the IS Shared Field/database entry in script
  • Avoid rate limits from activity
  • Reduce API calls required during agent work time
  • Keep our scripting simple (less API code)

Some of the benefits...

Example Script Review

  1. Example 1 - Get weather by Observation Station
  2. Example 2 - Get weather by lat/long
  3. Example 3 - Enforce temperature in scripting
  4. Example 4 - Get weather by address
  5. Example 5 - Using MergeComm

The example script is available on learn.calltheory.com - sign up for a free account for your team.
 

(Or email support@calltheory.com to request a copy)

Thank you

Questions?

https://store.calltheory.com/collections/services/products/calltheory-security-workshop-2022

Learn Security

Made with Slides.com