OSH Tutorial Part 4

Publishing Data

Types of data produced

  • Measurement data exists mainly in two forms within OSH
    • Real-time data is wrapped by events and transits through the event bus
    • Historical data is available on request from storage modules
  • Data can be produced by different types of modules
    • Sensor Drivers (events)
    • Stream Processing (events)
    • On-demand Processing (events or storage)
    • Storage impl. can wrap existing data directly
  • All sources generate data in a consistent in-memory format based on SWE Common models

How to get it out?

  • Different types of services can make real-time and historical data available through an external interface:
    • Sensor Observation Service (SOS) is part of core
    • SensorThings API (coming soon in v1.2)
    • MQTT with SWE payload (coming soon in v1.2)
    • Other services can be developed if needed
  • These services can obtain data from
    • Event bus
    • Storage
    • Both

Publishing through SOS

Sensor Driver

Data Storage

Event
Bus

Data + Sensor Events

SOS Interface

Data + Sensor Events

Historical Data and Sensor Descriptions

Data + Sensor Events

SOS - Storage Connection

Get Records by Time Range

SOS Interface

  • SOS never writes directly to storage, this is done through the associated virtual sensor

Get Records by FOI ID or BBOX

Get Sensor Description By Time

Get FOI by ID or BBOX

Observation Storage

Using the Web Admin

Sensor Observation Service

  • Provides access to observations collected by one or more sensors of any type and organized by offerings
  • Default observation format is O&M + SWE Common v2.0
  • Provides access to sensor descriptions (as SensorML) via DescribeSensor
  • Result handling extension for efficient encoding of observation arrays and streams
  • Websocket for efficient streaming to web browsers
<GetObservation service="SOS" version="2.0.0">
   <observedProperty>http://sensorml.com/ont/swe/property/Weather</observedProperty>
   <procedure>urn:osh:sensor:simweather:0123456879</procedure>
   <temporalFilter>
      <fes:After>
         <fes:ValueReference>phenomenonTime</fes:ValueReference>
         <gml:TimePeriod gml:id="tp1">
            <gml:beginPosition>2016-08-11T19:00:00Z</gml:beginPosition>
            <gml:endPosition>2016-08-11T19:10:00Z</gml:endPosition>
         </gml:TimePeriod>
      </fes:After>
   </temporalFilter>
   <spatialFilter>
      <fes:BBOX>
         <fes:ValueReference>om:featureOfInterest/sams:SF_SpatialSamplingFeature/sams:shape</fes:ValueReference>
         <gml:Envelope srsName="http://www.opengis.net/def/crs/EPSG/0/4326">
            <gml:lowerCorner>21 -94</gml:lowerCorner>
            <gml:upperCorner>22 -90</gml:upperCorner>
         </gml:Envelope>
      </fes:BBOX>
   </spatialFilter>			
</GetObservation>

Request/Response Examples

Request/Response Examples

<sos:GetObservationResponse xmlns:sos="http://www.opengis.net/sos/2.0" xmlns:om="http://www.opengis.net/om/2.0" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:swe="http://www.opengis.net/swe/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <sos:observationData>
    <om:OM_Observation gml:id="OBS_001">
      <om:type xlink:href="http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_ComplexObservation"/>
      <om:phenomenonTime>
        <gml:TimeInstant id="T1">
          <gml:timePosition>2016-08-11T19:58:00.629Z</gml:timePosition>
        </gml:TimeInstant>
      </om:phenomenonTime>
      <om:resultTime>
        <gml:TimeInstant id="T2">
          <gml:timePosition>2016-08-11T19:58:00.629Z</gml:timePosition>
        </gml:TimeInstant>
      </om:resultTime>
      <om:procedure xlink:href="urn:osh:sensor:simweather:0123456879"/>
      <om:observedProperty xlink:href="http://sensorml.com/ont/swe/property/Weather"/>
      <om:featureOfInterest xlink:href="http://www.opengis.net/def/nil/OGC/0/unknown"/>
      <om:result xsi:type="swe:DataRecordPropertyType">
        <swe:DataRecord definition="http://sensorml.com/ont/swe/property/Weather">
          <swe:description>Weather measurements</swe:description>
          <swe:field name="time">
            <swe:Time definition="http://www.opengis.net/def/property/OGC/0/SamplingTime" referenceFrame="http://www.opengis.net/def/trs/BIPM/0/UTC">
              <swe:label>Sampling Time</swe:label>
              <swe:uom href="http://www.opengis.net/def/uom/ISO-8601/0/Gregorian"/>
              <swe:value>2016-08-11T19:58:00.629Z</swe:value>
            </swe:Time>
          </swe:field>
          <swe:field name="temperature">
            <swe:Quantity definition="http://sensorml.com/ont/swe/property/AirTemperature">
              <swe:label>Air Temperature</swe:label>
              <swe:uom code="Cel"/>
              <swe:value>19.451170409324014</swe:value>
            </swe:Quantity>
          </swe:field>
          <swe:field name="pressure">
            <swe:Quantity definition="http://sensorml.com/ont/swe/property/AtmosphericPressure">
              <swe:label>Air Pressure</swe:label>
              <swe:uom code="hPa"/>
              <swe:value>1012.6821277249093</swe:value>
            </swe:Quantity>
          </swe:field>
          <swe:field name="windSpeed">
            <swe:Quantity definition="http://sensorml.com/ont/swe/property/WindSpeed">
              <swe:label>Wind Speed</swe:label>
              <swe:uom code="m/s"/>
              <swe:value>5.697366000667501</swe:value>
            </swe:Quantity>
          </swe:field>
          <swe:field name="windDirection">
            <swe:Quantity axisID="z" definition="http://sensorml.com/ont/swe/property/WindDirection" referenceFrame="http://sensorml.com/ont/swe/property/NED">
              <swe:label>Wind Direction</swe:label>
              <swe:uom code="deg"/>
              <swe:value>6.327918472858433</swe:value>
            </swe:Quantity>
          </swe:field>
        </swe:DataRecord>
      </om:result>
    </om:OM_Observation>
  </sos:observationData>
   ...
</sos:GetObservationResponse>

Very verbose!! Only appropriate for low rate sensors
(e.g. manually collected and analyzed samples)

Request/Response Examples

2016-08-11T19:58:00.629Z,19.451170409324014,1012.6821277249093,5.697366000667501,6.327918472858433
2016-08-11T19:58:01.629Z,19.657246631929652,1012.4822950871114,5.7507965859145,7.16403954199059
2016-08-11T19:58:02.629Z,19.641315211943287,1012.4430981205993,5.7426506120262575,6.483612829957986
2016-08-11T19:58:03.629Z,19.67716030103642,1012.4233384261645,5.766215960424121,5.668539726608906
2016-08-11T19:58:04.629Z,19.77713623280296,1012.4295783567908,5.78218926738179,6.4296613863899585
2016-08-11T19:58:05.629Z,19.87019300799815,1012.2924689802721,5.550148577113433,5.760477002404333
2016-08-11T19:58:06.629Z,19.78374137891352,1012.458532498272,5.635861416693368,6.515786920245821
2016-08-11T19:58:07.630Z,19.857491534092954,1012.3833536278792,5.5778941298629885,6.23318892372243
2016-08-11T19:58:08.629Z,19.880834642022297,1012.3347236787683,5.5833179014474394,6.761233439049914
2016-08-11T19:58:09.630Z,19.96657560107393,1012.3442809268807,5.578685735965688,6.0746296595311735
2016-08-11T19:58:10.629Z,19.812269865904433,1012.2952561292434,5.6146196221524685,6.602298214661535
2016-08-11T19:58:11.629Z,19.763686764682753,1012.351516315173,5.574592113056894,5.824571134524893
2016-08-11T19:58:12.629Z,19.56796732358099,1012.3936907108588,5.369304626290556,5.8093195294873095
2016-08-11T19:58:13.629Z,19.592950737899066,1012.3611736527021,5.355685855690663,5.583697364828362
2016-08-11T19:58:14.629Z,19.381197714569506,1012.4447822777427,5.337583607112957,6.441275015952943
2016-08-11T19:58:15.630Z,19.495553468821786,1012.4434256572165,5.447823346195531,7.186009714390203
2016-08-11T19:58:16.629Z,19.483919507009833,1012.2000727656485,5.459279865512013,7.641126821070655
2016-08-11T19:58:17.630Z,19.494912027035436,1012.2353353722979,5.616524356525373,6.766164792139883

So we like that better!

Request/Response Examples

<sos:GetResultTemplateResponse>
    <sos:resultStructure>
        <swe:DataRecord definition="http://sensorml.com/ont/swe/property/Weather">
            <swe:description>Weather measurements</swe:description>
            <swe:field name="time">
                <swe:Time
                    definition="http://www.opengis.net/def/property/OGC/0/SamplingTime" referenceFrame="http://www.opengis.net/def/trs/BIPM/0/UTC">
                    <swe:label>Sampling Time</swe:label>
                    <swe:uom xlink:href="http://www.opengis.net/def/uom/ISO-8601/0/Gregorian"/>
                </swe:Time>
            </swe:field>
            <swe:field name="temperature">
                <swe:Quantity definition="http://sensorml.com/ont/swe/property/AirTemperature">
                    <swe:label>Air Temperature</swe:label>
                    <swe:uom code="Cel"/>
                </swe:Quantity>
            </swe:field>
            <swe:field name="pressure">
                <swe:Quantity definition="http://sensorml.com/ont/swe/property/AtmosphericPressure">
                    <swe:label>Air Pressure</swe:label>
                    <swe:uom code="hPa"/>
                </swe:Quantity>
            </swe:field>
            <swe:field name="windSpeed">
                <swe:Quantity definition="http://sensorml.com/ont/swe/property/WindSpeed">
                    <swe:label>Wind Speed</swe:label>
                    <swe:uom code="m/s"/>
                </swe:Quantity>
            </swe:field>
            <swe:field name="windDirection">
                <swe:Quantity axisID="z"
                    definition="http://sensorml.com/ont/swe/property/WindDirection" referenceFrame="http://sensorml.com/ont/swe/property/NED">
                    <swe:label>Wind Direction</swe:label>
                    <swe:uom code="deg"/>
                </swe:Quantity>
            </swe:field>
        </swe:DataRecord>
    </sos:resultStructure>
    <sos:resultEncoding>
        <swe:TextEncoding blockSeparator="
"
            collapseWhiteSpaces="true" decimalSeparator="." tokenSeparator=","/>
    </sos:resultEncoding>
</sos:GetResultTemplateResponse>

And you get the metadata only once with:

OSH Tutorial - Part 4: Publishing Data

By Alex Robin

OSH Tutorial - Part 4: Publishing Data

Principles of data publication through OSH services

  • 422