Rob Purser
Development Manager for IoT and Hardware Interfacing for MATLAB at MathWorks. Leads the creation of software that connects MATLAB with the real world
Live Version at http://slides.com/rpurser/integrating_analytics#/
Lots Collected, Little Analyzed
- Collection without analysis has negative ROI
Most IoT data are not used currently. For example, only 1 percent of data from an oil rig with 30,000 sensors is examined.
-- McKinsey & Company
- As difficult as it is to collect and capture data,
gaining insights via analysis takes just as much effort
- Dependable, real-world on-line analytics take time to develop, tune, and test.
Not a Product Pitch...but
MATLAB: High-level data analytic language and interactive environment for data analysis, visualization, and algorithm development
Simulink: Block diagram environment for simulation and design. Simulate, generate code, and verify embedded systems
ThingSpeak: Analytic IoT Platform for storing sensor data and developing IoT applications
- Online analysis and visualization using the MATLAB Language
- Sensor data can be sent to ThingSpeak from any internet capable device using simple REST calls over HTTP or HTTPS.
ThingSpeak.com
- Running for more than 6 years
- >20,000 active users worldwide
- Designed for engineers and scientists developing analytic systems
- Free for pilot projects
- Leverages MATLAB skills in your organization
- Lots of examples and projects
A Project Focused Tour
Tide Prediction - Online Analytics
Nighttime Noise - Historical Analytics
Weather - IoT Analytic Workflow
Counting Cars - Edge Node Analytics
A Project Focused Tour
Tide Prediction - Online Analytics
Nighttime Noise - Historical Analytics
Weather - IoT Analytic Workflow
Counting Cars - Edge Node Analytics
Weather - IoT Analytic Workflow
MathWorks Weather Station -- Revisited
Classic IoT Maker Project with Machine Learning
Deep Dive: Google "mathworks weather revisit"
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
1
2
3
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
Weather - IoT Analytic Workflow
A Project Focused Tour
Tide Prediction - Online Analytics
Nighttime Noise - Historical Analytics
Weather - IoT Analytic Workflow
Counting Cars - Edge Node Analytics
Car Counter - Edge Node Analytics
Counting Cars
Traffic monitor using a Raspberry Pi & Webcam
Deep Dive: Google "count cars thingspeak"
Car Counter - Edge Node Analytics
Counting Cars
Traffic monitor using a Raspberry Pi & Webcam
Deep Dive: Google "count cars thingspeak"
Car Counter - Edge Node Analytics
Embedded devices have always used:
- Signal processing
- Matrix math
- Closed loop control
Powerful mobile processors enable:
- Sensor fusion & analytics
- Computer vision & image processing
- Trained machine learning
- Statistical modeling
- Small vocabulary voice recognition
Car Counter - Edge Node Analytics
Foreground Detection, 2-D Median Filter, Blob Analysis
Feeds custom counting block and Transmit to ThingSpeak
Car Counter - Edge Node Analytics
Foreground Detection, 2-D Median Filter, Blob Analysis
Feeds custom counting block and Transmit to ThingSpeak
Car Counter - Edge Node Analytics
Foreground Detection, 2-D Median Filter, Blob Analysis
Feeds custom counting block and Transmit to ThingSpeak
Car Counter - Edge Node Analytics
Foreground Detection, 2-D Median Filter, Blob Analysis
Feeds custom counting block and Transmit to ThingSpeak
Car Counter - Edge Node Analytics
Foreground Detection, 2-D Median Filter, Blob Analysis
Feeds custom counting block and Transmit to ThingSpeak
Car Counter - Edge Node Analytics
Embedded devices now have enough compute power to use algorithms that previously were once only practical on desktop class machines or FPGAs
Car Counter - Edge Node Analytics
A Project Focused Tour
Tide Prediction - Online Analytics
Nighttime Noise - Historical Analytics
Weather - IoT Analytic Workflow
Counting Cars - Edge Node Analytics
Night Noise - Exploratory Analysis
Night Noise Analysis
Identify unusual changes in ambient noise level at night
Night Noise - Exploratory Analysis
Median Filter
Night Noise - Exploratory Analysis
Overlaid with Expected Levels
Determined expected based on median at a given time
Night Noise - Exploratory Analysis
Normalized based on first 3 hours
Polynomial fit of expected noise levels
Night Noise - Exploratory Analysis
Set Expected Levels for a given night
Move polynomial up or down to compensate for variation
Night Noise - Exploratory Analysis
Identify alarm conditions
> 1 std deviation above expected
Night Noise - Exploratory Analysis
To do: Investigate what happened this night...
Night Noise - Exploratory Analysis
Things to keep in mind:
- You haven't seen all the scenarios
- Always consider how you will convert to on-line analytics
- Pilot on-line analytics and alarms
- One false alarm will destroy trust
Night Noise - Exploratory Analysis
A Project Focused Tour
Tide Prediction - Online Analytics
Nighttime Noise - Historical Analytics
Weather - IoT Analytic Workflow
Counting Cars - Edge Node Analytics
Tide Prediction - Online Analytics
On Line Tide Alerts
Tide Measurement and Prediction
Deep Dive: Google "tide matlab thingspeak"
Tide Prediction - Online Analytics
Source: SUNY StoneyBrook
23 astronomical components in tidal harmonics...
plus geography and weather -- it's always an approximation!
Tide Prediction - Online Analytics
Tidal variation between bays 6 miles apart
Tide Prediction - Online Analytics
Edge Node
- Arduino
- Ultrasonic Rangefinder and Temperature Sensor
- Data Reduction
- De-noising and "De-batting"
- Sends Data to ThingSpeak
Tide Prediction - Online Analytics
On-Line Analysis
- Median Filter
- Temperature Compensation
- Convert to Water Depth
- Tweet @tidealerts
- Hysteresis and state machine to avoid being too "tweekative"
Tide Prediction - Online Analytics
On-Line Analysis
- Median Filter
- Temperature Compensation
- Convert to Water Depth
- Tweet @tidealerts
- Hysteresis and state machine to avoid being too "tweekative"
Tide Prediction - Online Analytics
Thank You!
- Collection without analysis has negative ROI
- Analysis happens at every stage
- Dependable, real-world on-line analytics take time to develop, tune, and test
- Trust can be lost in a single momentRob Purser -- rob.purser@mathworks.com, @rpurser47
http://slides.com/rpurser/environmental_analysis
Integrating Analytics and Algorithms into your IoT System
By Rob Purser
Integrating Analytics and Algorithms into your IoT System
In this presentation, we will discuss three kinds of analytics common in IoT systems: sensor analytics at the edge node for data reduction, cloud-based online analytics for situational awareness, and historical data analytics for developing predictive and classification algorithms. You’ll see practical techniques for integrating all these types of analytics into your IoT system. We will demonstrate these techniques using various environmental data sources, including data from weather stations, noise monitors, tide gauges, and traffic sensors. These examples utilize embedded programming, image and signal processing, and machine learning techniques to demonstrate how analytics can be integrated at all phases in your IoT system.
