Automatic Large Scale Detection of Red Palm Weevil Infestation using Street View Images

Dima Kagan, Galit Fuhrmann Alpert, Michael Fire

ISPRS Journal of Photogrammetry and Remote Sensing

Dima Kagan

Red Palm Weevil

South American Palm Weevil

Palm Weevils

Palm Weevil is a type of beetle that infests palm trees, lays eggs and feeds on  the palm's tissue, creating tunnels inside the tree trunk that weaken its structure, causing extensive damage resulting in tree decline and eventual breakage. The beatles can travel up to 50 km a day, rapidly spreading geographically and raising tremendous risks to spatially widespread locations.

The Problem

• Today, the Red Palm Weevil has spread to 85 different countries and regions worldwide.
   
• In fact, the Food and Agriculture Organization of the United Nations estimates that in 2023, the combined cost of pest management and replacement of damaged palm trees, in Spain and Italy alone, will reach 200 million Euro.

Photo taken by Omer Tsur - AirWorks aerial photography

Palm Weevils Life Cycle

How Does It Look

How Does It Look

Existing Solutions

Publicly Available Data + Computer Vision

●Applicable at large scale.

●Relatively affordable.

●Can be used on any point in the world.

●Can be used anywhere especially in urban areas.

Street View

• Provide 360 degree images.

• Contain historical data.

• World wide availability.

• Many vendors:

  • Google

  • Microsoft

  • Yandex

  • Apple

  • Kartaview

  • etc.

Google Street View

• 0.007 USD per each (7.00 USD per 1000).

• Each image contains up to 120 degrees out of the panorama.

• Max image size is 640x640.

• We estimate that fully mapping of San Diego area using only street view images should cost around 51,000$.

Aerial Imagery

• Cheaper, sometimes even free.
• Show larger areas than street view.

Aerial Palm Detection

Faster R-CNN model was trained to detect palm trees for aerial imagery with mAP of 0.5.

Palm Tree Detection & Classification

Faster R-CNN (mAP 0.9)

Palm Tree Detection & Classification

• XResNet model was trained to detect infested palm trees.
• Initially, Binary Cross Entropy loss was used to filter unknown labels. However, it did not perform as well as accepted.
• A new "Unkown" class was added to the training data.
• As for out-of-domain data:
  • We manually added street view data.
  • Under-sampled uniformly 8 images of each class from Caltech 101.  
• The infested palm tree data was oversampled from 70 to 892.
• The palm tree health classifier achieved an F1 score of 0.84, precision of 0.83, recall of 0.85, and AUC of 0.948.

Street VS Aerial Mapping

• 546 aerial images.

• 756 aerial detected palm trees.

• 4,544 street level images.

• 72% match between aerial images and street level detected palm trees.

Normal Heights Village, San Diego

Case Study  

• Has a Palm Weevil problem.

• Has ground truth data of infested palm trees (reports).

• Has historical data in Google Street View.

Analysis

• Scanning the area of the ground truth data we detected 5,008 trees.
• Our classifier classified five of the six ground truth trees as infested.
• Additionally, out of 5,008 detected palms, in the classifier detected additional 13 infested palm trees, of which we identified eight at advanced infestation stages.

Searching on a Large Scale

22,438 aerial images

54,781 street view images

 36,001 palm trees

109 were classified as infested

24 were infested at an advanced stage

Jun 2014

Nov 2017

April 2018

April 2019

Jun 2014

Nov 2017

April 2018

April 2019

Jun 2014

Nov 2017

April 2018

April 2019

Jun 2014

Nov 2017

April 2018

April 2019

Limitations & Challenges

• Different species of palms trees has different symptoms.
• Fertilizer may delay the visual appearance of the symptoms.
• Aerial images are not taken at the same time that of street view images.
• Street view images are not taken at constant time intervals, and only at sporadic time points.
• When classifying vegetation there may be external or seasonal variables that can affect how the plant looks and as result influence the classifier.
• The current method and data do not support the detection of borderline cases or detecting the severity of the infestation.

Future Directions

Future Directions

Future Directions

Future Directions

Thank you questions?