nurse

line of sight problem ! -> Miss Tracking

Motivation

3

doctor

patient

doctor

patient

nurse

Motivation

4

Inside-out Tracking

• Literature research

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Discussion 

• Future Works 

Road Map

5

Road Map

6

• Literature research

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Discussion 

• Future Works 

Inside-out Tracking

Similar application for solving line-of-sight

State of the art tracking technology

State-of-the-Art Tracking - AprilTag , Aruco

Really Robustness in Occlusion ,Wrapping , Lens Distorsion 

Olson, E., AprilTag: A robust and flexible visual fiducial system, ICRA, 2011

S. Garrido-Jurado, Automatic generation and detection of highly reliable fiducial markers under occlusion, Pattern Recognition, 2014

7

Marker-based tracking setup

Extract information from tracking

Multi-markers tracking model

Get the detector transformation matrix

Complete transformation for reconstruction

Road Map

8

• Literature research

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Discussion 

• Future Works 

Technical Setup

GoPro Camera

Gamma Camera

Calibrator 

3D Marker Cube

9

Gamma Camera

Tracking System

?

Calibrator 

patient

Image reconstruction

10

Aruco Tracking lib. +  Multi-markers Model

Tracking Transformation T1
camera --> marker cube  

GoPro Camera

Gamma Camera

11

Tracking System

 Cube Target

Tracking Transformation T2
camera --> calibrator  

GoPro Camera

Gamma Camera

12

Tracking System

T1

 Cube Target

GoPro Camera

Gamma Camera

Tracking Transformation T3  calibrator --> Detector   

T2

13

Tracking System

T1

 Cube Target

GoPro Camera

Gamma Camera

T3 

T1

T2

T3   x

T1

T2  x

Ttarget = 

14

Tracking System

detector

-1

-1

 Cube Target

GoPro Camera

Gamma Camera

T3 

T1

T2

 Cube Target

15

Tracking System

-1

patient

Volume of interest

Ttarget

detector

GoPro Camera

Gamma Camera

T3 

T1

T2

T

 Cube Target

16

Tracking System

-1

patient

Volume of interest 

-1

Tdetector 

voi

Aruco Tracking lib. +  Multi-markers Model

(voi)

Translation Extracted from Tracking Alg.

17

Tracking Model

Applied Multi-Markers Model

18

Tracking Model

Translation Extracted from Tracking Alg. + 3D Model

19

3 markers were tracked:

red: #marker 1

green: #maker 2

blue: #marker 3

Tracking Model

Road Map

20

Inside-out Tracking & Outside-in Tracking   

Reconstruction Tool

AR Visualization

• Literature research

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Discussion

• Future Works 

Multi-Tracker-App.

21

Multi-Tracker-App.

22

Threshold Setting

Reconstruction Tool

GammaProbe Acquisition

Augmented Reality Visualization

1

3

2

Switch View

Multi-Tracker-App.

23

Threshold Setting

Reconstruction Tool

GammaProbe Acquisition

Augmented Reality Visualization

1

3

2

Road Map

24

Lymph node phantom 

Thyroid phantom

Breast cancer patient scan

• Literature research

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Discussion

• Future Works 

System Setup - Breast lymph node phantom scan 

25

Inside-out Tracking

Outside-in Tracking (NDI Tracker)

System Setup - Thyroid phantom scan 

26

Inside-out Tracking

Outside-in Tracking (NDI Tracker)

Multi-Tracker-App.

27

1. System Setup

2. Gamma camera start acquisition

3. Switching view to make sure the tracking validation

Multi-Tracker-App.

27

1. System Setup

2. Gamma camera start acquisition

3. Switching view to make sure the tracking validation

4. Stop acquisition

5. Choose inside-out tracker info. for reconstruction 

6. Thresholding the visualization  

Phantom Experiment Result

29

Real distance: 5.5cm

Real distance: 3.3cm

Real distance: 5.5cm

Real distance: 3.3cm

Phantom Experiment Result

29

 Mean                                       5.45            5.75

 Mean                                3.22            3.26

Patient Experiment Result

30

Discussion

No Line-of-sight issue ! 

Reference target (marker cube) can be easily positioned & tracked!

Simple, Easy, Low cost system setup with equivalent accuracy!

Patient & surgeon friendly

31

Discussion

How to deal: operator's handshake that leads to loose tracking?

What are the constrains for the reference target (marker cube) ?

Can the system auto-detect the error if happened during real scan?

Does the system reliably applicable in real clinical practice?

32

33

Road Map

• Literature research

• Tracking System

• Multi-Tracker-App.

• Experiment

• Evaluation & Results 

• Future Works 

Tracking Accuracy & Stabilization 

Mechanical Tracking as ground truth! 

Next Steps...

Marker-based Tracking:

• Tracking Algorithm

  • Accuracy assurance & Standard measurement

  • Discard loosely tracking : Error threshold

  • Mechanical tracking

• Marker Cube Design

  • ​Size & Positioning 

  • Evaluation of the limitation

34

Next Steps...

Inside-out Tracking System:

• Hardware setup

  • Dedicated combined optical-/gamma-camera

  • Custom printed the marker cube

  • Dedicated holder for marker cube  

• Clinical workflow​

  • ​Clinical integration issue address

35

Thank You for Your Attention!