Research Assistant - Digital Therapy, Data Analytics & Mobile Computing - CSS Health Lab.

Applicant: Chen-Hsuan (Iris), Shih 

Iris Shih 

iris.cshih@gmail.com

Iris Shih 

iris.cshih@gmail.com

 @ TUM

Research Project 

Marker-based Inside-out Tracking for Medical Applications
Using a Single Optical Camera 

Student: Chen-Hsuan, Shih (Iris)

Supervisor: Prof. Dr. Nassir Navab

Master of Biomedical Computing
Computer Aided Medical Procedure(CAMP),
Technical University of Munich, Germany

Advisor: Philipp Matthies

Motivation

Motivation

Outside-in Tracking

2

doctor

patient

nurse

line of sight problem ! -> Miss Tracking

Motivation

3

doctor

patient

doctor

patient

nurse

Motivation

4

Inside-out Tracking

• State-of-The-Art

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Evaluation

• Discussion 

• Future Works 

Road Map

5

Road Map

6

• State-of-The-Art

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Evaluation

• Discussion 

• Future Works 

State of the art tracking technology

Inside-out Tracking

Similar application for solving line-of-sight

State-of-the-Art: Marker-Based Tracking 

Really Robustness in Occlusion ,Wrapping , Lens Distorsion 

[1]Olson, E., AprilTag: A robust and flexible visual fiducial system, ICRA, 2011

[2] S. Garrido-Jurado, Automatic generation and detection of highly reliable fiducial markers under occlusion, Pattern Recognition, 2014

7

AprilTags [1]

ArUco [2]

Inside-out Tracking Application 

8

ClearGuide [4]

Panorama ultrasound for

[4]Clear Guide Medical. http://clearguidemedical.com/ (accessed 3/6/2015).

[3] Hedyeh Rafii-Tari. Panorama ultrasound for guiding epidural anesthesia: a feasibility study. IPCAI’11 2011

guiding epidural anesthesia [3]

Marker-based tracking setup

Extract information from tracking

Multi-marker tracking model

Get the detector transformation matrix

Complete transformation for reconstruction

Road Map

9

• State-of-The-Art

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Evaluation

• Discussion 

• Future Works 

Gamma Camera

Tracking System

?

Calibrator 

patient

Image reconstruction

11

 Cube Target

Tracking Transformation T1
camera --> marker cube  

GoPro Camera

Gamma Camera

12

Tracking System

 Cube Target

GoPro Camera

Gamma Camera

T3 

T1

T2

T3   x

T1

T2  x

Ttarget = 

14

Tracking System

detector

-1

-1

 Cube Target

GoPro Camera

Gamma Camera

T3 

T1

T2

 Cube Target

15

Tracking System

-1

patient

Volume of interest

Ttarget

detector

GoPro Camera

Gamma Camera

T3 

T1

T2

T

 Cube Target

16

Tracking System

-1

patient

Volume of interest 

-1

Tdetector 

voi

Multi-Marker Model + ArUco Tracking Framework

(voi)

Normal 2D Marker Tracking - Individual Coordinate

17

Tracking Model

Applied Multi-Marker Model

18

Tracking Model

s = 1/2 marker size

c = 1/2 cube size

Translation Extracted from Tracking Alg. with 3D Multi-Marker Model

19

3 markers were tracked:

red: #marker 1

green: #maker 2

blue: #marker 3

Tracking Model

Road Map

21

Inside-out Tracking & Outside-in Tracking   

Reconstruction Tool

AR Visualization

• State-of-The-Art

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Evaluation

• Discussion

• Future Works 

Multi-Tracker-App.

22

Threshold Setting

Reconstruction Tool

GammaProbe Acquisition

Augmented Reality Visualization

1

3

2

Switch View

Multi-Tracker-App.

23

Threshold Setting

Reconstruction Tool

GammaProbe Acquisition

Augmented Reality Visualization

1

3

2

Road Map

24

Lymph node phantom 

Thyroid phantom

Breast cancer patient scan

• State-of-The-Art

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Evaluation

• Discussion

• Future Works 

System Setup - Breast lymph node phantom scan 

25

Inside-out Tracking

Outside-in Tracking (NDI Tracker)

System Setup - Breast lymph node phantom scan 

25

Two hollow spheres in a water-bath:

• volumes of 3:5ml & 0:6ml each

• filled with a 3MBq solution of 99mTc

System Setup - Thyroid phantom scan 

26

Inside-out Tracking

Outside-in Tracking (NDI Tracker)

System Setup - Thyroid phantom scan 

26

• simulate hot or cold nodules

Custom-printed thyroid phantom :

• contains 4 chambers of different sizes

• filled with radioactivity or water
  

Inside-out Workflow

27

Multi-Tracker-App.

28

1. System Setup

2. Gamma camera start acquisition

3. Switching view to make sure the tracking validation

Multi-Tracker-App.

28

1. System Setup

2. Gamma camera start acquisition

3. Switching view to make sure the tracking validation

4. Stop acquisition

5. Choose inside-out tracker info. for reconstruction 

6. Thresholding the visualization  

Phantom Experiment Result

30

Real distance: 5.5cm

Real distance: 3.3cm

Real distance: 5.5cm

Real distance: 3.3cm

Phantom Experiment Result

30

 Mean                                       5.45            5.75

 Mean                                3.22            3.26

Patient Experiment Result

31

32

Tracking Accuracy

ArUco (Guppy) vs. Outside-in (NDI)

Distance between two sets of tracking points

Marker cube limitation

33

Multi-Marker Model Accuracy

34

35

Road Map

• State-of-The-Art

• Tracking System

• Multi-Tracker-App.

• Experiment

• Results & Evaluation

• Discussion 

• Future Works 

Inside-out Tracking  vs. Outside-in Tracking 

Reconstruction  & AR

Discussion

Reduced Line-of-sight issue ! 

Reference target (marker cube) can be easily positioned & tracked!

Simple, Easy, Low cost system setup with comparable accuracy!

Patient & surgeon friendly

36

Discussion

How to deal: operator's handshake that leads to bad tracking?

What are the constrains for the reference target? Is it necessary?

Can the system auto-detect the error if happened during real scan?

Is the system reliably applicable in real clinical practice?

37

38

Road Map

• Literature research

• Tracking System

• Multi-Tracker-App.

• Experiment

• Evaluation & Results 

• Future Works 

Tracking Accuracy & Stabilization 

Mechanical Tracking as ground truth 

Next Steps...

Marker-based Tracking:

• Tracking Algorithm

  • Accuracy assurance & Standard deviation

  • Discard outliers : Error threshold

  • Compare to ground truth. (e.g. Mechanical tracking)

39

Iris Shih 

iris.cshih@gmail.com

So much we can do by knowing and applying 
the correct Technology!

Iris Shih 

iris.cshih@gmail.com

Why I'm here...

Computer Science 

Healthcare Problem 

Iris Shih 

iris.cshih@gmail.com

Why I'm here...

Computer Science 

Healthcare Problem 

• Diagnosis Improvement

• Treatment Improvement 

• Earlier Diseases Detection/Prediction

• more

"Change Health Behavior through Technology", Health-IS

Iris Shih 

iris.cshih@gmail.com

Thank you