Marker-based Inside-out Tracking for Medical Applications
Using a Single Optical Camera 

Student: Chen-Hsuan, Shih (Iris)

Supervisors: Philipp Matthies, Benjamin Frisch 

Master of Biomedical Computing
Computer Aided Medical Procedure(CAMP),
Technical University of Munich, Germany

What? current status

            Why? the improvement is needed 

How? the technology

GOAL...

1

Motivation

doctor

patient

Outside-in Tracking

2

doctor

patient

nurse

line of sight problem ! -> Miss Tracking

Motivation

3

doctor

patient

nurse

Motivation

4

Inside-out Tracking

• Literature research

• Tracking Algorithm

• Evaluation

• OpenIGTLink & CAMPVis

• ​Integration

Road Map

5

Road Map

• Literature research

• Tracking Algorithm

• Evaluation

• OpenIGTLink & CAMPVis

• ​Integration

Wendler, T. 2007, Real-time fusion of ultrasound and gamma probe for navigated localization of liver metastases, MICCAI 

Vincent L. 2008, EPnP: An Accurate O(n) Solution to the PnP Problem, Springer Science+Business Media LLC

 

      more...

Olson, E. 2011, AprilTag: A robust and flexible visual fiducial system, ICRA

6

Olson, E., AprilTag: A robust and flexible visual fiducial system, ICRA, 2011

7

State of the Art Tracking - AprilTag

State of the Art Tracking - AprilTag

Advantages:

    Fast & Robustness in 

• Occlusion

• Wrapping 

• Lens Distorsion 

Olson, E., AprilTag: A robust and flexible visual fiducial system, ICRA, 2011

8

Marker-based tracking setup

Extract information from tracking

Multi-markers tracking model

Test video acquisition with different setting 

Get the detector transformation matrix

Road Map

• Literature research

• Tracking Algorithm

• Evaluation

• OpenIGTLink & CAMPVis

• ​Integration

9

Technical Setup

GoPro Camera

Gamma Camera

Calibrator 

3D Marker Cube

10

Gamma Camera

Input : Images 

Output : 4x4 Transformation Matrix

AprilTags Tracking Alg.  +  Multi-markers Model

Tracking System

11

?

Tracking Transformation T1
camera <-- marker cube  

Tracking System

GoPro Camera

Gamma Camera

3D Marker Cube

12

Tracking Transformation T2
camera --> calibrator  

Tracking System

GoPro Camera

Gamma Camera

T1

3D Marker Cube

13

GoPro Camera

Gamma Camera

Tracking Transformation T3  calibrator --> Detector   

T1

T2

Tracking System

3D Marker Cube

14

GoPro Camera

Gamma Camera

T3 

T1

T2

T3   x

T1

T2  x

Tdetector  =

3D Marker Cube

Tracking System

15

Algorithm Accuracy

Mechanical tracking as ground truth   

Comparison with different Tracking Models

Road Map

• Literature research

• Tracking Algorithm

• Evaluation

• OpenIGTLink & CAMPVis

• ​Integration

16

Applied Multi-Markers Model

17

Translation Extracted from Tracking Alg.

19

Communication through OpenIGTLink Protocol
Server <--> Client

Road Map

• Literature research

• Tracking Algorithm

• Evaluation

• OpenIGTLink & CAMPVis

• ​Integration

20

OpenIGTLink

http://openigtlink.org/

My Server

ReceiverClient

Send transformation matrices!

Connect to my server

21

OpenIGTLink

http://openigtlink.org/

My Server

ReceiverClient

Send transformation matrices!

Connect to my server

21

Receive my data :
Transformation matrices  

Inside-Out Tracking + 

Gamma Camera 2D

Ultrasound 2D, 3D

Gamma Camera + SPECT

Road Map

• Literature research

• Tracking Algorithm

• Evaluation

• OpenIGTLink & CAMPVis

• ​Integration

22

Inside-Out Tracking + 

Gamma Camera 2D

Ultrasound 2D, 3D

Gamma Camera + SPECT

Next Step...

• Literature research

• Tracking Algorithm

• Evaluation

• OpenIGTLink & CAMPVis

• ​Integration

Mechanical tracking as ground truth   

Comparison with different Tracking Models

23

Thank You for Your Attention!