Video Communication Demo
空拍影像偵測移動物體
- A. Hafiane, K. Palaniappan and G. Seetharaman, "UAV-Video Registration Using Block-Based Features," IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium, Boston, MA, 2008, pp. II-1104-II-1107.
- Q. Wei, S. Lao and L. Bai, "Panorama Stitching, Moving Object Detection and Tracking in UAV Videos," 2017 International Conference on Vision, Image and Signal Processing (ICVISP), Osaka, 2017, pp. 46-50.
Referenced
Papers
- Problem definitions
- UAV Streaming
- Find homography
- Algorithms
- Find feature points
- Match feature points
- Estimate homography
- Detect targets
- Experimental results
- PSNR plots
- Moving object detections
- Discussions
- True rejects from FLANN
- False accepts from differences
- Could be faster
Detailed
Slide MAps
Problem
Definitions
Situations:
UAV Streaming
SIMPLIFy:
Find Homography
homography
Algorithms
Steps
- Find feature points
- SIFT
- Beltrami
- Match feature points
- FLANN
- Motion vectors
- Estimate homography
- RANSAC + DLT
- Detect targets
- Difference methods
。
。
。
。
。
。
Step 1:
Find Feature Points
SIFT
Beltrami
R=\text{tr}(\mathrm{H})^2/\det(\mathrm{H}),\mathrm{H}=\begin{bmatrix}
\frac{\partial^2I}{\partial x^2}&
\frac{\partial^2I}{\partial x\partial y}\\
\frac{\partial^2I}{\partial x\partial y}&
\frac{\partial^2I}{\partial y^2}
\end{bmatrix}
R=tr(H)2/det(H),H=[∂x2∂2I∂x∂y∂2I∂x∂y∂2I∂y2∂2I]
B=1 + \text{tr}(\mathrm{A})+\det(\mathrm{A}),\mathrm{A}=\begin{bmatrix}
\big(\frac{\partial I}{\partial x}\big)^2&
\frac{\partial I}{\partial x}\frac{\partial I}{\partial y}\\
\frac{\partial I}{\partial x}\frac{\partial I}{\partial y}&
\big(\frac{\partial I}{\partial y}\big)^2
\end{bmatrix}
B=1+tr(A)+det(A),A=[(∂x∂I)2∂x∂I∂y∂I∂x∂I∂y∂I(∂y∂I)2]
Step 1:
Find Feature Points
Beltrami + Prominent feature blocks
。
。
。
。
。
。
。
。
。
。
Step 2:
Match Feature Points
Motion vectors
FLANN
- priority search k-means tree
- multiple randomized k-d trees
- minimal SAD
- maximal NCC
\displaystyle\arg\max_{\Delta X}\bigg|
\dfrac{\mathrm{W}_{X+\Delta X}\cdot\mathrm{W}_X}
{\big\|\mathrm{W}_{X+\Delta X}\big\|\big\|\mathrm{W}_X\big\|}\bigg|
argΔXmax∣∣∣∣∥∥WX+ΔX∥∥∥∥WX∥∥WX+ΔX⋅WX∣∣∣∣
\displaystyle\arg\min_{\Delta X}\sum|\mathrm{W}_{X+\Delta X}-\mathrm{W}_X|
argΔXmin∑∣WX+ΔX−WX∣
Step 3:
Estimate Homography
\displaystyle
\mathrm{\hat{h}_\Omega}=\arg\min_\mathrm{h}\|\mathrm{A_\Omega h}\|^2\quad\text{w.r.t.}~\|\mathrm{h}\|=1
h^Ω=arghmin∥AΩh∥2w.r.t. ∥h∥=1
RANSAC + DLT
\mathrm{A}_\Omega=\begin{bmatrix}
0_{1\times3}&-\mathrm{a}_1^\intercal&y'_1\mathrm{a}_1^\intercal\\
\mathrm{a}_1^\intercal&0_{1\times3}&-x'_1\mathrm{a}_1^\intercal\\
\vdots&\vdots&\vdots
\end{bmatrix},
\mathrm{a}_i=\begin{bmatrix}
x_i\\y_i\\\mathrm{1}
\end{bmatrix},
\mathrm{h}=\begin{bmatrix}
\mathrm{H_1}\\\mathrm{H_2}\\\mathrm{H_3}
\end{bmatrix}
AΩ=⎣⎡01×3a1⊺⋮−a1⊺01×3⋮y1′a1⊺−x1′a1⊺⋮⎦⎤,ai=⎣⎡xiyi1⎦⎤,h=⎣⎡H1H2H3⎦⎤
\displaystyle
\hat{\Omega}=\arg_\Omega\dfrac{|\Omega|}{N}>\gamma
\quad\text{w.r.t.}~\forall\mathrm{a}_i\in\Omega.~d(\mathrm{a}'_i,\mathrm{\hat{H}_\Omega\mathrm{a}}_i)<\epsilon
Ω^=argΩN∣Ω∣>γw.r.t. ∀ai∈Ω. d(ai′,H^Ωai)<ϵ
Step 3:
Estimate Homography
RANSAC + DLT
。
。
。
。
。
。
\quad,\quad\in\hat\Omega, \quad\notin\hat\Omega
,∈Ω^,∉Ω^
\mathrm{a}_1
a1
\mathrm{a}_2
a2
\mathrm{a}_3
a3
\quad=\mathrm{\hat{H}}\quad
=H^
\mathrm{a}'_2
a2′
\mathrm{a}_2
a2
\quad=\mathrm{\hat{H}}\quad
=H^
\mathrm{a}'_3
a3′
\mathrm{a}_3
a3
Step 4:
Detect Targets
Difference methods
\begin{aligned}
=&\\\land&
\end{aligned}
=∧
(D_{13}\lor D_{35})
(D13∨D35)
(D_{23}\lor D_{34})
(D23∨D34)
D
D
Experimental
Results
Demo Video
PSNR &
Speeds
SIFT+FLANN | Block-based | with FFmpeg |
---|---|---|
19.22 dB | 19.07 dB | 19.47 dB |
8.43 fps | 1.20 fps | 3.01 fps |
Detections
1
2
3
Discussions
True Rejects
From RANSAC
\displaystyle
\hat{\Omega}=\arg_\Omega\dfrac{|\Omega|}{N}>\gamma
\quad\text{w.r.t.}~\forall\mathrm{a}_i\in\Omega.
Ω^=argΩN∣Ω∣>γw.r.t. ∀ai∈Ω.
d(\mathrm{a}'_i,\mathrm{\hat{H}_\Omega\mathrm{a}}_i)<\epsilon
d(ai′,H^Ωai)<ϵ
False Accepts
From Differences
Solution to Errors
- As-projective-as-Possible
- Track from unreliable detections
Filter outliers
Nonlinear transform
p=0.9
p=0.9
p=0.1
p=0.1
Insufficient Speed
╔═════ main.py ═════╗ ▶ frame ╾─┐ ║ ║ ├──╫───╼ parser.py ║ ▶ MVs ╾─┘ ║ ┊ ║ (1-thread) ║ homographier.py ║ ║ ┊ ║ ║ detector.py ╾──╫──╼ output ▶ ║ ║ ╚═══════════════════╝
Insufficient Speed
- Total time
-
[14%] Read frames and motion vectors
- [48%] Parse motion vector info
-
[76%] Find homography
- [78%] Find unknown motion vectors
- [9%] Detect moving objects
-
[14%] Read frames and motion vectors
$ python3 -m cProfile main.py
Video Communication Demo
By RedBug312
Video Communication Demo
Credit: All the high-quality icons are made by Freepik from www.flaticon.com
- 1,419