X - frame
An attempt on frame interpolation with GAN
∞
\infin
What is frame interpolation?
A brief review on frames

Image from Lecture slides



α
\alpha
1−α
1-\alpha
Generate in-between frames
Synthetic video dataset
We have this claw and ball...
Sample Frames



α=0.7
\alpha = 0.7
α=0.6
\alpha = 0.6
α=0.2
\alpha = 0.2
α=0.9
\alpha = 0.9
WGAN-GP with U-Net

WGAN-GP with U-Net





Results
Start_Frame GT Predicted End_Frame
Forced learning?
Generator Pyramid Loss
...
s
real
...
s
fake
Discriminator
py_gen
Results
Start_Frame GT Predicted End_Frame




Discriminator Pyramid Loss
...
s
B-frame
...
s
A-frame
Discriminator
α
1-α
...
s
M-frame
py_dis
Results
Start_Frame GT Predicted End_Frame




The real stuff
movie <Zathura>
Sample Frames



α=0.94
\alpha = 0.94
α=0.40
\alpha = 0.40
α=0.55
\alpha = 0.55






WGAN-GP with U-Net

WGAN-GP with U-Net

Results




Start_Frame GT Predicted End_Frame








Conclusion
- Forced learning may be helpful at times
- Phase interpolation is learnable by a simple GAN
- Balanced WGAN loss + L2 loss
X - frame An attempt on frame interpolation with GAN ∞ \infin
Inf-Frames
By james01
Inf-Frames
frame interpolation with pyramid structured GAN
- 981