X - frame
An attempt on frame interpolation with GAN
\infin
What is frame interpolation?
A brief review on frames
Image from Lecture slides
\alpha
1-\alpha
Generate in-between frames
Synthetic video dataset
We have this claw and ball...
Sample Frames
\alpha = 0.7
\alpha = 0.6
\alpha = 0.2
\alpha = 0.9
WGAN-GP with U-Net
WGAN-GP with U-Net
Results
Start_Frame GT Predicted End_Frame
Forced learning?
Generator Pyramid Loss
...
s
real
...
s
fake
Discriminator
py_gen
Results
Start_Frame GT Predicted End_Frame
Discriminator Pyramid Loss
...
s
B-frame
...
s
A-frame
Discriminator
α
1-α
...
s
M-frame
py_dis
Results
Start_Frame GT Predicted End_Frame
The real stuff
movie <Zathura>
Sample Frames
\alpha = 0.94
\alpha = 0.40
\alpha = 0.55
WGAN-GP with U-Net
WGAN-GP with U-Net
Results
Start_Frame GT Predicted End_Frame
Conclusion
- Forced learning may be helpful at times
- Phase interpolation is learnable by a simple GAN
- Balanced WGAN loss + L2 loss
Inf-Frames
By Arvin Liu
