Python implementation of Typhoon algorithm: dense estimation of 2D-3D optical flow on wavelet bases.
Python implementation of Typhoon motion estimator: dense estimation of 2D/3D optical flow on wavelet bases, primarily aimed at fluid motion estimation.
At the moment, the wavelet-based data DFD term (Dérian et al., 2013) only is provided: the high-order regularizers (Kadri-Harouna et al., 2013) are not included in this implementation.
The reference implementation used in (Dérian et al., 2015) and (Dérian et al., 2017) is written in C++ and GPU-accelerated with CUDA, and contains the high-order regularizers. It is the property of Inria (FR) and the CSU Chico Research
Foundation (Ca, USA), and can be licensed from these institutions.
This Python implementation is not the same as the
reference for many reasons, and it is obviously much slower.
Tested with Anaconda Python 3.6.1, Numpy 1.12.1, Scipy 0.19.1, PyWavelet 0.5.2.
The Typhoon class can be imported from other modules/scripts to perform estimations as needed.
The script can also work as a standalone estimator in simple cases, e.g.:
python pytyphoon.py -i0 path/to/im0.jpg -i1 path/to/im1.jpg -wav 'db3' --display
will solve the problem for image pair (im0.jpg, im1.jpg) and wavelet Daubechies-3.
See python pytyphoon.py -h for the complete list of parameters.
Typhoon solves a dense variational optical flow problem, that is to say: (i) it provides one motion vector at every pixel of input images and (ii) it estimates the entire vector field altogether.
To do so, it looks for the motion field which minimizes the displaced frame difference (DFD):
This is achieved by minimizing the following functional:
where the integral is taken over the image.
The functional above is non-linear with respect to the motion field.
This has the advantage of better handling large displacement, but complicates the minimization process.
For the non-linear minimization to succeed, the solution should lie reasonable “close” to the first guess. This is where wavelets bases come into play: by providing a multiscale representation of the motion field, they enable to estimate the motion iteratively from its coarsest scales to the finests.
The minimization is handled by L-BFGS, which is efficient memory-wise and only requires the functional value and its gradient.
These demos are shipped with the project.
Simple 2d estimation using synthetic particle images (256x256 pixels) originally created for the FLUID project (image database #1). Run:
python pytyphoon.py --demo particles
Simple 3d estimation using synthetic images (64x64x64 pixels) obtained by filtering random normal noise at various scales. The displacements are integer shifts along each of the 3 dimensions. Run:
python pytyphoon.py --demo 3dshift
Simple 3d estimation using synthetic images (96x96x96 pixels) obtained by filtering random normal noise at various scales. The displacement field is a column vortex (first two axes) with an updraft increasing linearly along the third axis. Run:
python pytyphoon.py --demo 3dvortex
