RiVAE

Real to Imaginary Variational Auto Encoder

The goal of this project is to convert real and virtual images into a general imaginary image. It is the search for a medium between the real and the virtual from a visual interpretation of the environment. This project aims to assist in tasks of deep reinforcement learning by enabling a virtual training and at the same time an interpretation of reality.

Setup

All of requirements is show in the badgets above, but if you want to install all of them, enter the repository and execute the following line of code:

pip3 install -r requirements.txt

With the videos from a virtual environment inside the data/video folder we can now do the process of image extraction from the video. To perform the extraction run the following code:

python3 image_extractor.py

With the extracted images, now we can train our RiVAE network, to do this just run the following code:

python3 rivae.py

Structure

The way RiVAE works is somewhat simple. We extract the images from a virtual video, then go through an auto encoder and calculate the error of the decoder output image with the ground truth image to update the weights. Something very similar to a conventional variational auto encoder, the difference here is in the focus of the project.

If you liked this repository, please don’t forget to starred it!