How To Apply Computer Vision to Build an Emotion-Based Dog Filter in Python 3

Want an in-person tutorial with step-by-step walkthroughs and explanations? See the corresponding AirBnb experience for both beginner and experienced coders alike, at “Build a Dog Filter with Computer Vision”

This repository includes all source code for the tutorial on DigitalOcean with the same title, including:

• A real-time filter that adds dog masks to faces on a live feed.
• A dog filter that responds to your emotions. (Couldn’t find a pug mask, so I used a cat.) A generic dog for smiling “happy”, a dalmation for frowning “sad”, and a cat for dropped jaws “surprise”.
• Utilities used for portions of the tutorial, such as plotting and advesarial example generation.
• Simple convolutional neural network written in PyTorch, with pretrained model.
• Ordinary least squares and ridge regression models using randomized features.

created by Alvin Wan, December 2017

Getting Started

You can setup the repository using Python or view the web demo at dogfilter.alvinwan.com

For complete step-by-step instructions, see the tutorial on DigitalOcean. This codebase was developed and tested using Python 3.6. If you’re familiar with Python, then see the below to skip the tutorial and get started quickly:

(Optional) Setup a Python virtual environment with Python 3.6.

1. Install all Python dependencies.

pip install -r requirements.txt

1. Navigate into src.

cd src

1. Launch the script for an emotion-based dog filter:

python step_8_dog_emotion_mask.py

How it Works

See the below resources for explanations of related concepts:

• “Understanding Least Squares”
• “Understanding Neural Networks”

Acknowledgements

These models are trained on a Face Emotion Recognition (FER) dataset curated by Pierre-Luc Carrier and Aaron Courville in 2013, as published on Kaggle.