Signans - The Conversation Tool

In an attempt to maximize inclusion and expand the level of communication between signers and non-signers, this project employs the use of machine learning to translate American Sign Language fingerspelling alphabet into text and speech in real-time.

To load Signans on your project, follow this tutorial.
To load the Lite version, follow this tutorial.
To load the JS version, follow this tutorial.
To test Signans on your machine with OpenCV, follow the next steps.

1. Create Virtual Python Environment

Use the package manager pip to install virtualenv and create a virtual environment on your machine.

pip install virtualenv
python3.8 -m venv signans-python

On Windows, run:

signans-python\Scripts\activate.bat
pip install --upgrade pip
cd signans-python

On Unix or MacOS, run:

source signans-python/bin/activate
pip install --upgrade pip
cd signans-python

2. Clone Signans and Download Packages

Clone Signans GitHub repository:

git clone https://github.com/brunobpr/Signans

Download TensorFlow Models:

git clone https://github.com/tensorflow/models.git

Install the required packages:
```
pip install -r Signans/requirements.txt
```
Move the models to the python environment:

On Windows, run:

move "models\research\object_detection" "Lib\site-packages"
move "models\official" "Lib\site-packages"
cd Signans

On Unix or MacOS, run:

mv models/research/object_detection lib/python3.8/site-packages
mv models/official lib/python3.8/site-packages
cd Signans

3. Run the Script

python Signans.py

Once the video capturing window opens, use the ASL alphabet to finger-spell words or sentences. There are two extra signs, space and dot.

Space (left) means end of the word;
Dot (right) triggers the text-to-speech function.

To change the detection speed, run the scrip with the argument —speed or -s:

python Signans.py --speed {time in seconds}

Signans with Google Translator

To enable Google Translator, you need to set up an authentication key.
Once the authentication key is generated and downloaded, move it to signans-python/Signans and rename it as authentication.json

At the moment the supported languages are:
| Language | Code |
|:————: |———:|
|French | fr|
|Spanish | es|
|Portuguese | pt |
|Mandarin | zh|

To change the language, run the scrip with the argument —language or -l

python Signans.py --language {code}

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

License

Apache 2.0