项目作者: menon92

项目描述 :
Bangla Machine Translator
高级语言: Python
项目地址: git://github.com/menon92/BanglaTranslator.git
创建时间: 2020-05-03T08:56:27Z
项目社区:https://github.com/menon92/BanglaTranslator

开源协议:Apache License 2.0

下载


BanglaTranslator

Translate bangla to english. This model is train based on encoder decoder with attention mechanism. This repository may be a starting point to approaching bangla machine translation problem. If this repository helps others people who are working on bangla machine translation then it would be very greatfull for me.

Dataset

I use dataset provide in http://www.manythings.org/anki/ben-eng.zip . This dataset contain english bangla sentence pair in the following format,

  1. I'm counting on you. আমি আপনার উপর নির্ভর করে আছি।
  2. I want your opinion. আমি আপনার মতামত চাই।
  3. How is your daughter? আপনার মেয়ে কেমন আছে?

Project structure

  1. BanglaTranslator
  2. ├── assets
  3. └── banglafonts
  4. └── Siyamrupali.ttf
  5. ├── data
  6. ├── ben-eng
  7. ├── _about.txt
  8. └── ben.txt
  9. ├── docs
  10. └── U0980.pdf
  11. ├── models
  12. ├── input_language_tokenizer.json
  13. ├── target_language_tokenizer.json
  14. ├── translator
  15. ├── config.py
  16. ├── datasets.py
  17. ├── infer.py
  18. ├── __init__.py
  19. ├── models.py
  20. ├── train.py
  21. └── utils.py
  22. ├── infer-example.ipynb
  23. ├── README.md
  24. └── training-example.ipynb
  • assets contain bangla font that used in plotting
  • data contain english bangla pair dataset
  • docs contrain documeantaion bangla unicode poins and it’s char maping
  • models contrain saved tokenize and training checkpoints if you do training
  • translator is the core of the project that contrain all the required scripts for this project.
  • infer-example.ipynb An example notebook that shows how predict on single sentence using saved checkpoints
  • training-example.ipynb you can use this notebook to train bangla to english translator model

Dependency

  1. python 3.7
  2. tensorflow 2.x
  3. matplotlib
  4. sklearn
  5. tqdm
  6. jupyter notebook

Pre-train model

If you want to just test the model then you need to download pretrain model from from google drive link
and extract training_checkpoints.zip file under models directory

Test result

I test pre-train model and got result like bellow.

  • If you want to test it yourself please check infer-example.ipynb and also download pre-train model

Resources