Nlp>> BanglaTranslator>> 返回
项目作者： menon92

项目描述 ：
Bangla Machine Translator
高级语言： Python
项目地址: git://github.com/menon92/BanglaTranslator.git
创建时间： 2020-05-03T08:56:27Z
项目社区：https://github.com/menon92/BanglaTranslator
开源协议：Apache License 2.0
下载


BanglaTranslator

Translate bangla to english. This model is train based on encoder decoder with attention mechanism. This repository may be a starting point to approaching bangla machine translation problem. If this repository helps others people who are working on bangla machine translation then it would be very greatfull for me.

Dataset

I use dataset provide in http://www.manythings.org/anki/ben-eng.zip . This dataset contain english bangla sentence pair in the following format,

  1. I'm counting on you.    আমি আপনার উপর নির্ভর করে আছি।
  2. I want your opinion.    আমি আপনার মতামত চাই।
  3. How is your daughter?    আপনার মেয়ে কেমন আছে?

Project structure

  1. BanglaTranslator
  2. ├── assets
  3. │   └── banglafonts
  4. │       └── Siyamrupali.ttf
  5. ├── data
  6. │   ├── ben-eng
  7. │   │   ├── _about.txt
  8. │   │   └── ben.txt
  9. ├── docs
  10. │   └── U0980.pdf
  11. ├── models
  12. │   ├── input_language_tokenizer.json
  13. │   ├── target_language_tokenizer.json
  14. ├── translator
  15. │   ├── config.py
  16. │   ├── datasets.py
  17. │   ├── infer.py
  18. │   ├── __init__.py
  19. │   ├── models.py
  20. │   ├── train.py
  21. │   └── utils.py
  22. ├── infer-example.ipynb
  23. ├── README.md
  24. └── training-example.ipynb
  • assets contain bangla font that used in plotting
  • data contain english bangla pair dataset
  • docs contrain documeantaion bangla unicode poins and it’s char maping
  • models contrain saved tokenize and training checkpoints if you do training
  • translator is the core of the project that contrain all the required scripts for this project.
  • infer-example.ipynb An example notebook that shows how predict on single sentence using saved checkpoints
  • training-example.ipynb you can use this notebook to train bangla to english translator model

Dependency

  1. python 3.7
  2. tensorflow 2.x
  3. matplotlib
  4. sklearn
  5. tqdm
  6. jupyter notebook

Pre-train model

If you want to just test the model then you need to download pretrain model from from google drive link
and extract training_checkpoints.zip file under models directory

Test result

I test pre-train model and got result like bellow.

  • If you want to test it yourself please check infer-example.ipynb and also download pre-train model

Resources