项目作者: sbillburg

项目描述 :
implement CRNN in Keras with Spatial Transformer Network
高级语言: Python
项目地址: git://github.com/sbillburg/CRNN-with-STN.git
创建时间: 2018-05-04T05:17:29Z
项目社区:https://github.com/sbillburg/CRNN-with-STN

开源协议:

下载


CRNN-with-STN

implement CRNN in Keras with Spatial Transformer Network (STN) for Optical Character Recognition(OCR)

The model is easy to start a trainning, but the performance of recognition is not better than the original CRNN without STN.

You can run CRNN individually by just remove the STN components, and connect batchnorm_7 to x_shape. The CRNN can reach 90% of recognition accuracy.

Train on Synthetic Word Dataset realsed by M. Jaderberg et al. You can download the dataset HERE

How to Run

Just run the CRNN_with_STN.py script.
My environment is Tensorflow 1.4.0, Keras 2.0.9. If you don’t know how to install frameworks, please check
Installing Deep Learning Frameworks on Ubuntu with CUDA support
.

Reference

CRNN(Convolutional Recurrent Neural Network)

Shi, Baoguang, X. Bai, and C. Yao. “An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition.” IEEE Transactions on Pattern Analysis & Machine Intelligence PP.99(2016):1-1.[arxiv]

STN(Spatial Transformer Network)

Max Jaderberg, Karen Simonyan,Andrew Zisserman and Koray Kavukcuoglu. “Spatial Transformer Network” [arxiv]

2019.03.13 UPDATE

I have tried to put STN part in the front of the network, between batchnorm_1 and conv_2, than the network didn’t converge at all. Maybe I used STN in a wrong way?

Weights Download, 2019.05.31 UPDATE

1. ou can download trained weights CRNN_without_STN, and CRNN_with_STN. (Google Drive)

3. Added a DEMO for model predicting