WAF>> Document_Scanner>> 返回
项目作者： felixdittrich92

项目描述 ：
this repository contains utils for post processing scanned documents and classify any kind of document for the given labels
高级语言： Python
项目地址: git://github.com/felixdittrich92/Document_Scanner.git
创建时间： 2020-10-26T10:44:14Z
项目社区：https://github.com/felixdittrich92/Document_Scanner
开源协议：MIT License
下载




license
https://www.singularity-hub.org/static/img/hosted-singularity--hub-%23e32929.svg

Introduction

Document utilities

Implementation of a:

  1. CNN to classify the document quality (reading and ocr usage)

  2. Autoencoder to denoise scanned document images (100-300dpi)

  3. BertTransformer to classify multilabel documents

  • supported languages: german , english
  • autotranslated labels if document is english
  • supported labels: [‘boxen’, ‘china’, ‘corona’, ‘italien’, ‘politik’, ‘python’, ‘rezept’, ‘sport’, ‘typisierung’, ‘usa’]

Helper:

  1. a helper util to prepare a handy image for OCR, Autoencoder etc.

Requirements

  • Linux (Ubuntu 20.04)
  • Anaconda or miniconda
  • Nvidia GPU >= 8GB VRAM (recommended for custom training)

Installation

local (recommended)

  1. clone the repository
    1. git clone URL
  2. install requirements
    1. sudo apt-get install libxml2-dev libxslt1-dev antiword unrtf poppler-utils pstotext tesseract-ocr \
    2. libgtk2.0-dev libjpeg-dev swig
  • install Anaconda and create an environment
    ```
    $ cd ./Document_Scanner
    $ conda env create —name -f environment.yml
    $ conda activate

$ sudo apt install curl
$ curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
$ sudo apt-get install git-lfs

$ git lfs install
$ git lfs pull (need > 3GB)

  2. ### Singularity Container
  4. - in singularity container the text summarization from Doc_Classifier is not available 
  6. 1. install singularity [Singularity installation](https://gitlab.gwdg.de/mpg-singularity/mpg-singularity/-/wikis/Singularity-unter-Linux-installieren) or `pip3 install singularity`
  8. 2. download [Singularity container](https://singularity.gwdg.de/collections)
  9. - Collection: document-scanner
  10. - doc_classifier contains: classify documents
  11. - post_scan contains: predict quality of document images, denoising images, helper 
  13. 3. run container and read help section

$ singularity run-help doc_classifier.sif
$ singularity run-help post_scan.sif

$ singularity run —bind PATH TO FOLDER post_scan.sif (bind a folder or file with the container)

  2. ## Data
  3. if you are interested in getting the raw data contact me:
  4. [mailto](mailto:felixdittrich92@gmail.com)
  6. - Denoiser: 365 items

denoising
|
|_train: 1.png, 2.png, ..
|
|_train_cleaned: 1.png, 2.png, ..
|
|_test: 1.png, 2.png, ..

  2. - Evaluator: 275 items

Scans
|
|_best: 1.jpg, 2.jpg, ..
|
|_worst: 1.jpg, 2.jpg, ..

  2. - Text Classifier: 313 items

MachineLearning
|
|_boxenSport: name1.pdf, name2.pdf, ..
|
|_china: name1.pdf, name2.pdf, ..
|
|_chinaPolitik: name1.pdf, name2.pdf, ..
|
|_chinaRezept: name1.pdf, name2.pdf, ..
|
|_corona: name1.pdf, name2.pdf, ..
|
|_italienPolitik: name1.pdf, name2.pdf, ..
|
|_italienRezept: name1.pdf, name2.pdf, ..
|
|_politik: name1.pdf, name2.pdf, ..
|
|_python: name1.pdf, name2.pdf, ..
|
|_pythonTypisierung: name1.pdf, name2.pdf, ..
|
|_sport: name1.pdf, name2.pdf, ..
|
|_usaChinaCoronaPolitik: name1.pdf, name2.pdf, ..
|
|_usaChinaPolitik: name1.pdf, name2.pdf, ..
|
|_usaCoronaPolitik: name1.pdf, name2.pdf, ..
|
|_usaPolitik: name1.pdf, name2.pdf, ..

  2. ## Usage
  4. ### predict quality of document images

python3 predict_quality.py
—path=PATH TO SINGLE IMAGE OR A FOLDER WITH IMAGES (NO SUBFOLDER SUPPORT)

  1. <p float="left">
  2.   <img allign="left" src="images/original.jpg" width="400px"/> 
  3.   <img allign="right" src="images/autoencoder_result.jpg" width="400px"/> 
  4. </p>

Image name : original.jpg Image name : autoencoder_result.jpg
Predicted class : not good Predicted class : not good
Score : 0.007537 Score : 0.235914 

  1. <p float="left">
  2.   <img allign="left" src="images/output.jpg" width="400px"/> 
  3.   <img allign="right" src="images/lineart_300dpi.jpg" width="400px"/> 
  4. </p>

Image name : output.jpg Image name : lineart_300dpi.jpg
Predicted class : good Predicted class : good
Score : 0.728959 Score : 0.999801 

  1. ### denoising image

python3 denoising_image.py
—input=PATH TO SINGLE IMAGE
—output=NAME OF THE OUTPUT FILE (Example: “out.png”)
—quality=THE QUALITY FOR THE OUTPUT IMAGE 0-100 (Optional)
—res_width=THE WIDTH SIZE FOR THE OUTPUT IMAGE (ASPECT RATIO IS CALCULATED) (Optional)

  1. #### ORIGINAL / RESULT:
  2. <p float="left">
  3.   <img allign="left" src="images/color_300dpi.jpg" width="400px"/>
  4.   <img allign="right" src="images/output.jpg" width="400px"/>
  5. </p>
  7. ### classify documents
  8. Note: 
  9. - if you use summarization it takes a few minutes more
  10. - the labels will be auto translated

python3 classify_text.py
—input=PATH TO SINGLE DOCUMENT OR FOLDER WITH DOCUMENTS (NO SUBFOLDER SUPPORT)
—meta=TRUE: CREATE META FILE WITH LANGUAGE AND CLASSES / KEYWORDS FROM TEXT
—summarize=TRUE: CREATE TEXT SUMMARIZATION AND SAVES IN METAFILE (OPTIONAL IF META: TRUE)

  1. #### GERMAN TEXT:
  2. Mit 40 Prozent der Stimmen hat der Par-
  3. tito Democratico (PD)1 bei den Europawahlen 
  4. im Mai dieses Jahres einen herausragenden 
  5. und in dieser Höhe auch nicht erwarteten 
  6. Sieg errungen. Dieser Erfolg ist vor allem mit 
  7. dem Namen Matteo Renzi verbunden. Der 
  8. frühere Bürgermeister von Florenz hat einen 
  9. rasanten politischen Aufstieg hinter sich. Aus 
  10. dem eher christdemokratischen Teil des PD 
  11. kommend (vor dem Zusammenschluss zur 
  12. PD war er Mitglied von „La Margherita“2), gilt 
  13. der junge, dynamisch und unkonventionell 
  14. wirkende Renzi als neuer Hoffnungsträger 
  15. der italienischen Politik. Er wurde im Dezem-
  16. ber 2013 zum Vorsitzenden des PD gewählt 
  17. und löste im Februar den eher unglücklich 
  18. agierenden Enrico Letta als Regierungschef 
  19. ab. Allerdings war dieser Machtwechsel nicht 
  20. Ergebnis eines demokratischen Prozesses, 
  21. sondern eines parteitaktischen Manövers. 
  22. Die Europawahl gilt als nachgeholte demo-
  23. kratische Legitimation. Mit Renzi ist zugleich 
  24. die Hoffnung auf eine stabile Mitte-Links-
  25. Regierung als Alternative zur jahrelangen 
  26. Vorherrschaft Berlusconis verbunden. Renzis 
  27. Erfolg erklärt sich vor dem Hintergrund ei-
  28. ner völlig desolaten politischen Situation in 
  29. Italien. Die politischen Parteien haben sich 
  30. zum großen Teil diskreditiert und nahezu 
  31. jede Glaubwürdigkeit verloren. In den Augen 
  32. vieler handelt es sich bei den Politikern um 
  33. eine nur auf ihren Vorteil bedachte Kaste“. 
  34. Und in der Tat reißen die Skandale und Kor-
  35. ruptionsfälle nicht ab. Erst dieser Tage wur-
  36. de die Hauptstadt Rom von einem großen 
  37. Korruptionsskandal in der Stadtverwaltung 
  38. erschüttert, nachdem wenige Jahre zuvor be-
  39. reits die Provinzverwaltung .....
  41. #### PREDICTIONS:

Document name : 205_bierbaum.pdf

Class : boxen
Probability : 0.008778

Class : china
Probability : 0.003453

Class : corona
Probability : 0.018391

Class : italien
Probability : 0.992215

Class : politik
Probability : 0.841440

Class : python
Probability : 0.012592

Class : rezept
Probability : 0.189319

Class : sport
Probability : 0.009547

Class : typisierung
Probability : 0.012490

Class : usa
Probability : 0.006952

  1. #### METAFILE:

{
“Filename”: “205_bierbaum.pdf”,
“Date”: “Sep-28-2020”,
“Time”: “11:06:30”,
“Language”: “de”,
“Labels”: [“italien”, “politik”],
“Keywords”: [“geben deutschen italien politik deutschland deutsche menschen regierung ganz politischen”],
“Summary”: “Partito democratico pd ist mitglied des parteivorstandes die linke vorsitzender der internationalen kommission and professor fuer betriebswirtschaft. Die europawahl gilt als nachgeholte demokratische legitimation mit renzi ist zugleich die hoffnung auf eine stabile mittelinksregierung.”
}

  2. #### ENGLISH TEXT:
  3. So can the EU; now indeed the EU must prove 
  4. that it can do better than after the 2008 financial 
  5. crisis and protect the livelihood of its citizens 
  6. rather than just the banks. After the 2016 British 
  7. vote for Brexit, leaders of all colours finally 
  8. discovered the need for a social Europe – and 
  9. then forgot about it again. The EU cannot afford 
  10. to make this mistake twice: strong Keynesian 
  11. policies must create an economically vibrant and 
  12. socially just Union. (How the UK will fare, on its 
  13. own now and under a Prime Minister who 
  14. unfortunately seems to have taken his cue from 
  15. Trump, is another question). 
  17. If one power falls behind the other two, it will 
  18. lose global markets and influence to the benefit 
  19. of the others. But, paradoxically, if one really 
  20. collapses, that risks pulling down the others as 
  21. well, so to some extent all three have an interest 
  22. in a minimal recovery of the others (though not 
  23. all of their leaders may see it as such). 
  25. At the same time, the corona crisis will accelerate 
  26. US and even EU action, which they were 
  27. envisaging already, to reduce the interdependence 
  28. with China (and others) by reviewing the supply 
  29. chains in critical sectors. Chinese attempts to use 
  30. the crisis to increase their presence in Europe and 
  31. America (provided Beijing can mobilise the 
  32. means in view of its undoubtedly underreported 
  33. domestic problems) will be viewed with a lot 
  34. more suspicion this time. Nevertheless, this is 
  35. about reorganizing globalisation, not undoing it – 
  36. at basis, deep economic interdependence will 
  37. remain.
  39. #### PREDICTIONS:

Document name : SPB126-sven-corona-260320.pdf

Class : boxen
Probability : 0.002254

Class : china
Probability : 0.014674

Class : corona
Probability : 0.996145

Class : italien
Probability : 0.003417

Class : political <- auto translated
Probability : 0.988726

Class : python
Probability : 0.002591

Class : rezept
Probability : 0.002722

Class : sport
Probability : 0.002718

Class : typisierung
Probability : 0.002515

Class : usa
Probability : 0.990106

  1. #### METAFILE:

{
“Filename”: “SPB126-sven-corona-260320.pdf”,
“Date”: “Sep-28-2020”,
“Time”: “14:16:51”,
“Language”: “en”,
“Labels”: [“corona”, “political”, “usa”],
“Keywords”: [“chinese chinas trade education growth security market military reforms percent”],
“Summary”: “ corona crisis is test of legitimacy for every government though not necessarily for the political system as such. Crisis may strengthen opposition to xi jinping within the chinese communist party which might have consequences for the succession. Great powers instrumentalise everything so it was only to be expected that they would instrumentalise the coronavirus.”
}

  2. ## Helper
  3. - NOTE: the image needs a clear background / four edges must be recognizable
  4. - rotate the image correctly and crop the relevant part / add sharpening with --sharp=True

python3 handy_helper/helper.py
—input=PATH TO SINGLE IMAGE
—output=NAME OF THE OUTPUT FILE (Example: “out.png”)
—sharp=True (Optional)
—quality=THE QUALITY FOR THE OUTPUT IMAGE 0-100 (Optional)
—res_width=THE WIDTH SIZE FOR THE OUTPUT IMAGE (ASPECT RATIO IS CALCULATED) (Optional)

  1. #### ORIGINAL / RESULT
  3. <p float="left">
  4.   <img allign="left" src="images/handy.jpg" width="400px"/> 
  5.   <img allign="right" src="images/handy_out.jpg" width="380px"/> 
  6. </p>
  8. ## Custom Training
  9. - NOTE: If you want to train your own data, look in the data section in which structure the data must be available
  11. ### Evaluator

evaluator/Dataset.py -> Line 16 -> path to your Dataset
evaluator/CNN.py -> Linie 141/181 -> change save path
cd Document_Scanner/evaluator
python3 CNN.py
predict_quality.py -> Line 18 -> change to your model stored path

  1. ### Denoiser

denoiser/Autoencoder.py -> Line 30/31 -> path to your Dataset
denoiser/Autoencoder.py -> Line 124 / 158 -> change save path
cd Document_Scanner/denoiser
python3 Autoencoder.py
denoising_image.py -> Line 25 -> change to your model stored path

  1. ### Multilabel Classifier

cd Document_Scanner
python3 text_classifier/train_custom.py
—path=PATH TO THE FOLDER WITH DOCUMENTS
(OPTIONAL)
—save_dataset=SAVE THE DATASET FOR RENEWED TRAINING (BOOL)
—maxlen=EACH DOCUMENT CAN BE OF MOST MAXLEN WORDS
—max_features=MAX NUM OF WORDS TO CONSIDER IN VOCABULARY
—batch_size=SIZE OF PARALLEL TRAINED DATA STACK
—epochs=SIZE OF TRAINING ITERATIONS
—lda_n_features=NUMBER OF EXTRACTED TOPICS
—lda-threshold=PROBABILITIES BELOW THIS VALUE ARE FILTERED OUT

python3 classify_text.py
—custom=True: USES YOUR OWN TRAINED MODEL
—input=PATH TO SINGLE DOCUMENT OR FOLDER WITH DOCUMENTS (NO SUBFOLDER SUPPORT)
—meta=TRUE: CREATE META FILE WITH LANGUAGE AND CLASSES / KEYWORDS FROM TEXT
—summarize=TRUE: CREATE TEXT SUMMARIZATION AND SAVES IN METAFILE (OPTIONAL IF META: TRUE)

  2. - Example:

(doc) USER@PC:~/Desktop/Document_Scanner > python3 text_classifier/train_custom.py —path=/home/user/Desktop/MachineLearning/ —save_dataset=True
(doc) USER@PC:~/Desktop/Document_Scanner > python3 classify_text.py —path=/home/user/Desktop/MachineLearning/boxenSport/ —meta=True —summarize=True —custom=True
```

  • For completely fresh training delete custom_data folder
de_test_1648705422467.pdf
en_test_1648705422503.pdf