场景模型>> Big-gram-Grammer-Generator-with-Smoothing>> 返回
项目作者： anjanatiha

项目描述 ：
Generates Bi-gram Grammar with Add One Smoothing and shows Zip-f's Law on Brown Corpus
高级语言： Jupyter Notebook
项目地址: git://github.com/anjanatiha/Big-gram-Grammer-Generator-with-Smoothing.git
创建时间： 2018-06-27T00:45:34Z
项目社区：https://github.com/anjanatiha/Big-gram-Grammer-Generator-with-Smoothing
开源协议：MIT License
下载


Bigram Grammer Generator with Add One Smoothing

Domain : Natural Language Processing
Sub-Domain : Language Processing, Text Processing, Language Modeling
Techniques : Language Modeling, Bigram, Add-one Smoothing, Zip Law
Application Domain : Language Generation

Description

  1. From the SnapshotBROWN.pos.all.txt file extracted all word types and their frequencies.
  2. Sorted the list of word types in decreasing order based on their frequency.
    Drew a chart showing the relationship between the rank in the ordered list and the frequency (Zipf’s Law). (Do not stem but do ignore punctuation.)
  3. Generated a Bigram Grammar from the above file.
  4. Performed add-one smoothing.
  5. Showed the grammar before and after smoothing for the sentence “A similar resolution passed in the Senate”.
Languages : Python
Tools/IDE : Anaconda
Libraries :
Duration :

Current Version : v1.0.0.3

Last Update : 04.07.2018 (Time : 05:45 P.M)