This is a data-crawler made in Scrapy. The data from local newspapers is crawled for data processing purpose. There are a number of spiders through which we can start the data crawling. The crawled data is stored in csv files. The files can also be stored in other formats like .json, .txt etc as per the convenience.
This is a data-crawler made in Scrapy. The data from local newspapers is crawled for data processing purpose. There are a number of spiders through which we can start the data crawling. The crawled data is stored in csv files. The files can also be stored in other formats like .json, .txt etc as per the convenience.
scrapy startproject <project_name>
After navigating to the parent folder <project_name> :
scrapy crawl <spider_name>scrapy crawl <spider_name> -o <filename>.csvscrapy crawl <spider_name> -o <filename>.json