机器人>> gflare-tk>> 返回
项目作者： beb7

项目描述 ：
Open-Source Python Based SEO Web Crawler
高级语言： Python
项目地址: git://github.com/beb7/gflare-tk.git
创建时间： 2020-05-09T12:52:04Z
项目社区：https://github.com/beb7/gflare-tk
开源协议：GNU General Public License v3.0
下载


Greenflare SEO Web Crawler

PyPI version
Supported Python Versions
Downloads
License: GPL v3

Greenflare is a lightweight free and open-source SEO web crawler for Linux, Mac, and Windows, and is dedicated to delivering high quality
SEO insights and analysis solutions to the world.

Features

  • Cross-platform (Linux, Mac, and Windows)
  • Low hardware requirements
  • Scalable (tested against sites with 4M+ URLs)
  • Reports on on-page SEO elements (i.e. page title, meta robots, canonical tag)
  • Analysis of HTTP header responses (i.e. X-Robots-Tag, Canonical HTTP Header)
  • Status code reporting (i.e. 301, 404, 503 etc.)
  • robots.txt parser (implemented against the suggested REP standard by Google)
  • Custom extraction through XPath or CSS
  • Custom exclusion of URLs through various patterns
  • Quick filtering and sorting of crawl data
  • View broken internal links (3xx, 4xx, 5xx)
  • Greenflare databases (.gflaredb) are sqlite tables
  • Export any view to CSV

Getting Started

The quickest way to get started using Greenflare is to download one of
our pre-built installers. Choose the version for your OS from our Download page:

https://greenflare.io/download

Python Package

Greenflare is also available as a pypi package:

pip install greenflare

The use of a virtual environment (venv) is recommended.
Linux users may chose to install ttkthemes for an improved visual experience.

Developers

Are you interested in becoming more involved in the development of
Greenflare? Please submit a pull request if you want to help to build new amazing features or to fix nasty bugs!
Alternatively, please email ben at greenflare dot io

Report a bug

Please report bugs by creating a new issue directly on GitHub:

https://github.com/beb7/gflare-tk/issues