项目作者: tomholford

项目描述 :
CLI tool written in Deno to scrape URLs and download linked media files
高级语言: TypeScript
项目地址: git://github.com/tomholford/media-downloader.git
创建时间: 2020-06-19T06:31:51Z
项目社区:https://github.com/tomholford/media-downloader

开源协议:

下载


media-downloader

image

A basic media downloading tool written in Typescript for
Deno. Pass in a URL, select which filetypes you’re
interested in, and it will scrape the HTML and download all the linked files
that match the filter.

Prerequisites

Usage

  • -u - (required) the URL to scrape for media links
  • -p - (optional) the output path for writing files; defaults to ./output/
  • -t - (optional) the filetype(s) to download; defaults to several common
    image formats
  • -R - (optional) after finishing downloading, scan the output directory and
    remove duplicates

Example

  1. deno run --allow-net --allow-write --allow-read --unstable media_downloader.ts -t jpg -t png -u https://dribbble.com/shots

Install

  1. Clone this repo
  2. Run this command
  1. deno install --allow-net --allow-write --allow-read --unstable --name media-downloader main.ts
  1. Then, use it like so:
  1. media-downloader -u https://example.com

macOS

You will also need to add the deno executables path to your bash_profile:

  1. export PATH="/Users/[your username]/.deno/bin:$PATH"

Supported Sites

This tool has been confirmed to work with imageboards [e.g.,
4chan) and thisvid. Feel free to
open a PR to contribute more sites to this list.

Roadmap

  • Support <img> tags
  • Multiple parallel requests
  • progress bar
  • CI
  • release binaries
  • tests