#data-processing #request-response #spider #web-framework #web-crawling

app dyer-cli

dyer-cli is a great tool created to guide you use dyer fast and at ease, helps you build a robust crawler, data processor, netwrok program fast and correctly

9 releases (5 breaking)

0.6.0 Jul 27, 2022
0.5.0 May 19, 2022
0.4.0 Nov 28, 2021
0.3.0 Oct 13, 2021
0.1.2 Apr 13, 2021

#1293 in Web programming

Download history 829/week @ 2023-12-07 800/week @ 2023-12-14 795/week @ 2023-12-21 720/week @ 2023-12-28 834/week @ 2024-01-04 799/week @ 2024-01-11 800/week @ 2024-01-18 809/week @ 2024-01-25 789/week @ 2024-02-01 776/week @ 2024-02-08 559/week @ 2024-02-15 890/week @ 2024-02-22 889/week @ 2024-02-29 869/week @ 2024-03-07 899/week @ 2024-03-14 690/week @ 2024-03-21

3,482 downloads per month

MIT license

45KB
1K SLoC

Introduction

Dyer-cli is a great tool created to guide you use dyer fast and at ease, helps you build a robust crawler, data processor, netwrok program fast and correctly.

Installation

Dyer-cli is built completely by Rust programming language without extra dependencies, So rust must be installed beforehand, to test it with:

rustup --version

if you ever see some infomation like that

rustup 1.23.1 (3df2264a9 2020-11-30)

then you are ready to go, the following code would suffice.

cargo install dyer-cli

the command will download the source code and complie it to build a executable file inside your $HOME/.cargo/bin, make sure it's in your $PATH

Commands

Dyer-cli provides some commands that helps you initialize, debug programm, more commands are to go.

dyer new

This command helps you initialize a project with log level Info, other log levels vares from Error, Warn, Info, Debug, and Trace, and its structure is

|___Cargo.toml
|___Readme.md
|___data/
|___data/tasks/
|___src/
    |___src/entity.rs
    |___src/parser.rs
    |___src/actor.rs
    |___src/middleware.rs
    |___src/pipeline.rs

dyer check

Alias dyer c, A warper of cargo check, if you run it the first time,dyer-cli will download the crates and then check the code.

dyer fix

Alias dyer f, A wraper of cargo fix, if some warning happens such as unused import or dead code the command does a lot for you. However it won't help if some errors occur, if so, you have to debug the code manually.

dyer run

Alias dyer r, A wraper of cargo run, when the program compiles, run it.

dyer build

Alias dyer b, A wraper of cargo build, build the program.

dyer test

Alias dyer t, A wraper of cargo test, test the program.

dyer clean

A wraper of cargo clean, clean the directory.

Dependencies

~2.2–3MB
~54K SLoC