12 stable releases

Uses new Rust 2021

1.0.11 Oct 31, 2022
1.0.10 Aug 22, 2022
1.0.8 Jul 8, 2022
1.0.7 Jun 8, 2022
1.0.3 Nov 23, 2021

#2342 in Magic Beans

Download history 351/week @ 2022-08-05 293/week @ 2022-08-12 286/week @ 2022-08-19 88/week @ 2022-08-26 295/week @ 2022-09-02 256/week @ 2022-09-09 62/week @ 2022-09-16 27/week @ 2022-09-23 102/week @ 2022-09-30 52/week @ 2022-10-07 36/week @ 2022-10-14 32/week @ 2022-10-21 190/week @ 2022-10-28 71/week @ 2022-11-04 101/week @ 2022-11-11 74/week @ 2022-11-18

441 downloads per month
Used in 5 crates (4 directly)

Apache-2.0

34KB
1K SLoC

crypto-crawler-rs

A rock-solid cryprocurrency crawler.

Quickstart

Use the carbonbot binary to crawl data.

If you need more control and customization, use this crypto-crawler library.

Architecture

  • crypto-crawler is the crawler library to crawl websocket and restful messages from exchanges
  • carbonbot is the main CLI tool to run crawlers.
  • crypto-ws-client is the underlying websocket client library, providing a set of universal APIs for different exchanges.
  • crypto-rest-client is the underlying RESTful client library, providing universal APIs to get public data from different exchanges.
  • crypto-markets is a RESTful library to retreive market meta data from cryptocurrency echanges.
  • crypto-client is a RESTful client library to place and cancel orders.
  • Support multiple languages. Some libraries support multiple languages, which is achieved by first providing a FFI binding, then a languge specific wrapper. For example, crypto-crawler provides a C-style FFI binding first, and then provides a Python wrapper and a C++ wrapper based on the FFI binding.

How to parse raw messages

Use the crypto-msg-parser library to parse raw messages.

Crawlers should always preserve the original data without any parsing.

Dependencies

~0.7–1.4MB
~31K SLoC