26 releases (17 stable)

1.1.6 Aug 14, 2023
1.1.5 Oct 31, 2022
1.1.3 Aug 17, 2022
1.1.1 Jul 4, 2022
0.1.2 Mar 19, 2021

#128 in #trading

Download history 83/week @ 2024-12-01 130/week @ 2024-12-08 91/week @ 2024-12-15 48/week @ 2024-12-22 47/week @ 2024-12-29 85/week @ 2025-01-05 102/week @ 2025-01-12 76/week @ 2025-01-19 57/week @ 2025-01-26 121/week @ 2025-02-02 116/week @ 2025-02-09 69/week @ 2025-02-16 65/week @ 2025-02-23 90/week @ 2025-03-02 76/week @ 2025-03-09 47/week @ 2025-03-16

287 downloads per month
Used in 9 crates (8 directly)

Apache-2.0

7KB
99 lines

crypto-crawler-rs

Discord

A rock-solid cryprocurrency crawler.

Quickstart

Use the carbonbot binary to crawl data.

If you need more control and customization, use this library.

Architecture

  • crypto-crawler is the crawler library to crawl websocket and restful messages from exchanges
  • carbonbot is the main CLI tool to run crawlers.
  • crypto-ws-client is the underlying websocket client library, providing a set of universal APIs for different exchanges.
  • crypto-rest-client is the underlying RESTful client library, providing universal APIs to get public data from different exchanges.
  • crypto-markets is a RESTful library to retreive market meta data from cryptocurrency echanges.
  • crypto-client is a RESTful client library to place and cancel orders.
  • Support multiple languages. Some libraries support multiple languages, which is achieved by first providing a FFI binding, then a languge specific wrapper. For example, crypto-crawler provides a C-style FFI binding first, and then provides a Python wrapper and a C++ wrapper based on the FFI binding.

How to parse raw messages

Use the crypto-msg-parser library to parse raw messages.

Crawlers should always preserve the original data without any parsing.

Dependencies

~0.4–1MB
~21K SLoC