1 unstable release
0.0.1 | Jan 3, 2023 |
---|
#81 in #analytics
Used in barreleye
130KB
4.5K
SLoC
Open-Source Blockchain Analytics ✨
Website • Join Waitlist • Chat • Twitter
Barreleye is an open-source blockchain analytics tool. It's entity based, so it can answer questions like what assets an entity has and where they came from.
Note This is an actively developed work-in-progress and not yet ready for production. Use at your own risk.
✨ Features
- ⛵️ Easy to use — start on a single machine, scale up as needed
- 🚢 Scalable — optimized for demanding business use-cases
- 🥳 Self-hosted — API-based interface that can be integrated into other systems
- 🪢 Multi-chain — designed to support multiple blockchain architectures
🧰 Requirements
- Rust v1.75+ — if you're compiling from source
- ClickHouse v23.5+ — for warehouse data storage
- Blockchain nodes — for indexing (eg: Bitcoin, Ethereum, etc)
Note ⚠️ EVM-based chains are not yet supported (this is a work-in-progress)
👩💻 Get Started
Clone, build & install:
git clone https://github.com/barreleye/barreleye
cd barreleye
cargo build
cargo install
Run locally (pointing to your ClickHouse instance):
barreleye \
--warehouse http://username:password@localhost:8123/database_name
This will do the following:
- Start the server, which will handle future API requests
- Start the indexer, which will:
For production you'll probably want to store extracted blockchain data in the cloud (eg: Amazon S3, Cloudflare R2, etc), as opposed to your local files:
barreleye \
--warehouse http://username:password@localhost:8123/database_name \
--storage http://s3.us-east-1.amazonaws.com/bucket_name/
You can also use a hosted RDBMS like PostgreSQL or MySQL instead of SQLite:
barreleye \
--warehouse http://username:password@localhost:8123/database_name \
--storage http://s3.us-east-1.amazonaws.com/bucket_name/ \
--database postgres://username:password@postgres-host:5432/database_name
📦 Modes
Barreleye is bundled with the indexer and the server in the same program. The indexer is responsible for crawling blockchains and retrieving all the necessary data, while the server is focused on handling API requests (data management + analytics requests).
By default, both the indexer and the server are enabled and will run in parallel:
barreleye
To run only the indexer:
barreleye --mode indexer
To run only the server:
barreleye --mode http
Note ⚠️ Indexer is designed to run in failover mode. Only the primary instance will run at once; the others will wait for the primary to fail in order to promote a secondary.
💾 Add Data
Barreleye does not come with any pre-defined data. Instead, it gives you the ability to add and manage data yourself. The API calls below give an overview of how to manage data.
A default API key is generated when you first start Barreleye, so to get it — connect to your RDBMS and retrieve the only key that has been auto-inserted:
select secret_key from api_keys where id='key_default';
-- the result will be $YOUR_API_KEY in examples below
Note Be sure to write down that API key somewhere else, because after the first successful API call, that plaintext value will be erased.
Add Blockchains
Add a Bitcoin RPC node:
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"id": "net_bitcoin",
"name": "Bitcoin",
"architecture": "bitcoin",
"blockTime": 600000,
"rpcEndpoint": "http://username:password@127.0.0.1:8332"
}' \
http://localhost:4000/v1/networks
Add an EVM-based RPC node (archive node is required):
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"id": "net_ethereum",
"name": "Ethereum",
"architecture": "evm",
"chainId": 1,
"blockTime": 12000,
"rpcEndpoint": "http://127.0.0.1:8545"
}' \
http://localhost:4000/v1/networks
Add Tokens
To add native Bitcoin currency:
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"network": "net_bitcoin",
"name": "bitcoin",
"symbol": "BTC",
"decimals": 8
}' \
http://localhost:4000/v1/tokens
To add native Ethereum currency:
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"network": "net_ethereum",
"name": "Ether",
"symbol": "ETH",
"decimals": 18
}' \
http://localhost:4000/v1/tokens
To add an ERC-20 token:
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"network": "net_ethereum",
"name": "USD Coin",
"symbol": "USDC",
"address": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48",
"decimals": 6
}' \
http://localhost:4000/v1/tokens
Add Tags
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"id": "tag_exchange",
"name": "Exchange",
"riskLevel": "low"
}' \
http://localhost:4000/v1/tags
Add Entities
An entity can be an item that contains one or several blockchain addresses:
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"id": "ent_coinbase",
"name": "Coinbase",
"description": "",
"tags": ["tag_exchange"]
}' \
http://localhost:4000/v1/entities
To add addresses:
curl -X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $YOUR_API_KEY" \
-d '{
"entity": "ent_coinbase",
"network": "net_ethereum",
"addresses": [
{
"address": "0x71660c4005BA85c37ccec55d0C4493E66Fe775d3",
"description": "Address #1"
}, {
"address": "0x503828976d22510aad0201ac7ec88293211d23da",
"description": "Address #2"
}
]
}' \
http://localhost:4000/v1/addresses
📊 Analytics
To query information about a particular blockchain address:
curl -X GET \
-H 'Content-Type: application/json' \
http://localhost:4000/v1/info?q=$BLOCKCHAIN_ADDRESS
🗒 Random Notes
- Be aware of your RPC node limits. Indexer makes a significant amount of RPC calls to index historical and new blocks.
- For indexing, you might have to set ClickHouse's
max_server_memory_usage_to_ram_ratio
to2
(read more)
🥹 Get Involved
To stay in touch with Barreleye:
- Star this repo ★
- Follow on Twitter
- Join on Discord
- Contribute -- pull requests are welcome (for major changes, please open an issue first to discuss what you would like to change)
⚖️ License
Source code for Barreleye is variously licensed under a number of different licenses. A copy of each license can be found in each repository.
- Core code for Barreleye, located in this repository, is released under the Apache 2.0.
- Libraries and SDKs, each located in its own distinct repository, are released under either the Apache License 2.0 or MIT License.
Dependencies
~123MB
~2.5M SLoC