#indexer #data-integrity #graph-protocol #waku #real-time #p2p

bin+lib subgraph-radio

Subgraph Radio monitors subgraph PublicPoI and UpgradeIntent messages in real time using Graphcast SDK

11 releases (1 stable)

1.0.1 Mar 4, 2024
0.1.10 Nov 2, 2023
0.1.9 Oct 31, 2023
0.1.7 Sep 15, 2023
0.1.2 Jul 21, 2023

#584 in Network programming

Download history 4/week @ 2024-02-26 153/week @ 2024-03-04 25/week @ 2024-03-11 24/week @ 2024-03-25 12/week @ 2024-04-01

67 downloads per month

Apache-2.0

200KB
5K SLoC

Subgraph Radio

Docs crates.io

Introduction

This is a Graphcast Radio focused on sending gossips about particular subgraphs on a P2P network. The available message types are Public Proof of Indexing (PoI) messages from an indexer, or a version intent message from a subgraph owner.

Reaching Public PoI consensus and ensuring data availability during subgraph upgrades is critical to the indexing service. Both messages should find value from indexers, subgraph owners, and ultimately data consumers.

Documentation | Packages | Chat

Public PoI message

The key requirement for an Indexer to earn indexing rewards is to submit a valid Proof of Indexing (POI) promptly. The importance of valid POIs causes many Indexers to alert each other on subgraph health in community discussions. To alleviate the Indexer workload, this Radio utilized Graphcast SDK to exchange and aggregate POI along with a list of Indexer on-chain identities that can be used to trace reputations. With the pubsub pattern, the Indexer can get notified as soon as other indexers (with majority of stake) publish a POI different from the local POI.

Upgrade Intent message

When developers publish a new version (subgraph deployment) to their subgraph, data service instability may occur while their API queries the pre-existing version. Indexers may require some time to sync a subgraph to the chainhead after they have stopped syncing the previous deployment. To decrease the upgrade friction, developers can send a message before publishing the subgraph on current deployment hash's Graphcast channel, which includes the subgraph id, corresponding new deployment hash, and the represented graph account that must be validated to be the subgraph's owner.

Indexers running the subgraph radio and listening to that channel will in turn receive the message. Given valid configurations to notification, indexer management server, and auto_upgrade_coverage coverage level, the radio will send a notification to the indexer and/or an offchain request to their indexer management server that should automatically presync the deployment on graph node.

Note that it is still at the subgraph developers' discretion to await for the indexers to sync upto chainhead, in which point they can publish the staged version without disrupting API usage.

🧪 Testing

To run unit tests for the Radio. We recommend using nextest as your test runner. Once you have it installed you can run the tests using the following commands:

cargo nextest run

Contributing

We welcome and appreciate your contributions! Please see the Contributor Guide, Code Of Conduct and Security Notes for this repository.

Dependencies

~109MB
~2M SLoC