ASIMOV Bright Data Module

ASIMOV module for data import powered by the Bright Data web data platform.

✨ Features

• Imports structured data from Airbnb, Amazon, Crunchbase, eBay, Facebook, Google, Indeed, Instagram, LinkedIn, Walmart, X (aka Twitter), Yahoo, and YouTube.
• Collects the raw JSON data via the Bright Data API (requires an API key).
• Constructs a semantic knowledge graph based on the KNOW Ontology.
• Supports plain JSON output as well as RDF output formats such as JSON-LD, Turtle, and N-Triples.

🛠️ Prerequisites

• Rust 1.85+ (2024 edition)

⬇️ Installation

Installation from Source Code

cargo install asimov-brightdata-module

👉 Examples

export BRIGHTDATA_API_KEY="..."

Fetching X Profiles

asimov-brightdata-fetcher https://x.com/bendiken
asimov-brightdata-fetcher https://x.com/asimov_protocol

Fetching LinkedIn Profiles

asimov-brightdata-fetcher https://www.linkedin.com/in/arto/
asimov-brightdata-fetcher https://www.linkedin.com/company/asimov-protocol/

Fetching Crunchbase Profiles

asimov-brightdata-fetcher https://www.crunchbase.com/organization/near-f896

Fetching Amazon Products

asimov-brightdata-fetcher https://www.amazon.com/Master-Algorithm-Ultimate-Learning-Machine/dp/0465094279

⚙ Configuration

Environment Variables

• BRIGHTDATA_API_KEY: (required) the Bright Data API key to use

📚 Reference

Installed Binaries

• asimov-brightdata-cataloger: discovers entities via the Bright Data API (not implemented yet)
• asimov-brightdata-fetcher: collects JSON data from the Bright Data API
• asimov-brightdata-importer: collects and transforms JSON into RDF formats (not implemented yet)

Supported Datasets

👨‍💻 Development

git clone https://github.com/asimov-modules/asimov-brightdata-module.git