13 releases (7 breaking)
new 0.8.0 | Nov 19, 2024 |
---|---|
0.7.0 | Nov 17, 2024 |
0.6.0 | Nov 12, 2024 |
0.5.0 | Nov 10, 2024 |
0.1.2 | Nov 3, 2024 |
#167 in Database interfaces
1,112 downloads per month
Used in learnerd
165KB
1.5K
SLoC
Features
-
Paper Metadata Management
- Support for arXiv, IACR, and DOI sources
- Automatic source detection from URLs or identifiers
- Full metadata extraction including authors and abstracts
-
Local Database
- SQLite-based storage with full-text search
- Configurable document storage
- Platform-specific defaults
-
CLI Tool (
learnerd
)- Paper addition and retrieval
- Search functionality
- Document management
- Daemon support for background operations
Installation
Library
[dependencies]
learner = { version = "*" } # Uses latest version
CLI Tool
cargo install learnerd
which will install a binary you can reference with the command learner
.
Usage
Library Usage
use learner::{Paper, Database};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let db = Database::open(Database::default_path()).await?;
// Add papers from various sources
let paper = Paper::new("https://arxiv.org/abs/2301.07041").await?;
paper.save(&db).await?;
// Download associated document
let storage = Database::default_storage_path();
paper.download_pdf(&storage).await?;
Ok(())
}
CLI Usage
# Initialize database
learner init
# Add papers
learner add 2301.07041
learner add "https://arxiv.org/abs/2301.07041"
learner add "10.1145/1327452.1327492"
# Manage documents
learner download arxiv 2301.07041
learner get arxiv 2301.07041
learner search "neural networks"
Daemon Management
learnerd
can run as a background service for paper monitoring and updates.
System Service
# Install and start
sudo learnerd daemon install
sudo systemctl enable --now learnerd # Linux
sudo launchctl load /Library/LaunchDaemons/learnerd.daemon.plist # macOS
# Remove
sudo learnerd daemon uninstall
Logs
- Linux: /var/log/learnerd/
- macOS: /Library/Logs/learnerd/
Files: learnerd.log
(main, rotated daily), stdout.log
, stderr.log
Troubleshooting
- Permission Errors: Check ownership of log directories
- Won't Start: Check system logs and remove stale PID file if present
- Installation: Run commands as root/sudo
Project Structure
-
learner
- Core library- Paper metadata extraction and management
- Database operations and search
- PDF handling and source-specific clients
- Error handling and type safety
-
learnerd
- CLI application- Paper and document management interface
- System daemon capabilities
- Logging and diagnostics
Roadmap
Core Features
- PDF management
- Content extraction
- Paper removal
- Batch operations
- Export functionality
- Enhanced search
- Custom metadata
Advanced Features
- LLM integration
- Version control and annotations
- Paper discovery
- Citation analysis
Contributing
Contributions welcome! Please open an issue before making major changes.
CI Workflow
Our automated pipeline ensures:
-
Code Quality
- rustfmt and taplo for consistent formatting
- clippy for Rust best practices
- cargo-udeps for dependency management
- cargo-semver-checks for API compatibility
-
Testing
- Full test suite across workspace and platforms
All checks must pass before merging pull requests.
Development
This project uses just as a command runner.
# Setup
cargo install just
just setup
# Common commands
just test # run tests
just fmt # format code
just ci # run all checks
just build-all # build all targets
[!TIP] Running
just setup
andjust ci
locally is a quick way to get up to speed and see that the repo is working on your system!
License
This project is licensed under the MIT License - see the LICENSE file for details.
Acknowledgments
- arXiv API for paper metadata
- IACR for cryptography papers
- CrossRef for DOI resolution
- SQLite for local database support
Dependencies
~61MB
~1M SLoC