#hdfs #hadoop #webhdfs

bin+lib webhdfs

Hadoop WebHDFS client library

6 releases

0.3.2 Feb 13, 2021
0.3.1 Sep 28, 2020
0.2.2 Feb 7, 2020
0.2.1 Oct 6, 2019
0.1.0 Aug 4, 2019

#208 in Asynchronous

Apache-2.0

115KB
2.5K SLoC

Rust 2K SLoC // 0.1% comments Shell 492 SLoC // 0.1% comments

webhdfs

Hadoop webhdfs client library for Rust.

Crates.io Build Status

Documentation

  • Built on Tokio and Hyper.
  • Provides both synchronous and asynchronous APIs.

See src/bin/webhdfs.rs or tests/it.rs for usage examples.

NOTE: APIs are not yet stabilized and may change towards version 1.0.

TODO list (2nd checkbox is IT coverage)

  • Async read and write ([X])
  • Sync read and write ([X])
  • Stat and dir ([X])
  • Basic filesystem mutations (concat, mkdirs, rename, delete, symlink)
  • File-based configuration
  • Other file and directory operations (permissions, owner, ACL, times, checksum)
  • XAttr operations
  • Snapshot operations
  • Basic delegation token support
  • Full delegation token support
  • webhdfs tool
  • failover support ([X])

Building and testing

cargo test --lib -- --nocapture

Integration tests

See Integration tests

Changelog

See CHANGES.md

Dependencies

~8.5MB
~159K SLoC