#hdfs #hadoop #webhdfs

bin+lib webhdfs

Hadoop WebHDFS client library

7 releases

0.3.3 Mar 26, 2021
0.3.2 Feb 13, 2021
0.3.1 Sep 28, 2020
0.2.2 Feb 7, 2020
0.1.0 Aug 4, 2019

#280 in Filesystem

Download history 22/week @ 2022-08-15 27/week @ 2022-08-22 31/week @ 2022-08-29 89/week @ 2022-09-05 80/week @ 2022-09-12 40/week @ 2022-09-19 23/week @ 2022-09-26 41/week @ 2022-10-03 46/week @ 2022-10-10 58/week @ 2022-10-17 74/week @ 2022-10-24 85/week @ 2022-10-31 95/week @ 2022-11-07 134/week @ 2022-11-14 94/week @ 2022-11-21 161/week @ 2022-11-28

484 downloads per month

Apache-2.0

115KB
2.5K SLoC

Rust 2K SLoC // 0.1% comments Shell 492 SLoC // 0.1% comments

webhdfs

Hadoop webhdfs client library for Rust.

Crates.io Build Status

Documentation

  • Built on Tokio and Hyper.
  • Provides both synchronous and asynchronous APIs.

See src/bin/webhdfs.rs or tests/it.rs for usage examples.

NOTE: APIs are not yet stabilized and may change towards version 1.0.

TODO list (2nd checkbox is IT coverage)

  • Async read and write ([X])
  • Sync read and write ([X])
  • Stat and dir ([X])
  • Basic filesystem mutations (concat, mkdirs, rename, delete, symlink)
  • File-based configuration
  • Other file and directory operations (permissions, owner, ACL, times, checksum)
  • XAttr operations
  • Snapshot operations
  • Basic delegation token support
  • Full delegation token support
  • webhdfs tool
  • failover support ([X])

Building and testing

cargo test --lib -- --nocapture

Integration tests

See Integration tests

Changelog

See CHANGES.md

Dependencies

~5–12MB
~198K SLoC