16 releases

0.3.2 Mar 21, 2024
0.3.1 Sep 21, 2023
0.3.0 Jul 11, 2023
0.2.0 Jan 10, 2023
0.1.4 Jun 3, 2022

#151 in Filesystem

Download history 3690/week @ 2024-01-25 3439/week @ 2024-02-01 426/week @ 2024-02-08 2143/week @ 2024-02-15 2181/week @ 2024-02-22 1824/week @ 2024-02-29 1369/week @ 2024-03-07 1842/week @ 2024-03-14 1701/week @ 2024-03-21 1369/week @ 2024-03-28 1242/week @ 2024-04-04 1876/week @ 2024-04-11 1315/week @ 2024-04-18 1693/week @ 2024-04-25 1235/week @ 2024-05-02 1940/week @ 2024-05-09

6,383 downloads per month
Used in 2 crates (via opendal)


50K SLoC

C 49K SLoC // 0.1% comments Rust 1.5K SLoC // 0.0% comments

hdrs   Build Status Latest Version

HDFS Native Client in Rust based on hdfs-sys.

Quick Start

use std::io::{Read, Write};

use hdrs::Client;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let fs = Client::connect("hdfs://")?;

    let mut f = fs.open_file().write(true).create(true).open("/tmp/hello.txt")?;
    let n = f.write("Hello, World!".as_bytes())?;

    let mut f = fs.open_file().read(true).open("/tmp/hello.txt")?;
    let mut buf = vec![0; 1024];
    let n = f.read(&mut buf)?;

    let _ = fs.remove_file("/tmp/hello.txt")?;



hdrs depends on hdfs-sys which links libjvm to work.

Please make sure JAVA_HOME is set correctly:

export JAVA_HOME=/path/to/java
  • Enable vendored feature to compile libhdfs and link in static.
  • Specify HDFS_LIB_DIR or HADOOP_HOME to load from specified path instead of compile.
  • Specify HDFS_STATIC=1 to link libhdfs in static.
  • And finally, we will fallback to compile libhdfs and link in static.


hdrs depends on hdfs-sys which uses JNI to call functions provided by jars that provided by hadoop releases.

Please also make sure HADOOP_HOME, LD_LIBRARY_PATH, CLASSPATH is set correctly during runtime:

export HADOOP_HOME=/path/to/hadoop
export CLASSPATH=$(${HADOOP_HOME}/bin/hadoop classpath --glob)

If libhdfs is configued to link dynamiclly, please also add ${HADOOP_HOME}/lib/native in LD_LIBRARY_PATH to make sure linker can find libhdfs.so:

export LD_LIBRARY_PATH=${JAVA_HOME}/lib/server:${HADOOP_HOME}/lib/native:${LD_LIBRARY_PATH}

Version Requirement

hdrs requires at least hadoop 2.3 to work: hadoop 2.2 doesn't handle FileNotFound correctly.

hdrs requires at least hadoop 2.6 to work: Older version of hadoop doesn't handle errno correctly. In older versions, hadoop will set errno to 3 if input path is an empty dir.


Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.

Getting help

Submit issues for bug report or asking questions in discussion.


This project is highly inspired by clang-sys


Licensed under Apache License, Version 2.0.


~50K SLoC