#nydus #blob #container-image #metadata #fuse #chunk #rafs

nydus-rafs

The RAFS filesystem format for Nydus Image Service

7 releases

0.3.2 Aug 8, 2023
0.3.1 Jun 17, 2023
0.2.2 Feb 24, 2023
0.2.0 Jan 31, 2023
0.1.0 Jun 16, 2022

#139 in Filesystem

Download history 856/week @ 2024-01-05 818/week @ 2024-01-12 971/week @ 2024-01-19 1417/week @ 2024-01-26 1477/week @ 2024-02-02 848/week @ 2024-02-09 1127/week @ 2024-02-16 1605/week @ 2024-02-23 1493/week @ 2024-03-01 1193/week @ 2024-03-08 1446/week @ 2024-03-15 1162/week @ 2024-03-22 1017/week @ 2024-03-29 1058/week @ 2024-04-05 1168/week @ 2024-04-12 1099/week @ 2024-04-19

4,569 downloads per month
Used in 5 crates (4 directly)

Apache-2.0 OR BSD-3-Clause

1.5MB
33K SLoC

nydus-rafs

The RAFS Fuse filesystem for Nydus Image Service.

Support

Platforms:

  • x86_64
  • aarch64

Operating Systems:

  • Linux
  • MacOS

License

This code is licensed under Apache-2.0 or BSD-3-Clause.


lib.rs:

RAFS: a chunk dedup, on-demand loading, readonly fuse filesystem.

The Rafs filesystem is blob based readonly filesystem with chunk deduplication. A Rafs filesystem is composed up of a metadata blob and zero or more data blobs. A blob is just a plain object containing data chunks. Data chunks may be compressed, encrypted and deduplicated by chunk content digest value. When Rafs file is used for container images, Rafs metadata blob contains all filesystem metadatas, such as directory, file name, permission etc. Actually file contents are divided into chunks and stored into data blobs. Rafs may built one data blob for each container image layer or build a single data blob for the whole image, according to building options.

There are several versions of Rafs filesystem defined:

  • V4: the original Rafs filesystem format
  • V5: an optimized version based on V4 with metadata direct mapping, data prefetching etc.
  • V6: a redesigned version to reduce metadata blob size and inter-operable with in kernel erofs, better support of virtio-fs.

The nydus-rafs crate depends on the nydus-storage crate to access metadata and data blobs and improve performance by caching data on local storage. The nydus-rafs itself includes two main sub modules:

  • fs: the Rafs core to glue fuse, storage backend and filesystem metadata.
  • metadata: defines and accesses Rafs filesystem metadata.

For more information, please refer to Dragonfly Image Service

Dependencies

~15–33MB
~519K SLoC