#file #memory #shared-memory #memory-mapped #fd #memfd #sealing

memfile

memory backed files using memfd_create with file sealing support

9 releases

0.3.2 Dec 18, 2023
0.3.1 Nov 28, 2023
0.2.1 Oct 30, 2021
0.2.0 Apr 1, 2021
0.1.3 Feb 2, 2021

#151 in Unix APIs

Download history 873/week @ 2023-12-23 1467/week @ 2023-12-30 3532/week @ 2024-01-06 2477/week @ 2024-01-13 1591/week @ 2024-01-20 2221/week @ 2024-01-27 2291/week @ 2024-02-03 2230/week @ 2024-02-10 1698/week @ 2024-02-17 2248/week @ 2024-02-24 2053/week @ 2024-03-02 2176/week @ 2024-03-09 2344/week @ 2024-03-16 2145/week @ 2024-03-23 1956/week @ 2024-03-30 1490/week @ 2024-04-06

8,424 downloads per month
Used in 6 crates (5 directly)

BSD-2-Clause

35KB
631 lines

Docs.rs CI

memfile

This contains thin wrappers around memfd_create and the associated file sealing API.

The MemFile struct represents a file created by the memfd_create syscall. Such files are memory backed and fully anonymous, meaning no other process can see them (well... except by looking in /proc on Linux).

After creation, the file descriptors can be shared with child processes, or even sent to another process over a Unix socket. The files can then be memory mapped to be used as shared memory.

This is all quite similar to shm_open, except that files created by shm_open are not anoynmous. Depending on your application, the anonymous nature of memfd may be a nice property. Additionally, files created by shm_open do not support file sealing.

OS support:

Currently supported operating systems are:

  • Linux
  • Android
  • FreeBSD

The Seal::FutureWrite seal is only supported on Linux.

Please feel free to open a PR to add support for your favorite OS if it provides memfd_create.

File sealing

You can enable file sealing for MemFile by creating them with CreateOptions::allow_sealing(true). This allows you to use MemFile::add_seals to add seals to the file. You can also get the list of seals with MemFile::get_seals.

Once a seal is added to a file, it can not be removed. Each seal prevents certain types of actions on the file. For example: the Seal::Write seal prevents writing to the file, both through syscalls and memory mappings, and the Seal::Shrink and Seal::Grow seals prevent the file from being resized.

This is quite interesting for Rust, as it is the only guaranteed safe way to map memory: when a file is sealed with Seal::Write and Seal::Shrink, the file contents can not change, and the file can not be shrinked. The latter is also important, because trying to read from a memory mapping a of file that was shrinked too far will raise a SIGBUS signal and likely crash your application.

Another interesting option is to first create a shared, writable memory mapping for your MemFile, and then add the Seal::FutureWrite and Seal::Shrink seals. In that case, only the existing memory mapping can be used to change the contents of the file, even after the seals have been added. When sharing the file with other processes, it prevents those processes from shrinking or writing to the file, while the original process can still change the file contents.

Example

use memfile::{MemFile, CreateOptions, Seal};
use std::io::Write;

let mut file = MemFile::create("foo", CreateOptions::new().allow_sealing(true))?;
file.write_all(b"Hello world!")?;
file.add_seals(Seal::Write | Seal::Shrink | Seal::Grow)?;
// From now on, all writes or attempts to created shared, writable memory mappings will fail.

Dependencies

~42KB