6 releases
0.1.6 | Aug 9, 2024 |
---|---|
0.1.5 | Jun 3, 2024 |
0.1.4 | Apr 27, 2024 |
0.1.2 | Dec 4, 2023 |
0.1.0 | Nov 30, 2023 |
1,688 downloads per month
Used in mem_dbg
33KB
349 lines
mem_dbg
Traits and associated procedural macros to display recursively the layout and memory usage of a value.
The trait MemDbg
can be used to display the recursive layout of a value,
together with the size of each part and the associated padding bytes. We provide
implementations for most basic types, a derive macro for structs and enums
whose fields implement MemDbg
, and support for a few other crates.
To compute the size, we provide the trait MemSize
and a derive macro that
can be used to compute the size of a value in bytes as the standard library
function std::mem::size_of
returns the stack size of a type in bytes, but it
does not take into consideration heap memory.
Why MemSize
Other traits partially provide the functionality of MemSize
, but either they
require implementing manually a trait, which is prone to error, or they do not
provide the flexibility necessary for MemDbg
. Most importantly, MemSize
uses the type system to avoid iterating over the content of a container (a
vector, etc.) when it is not necessary, making it possible to compute instantly
the size of values occupying hundreds of gigabytes of heap memory.
This is the result of the benchmark bench_hash_map
contained in the examples
directory. It builds a hash map with a hundred million entries and then measures
its heap size:
Allocated: 2281701509
get_size: 1879048240 152477833 ns
deep_size_of: 1879048240 152482000 ns
size_of: 2281701432 152261958 ns
mem_size: 2281701424 209 ns
The first line is the number of bytes allocated by the program as returned by
cap
. Then, we display the result of get-size
, deepsize
, size-of
,
and our own MemSize
. Note that the first two crates are just measuring the
space used by the items, and not by the data structure (i.e., they are not
taking into account the load factor and the power-of-two size constraint of the
hash map). Moreover, all other crates are about six orders of magnitude slower
than our implementation, due to the necessity to iterate over all elements.
Padding
The trait MemDbg
is useful to display the layout of a value and understand
how much memory is used by each part. In particular, it exploits the new stable
macro std::mem::offset_of
to display the padding of each field in square
brackets; moreover, the flag DbgFlags::RUST_LAYOUT
makes it possible to
display structures in the layout used by the Rust compiler, rather than
that given by declaration order.
These features are also available for enums using the feature offset_of_enum
,
which however needs the nightly compiler, as it enables the unstable features
offset_of_enum
and offset_of_nested
.
Features
offset_of_enum
: support for padding and for theDbgFlags::RUST_LAYOUT
flag for enums. Requires the nightly compiler as it enables the unstable featuresoffset_of_enum
andoffset_of_nested
. Callingmem_dbg
with the flagDbgFlags::RUST_LAYOUT
without this feature enabled will result in a panic.half
: support for thehalf
crate.maligned
: support for themaligned
crate.mmap-rs
: support for themmap-rs
crate.rand
: support for therand
crate.
Example
# #![cfg_attr(feature = "offset_of_enum", feature(offset_of_enum, offset_of_nested))]
# fn main() -> Result<(), Box<dyn std::error::Error>> {
use mem_dbg::*;
#[derive(MemSize, MemDbg)]
struct Struct<A, B> {
a: A,
b: B,
test: isize,
}
#[derive(MemSize, MemDbg)]
struct Data<A> {
a: A,
b: Vec<i32>,
c: (u8, String),
}
#[derive(MemSize, MemDbg)]
enum TestEnum {
Unit,
Unit2(),
Unit3 {},
Unnamed(usize, u8),
Named { first: usize, second: u8 },
}
let b = Vec::with_capacity(100);
let s = Struct {
a: TestEnum::Unnamed(0, 16),
b: Data {
a: vec![0x42_u8; 700],
b,
c: (1, "foo".to_owned()),
},
test: -0xbadf00d,
};
println!("size: {}", s.mem_size(SizeFlags::default()));
println!("capacity: {}", s.mem_size(SizeFlags::CAPACITY));
println!();
s.mem_dbg(DbgFlags::empty())?;
println!();
println!("size: {}", s.mem_size(SizeFlags::default()));
println!("capacity: {}", s.mem_size(SizeFlags::CAPACITY));
println!();
s.mem_dbg(DbgFlags::default() | DbgFlags::CAPACITY | DbgFlags::HUMANIZE)?;
#[cfg(feature = "offset_of_enum")]
{
println!();
println!("size: {}", s.mem_size(SizeFlags::default()));
println!("capacity: {}", s.mem_size(SizeFlags::CAPACITY));
println!();
s.mem_dbg(DbgFlags::empty() | DbgFlags::RUST_LAYOUT)?;
}
# Ok(())
# }
The previous program prints:
size: 807
capacity: 1207
807 B ⏺
16 B ├╴a
│ ├╴Variant: Unnamed
8 B │ ├╴0
1 B │ ╰╴1
783 B ├╴b
724 B │ ├╴a
24 B │ ├╴b
35 B │ ╰╴c
1 B │ ├╴0 [7B]
27 B │ ╰╴1
8 B ╰╴test
size: 807
capacity: 1207
1.207 kB 100.00% ⏺: readme::main::Struct<readme::main::TestEnum, readme::main::Data<alloc::vec::Vec<u8>>>
16 B 1.33% ├╴a: readme::main::TestEnum
│ ├╴Variant: Unnamed
8 B 0.66% │ ├╴0: usize
1 B 0.08% │ ╰╴1: u8
1.183 kB 98.01% ├╴b: readme::main::Data<alloc::vec::Vec<u8>>
724 B 59.98% │ ├╴a: alloc::vec::Vec<u8>
424 B 35.13% │ ├╴b: alloc::vec::Vec<i32>
35 B 2.90% │ ╰╴c: (u8, alloc::string::String)
1 B 0.08% │ ├╴0: u8 [7B]
27 B 2.24% │ ╰╴1: alloc::string::String
8 B 0.66% ╰╴test: isize
If run with the feature offset_of_enum
, it prints:
size: 807
capacity: 1207
807 B ⏺
16 B ├╴a
│ ├╴Variant: Unnamed
8 B │ ├╴0
1 B │ ╰╴1 [6B]
783 B ├╴b
724 B │ ├╴a
24 B │ ├╴b
35 B │ ╰╴c
1 B │ ├╴0 [7B]
27 B │ ╰╴1
8 B ╰╴test
size: 807
capacity: 1207
1.207 kB 100.00% ⏺: readme::main::Struct<readme::main::TestEnum, readme::main::Data<alloc::vec::Vec<u8>>>
16 B 1.33% ├╴a: readme::main::TestEnum
│ ├╴Variant: Unnamed
8 B 0.66% │ ├╴0: usize
1 B 0.08% │ ╰╴1: u8 [6B]
1.183 kB 98.01% ├╴b: readme::main::Data<alloc::vec::Vec<u8>>
724 B 59.98% │ ├╴a: alloc::vec::Vec<u8>
424 B 35.13% │ ├╴b: alloc::vec::Vec<i32>
35 B 2.90% │ ╰╴c: (u8, alloc::string::String)
1 B 0.08% │ ├╴0: u8 [7B]
27 B 2.24% │ ╰╴1: alloc::string::String
8 B 0.66% ╰╴test: isize
size: 807
capacity: 1207
807 B ⏺
783 B ├╴b
724 B │ ├╴a
24 B │ ├╴b
35 B │ ╰╴c
1 B │ ├╴0 [7B]
27 B │ ╰╴1
16 B ├╴a
│ ├╴Variant: Unnamed
1 B │ ├╴1 [6B]
8 B │ ╰╴0
8 B ╰╴test
Caveats
-
We support out-of-the-box most basic types, and tuples up to size ten. The derive macros
MemSize
/MemDbg
will generate implementations for structs and enums whose fields implement the associated interface: if this is not the case (e.g., because of the orphan rule) one can implement the traits manually. -
If you invoke the methods of this crate on a shared reference, the compiler will automatically dereference it, and the method will be invoked on the referenced type:
# fn main() -> Result<(), Box<dyn std::error::Error>> {
use mem_dbg::*;
let mut x: [i32; 4] = [0, 0, 0, 0];
assert_eq!(
(&x).mem_size(SizeFlags::default()),
std::mem::size_of::<[i32; 4]>()
);
assert_eq!(
(&mut x).mem_size(SizeFlags::default()),
std::mem::size_of::<&mut [i32; 4]>()
);
assert_eq!(
<&[i32; 4] as MemSize>::mem_size(&&x, SizeFlags::default()),
std::mem::size_of::<&[i32; 4]>()
);
# Ok(())
# }
-
Computation of the size of arrays, slices, and vectors will be performed by iterating over their elements unless the type is a copy type that does not contain non-
'static
references and it is declared as such using the attribute#[copy_type]
. SeeCopyType
for more details. -
The content of vectors and slices is not expanded recursively as the output might be too complex; this might change in the future (e.g., via a flag) should interesting use cases arise.
-
BTreeMap
/BTreeSet
are not currently supported as we still have to figure out a way to precisely measure their memory size and capacity.
Dependencies
~240–690KB
~16K SLoC