#allocation #header #memory #tracker

near-rust-allocator-proxy

Rust allocator proxy with added header

2 unstable releases

Uses new Rust 2021

0.4.0 Jan 19, 2022
0.3.1 Dec 25, 2021
0.3.0 Oct 11, 2021
0.2.9 Feb 23, 2021

#187 in Magic Beans

Download history 333/week @ 2022-03-15 296/week @ 2022-03-22 214/week @ 2022-03-29 318/week @ 2022-04-05 398/week @ 2022-04-12 413/week @ 2022-04-19 324/week @ 2022-04-26 273/week @ 2022-05-03 303/week @ 2022-05-10 229/week @ 2022-05-17 380/week @ 2022-05-24 459/week @ 2022-05-31 367/week @ 2022-06-07 197/week @ 2022-06-14 328/week @ 2022-06-21 248/week @ 2022-06-28

1,190 downloads per month
Used in 3 crates

Apache-2.0

18KB
388 lines

Track Rust memory usage by adding a 32 bytes header to all allocations. See https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html

Usage

You can use code below to enable usage of this library:

use near_rust_allocator_proxy::allocator::MyAllocator;
use jemallocator::Jemalloc;

#[global_allocator]
static ALLOC: MyAllocator<Jemalloc> = MyAllocator::new(Jemalloc);

Design

  • header - For each memory allocation we add a 32 bytes header. This allows figuring out how memory was allocated by looking at memory dump of the process.
  • per thread memory usage stats - thread_memory_usage(tid) method can be used to get amount of memory allocated by thread
  • PRINT_STACK_TRACE_ON_MEMORY_SPIKE - if set to true a stack trace will be used on memory spike

Constants

  • ENABLE_STACK_TRACE - if enabled backtrace will get executed on each allocation and stack pointer will be added to the header
  • MIN_BLOCK_SIZE - if allocation size of below MIN_BLOCK_SIZE, we will only run backtrace SMALL_BLOCK_TRACE_PROBABILITY percentage of time
  • SMALL_BLOCK_TRACE_PROBABILITY - probability of running a stack trace for small allocations
  • REPORT_USAGE_INTERVAL - if printing memory spikes is enabled print if memory usage exceeded this value in bytes
  • PRINT_STACK_TRACE_ON_MEMORY_SPIKE - if true print stack trace when memory usage exceeds REPORT_USAGE_INTERVAL on given Rust thread

Header representation

Allocation structure:

  • magic - unique 8 bytes identifier, which is used to mark memory allocations
  • size - size in bytes
  • tid - thread id
  • stack - stack trace during time of allocation
#[repr(C)]
struct AllocHeader {
    magic: u64,
    size: u64,
    tid: u64,
    stack: [*mut c_void; STACK_SIZE],
}

TODO

  • Add methods to set the configuration instead of having to change the constants.
  • Add tests

Dependencies

~5.5MB
~107K SLoC