#volume #data #archive #real-time #download #data-model #radar

nexrad-data

Models and functions for accessing NEXRAD data from public sources

3 releases

0.1.0 Sep 18, 2024
0.1.0-rc2 Aug 12, 2024

#215 in Compression

Download history 217/week @ 2024-08-10 18/week @ 2024-08-17 174/week @ 2024-09-14 34/week @ 2024-09-21 9/week @ 2024-09-28

155 downloads per month

MIT license

135KB
3K SLoC

NEXRAD Data

Crates.io Docs.rs Rust CI Rust CD

Provides structure definitions and decoding functions for NEXRAD Archive II volume files, along with functions for downloading both archival and real-time data from open cloud providers like AWS OpenData.

Volume Definitions

The nexrad-data::volume module provides model definitions for the NEXRAD Archive II volume file format described in the Radar Operations Center's ICD 2620010H for the Archive II/User (as of build 19.0 March 3, 2020). A volume::File can be constructed with archive or real-time data. It can decode the archive volume header and provide access to LDM volume::Records which can be decompressed and decoded into a series of messages.

AWS Open Data

NOAA uploads archive and real-time NEXRAD data to AWS Open Data S3 buckets which are publicly available. The nexrad-data::aws module provides functions for listing and downloading NEXRAD data from these buckets.

Archive Data

Historical volumes are archived by date and radar site in the noaa-nexrad-level2 bucket. The nexrad-data::aws::archive module provides functions for accessing these volumes. The archive::list_files function queries volumes for a given date and radar site, returning identifiers for each volume. The archive::download_file function downloads a volume file by its identifier.

Real-Time Data

Real-time volume data is uploaded in chunks to the unidata-nexrad-level2-chunks bucket. 999 volume directories are rotated through with chunks being added to each directory until they comprise a full volume. The nexrad-data::aws::realtime module provides functions for accessing these chunks. The realtime::list_chunks_in_volume function queries a volume for its chunks, returning identifiers for each chunk. The realtime::download_chunk function downloads a chunk by its identifier. The realtime::get_latest_volume function can be used to identify which of the 999 volume directories contain the latest data, and the realtime::estimate_next_chunk_time function can be used to estimate when the next chunk will be uploaded. Finally, realtime::poll_chunks function will asynchronously poll for new chunks in the latest volume directory, downloading them as they become available.

Features

The APIs in this crate should be configured to require only the dependencies they need, leaving the consumer to include as much or little as they desire. By default, all features are included. The following named features are helpful groupings of dependencies/behavior:

  • decode - Enables both decoding of the volume headers and of decoding the LDM records' NEXRAD messages using nexrad-decode.
  • aws - Enables accessing archive and real-time NEXRAD data from AWS Open Data.
  • nexrad-model - Provides mappings to a common radar data model, particularly for mapping volume::File into a Scan.

Dependencies

~2–14MB
~163K SLoC