#data #download #decode #level #real-time #ii #aws

nexrad

Download and decode functions for NEXRAD radar data

3 releases

0.0.3 Oct 1, 2023
0.0.2 Sep 30, 2023
0.0.1 Sep 28, 2023

#1 in #ii

Download history 83/week @ 2024-01-08 25/week @ 2024-02-12 16/week @ 2024-02-19 28/week @ 2024-02-26 8/week @ 2024-03-11 59/week @ 2024-04-01

67 downloads per month

MIT license

64KB
702 lines

NEXRAD

Crate Rust CI Rust CD

Download and decode functions for NEXRAD radar data.

Summary

This library provides functions to download and decode NEXRAD Level II data from AWS uploaded in near real-time by NOAA.

An EF4 tornado near Des Moines, IA on March 5, 2022 rendered using this library's "render" example.

Downloading

The download feature may be enabled to download NEXRAD Level II data from AWS. For more information on this data source, see the Registry of Open Data's page. As the radar rotates or "sweeps" it collects data which is aggregated into messages. The messages are broken into 5-minute "chunks" before being compressed and uploaded to AWS.

The data is organized by site and date. Here is an example of downloading the first file for April 6, 2023 from KDMX:

let site = "KDMX";
let date = NaiveDate::from_ymd_opt(2023, 4, 6).expect("is valid date");

let metas = list_files(site, &date).await?;
if let Some(meta) = metas.first() {
    println!("Downloading {}...", meta.identifier());
    let downloaded_file = download_file(meta).await?;
    
    println!("Data file size (bytes): {}", downloaded_file.len());
    println!("Data file is compressed: {}", is_compressed(downloaded_file));
} else {
    println!("No files found for the specified date/site to download.");
}

In this example, list_files is being used to query which files are available for the specified site and date, and download_file is used to download the contents of the first file. The downloaded file will need to be decompressed and decoded before the data can be inspected.

Decompression

Raw data files are compressed with bzip2 and must be decompressed prior to decoding. Here is an example of decompressing a file:

let compressed_file = ...;

println!("Data file size (bytes): {}", compressed_file.data().len());
println!("Data file is compressed: {}", compressed_file.compressed());

let decompressed_file = decompress_file(&compressed_file)?;
println!("Decompressed data file size (bytes): {}", decompressed_file.data().len());
println!("Decompressed data file is compressed: {}", decompressed_file.compressed());

Decoding

A decompressed data file consists of binary-encoded messages containing sweep data. Here is an example of decoding a file:

let decompressed_file = ...;

let decoded = decode_file(&decompressed_file)?;
println!("Decoded file with {} elevations.", decoded.elevation_scans().len());

Rendering

A downloaded file can be rendered to an image using the render example. Here is an example usage and the result:

cargo run --example render KDMX20220305_233003_V06

Acknowledgements

I consulted the following resources when developing this library:

https://github.com/bwiggs/go-nexrad

https://trmm-fc.gsfc.nasa.gov/trmm_gv/software/rsl/

Dependencies

~2.3–9MB
~149K SLoC