3 releases
0.1.0 | Sep 18, 2024 |
---|---|
0.1.0-rc2 | Aug 12, 2024 |
#216 in Compression
72 downloads per month
135KB
3K
SLoC
NEXRAD Data
Provides structure definitions and decoding functions for NEXRAD Archive II volume files, along with functions for downloading both archival and real-time data from open cloud providers like AWS OpenData.
Volume Definitions
The nexrad-data::volume
module provides model definitions for the NEXRAD Archive II volume file format described in
the Radar Operations Center's ICD 2620010H for the Archive II/User (as of build 19.0 March 3, 2020). A volume::File
can be constructed with archive or real-time data. It can decode the archive volume header and provide access to LDM
volume::Record
s which can be decompressed and decoded into a series of messages.
AWS Open Data
NOAA uploads archive and real-time NEXRAD data to AWS Open Data S3 buckets which are publicly available. The
nexrad-data::aws
module provides functions for listing and downloading NEXRAD data from these buckets.
Archive Data
Historical volumes are archived by date and radar site in the noaa-nexrad-level2
bucket. The
nexrad-data::aws::archive
module provides functions for accessing these volumes. The archive::list_files
function
queries volumes for a given date and radar site, returning identifiers for each volume. The archive::download_file
function downloads a volume file by its identifier.
Real-Time Data
Real-time volume data is uploaded in chunks to the unidata-nexrad-level2-chunks
bucket. 999 volume directories are
rotated through with chunks being added to each directory until they comprise a full volume. The
nexrad-data::aws::realtime
module provides functions for accessing these chunks. The realtime::list_chunks_in_volume
function queries a volume for its chunks, returning identifiers for each chunk. The realtime::download_chunk
function
downloads a chunk by its identifier. The realtime::get_latest_volume
function can be used to identify which of the 999
volume directories contain the latest data, and the realtime::estimate_next_chunk_time
function can be used to
estimate when the next chunk will be uploaded. Finally, realtime::poll_chunks
function will asynchronously poll for
new chunks in the latest volume directory, downloading them as they become available.
Features
The APIs in this crate should be configured to require only the dependencies they need, leaving the consumer to include as much or little as they desire. By default, all features are included. The following named features are helpful groupings of dependencies/behavior:
decode
- Enables both decoding of the volume headers and of decoding the LDM records' NEXRAD messages usingnexrad-decode
.aws
- Enables accessing archive and real-time NEXRAD data from AWS Open Data.nexrad-model
- Provides mappings to a common radar data model, particularly for mappingvolume::File
into aScan
.
Dependencies
~2–14MB
~160K SLoC