#flat-file #cache #osm #latitude-longitude #node #multi-threading #pair

osmnodecache

Flat file OSM node cache to store (latitude,longitude) pairs as indexed entries

12 releases (7 breaking)

0.8.4 Oct 3, 2023
0.8.0 Mar 23, 2023
0.7.0 Feb 10, 2022
0.6.0 Dec 17, 2021
0.2.0 Nov 28, 2021

#106 in Geospatial


Used in osm2rdf

MIT/Apache

27KB
600 lines

osm-node-cache

GitHub crates.io version docs.rs docs crates.io version CI build

Flat file node cache stores lat,lon coordinate pairs as u64 values with their index being the position in the file. In other words - 0th u64 value is stored as the first 8 bytes, etc.

The library allows multithreaded access to the cache, and can dynamically grow the cache file.

// This example uses osmpbf crate
use std::path::PathBuf;
use rayon::iter::{ParallelBridge, ParallelIterator};
use osmnodecache::{DenseFileCache, CacheStore as _};
use osmpbf::{BlobReader, BlobDecode};

fn main() {
  let reader = BlobReader::from_path("planet.osm.pbf").unwrap();
  let file_cache = DenseFileCache::new(PathBuf::from("node.cache")).unwrap();

  reader.par_bridge().for_each_with(
    file_cache,
    |fc, blob| {
      let mut cache = fc.get_accessor();
      if let BlobDecode::OsmData(block) = blob.unwrap().decode().unwrap() {
        for node in block.groups().flat_map(|g| g.dense_nodes()) {
          cache.set_lat_lon(node.id as usize, node.lat(), node.lon());
        }
      };
    });
}

Development

  • This project is easier to develop with just, a modern alternative to make. Install it with cargo install just.
  • To get a list of available commands, run just.
  • To run tests, use just test.
  • On git push, it will run a few validations, including cargo fmt, cargo clippy, and cargo test. Use git push --no-verify to skip these checks.

License

Licensed under either of

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Dependencies

~2.1–7.5MB
~64K SLoC