#keyed #osm #leveldb #osm-pbf #file-format #data-processing #convert

nightly bin+lib peermaps-ingest

Convert OSM data into the peermaps on-disk format

10 stable releases (3 major)

4.0.1 Mar 28, 2022
4.0.0 Oct 7, 2021
3.0.1 Oct 2, 2021
2.2.0 Sep 29, 2021
1.0.2 Jun 18, 2021

#775 in Database interfaces

MIT/Apache

71KB
2K SLoC

peermaps-ingest

Converts OSM data into the peermaps on-disk format.

This is done in two passes of the data. The first pass writes records from a pbf file into a leveldb database keyed by id. The second pass writes all the features that can be rendered into an eyros database with payloads in the georender format.

command-line usage

usage: target/release/peermaps-ingest COMMAND {OPTIONS}

ingest - scans and processes a pbf
  -f, --pbf     osm pbf file to ingest or "-" for stdin (default)
  -e, --edb     eyros db dir to write spatial data
  -o, --outdir  write eyros db in this dir in edb/

  --no-ingest-node      skip over processing nodes
  --no-ingest-way       skip over processing ways
  --no-ingest-relation  skip over processing relations
  --defaults            Print default values for ingest parameters.

  This step will optimize when --optimize is provided.

scan - scans a pbf, outputting a scan file
  -f, --pbf     osm pbf file to ingest or "-" for stdin (default)
  -o, --outdir  write a scan file in this dir
  --scan_file   write scan file with explicit path

ingest-from-scan - process a pbf from an existing scan
  -f, --pbf     osm pbf file to ingest or "-" for stdin (default)
  -e, --edb     eyros db dir to write spatial data
  -o, --outdir  write eyros db in this dir in edb/ and read scan file
  --scan_file   read scan file with explicit path

  --no-ingest-node      skip over processing nodes
  --no-ingest-way       skip over processing ways
  --no-ingest-relation  skip over processing relations
  --defaults            Print default values for ingest parameters.

optimize - recursively rebuild tree sections to improve query performance
  --optimize=X,Y  divide into a grid of X*Y sublevels to rebuild the tree
  -e, --edb       eyros db dir to write spatial data
  -o, --outdir    write eyros db in this dir in edb/ and read scan file

-h, --help     Print this help message
-v, --version  Print the version string (3.0.0)

install

To get the command-line program:

cargo install peermaps-ingest

Dependencies

~13–24MB
~361K SLoC