#pavexc #json #rustdoc-json #json-file #output #fields #parse-json

pavexc_rustdoc_types

The slimmed down schema used by pavexc to work with rustdoc's JSON output

6 releases

new 0.1.57 Nov 27, 2024
0.1.56 Nov 26, 2024

#286 in Database interfaces

Download history 202/week @ 2024-11-06 23/week @ 2024-11-13 241/week @ 2024-11-20

475 downloads per month
Used in 2 crates (via pavexc)

Apache-2.0

49KB
455 lines

pavexc_rustdoc_types

It's a fork of rustdoc_types with a specific purpose: maximise performance when working with JSON docs in pavexc.
In particular, we want to minimise the size of the JSON files stored in pavexc SQLite database as well as the time it takes to parse the JSON files returned by rustdoc.

Changes

The changes are rather minimal: we comment out some fields that are not used in pavexc but account for a significant portion of the JSON file size. They are all marked out with a // EXCLUDED! comment.

We rely on this script:

total_size=$(wc -c < file.json); docs_size=$(jq '.. | .field_name1? // .field_name2? // empty' file.json | wc -c); echo "scale=2; $docs
_size*100/$total_size" | bc

to measure the impact of removing field_name1 and field_name2 from the JSON file. The script calculates the percentage of the JSON file size that is taken up by field_name1 and field_name2.

We strive to remove fields that account for a non-negligible portion of the JSON file size, to minimise the overhead of syncing the fork with the upstream repository.

Dependencies

~0.4–1MB
~22K SLoC