16 releases
new 1.0.0-beta.42 | Oct 30, 2024 |
---|---|
1.0.0-beta.38 | Jul 2, 2024 |
1.0.0-beta.35 | Feb 26, 2024 |
1.0.0-beta.32 | Dec 12, 2023 |
0.0.0-placeholder.0 | Apr 18, 2023 |
#4 in #checkpoint
51 downloads per month
Used in zebrad
1.5MB
30K
SLoC
Zebra Utilities
Tools for maintaining and testing Zebra:
- zebra-checkpoints
- zebrad-hash-lookup
- zebrad-log-filter
- zcash-rpc-diff
- scanning-results-reader
- openapi-generator
Binaries are easier to use if they are located in your system execution path.
zebra-checkpoints
This command generates a list of zebra checkpoints, and writes them to standard output. Each checkpoint consists of a block height and hash.
Automatic Checkpoint Generation in CI
Zebra's GitHub workflows automatically generate checkpoints after every main
branch update.
These checkpoints can be copied into the main-checkpoints.txt
and test-checkpoints.txt
files.
To find the latest checkpoints on the main
branch:
- Find the latest completed
CI Docker
workflow run onmain
. Due to GitHub UI issues, some runs will show as waiting, cancelled, or failed, but the checkpoints have still been generated. - Go to the
Result of checkpoints-mainnet
step in theRun checkpoints-mainnet
job, in theGenerate checkpoints mainnet
job - Scroll down until you see the list of checkpoints, it should start around line 200
- Add those checkpoints to the end of
zebra-consensus/src/checkpoint/main-checkpoints.txt
- Repeat steps 2 to 4 for
Generate checkpoints testnet
- Open a pull request at https://github.com/ZcashFoundation/zebra/pulls
Manual Checkpoint Generation
To create checkpoints, you need a synchronized instance of zebrad
or zcashd
.
zebrad
can be queried directly or via an installed zcash-cli
RPC client.
zcashd
must be queried via zcash-cli
, which performs the correct RPC authentication.
Checkpoint Generation Setup
Make sure your zebrad
or zcashd
is listening for RPC requests,
and synced to the network tip.
If you are on a Debian system, zcash-cli
can be installed as a package.
zebra-checkpoints
is a standalone rust binary, you can compile it using:
cargo install --locked --features zebra-checkpoints --git https://github.com/ZcashFoundation/zebra zebra-utils
Checkpoint Generation Commands
You can update the checkpoints using these commands:
zebra-checkpoints --last-checkpoint $(tail -1 zebra-consensus/src/checkpoint/main-checkpoints.txt | cut -d" " -f1) | tee --append zebra-consensus/src/checkpoint/main-checkpoints.txt &
zebra-checkpoints --last-checkpoint $(tail -1 zebra-consensus/src/checkpoint/test-checkpoints.txt | cut -d" " -f1) -- -testnet | tee --append zebra-consensus/src/checkpoint/test-checkpoints.txt &
wait
When updating the lists there is no need to start from the genesis block. The program option
--last-checkpoint
will let you specify at what block height you want to start. Usually, the
maintainers will copy the last height from each list, and start from there.
Other useful options are:
--transport direct
: connect directly to azebrad
instance--addr
: supply a custom RPC address and port for the node-- -testnet
: connect thezcash-cli
binary to a testnet node instance
You can see all the zebra-checkpoints
options using:
target/release/zebra-checkpoints --help
For more details about checkpoint lists, see the zebra-checkpoints
README.
Checkpoint Generation for Testnet
To update the testnet checkpoints, zebra-checkpoints
needs to connect to a testnet node.
To launch a testnet node, you can either:
- start
zebrad
with azebrad.toml
withnetwork.network
set toTestnet
, or - run
zcashd -testnet
.
Then use the commands above to regenerate the checkpoints.
Submit new checkpoints as pull request
- If you started from the last checkpoint in the current list, add the checkpoint list to the end of the existing checkpoint file. If you started from genesis, replace the entire file.
- Open a pull request with the updated Mainnet and Testnet lists at: https://github.com/ZcashFoundation/zebra/pulls
zebrad-hash-lookup
Given a block hash the script will get additional information using zcash-cli
.
$ echo "00000001f53a5e284393dfecf2a2405f62c07e2503047a28e2d1b6e76b25f863" | zebrad-hash-lookup
high: 3299
time: 2016-11-02T13:24:26Z
hash: 00000001f53a5e284393dfecf2a2405f62c07e2503047a28e2d1b6e76b25f863
prev: 00000001dbbb8b26eb92003086c5bd854e16d9f16e2e5b4fcc007b6b0ae57be3
next: 00000001ff3ac2b4ccb57d9fd2d1187475156489ae22337ca866bbafe62991a2
$
This program is commonly used as part of zebrad-log-filter
where hashes will be captured from zebrad
output.
zebrad-log-filter
The program is designed to filter the output from the zebra terminal or log file. Each time a hash is seen the script will capture it and get the additional information using zebrad-hash-lookup
.
Assuming zebrad
, zcash-cli
, zebrad-hash-lookup
and zebrad-log-filter
are in your path the program can used as:
$ zebrad -v start | zebrad-log-filter
...
block::Hash("
high: 2800
time: 2016-11-01T16:17:16Z
hash: 00000001ecd754790237618cb79c4cd302e52571ecda7a80e6113c5e423c0e55
prev: 00000003ed8623d9499f4bf80f8bc410066194bf6813762b31560f9319205bf8
next: 00000001436277884eef900772f0fcec9566becccebaab4713fd665b60fab309
"))) max_checkpoint_height=Height(419581)
...
zcash-rpc-diff
This program compares zebrad
and zcashd
RPC responses.
Make sure you have zcashd and zebrad installed and synced.
The script:
- gets the
zebrad
andzcashd
tip height and network - sends the RPC request to both of them using
zcash-cli
- compares the responses using
diff
- leaves the full responses in files in a temporary directory, so you can check them in detail
- if possible, compares different RPC methods for consistency
Assuming zebrad
's RPC port is 28232, you should be able to run:
$ zebra-utils/zcash-rpc-diff 28232 getinfo
Checking zebrad network and tip height...
Checking zcashd network and tip height...
Request:
getinfo
Querying zebrad main chain at height 1649797...
Querying zcashd main chain at height 1649797...
Response diff (between zcashd port and port 28232):
--- /run/user/1000/tmp.g9CJecu2Wo/zebrad-main-1649797-getinfo.json 2022-04-29 14:08:46.766240355 +1000
+++ /run/user/1000/tmp.g9CJecu2Wo/zcashd-main-1649797-getinfo.json 2022-04-29 14:08:46.769240315 +1000
@@ -1,4 +1,16 @@
{
- "build": "1.0.0-beta.8+54.ge83e93a",
- "subversion": "/Zebra:1.0.0-beta.8/"
+ "version": 4070050,
+ "build": "v4.7.0-gitian",
+ "subversion": "/MagicBean:4.7.0/",
... more extra zcashd fields ...
}
Sometimes zcashd will have extra fields (+
) or different data (-
and +
).
And sometimes it will have the same data, but in a different order.
The script will warn you if the heights or networks are different, then display the results of querying the mismatched node states.
The script accepts any RPC, with any number of arguments. If a node doesn't implement an RPC, the script will exit with an error.
Configuration
The script uses the configured zcash-cli
RPC port,
and the zebrad
port supplied on the command-line.
It doesn't actually check what kind of node it is talking to,
so you can compare two zcashd
or zebrad
nodes if you want.
(Just edit the zcash.conf
file used by zcash-cli
, or edit the script.)
You can override the binaries the script calls using these environmental variables:
$ZCASH_CLI
$DIFF
$JQ
Scanning Results Reader
A utility for displaying Zebra's scanning results.
How It Works
- Opens Zebra's scanning storage and reads the results containing scanning keys and TXIDs.
- Fetches the transactions by their TXIDs from Zebra using the
getrawtransaction
RPC. - Decrypts the tx outputs using the corresponding scanning key.
- Prints the memos in the outputs.
How to Try It
Scan the Block Chain with Zebra
-
Follow the Build & Install and Configuration instructions in the Zebra Book.
-
Make sure Zebra runs on Mainnet and listens on the default RPC port by having the following in the config file:
[network] network = 'Mainnet' [rpc] listen_addr = "127.0.0.1:8232"
-
Run Zebra with your config file. You can follow the Scanning the Block Chain section in the book for more details.
Run the Reader
-
To print the memos in outputs decryptable by the provided scanning keys, run the reader while also running Zebra. For example:
cargo run --release --features shielded-scan --bin scanning-results-reader
OpenAPI generator
This utility generates an openapi.yaml
specification by extracting information from RPC method documentation in the zebra-rpc
crate code.
Usage
To use the generator tool, build and run it with the following command:
cargo run --bin openapi-generator --features="openapi-generator"
This command will create or update an openapi.yaml
file at the root of the Zebra project repository.
The latest specification generated using this utility can be found here.
Documentation standard
In order for the script to work, each RPC method documentation needs to follow a specific well-defined format. For example, here is the in-code documentation for the getblock
method, which takes arguments:
/// Returns the requested block by hash or height, as a [`GetBlock`] JSON string.
/// If the block is not in Zebra's state, returns
/// [error code `-8`.](https://github.com/zcash/zcash/issues/5758)
///
/// zcashd reference: [`getblock`](https://zcash.github.io/rpc/getblock.html)
/// method: post
/// tags: blockchain
///
/// # Parameters
///
/// - `hash_or_height`: (string, required, example="1") The hash or height for the block to be returned.
/// - `verbosity`: (number, optional, default=1, example=1) 0 for hex encoded data, 1 for a json object, and 2 for json object with transaction data.
///
/// # Notes
///
/// With verbosity=1, [`lightwalletd` only reads the `tx` field of the
/// result](https://github.com/zcash/lightwalletd/blob/dfac02093d85fb31fb9a8475b884dd6abca966c7/common/common.go#L152),
/// and other clients only read the `hash` and `confirmations` fields,
/// so we only return a few fields for now.
///
/// `lightwalletd` and mining clients also do not use verbosity=2, so we don't support it.
#[rpc(name = "getblock")]
fn get_block(
&self,
hash_or_height: String,
verbosity: Option<u8>,
) -> BoxFuture<Result<GetBlock>>;
An example of a method with no arguments can be the getinfo
call:
#[rpc(name = "getinfo")]
/// Returns software information from the RPC server, as a [`GetInfo`] JSON struct.
///
/// zcashd reference: [`getinfo`](https://zcash.github.io/rpc/getinfo.html)
/// method: post
/// tags: control
///
/// # Notes
///
/// [The zcashd reference](https://zcash.github.io/rpc/getinfo.html) might not show some fields
/// in Zebra's [`GetInfo`]. Zebra uses the field names and formats from the
/// [zcashd code](https://github.com/zcash/zcash/blob/v4.6.0-1/src/rpc/misc.cpp#L86-L87).
///
/// Some fields from the zcashd reference are missing from Zebra's [`GetInfo`]. It only contains the fields
/// [required for lightwalletd support.](https://github.com/zcash/lightwalletd/blob/v0.4.9/common/common.go#L91-L95)
fn get_info(&self) -> Result<GetInfo>;
Find more examples inside the zebra-rpc/src/methods.rs
and the zebra-rpc/src/methods/get_block_template_rpcs.rs
files.
The generator will detect new methods added if they are members of the Rpc
trait for the zebra-rpc/src/methods.rs
file and inside the GetBlockTemplateRpc
in the file zebra-rpc/src/methods/get_block_template_rpcs.rs
.
Dependencies
~35–69MB
~1M SLoC