feat(rpc): add lightwalletd-compatible gRPC server implementing CompactTxStreamer#10953
Draft
arya2 wants to merge 3 commits into
Draft
feat(rpc): add lightwalletd-compatible gRPC server implementing CompactTxStreamer#10953arya2 wants to merge 3 commits into
arya2 wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an opt-in CompactTxStreamer gRPC server so Zebra can directly serve light clients.
Changes:
- Implements gRPC handlers using Zebra RPC, state, tip, and mempool services.
- Adds configuration, startup wiring, vendored protobufs, and generated bindings.
- Updates user and crate changelogs.
Assessment: The implementation targets an outdated lightwalletd schema and has unresolved correctness, DoS, backpressure, and test-coverage issues. The supplied metadata also lacks a link to the pre-discussed issue or maintainer acknowledgment.
Reviewed changes
Copilot reviewed 12 out of 15 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
CHANGELOG.md |
Documents the new server. |
zebra-rpc/CHANGELOG.md |
Records the crate API addition. |
zebra-rpc/build.rs |
Builds or copies protobuf artifacts. |
zebra-rpc/proto/compact_formats.proto |
Defines compact block messages. |
zebra-rpc/proto/service.proto |
Defines CompactTxStreamer. |
zebra-rpc/proto/__generated__/cash.z.wallet.sdk.rpc.rs |
Generated tonic bindings. |
zebra-rpc/src/config/rpc.rs |
Adds the listen address setting. |
zebra-rpc/src/lib.rs |
Exposes the module. |
zebra-rpc/src/lightwalletd.rs |
Converts chain data to compact messages. |
zebra-rpc/src/lightwalletd/methods.rs |
Implements the gRPC methods. |
zebra-rpc/src/lightwalletd/server.rs |
Initializes the tonic server. |
zebra-rpc/src/server/tests/vectors.rs |
Updates configuration fixtures. |
zebrad/src/commands/start.rs |
Starts and monitors the server. |
Comment on lines
+41
to
+45
| message CompactTx { | ||
| // Index and hash will allow the receiver to call out to chain | ||
| // explorers or other data structures to retrieve more information | ||
| // about this transaction. | ||
| uint64 index = 1; // the index within the full block |
| continue; | ||
| } | ||
|
|
||
| let mut matches = txids.iter().filter(|txid| txid.0.ends_with(suffix)); |
| let (sapling_response, orchard_response) = futures::join!(sapling_request, orchard_request); | ||
|
|
||
| let sapling_tree_size = match sapling_response { | ||
| Ok(ReadResponse::SaplingTree(tree)) => tree.map(|tree| tree.count()).unwrap_or_default(), |
| }; | ||
|
|
||
| let orchard_tree_size = match orchard_response { | ||
| Ok(ReadResponse::OrchardTree(tree)) => tree.map(|tree| tree.count()).unwrap_or_default(), |
|
|
||
| let subtree_root = SubtreeRoot { | ||
| root_hash, | ||
| completing_block_hash: completing_block_hash.0.to_vec(), |
Comment on lines
+739
to
+741
| if response_sender.send(response).await.is_err() || is_err { | ||
| return; | ||
| } |
Comment on lines
+481
to
+483
| if response_sender.send(Ok(subtree_root)).await.is_err() { | ||
| return; | ||
| } |
| &self, | ||
| request: Request<AddressList>, | ||
| ) -> Result<Response<Balance>, Status> { | ||
| let addresses = request.into_inner().addresses; |
Comment on lines
+798
to
+800
| let response = rpc | ||
| .get_address_utxos(GetAddressUtxosRequest::new(args.addresses, false)) | ||
| .await |
Comment on lines
+48
to
+50
| #[tonic::async_trait] | ||
| impl<Rpc, ReadStateService, Mempool, Tip> CompactTxStreamer | ||
| for LightwalletdRPC<Rpc, ReadStateService, Mempool, Tip> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Zcash light clients currently need a separate
lightwalletdinstance sitting between them and a full node. This PR lets Zebra serve light clients directly by implementing the lightwalletdCompactTxStreamergRPC interface inzebra-rpc.Solution
Adds a new
zebra_rpc::lightwalletdmodule: a tonic gRPC server implementing all 18CompactTxStreamermethods, enabled with a newrpc.lightwalletd_listen_addrconfig field (disabled by default).service.proto/compact_formats.protofiles fromzcash/lightwalletdare vendored intozebra-rpc/proto/and compiled by the existing build-script mechanism, with generated code checked intoproto/__generated__/so crates.io builds don't needprotoc.GetTreeState,GetSubtreeRoots,GetTaddressBalance,GetTaddressTxids,GetAddressUtxos,SendTransaction,GetLightdInfo) call the existing JSON-RPCRpcImplmethods in-process — the same methods lightwalletd calls on zcashd/Zebra over HTTP.GetBlock,GetBlockRange, and the nullifier variants) and mempool streams (GetMempoolTx,GetMempoolStream) are built directly from the read-state and mempool services, following the existing indexer gRPC server's patterns. Compact-block tree sizes are fetched by the block's own hash so a concurrent reorg can't makeChainMetadatainconsistent with the block.Excludeentries inGetMempoolTxare little-endian txid suffixes, andGetMempoolStreamsendsheight: 0for mempool transactions (Figure out what to do in librustzcash about the brokenGetTransactionmethod of lightwalletd zcash/librustzcash#1484).Pingis disabled, matching production lightwalletd instances.Tests
BlockIDconversion andExcludelist matching (byte order, ambiguous and empty entries).zebra-rpctests pass (cargo nextest run -p zebra-rpc --release: 113 passed).lightwalletd_listen_addrset, mined 5 blocks via thegenerateRPC, and exercised 12 methods with a tonicCompactTxStreamerClient(GetLightdInfo,GetLatestBlock,GetBlockby height and by hash,GetBlockRange,GetTreeState,GetLatestTreeState,GetMempoolTx,GetTaddressBalance,GetTaddressTxids,GetAddressUtxos), verifying hashes, heights, coinbase balances, and tree states.cargo fmt --checkandcargo clippy --workspace --all-targetsare clean for the touched crates.Specifications & References
GetTransactionmethod of lightwalletd zcash/librustzcash#1484 (RawTransaction height semantics)Follow-up Work
GetBlockRangeissues 3 sequential state reads per block; pipelining them would matter for full-chain light-client sync throughput.pub use wire::*re-exports all generated proto types fromzebra-rpc; a curated re-export would shrink the semver-checked public API surface.fs::read) means checked-in*_descriptor.binfiles now refresh from localprotocoutput, which regeneratedindexer_descriptor.binin this PR and may churn across contributor protoc versions.AI Disclosure
PR Checklist
type(scope): description