Skip to content

Internal panic in reqwest when transaction body is large #404

@plaidfinch

Description

@plaidfinch

There is an open bug in reqwest (#668): it is an internal expectation that any valid Url is also a valid Uri. Per the current logic for parsing Uris and Urls, this is not always the case. The corner case that affects us is that when the length of the Url exceeds a certain threshold (unclear of the exact limit at time of writing), an internal .expect() fails in request, resulting in:

thread 'main' panicked at 'a parsed Url should always be a valid Uri: InvalidUri(TooLong)', /Users/$USER/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/reqwest-0.11.9/src/into_url.rs:70:14

We encounter this issue whenever we try to submit a transaction to the Tendermint RPC API which is sufficiently long to trigger this condition when included as hex-encoding in the URI, because we are using the URI-over-HTTP RPC API from Tendermint:

let rsp: serde_json::Value = reqwest::get(format!(
r#"http://{}:{}/broadcast_tx_sync?tx=0x{}"#,
self.node,
self.rpc_port,
hex::encode(&transaction.encode_to_vec())
))
.await?
.json()
.await?;

When the hex-encoding of the transaction is too long, the direct inclusion of that encoding in the URI of the GET request causes reqwest to panic.

This issue points to two things that need to be solved:

  1. Transactions are limited in size by the arbitrary bound of the size of a URI, and
  2. Transactions are often very very large.

This issue addresses (1); a follow-up issue will address (2).

Proposed solution to (1)

We should use the JSONRPC-over-HTTP endpoint for Tendermint instead of the URI-over-HTTP endpoint. This encodes the transaction body in the body of a POST, which can be arbitrarily large. See the documentation for the Tendermint RPC: https://docs.tendermint.com/master/rpc/.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-clientArea: Design and implementation for client functionalityC-bugCategory: a bugE-easyEffort: Easy

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions