bump all major dependencies

major bumps are:

1. axum: 0.7.9
2. celestia-types: 0.10.0
3. cnidarium: 0.83.0 (instead of upstream rev ac7abacc)
4. http: 1.2.0
5. hyper: 1.6.0
6. ibc-types: 0.15.0
7. jsonrpsee: 0.24.8
8. metrics: 0.24.1
9. metrics-exporter-prometheus: 0.16.2
10. opentelemetry: 0.27.1
11. opentelemetry-otlp: 0.27.0
12. opentelemetry-semantic-conventions: 0.27.0
13. opentelemetry_sdk: 0.27.1
14. pbjson-build: 0.7.0
15. pbjson-types: 0.7.0
16. penumbra-ibc (renamed to penumbra-sdk-ibc): 1.0.1
16. penumbra-proto (renamed to penumbra-sdk-proto): 1.0.1
17. penumbra-tower-trace (renamed to penumbra-sdk-tower-trace): 1.0.1
18. prost: 0.13.4
19. prost-build: 0.13.4
20. reqwest: 0.12.12
21. tendermint: 0.40.1
22. tendermint-config: 0.40.1
23. tendermint-proto: 0.40.1
24. tendermint-rpc: 0.40.1
25. tonic: 0.12.3
26. tonic-build: 0.12.3
27. tower: 0.5.2
28. tower-http: 0.6.2
29. tracing-opentelemetry: 0.28.0

removed:

1. celestia-tendermint: celestia-types
now relies on upstream tendermint and
celestia-tendermint is no longer maintained.
2. opentelemetry-stdout: not to be used in
production code and the APIs we used were removed.
3. isahc: replaced by reqwest because it has not
seen updates in a long time and was not updated
to the work with hyper and http 1.0 stabilization.

notable overrides:

this patch relies on changes to a number of
dependencies that have not yet been upstreamed:

1. penumbra-sdk-ibc, penumbra-sdk-proto,
penumbra-sdk-tower-trace need a bump to their
dependencies on tonic, tower, tower-actor. Tracked
in
penumbra-zone/penumbra#5055
2. tower-actor: needs a bump to tower. Tracked in
penumbra-zone/tower-actor#5
3. tonic: has its tower dependency bumped to
0.5.2 upstream but is still awaiting a release to
0.13.0. The tower patch was merged in
hyperium/tonic@58345a6

Several crates that had a custom defined
dependency now use the workspace definition.

notable conductor changes:

upgrade the following crates:

celestia-types no longer has celestia-tendermint
as a dependency and instead uses tendermint 0.40
(the same as Astria uses). This required changing
a bunch imports and type paths.

`Blob::new` now requires an `AppVersion` argument
(only needed in tests. Mainnet is running v3.2.0,
so the blackbox tests sets the same, but the
conductor business logic is oblivious to that).

The DataAvailability no longer has public fields
and instead provides a constructor (only in tests,
Conductor business logic is again oblivious).

From celestia-node v0.15.0 blob.GetAll no longer
returns an error if a single requested namespace
(out of a range of namespaces) does not contain
blobs. Instead, the namespace does not show up in
the list of blobs. If there are no blobs at all
then the RPC omits the field entirely, which
the Rust celestia-rpc crate represents as a
`Option<Vec<Blob>>`. All code related to
mapping errors like `BlobNotFound` to an empty
vec have been removed. The current version of
celestia-node on Astria's mainnet is 0.20.0.

tower introduced the BoxCloneSyncService, which
makes constructing a rate-limited, buffered
service a bit easier.

notable sequencer-client changes:

changes except for tests worked out of the box,
but tokio for some reason was a direct dependency.
Changed it to derive from the workspace.

tendermint_rpc::endpoint::broadcast::Response
has a new field codespace, which needed to be
populated in test code.

noteable composer changes:

the api server needed refactoring because axum
had breaking changes. To avoid dealing with
the type parameters in the new axum::serve::Serve
future, composer turns it into a trait object and
wraps it in a api::Serve future.

The geth related blackbox tests also started
to be more flaky for unknown reasons - extending
the timeout duration for the tests mostly fixes
the flakiness. But this will have to be
be investigated in a follow-up PR.

notable bridge-withdrawer changes:

axum::Server was replaced by axum::serve::Serve,
which requires a tokio::net::TcpListener to be
instantiated. We have introduced `api::Serve` as a
type-erased wrapper to `axum::serve::Serve` to
avoid having to specify its type parameteres while
still providing access to its bound socket.
However, because of these changes
BridgeWithdrawer::new is now async.

the http dependency is now specified through the
workspace.

The shutdown signal for the API server was also
changed to be a CancellationToken.

notable sequencer-relayer changes:

axum underwent heavy changes and has replaced
axum::Server by axum::serve::Serve future. This
patch wraps the future to avoid having to name its
type, erasing it, while still giving access to the
bound tcp socket address. Because its constructor
takes a tokio::net::TcpListener, the API server
setup is now async, which in turn required
SequencerRelayer::new to become async.
Due to type erasure + wrapping, the shutdown
signal closure now has to be passed in early on
instead of right before entering
sequencer-relayer's runloop. While doing so,
the oneshot channel was replaced by a cancellation
token.

Because Celestia blobs now require an app version,
the relayer write path now sets it to version 3
(because astria mainnet is targetting celestia-app
3.2.0, and the consensus node is targetting
3.3.0).

In tests, isahc was replaced by reqwest. The
entire workspace is migrating to hyper 1+ and
http 1+ and isahc has not seen upgrades in several
months and there is no work underway to make it
compatible with http 1+.

dependency tower was removed because it was not
used.

notable auctioneer changes:

there was no need for the direct dependencies
on http-body and http-body-util because tonic
already provided reexports and/or type aliases.

notable telemetry changes:

opentelemetry had a couple of breaking changes
that required reworking how the otel pipeline
is set up.

the most visible change is opentelemetry-stdout
no longer allowing to write to stderr or another
sink. because the otel stdout writer was never
used and because opentelemetry-stdout itself
notes that is should not be used in production
contexts this patch removes it entirely (and
with it the the need to explicitly set pretty
printed logs; if services are connected to a
TTY or the FORCE_STDOUT flag is set, they will
always write to stdout).

Author: SuperFluffy

Cargo.lock

@@ -60,56 +60,58 @@ libraries = [{ path = "lint/tracing_debug_field" }]
 
 [workspace.dependencies]
 async-trait = "0.1.52"
-axum = "0.6.16"
+axum = "0.7.9"
 base64 = "0.21"
 base64-serde = "0.7.0"
 bytes = "1"
-celestia-tendermint = "0.32.1"
-celestia-types = "0.1.1"
+celestia-types = "0.10.0"
 clap = "4.5.4"
 const_format = "0.2.32"
 divan = "0.1.14"
 ethers = { version = "2.0.11", default-features = false }
 futures = "0.3"
 hex = "0.4"
 hex-literal = "0.4.1"
-hyper = "0.14"
-ibc-types = "0.12"
+http = "1.2.0"
+hyper = "1.6.0"
+ibc-types = "0.15.0"
 indexmap = "2.1.0"
 insta = "1.36.1"
 itertools = "0.12.1"
 itoa = "1.0.10"
 jiff = "0.2.4"
-jsonrpsee = { version = "0.20" }
-pbjson-types = "0.6"
+jsonrpsee = "0.24.8"
+pbjson-types = "0.7.0"
+
 # Note that when updating the penumbra versions, vendored types in `proto/sequencerapis/astria_vendored` may need to be updated as well.
-penumbra-ibc = { git = "https://github.com/penumbra-zone/penumbra.git", rev = "686fa5b53e8ad306736d2de61d1ffb6d11722e2b", default-features = false }
-penumbra-proto = { git = "https://github.com/penumbra-zone/penumbra.git", rev = "686fa5b53e8ad306736d2de61d1ffb6d11722e2b" }
-penumbra-tower-trace = { git = "https://github.com/penumbra-zone/penumbra.git", rev = "686fa5b53e8ad306736d2de61d1ffb6d11722e2b" }
+penumbra-ibc = { package = "penumbra-sdk-ibc", version = "1.3.1", default-features = false }
+penumbra-proto = { package = "penumbra-sdk-proto", version = "1.3.1" }
+
 pin-project-lite = "0.2.13"
-prost = "0.12"
+prost = "0.13.4"
 rand = "0.8.5"
 regex = "1.9"
 # disable default features and explicitly enable rustls-tls to ensure openssl is disabled
 # in the entire workspace
-reqwest = { version = "0.11", default-features = false, features = [
+reqwest = { version = "0.12.12", default-features = false, features = [
   "rustls-tls",
 ] }
 serde = "1"
 serde_json = "1"
 sha2 = "0.10"
 tempfile = "3.6.0"
-tendermint = "0.34.0"
-tendermint-config = "0.34.0"
-tendermint-proto = "0.34.0"
-tendermint-rpc = "0.34.0"
+tendermint = "0.40.2"
+tendermint-config = "0.40.2"
+tendermint-proto = "0.40.2"
+tendermint-rpc = "0.40.2"
 thiserror = "1"
-tokio = "1.28"
+tokio = { version = "1.28", default-features = false }
 tokio-stream = { version = "0.1.14" }
 tokio-test = "0.4.2"
 tokio-util = "0.7.13"
-tonic = "0.10"
+tonic = "0.12.3"
 tracing = "0.1"
+tower = { version = "0.5.2", default-features = false }
 tryhard = "0.5.1"
 which = "4.4.0"
 wiremock = "0.5"

Cargo.toml

@@ -20,7 +20,6 @@ data:
   ASTRIA_COMPOSER_NO_METRICS: "{{ not .Values.metrics.enabled }}"
   ASTRIA_COMPOSER_METRICS_HTTP_LISTENER_ADDR: "0.0.0.0:{{ .Values.ports.metrics }}"
   ASTRIA_COMPOSER_FORCE_STDOUT: "{{ .Values.global.useTTY }}"
-  ASTRIA_COMPOSER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   NO_COLOR: "{{ .Values.global.useTTY }}"
   ASTRIA_COMPOSER_NO_OTEL: "{{ not .Values.otel.enabled }}"
   OTEL_EXPORTER_OTLP_ENDPOINT: "{{ tpl .Values.otel.endpoint . }}"
@@ -31,6 +30,7 @@ data:
   OTEL_EXPORTER_OTLP_TRACE_HEADERS: "{{ tpl .Values.otel.traceHeaders . }}"
   OTEL_SERVICE_NAME: "{{ tpl .Values.otel.serviceName . }}"
   {{- if not .Values.global.dev }}
+    ASTRIA_COMPOSER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   {{- else }}
   {{- end }}
 ---

charts/composer/templates/configmap.yaml

@@ -21,7 +21,6 @@ data:
   ASTRIA_BRIDGE_WITHDRAWER_NO_METRICS: "{{ not .Values.metrics.enabled }}"
   ASTRIA_BRIDGE_WITHDRAWER_METRICS_HTTP_LISTENER_ADDR: "0.0.0.0:{{ .Values.ports.metrics }}"
   ASTRIA_BRIDGE_WITHDRAWER_FORCE_STDOUT: "{{ .Values.global.useTTY }}"
-  ASTRIA_BRIDGE_WITHDRAWER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   ASTRIA_BRIDGE_WITHDRAWER_NO_OTEL: "{{ not .Values.otel.enabled }}"
 
   NO_COLOR: "{{ .Values.global.useTTY }}"
@@ -33,6 +32,7 @@ data:
   OTEL_EXPORTER_OTLP_TRACE_HEADERS: "{{ tpl .Values.otel.traceHeaders . }}"
   OTEL_SERVICE_NAME: "{{ tpl .Values.otel.serviceName . }}"
   {{- if not .Values.global.dev }}
+    ASTRIA_BRIDGE_WITHDRAWER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   {{- else }}
   ASTRIA_BRIDGE_WITHDRAWER_NO_FROST_THRESHOLD_SIGNING: "{{ .Values.config.noFrostThresholdSigning }}"
   ASTRIA_BRIDGE_WITHDRAWER_FROST_MIN_SIGNERS: "{{ .Values.config.frostMinSigners }}"

charts/evm-bridge-withdrawer/templates/configmaps.yaml

@@ -19,7 +19,6 @@ data:
   ASTRIA_CONDUCTOR_METRICS_HTTP_LISTENER_ADDR: "0.0.0.0:{{ .Values.ports.conductorMetrics }}"
   ASTRIA_CONDUCTOR_SEQUENCER_REQUESTS_PER_SECOND: "{{ .Values.config.conductor.sequencerRequestsPerSecond }}"
   ASTRIA_CONDUCTOR_FORCE_STDOUT: "{{ .Values.global.useTTY }}"
-  ASTRIA_CONDUCTOR_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   NO_COLOR: "{{ .Values.global.useTTY }}"
   ASTRIA_CONDUCTOR_NO_OTEL: "{{ not .Values.otel.enabled }}"
   ASTRIA_CONDUCTOR_NO_CELESTIA_AUTH: "{{ not .Values.config.celestia.token }}"
@@ -31,6 +30,7 @@ data:
   OTEL_EXPORTER_OTLP_TRACE_HEADERS: "{{ tpl .Values.otel.traceHeaders .}}"
   OTEL_SERVICE_NAME: "{{ tpl .Values.otel.serviceNamePrefix . }}-conductor"
   {{- if not .Values.global.dev }}
+    ASTRIA_CONDUCTOR_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   {{- else }}
   {{- end }}
 ---

charts/evm-rollup/templates/configmap.yaml

@@ -15,7 +15,6 @@ data:
   ASTRIA_SEQUENCER_RELAYER_NO_METRICS: "{{ not .Values.config.relayer.metrics.enabled }}"
   ASTRIA_SEQUENCER_RELAYER_METRICS_HTTP_LISTENER_ADDR: "0.0.0.0:{{ .Values.ports.metrics }}"
   ASTRIA_SEQUENCER_RELAYER_FORCE_STDOUT: "{{ .Values.global.useTTY }}"
-  ASTRIA_SEQUENCER_RELAYER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   NO_COLOR: "{{ .Values.global.useTTY }}"
   ASTRIA_SEQUENCER_RELAYER_NO_OTEL: "{{ not .Values.otel.enabled }}"
   OTEL_EXPORTER_OTLP_ENDPOINT: "{{ tpl .Values.otel.endpoint . }}"
@@ -29,6 +28,7 @@ data:
   ASTRIA_SEQUENCER_RELAYER_SEQUENCER_CHAIN_ID: "{{ include "sequencer-relayer.sequencerChainId" . }}"
   ASTRIA_SEQUENCER_RELAYER_CELESTIA_CHAIN_ID: "{{ include "sequencer-relayer.celestiaChainId" . }}"
   {{- if not .Values.global.dev }}
+    ASTRIA_SEQUENCER_RELAYER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   {{- else }}
   {{- end }}
 ---

charts/sequencer-relayer/templates/configmaps.yaml

@@ -64,7 +64,6 @@ data:
   ASTRIA_SEQUENCER_NO_METRICS: "{{ not .Values.sequencer.metrics.enabled }}"
   ASTRIA_SEQUENCER_METRICS_HTTP_LISTENER_ADDR: "0.0.0.0:{{ .Values.ports.sequencerMetrics }}"
   ASTRIA_SEQUENCER_FORCE_STDOUT: "{{ .Values.global.useTTY }}"
-  ASTRIA_SEQUENCER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   NO_COLOR: "{{ .Values.global.useTTY }}"
   ASTRIA_SEQUENCER_NO_OTEL: "{{ not .Values.sequencer.otel.enabled }}"
   OTEL_EXPORTER_OTLP_ENDPOINT: "{{ .Values.sequencer.otel.endpoint }}"
@@ -79,6 +78,7 @@ data:
   # variables which are being deleted in the future.
   {{- if not (or .Values.global.dev (ne (include "sequencer.imageTag" .) .Chart.AppVersion)) }}
   # These are variables which are added in newer releases and are not present in the current release.
+  ASTRIA_SEQUENCER_PRETTY_PRINT: "{{ .Values.global.useTTY }}"
   {{- else }}
   {{- end }}
 ---

charts/sequencer/templates/configmaps.yaml

@@ -40,11 +40,10 @@ tryhard = { workspace = true }
 tonic = { workspace = true }
 tokio-stream = { workspace = true, features = ["sync"] }
 
-http = "0.2.11"
-http-body = "0.4.5"
+http = { workspace = true }
 pin-project-lite = "0.2.15"
-tower = { version = "0.5.1", features = ["util"] }
-tower-http = { version = "0.4.4", features = ["map-response-body", "trace"] }
+tower = { version = "0.5.2", features = ["util"] }
+tower-http = { version = "0.6.2", features = ["map-response-body", "trace"] }
 
 [dev-dependencies]
 config = { package = "astria-config", path = "../astria-config", features = [

crates/astria-auctioneer/Cargo.toml

@@ -36,12 +36,6 @@ ASTRIA_AUCTIONEER_LOG="info"
 # If false span data is written to stdout only if it is connected to a tty.
 ASTRIA_AUCTIONEER_FORCE_STDOUT=false
 
-# If true uses an exceedingly pretty human readable format to write to stdout.
-# If false uses JSON formatted OTEL traces.
-# This does nothing unless stdout is connected to a tty or
-# `ASTRIA_AUCTIONEER_FORCE_STDOUT` is set to `true`.
-ASTRIA_AUCTIONEER_PRETTY_PRINT=false
-
 # If set to any non-empty value removes ANSI escape characters from the pretty
 # printed output. Note that this does nothing unless `ASTRIA_AUCTIONEER_PRETTY_PRINT`
 # is set to `true`.

crates/astria-auctioneer/local.env.example

@@ -45,8 +45,6 @@ pub struct Config {
     pub no_metrics: bool,
     /// The endpoint which will be listened on for serving prometheus metrics
     pub metrics_http_listener_addr: String,
-    /// Writes a human readable format to stdout instead of JSON formatted OTEL trace data.
-    pub pretty_print: bool,
 }
 
 impl config::Config for Config {