Update asynchronous dependencies #758

Kixiron · 2020-05-14T13:27:24Z

Step one of #747

Updated tokio and added the rt-threaded feature
Updated futures
Updated rusoto_s3, rusoto_core and rusoto_credential

src/storage/s3.rs

jyn514 · 2020-05-14T13:38:54Z

The 600 added lines in Cargo.lock are making me nervous. Applying your changes to Cargo.toml and running cargo update gives me the following output:

`cargo update`

    Updating crates.io index
      Adding addr2line v0.12.0
    Updating arc-swap v0.4.5 -> v0.4.6
      Adding async-trait v0.1.31
    Updating backtrace v0.3.46 -> v0.3.48
    Removing backtrace-sys v0.1.35
      Adding base-x v0.2.6
      Adding base64 v0.12.1
      Adding block-buffer v0.7.3
      Adding block-padding v0.1.5
    Updating bstr v0.2.12 -> v0.2.13
    Updating bumpalo v3.2.1 -> v3.3.0
      Adding byte-tools v0.3.1
      Adding bytes v0.5.4
    Updating cc v1.0.50 -> v1.0.52
    Updating clap v2.33.0 -> v2.33.1
    Updating criterion v0.3.1 -> v0.3.2
    Updating criterion-plot v0.4.1 -> v0.4.2
      Adding crypto-mac v0.7.0
    Updating derive_more v0.99.5 -> v0.99.6
      Adding digest v0.8.1
    Updating dirs v1.0.5 -> v2.0.2
      Adding dirs-sys v0.3.4
      Adding discard v1.0.4
    Updating encoding_rs v0.8.22 -> v0.8.23
    Updating failure v0.1.7 -> v0.1.8
    Updating failure_derive v0.1.7 -> v0.1.8
    Updating filetime v0.2.9 -> v0.2.10
      Adding futures v0.3.5
      Adding futures-channel v0.3.5
      Adding futures-core v0.3.5
      Adding futures-executor v0.3.5
      Adding futures-io v0.3.5
      Adding futures-macro v0.3.5
      Adding futures-sink v0.3.5
      Adding futures-task v0.3.5
      Adding futures-util v0.3.5
      Adding generic-array v0.12.3
      Adding gimli v0.21.0
    Updating git2 v0.13.2 -> v0.13.5
      Adding h2 v0.2.5
    Updating hermit-abi v0.1.10 -> v0.1.12
    Updating hex v0.3.2 -> v0.4.2
      Adding hmac v0.7.1
      Adding http v0.2.1
      Adding http-body v0.3.1
      Adding hyper v0.13.5
      Adding hyper-tls v0.4.1
      Adding iron v0.6.1
    Updating itertools v0.8.2 -> v0.9.0
    Updating js-sys v0.3.37 -> v0.3.39
    Updating libc v0.2.68 -> v0.2.70
    Updating libgit2-sys v0.12.3+1.0.0 -> v0.12.5+1.0.0
      Adding md5 v0.7.0
    Updating mio v0.6.21 -> v0.6.22
    Updating mio-uds v0.6.7 -> v0.6.8
    Updating net2 v0.2.33 -> v0.2.34
    Updating num_cpus v1.12.0 -> v1.13.0
      Adding object v0.19.0
    Updating once_cell v1.3.1 -> v1.4.0
    Updating oorandom v11.1.0 -> v11.1.1
      Adding opaque-debug v0.2.3
    Updating openssl-sys v0.9.55 -> v0.9.56
    Updating parking_lot_core v0.7.1 -> v0.7.2
      Adding pin-project v0.4.16
      Adding pin-project-internal v0.4.16
      Adding pin-project-lite v0.1.5
      Adding pin-utils v0.1.0
    Updating plotters v0.2.12 -> v0.2.14
      Adding proc-macro-nested v0.1.4
    Updating proc-macro2 v1.0.10 -> v1.0.12
    Updating quote v1.0.3 -> v1.0.5
    Updating regex v1.3.6 -> v1.3.7
    Updating rusoto_core v0.40.0 -> v0.43.0
    Updating rusoto_credential v0.40.0 -> v0.43.0
    Updating rusoto_s3 v0.40.0 -> v0.43.0
      Adding rusoto_signature v0.43.0
    Updating rustwide v0.6.0 -> v0.6.1
    Updating ryu v1.0.3 -> v1.0.4
    Updating schannel v0.1.18 -> v0.1.19
    Updating security-framework v0.4.2 -> v0.4.4
    Updating security-framework-sys v0.4.2 -> v0.4.3
    Updating serde v1.0.106 -> v1.0.110
    Updating serde_derive v1.0.106 -> v1.0.110
    Updating serde_json v1.0.51 -> v1.0.53
      Adding sha1 v0.6.0
      Adding sha2 v0.8.1
    Updating siphasher v0.3.2 -> v0.3.3
    Updating smallvec v1.3.0 -> v1.4.0
      Adding standback v0.2.8
      Adding stdweb v0.4.20
      Adding stdweb-derive v0.5.3
      Adding stdweb-internal-macros v0.2.9
      Adding stdweb-internal-runtime v0.1.5
    Updating structopt v0.3.13 -> v0.3.14
    Updating structopt-derive v0.4.6 -> v0.4.7
      Adding subtle v1.0.0
    Updating syn v1.0.17 -> v1.0.21
    Removing time v0.1.42
      Adding time v0.1.43
      Adding time v0.2.16
      Adding time-macros v0.1.0
      Adding time-macros-impl v0.1.1
    Updating tinytemplate v1.0.3 -> v1.0.4
      Adding tokio v0.2.21
      Adding tokio-macros v0.2.5
      Adding tokio-tls v0.3.1
      Adding tokio-util v0.3.1
      Adding tower-service v0.3.0
    Updating typenum v1.11.2 -> v1.12.0
    Updating vec_map v0.8.1 -> v0.8.2
      Adding want v0.3.0
    Updating wasm-bindgen v0.2.60 -> v0.2.62
    Updating wasm-bindgen-backend v0.2.60 -> v0.2.62
    Updating wasm-bindgen-macro v0.2.60 -> v0.2.62
    Updating wasm-bindgen-macro-support v0.2.60 -> v0.2.62
    Updating wasm-bindgen-shared v0.2.60 -> v0.2.62
    Updating web-sys v0.3.37 -> v0.3.39
    Updating winapi-util v0.1.4 -> v0.1.5
    Updating xml-rs v0.7.0 -> v0.8.3
      Adding zeroize v1.1.0

Running cargo update first and then applying your changes shows that dependencies have only been added, not removed:

Added dependencies

    Updating crates.io index
      Adding async-trait v0.1.31
      Adding base-x v0.2.6
      Adding base64 v0.12.1
      Adding block-buffer v0.7.3
      Adding block-padding v0.1.5
      Adding byte-tools v0.3.1
      Adding bytes v0.5.4
      Adding crypto-mac v0.7.0
      Adding digest v0.8.1
    Updating dirs v1.0.5 -> v2.0.2
      Adding dirs-sys v0.3.4
      Adding discard v1.0.4
      Adding futures v0.3.5
      Adding futures-channel v0.3.5
      Adding futures-core v0.3.5
      Adding futures-executor v0.3.5
      Adding futures-io v0.3.5
      Adding futures-macro v0.3.5
      Adding futures-sink v0.3.5
      Adding futures-task v0.3.5
      Adding futures-util v0.3.5
      Adding generic-array v0.12.3
      Adding h2 v0.2.5
    Updating hex v0.3.2 -> v0.4.2
      Adding hmac v0.7.1
      Adding http v0.2.1
      Adding http-body v0.3.1
      Adding hyper v0.13.5
      Adding hyper-tls v0.4.1
      Adding md5 v0.7.0
      Adding opaque-debug v0.2.3
      Adding pin-project v0.4.16
      Adding pin-project-internal v0.4.16
      Adding pin-project-lite v0.1.5
      Adding pin-utils v0.1.0
      Adding proc-macro-nested v0.1.4
    Updating rusoto_core v0.40.0 -> v0.43.0
    Updating rusoto_credential v0.40.0 -> v0.43.0
    Updating rusoto_s3 v0.40.0 -> v0.43.0
      Adding rusoto_signature v0.43.0
      Adding sha1 v0.6.0
      Adding sha2 v0.8.1
      Adding standback v0.2.8
      Adding stdweb v0.4.20
      Adding stdweb-derive v0.5.3
      Adding stdweb-internal-macros v0.2.9
      Adding stdweb-internal-runtime v0.1.5
      Adding subtle v1.0.0
      Adding time v0.2.16
      Adding time-macros v0.1.0
      Adding time-macros-impl v0.1.1
      Adding tokio v0.2.21
      Adding tokio-macros v0.2.5
      Adding tokio-tls v0.3.1
      Adding tokio-util v0.3.1
      Adding tower-service v0.3.0
      Adding want v0.3.0
    Updating xml-rs v0.7.0 -> v0.8.3
      Adding zeroize v1.1.0

I'm concerned by firstly the number of added dependencies, but also that we haven't gotten rid of tokio 0.1. Can you try and figure out why that's the case? cargo tree shows the following:

$ cargo tree -i -p tokio:0.1.22
tokio v0.1.22
├── hyper v0.12.35
│   ├── hyper-tls v0.3.2
│   │   └── reqwest v0.9.24
│   │       ├── cratesfyi v0.6.0 (/home/joshua/src/rust/docs.rs)
│   │       └── rustwide v0.6.1
│   │           └── cratesfyi v0.6.0 (/home/joshua/src/rust/docs.rs) (*)
│   └── reqwest v0.9.24 (*)
├── reqwest v0.9.24 (*)
└── rustwide v0.6.1 (*)

jyn514 · 2020-05-14T13:39:46Z

Note that reqwest had a lot of duplicate dependencies in 0.9 that were removed in 0.10, so it might be worth updating that first.

Cargo.toml

Nemo157 · 2020-05-14T14:27:24Z

Looks like updating rustwide as well will be necessary to get rid of both hyper 0.9 and tokio 0.1.

Kixiron · 2020-05-15T17:22:32Z

Waiting on rustwide/#23

jyn514 · 2020-05-18T21:32:16Z

The new changes from cargo

(bash@joshua-ThinkPad-P51s) ~/.../rust/docs.rs ▶️ cargo update
    Updating crates.io index
      Adding async-trait v0.1.31
      Adding base-x v0.2.6
      Adding base64 v0.12.1
      Adding block-buffer v0.7.3
      Adding block-padding v0.1.5
      Adding byte-tools v0.3.1
      Adding bytes v0.5.4
      Adding crypto-mac v0.7.0
      Adding digest v0.8.1
    Updating dirs v1.0.5 -> v2.0.2
      Adding dirs-sys v0.3.4
      Adding discard v1.0.4
      Adding futures v0.3.5
      Adding futures-channel v0.3.5
      Adding futures-core v0.3.5
      Adding futures-executor v0.3.5
      Adding futures-io v0.3.5
      Adding futures-macro v0.3.5
      Adding futures-sink v0.3.5
      Adding futures-task v0.3.5
      Adding futures-util v0.3.5
      Adding generic-array v0.12.3
      Adding h2 v0.2.5
    Updating hex v0.3.2 -> v0.4.2
      Adding hmac v0.7.1
      Adding http v0.2.1
      Adding http-body v0.3.1
      Adding hyper v0.13.5
      Adding hyper-tls v0.4.1
      Adding md5 v0.7.0
      Adding opaque-debug v0.2.3
      Adding pin-project v0.4.17
      Adding pin-project-internal v0.4.17
      Adding pin-project-lite v0.1.5
      Adding pin-utils v0.1.0
      Adding proc-macro-nested v0.1.4
      Adding reqwest v0.10.4
    Updating rusoto_core v0.40.0 -> v0.43.0
    Updating rusoto_credential v0.40.0 -> v0.43.0
    Updating rusoto_s3 v0.40.0 -> v0.43.0
      Adding rusoto_signature v0.43.0
    Updating rustwide v0.6.1 -> v0.7.0
      Adding serde_urlencoded v0.6.1
      Adding sha1 v0.6.0
      Adding sha2 v0.8.1
      Adding standback v0.2.8
      Adding stdweb v0.4.20
      Adding stdweb-derive v0.5.3
      Adding stdweb-internal-macros v0.2.9
      Adding stdweb-internal-runtime v0.1.5
      Adding subtle v1.0.0
      Adding time v0.2.16
      Adding time-macros v0.1.0
      Adding time-macros-impl v0.1.1
      Adding tokio v0.2.21
      Adding tokio-macros v0.2.5
      Adding tokio-tls v0.3.1
      Adding tokio-util v0.3.1
      Adding tower-service v0.3.0
      Adding want v0.3.0
      Adding wasm-bindgen-futures v0.4.12
    Updating xml-rs v0.7.0 -> v0.8.3
      Adding zeroize v1.1.0

We still aren't removing any dependencies. This seems to be because rustwide is still using reqwest 0.9.

$ cargo tree -i -p tokio:0.1.22
tokio v0.1.22
├── hyper v0.12.35
│   ├── hyper-tls v0.3.2
│   │   └── reqwest v0.9.24
│   │       └── rustwide v0.7.0
│   │           └── cratesfyi v0.6.0 (/home/joshua/src/rust/docs.rs)
│   └── reqwest v0.9.24 (*)
├── reqwest v0.9.24 (*)
└── rustwide v0.7.0 (*)

jyn514 · 2020-05-21T14:44:31Z

This looks a lot better :)

Changes

$ cargo update
    Updating crates.io index
      Adding async-trait v0.1.31
      Adding base-x v0.2.6
    Updating base64 v0.10.1 -> v0.12.1
      Adding block-buffer v0.7.3
      Adding block-padding v0.1.5
      Adding byte-tools v0.3.1
      Adding bytes v0.5.4
    Removing cookie v0.12.0
    Removing cookie_store v0.7.0
    Removing crossbeam-queue v0.1.2
    Removing crossbeam-utils v0.6.6
      Adding crypto-mac v0.7.0
      Adding digest v0.8.1
    Updating dirs v1.0.5 -> v2.0.2
      Adding dirs-sys v0.3.4
      Adding discard v1.0.4
    Removing error-chain v0.12.2
    Updating futures v0.1.29 -> v0.3.5
      Adding futures-channel v0.3.5
      Adding futures-core v0.3.5
    Removing futures-cpupool v0.1.8
      Adding futures-executor v0.3.5
      Adding futures-io v0.3.5
      Adding futures-macro v0.3.5
      Adding futures-sink v0.3.5
      Adding futures-task v0.3.5
      Adding futures-util v0.3.5
      Adding generic-array v0.12.3
    Updating h2 v0.1.26 -> v0.2.5
    Updating hex v0.3.2 -> v0.4.2
      Adding hmac v0.7.1
    Updating http v0.1.21 -> v0.2.1
    Updating http-body v0.1.0 -> v0.3.1
    Updating hyper v0.12.35 -> v0.13.5
    Updating hyper-tls v0.3.2 -> v0.4.1
      Adding md5 v0.7.0
      Adding opaque-debug v0.2.3
    Removing parking_lot v0.9.0
    Removing parking_lot_core v0.6.2
      Adding pin-project v0.4.17
      Adding pin-project-internal v0.4.17
      Adding pin-project-lite v0.1.5
      Adding pin-utils v0.1.0
      Adding proc-macro-nested v0.1.4
    Removing publicsuffix v1.5.4
    Updating reqwest v0.9.24 -> v0.10.4
    Updating rusoto_core v0.40.0 -> v0.43.0
    Updating rusoto_credential v0.40.0 -> v0.43.0
    Updating rusoto_s3 v0.40.0 -> v0.43.0
      Adding rusoto_signature v0.43.0
    Updating rustwide v0.7.0 -> v0.7.1
    Updating serde_urlencoded v0.5.5 -> v0.6.1
      Adding sha1 v0.6.0
      Adding sha2 v0.8.1
    Removing smallvec v0.6.13
      Adding standback v0.2.8
      Adding stdweb v0.4.20
      Adding stdweb-derive v0.5.3
      Adding stdweb-internal-macros v0.2.9
      Adding stdweb-internal-runtime v0.1.5
    Removing string v0.2.1
      Adding subtle v1.0.0
      Adding time v0.2.16
      Adding time-macros v0.1.0
      Adding time-macros-impl v0.1.1
    Updating tokio v0.1.22 -> v0.2.21
    Removing tokio-buf v0.1.1
    Removing tokio-codec v0.1.2
    Removing tokio-current-thread v0.1.7
    Removing tokio-executor v0.1.10
    Removing tokio-fs v0.1.7
    Removing tokio-io v0.1.13
      Adding tokio-macros v0.2.5
    Removing tokio-process v0.2.5
    Removing tokio-reactor v0.1.12
    Removing tokio-signal v0.2.9
    Removing tokio-sync v0.1.8
    Removing tokio-tcp v0.1.4
    Removing tokio-threadpool v0.1.18
    Removing tokio-timer v0.2.13
      Adding tokio-tls v0.3.1
    Removing tokio-udp v0.1.6
    Removing tokio-uds v0.2.6
      Adding tokio-util v0.3.1
      Adding tower-service v0.3.0
    Removing try_from v0.3.2
    Removing uuid v0.7.4
    Updating want v0.2.0 -> v0.3.0
      Adding wasm-bindgen-futures v0.4.12
    Updating xml-rs v0.7.0 -> v0.8.3
      Adding zeroize v1.1.0

src/storage/mod.rs

src/storage/s3.rs

src/storage/mod.rs

Kixiron · 2020-05-22T03:36:00Z

Why is wasm-bindgen-futures in our stack? Do we have anything wasm-related?

Nemo157 · 2020-05-22T07:31:25Z

The Cargo.lock contains crates for all targets, it’s probably pulled in as a wasm-only dependency by something.

jyn514 · 2020-05-22T21:17:18Z

🎉 🎉

Nemo157 reviewed May 14, 2020

View reviewed changes

src/storage/s3.rs Outdated Show resolved Hide resolved

Nemo157 reviewed May 14, 2020

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

Kixiron requested a review from Nemo157 May 15, 2020 15:42

Kixiron force-pushed the lip-asyncin branch from 7071a5c to d0359bf Compare May 18, 2020 21:30

Nemo157 mentioned this pull request May 20, 2020

docs.rs should ignore .cargo/config #650

Closed

jyn514 mentioned this pull request May 20, 2020

Bump rustwide to 0.7 #763

Merged

Kixiron force-pushed the lip-asyncin branch from d0359bf to e7c613a Compare May 20, 2020 14:00

jyn514 self-assigned this May 21, 2020

jyn514 reviewed May 21, 2020

View reviewed changes

src/storage/mod.rs Show resolved Hide resolved

src/storage/s3.rs Outdated Show resolved Hide resolved

src/storage/s3.rs Show resolved Hide resolved

src/storage/mod.rs Show resolved Hide resolved

jyn514 mentioned this pull request May 22, 2020

Add Support for Retrying Requests Upon a "Request Throttled" Response rusoto/rusoto#234

Open

Kixiron added 6 commits May 22, 2020 09:47

Updated deps

d966dec

Enabled the threaded tokio runtime

bf281cd

Upgraded reqwest

f33e5cf

Worked on async uploading and switched to futures-util

8324cb8

Updated rustwide

b152777

Moved unload increment

7f2f642

Kixiron force-pushed the lip-asyncin branch from e7c613a to 7f2f642 Compare May 22, 2020 14:57

jyn514 merged commit 5acd658 into rust-lang:master May 22, 2020

jyn514 mentioned this pull request May 31, 2020

Updated rustwide to 0.7.1 #792

Merged

Kixiron mentioned this pull request Jun 1, 2020

Update Async Dependencies 2 #799

Merged

jyn514 mentioned this pull request Jun 27, 2020

Revert 758 Update async dependencies #768

Merged

Kixiron mentioned this pull request Jul 25, 2020

Update tokio? #910

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update asynchronous dependencies #758

Update asynchronous dependencies #758

Kixiron commented May 14, 2020

jyn514 commented May 14, 2020 •

edited

Loading

jyn514 commented May 14, 2020 •

edited

Loading

Nemo157 commented May 14, 2020

Kixiron commented May 15, 2020

jyn514 commented May 18, 2020

jyn514 commented May 21, 2020

Kixiron commented May 22, 2020

Nemo157 commented May 22, 2020

jyn514 commented May 22, 2020

Update asynchronous dependencies #758

Update asynchronous dependencies #758

Conversation

Kixiron commented May 14, 2020

jyn514 commented May 14, 2020 • edited Loading

jyn514 commented May 14, 2020 • edited Loading

Nemo157 commented May 14, 2020

Kixiron commented May 15, 2020

jyn514 commented May 18, 2020

jyn514 commented May 21, 2020

Kixiron commented May 22, 2020

Nemo157 commented May 22, 2020

jyn514 commented May 22, 2020

jyn514 commented May 14, 2020 •

edited

Loading

jyn514 commented May 14, 2020 •

edited

Loading