build(toolchain): bump to 2022-10-16

[xxchan]

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

[xxchan]

The ICE is fixed. Now we have TAIT bug 🤪 rust-lang/rust#101750

[BugenZhao]

Really want rust-lang/cargo#11114 after bumping. 🤤

[BugenZhao]

👀

[BugenZhao]

There shouldn't be any difference for enum construction.

[xxchan]

rust-lang/rust-clippy#9663

[skyzh]

Build of risingwave_stream will stuck on macOS. Still investigating why.

[skyzh]

INFO rustc_trait_selection::traits::query::normalize normalize::<rustc_middle::mir::ConstantKind>: result=Ok(Val(ZeroSized, for<'a, 'b, 'c> fn(std::pin::Pin<&'a mut futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]>>, &'b mut std::task::Context<'c>) -> std::task::Poll<std::option::Option<<futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]> as tokio_stream::Stream>::Item>> {<futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]> as tokio_stream::Stream>::poll_next})) with 4052 obligations
INFO rustc_trait_selection::traits::query::normalize normalize::<rustc_middle::ty::subst::GenericArg>: result=Ok(unsafe fn(&mut futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]>) -> std::pin::Pin<&mut futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]>> {std::pin::Pin::<&mut futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]>>::new_unchecked}) with 4052 obligations
INFO rustc_trait_selection::traits::query::normalize normalize::<rustc_middle::mir::ConstantKind>: result=Ok(Val(ZeroSized, unsafe fn(&mut futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]>) -> std::pin::Pin<&mut futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]>> {std::pin::Pin::<&mut futures_async_stream::try_stream::from_generator::GenTryStream<[static generator@src/stream/src/executor/hash_join.rs:760:5: 760:61]>>::new_unchecked})) with 4052 obligations

It seems that Rust is very slow with resolving futures_async_stream. It took about 1sec for each resolving.

[skyzh]

To reproduce, RUSTC_LOG=debug ./risedev d

[yuhao-su]

Do you have time to look into why HashJoin is causing rustc to stuck?

I'll take a look.

[skyzh]

Unluckily, the bombing obligations seem to originate from StateTableIter

INFO rustc_trait_selection::traits::query::normalize normalize::<rustc_middle::ty::subst::GenericArg>: result=Ok(std::future::from_generator::GenFuture<[static generator@src/stream/src/common/mod.rs:31:45: 37:2]>) with 4052 obligations
INFO rustc_trait_selection::traits::query::normalize normalize::<rustc_middle::ty::subst::GenericArg>: result=Ok(std::future::from_generator::GenFuture<[static generator@risingwave_storage::table::streaming_table::state_table::StateTable<risingwave_storage::monitor::MonitoredStateStore<risingwave_storage::hummock::HummockStorage>>::iter::{closure#0}]>) with 4052 obligations
INFO rustc_trait_selection::traits::query::normalize normalize::<rustc_middle::ty::subst::GenericArg>: result=Ok(std::future::from_generator::GenFuture<[static generator@risingwave_storage::table::streaming_table::state_table::StateTable<risingwave_storage::monitor::MonitoredStateStore<risingwave_storage::hummock::HummockStorage>>::iter_with_pk_prefix::{closure#0}]>) with 4052 obligations

which cases all executors to have a lot of obligations, whereas hash join needs a lot of specialization, and cases the most problems.

[skyzh]

After changing iter_key_and_val to boxed, risingwave_stream compiles in 40 seconds (still seems slow). cc @BugenZhao now it's your turn to investigate what's happening 🥵

    pub async fn iter_key_and_val<'a>(
        &'a self,
        pk_prefix: &'a Row,
    ) -> StorageResult<RowStreamWithPk<'a, S>> {
        let (mem_table_iter, storage_iter_stream) = self
            .iter_with_pk_prefix_inner(pk_prefix, self.epoch())
            .await?;
        let storage_iter = storage_iter_stream.into_stream().boxed();

        Ok(
            StateTableRowIter::new(mem_table_iter, storage_iter, self.row_deserializer.clone())
                .into_stream().boxed(),
        )
    }

[skyzh]

My wild guess is that while rustc allows more lifetime generics in the latest version, some wrongly-labeled lifetimes will cause compile issues. Every time we add 'a at &self, there must be something going wrong.

[skyzh]

The issue now narrows down to risingwave_storage. It seems that it produces 4052 obligations...

Most of them are OutlivesPredicate (lifetime issues)

[TennyZhuang]

turn to @wenym1

[TennyZhuang]

IIRC storage have many unnecessary generics, but I’m not sure whether it’s related.

[skyzh]

I feel this issue more related to StateTable implementation than HummockStorage itself. The obligations will only bomb when processing StateTable-related code.

[BugenZhao]

Will implementing state table iterator manually with poll_next help?

[skyzh]

I'm not sure, but I'd suggest investigating where's the obligations from.

[MrCroxx]

Also the lifetime issue doesn't look like a false positive.

        let rotate_last_mut = |buffers: &'a mut Vec<_>| {
            buffers.push(DioBuffer::with_capacity_in(
                self.buffer_capacity,
                &DIO_BUFFER_ALLOCATOR,
            ));
            buffers.last_mut().unwrap()
        };

is defined for 'a mut Vec

where

rotate_last_mut(&mut self.buffers),

is following the lifetime of &mut self. buffers' lifetime has no relationship to 'a. The compiler's report seems correct.

cc @MrCroxx would you please help fix this?

Sure. I'll handle it.

[lmatz]

#1334 can be closed after this

[skyzh]

I'll take over this PR and get it merged with my pace.

[vickiegpt]

Thank you guys very much

[skyzh]

close in favor of #6025