Combined scan + flat_map (scan_flat)

[SoniEx2]

On one hand, flat_map is very useful. On the other hand, scan is very useful. What if we combined both?

I was trying to write a simple FSM, so I tried this ugly abomination (which doesn't work, it just loops):

enum ShittyFSM {
  Default,
  Escape,
  End,
}

pub fn parse(pattern: &[u8]) {
  let parsed: Vec<_> = pattern.iter().scan(Some(ShittyFSM::Default), |state, &x| {
    loop {
      let r = match state.take().unwrap() {
        ShittyFSM::Default => {
          if x == b'%' {
            (ShittyFSM::Escape, None)
          } else {
            (ShittyFSM::Default, Some(x))
          }
        },
        ShittyFSM::Escape => {
          match x {
            _ => {
              (ShittyFSM::Default, Some(x))
            }
          }
        },
        ShittyFSM::End => {
          return None;
        }
      };
      let (newstate, potential_return) = r;
      *state = Some(newstate);
      if let Some(ret) = potential_return {
        return Some(ret);
      }
    }
  }).collect();
  println!("{:?}", parsed);
}

fn main() {
  parse(b"Hello, %world!");
}

Then I noticed it doesn't work. So I had to change it up:

enum ShittyFSM {
  Default,
  Escape,
}

pub fn parse(pattern: &[u8]) {
  let parsed: Vec<_> = pattern.iter().scan(Some(ShittyFSM::Default), |state, &x| {
    let r = match state.take().unwrap() {
      ShittyFSM::Default => {
        if x == b'%' {
          (ShittyFSM::Escape, None)
        } else {
          (ShittyFSM::Default, Some(x))
        }
      },
      ShittyFSM::Escape => {
        match x {
          _ => {
            (ShittyFSM::Default, Some(x))
          }
        }
      },
    };
    let (newstate, potential_return) = r;
    *state = Some(newstate);
    Some(potential_return)
  }).flat_map(|x| x).collect();
  println!("{:?}", parsed);
}

fn main() {
  parse(b"Hello, %world!");
}

This works, looks just as ugly, and has a .flat_map(|x| x). But hey, this is a FSM! But hey, since this is a FSM, this could be useful for others! But hey, what can we do to improve this?

Well, we could remove the .flat_map(|x| x), for one.

So I propose scan_flat. It scans the iterator flat.

fn scan_flat<St, U, F>(self, initial_state: St, f: F) -> ScanFlat<Self, St, U, F>
  where F: FnMut(&mut St, Self::Item) -> U, U: IntoIterator

(it could probably just return a FlatMap and be an alias for .scan(...).flat_map(|x| x), but idk.)

---

In a perfect world I'd be able to return a function pointer for the next state (fn default_case(key: u8) -> whatever { if key == b'%' { (*escape_case, None) } else { (*default_case, Some(key)) } }), but this is fine.

[scottmcm]

Looks like this would also like .flatten(), like is discussed in #2306 (comment)?

[Centril]

.flatten() is now available in the nightly compiler.

Closing this since I don't think this requires an RFC. If you want to add this operation to Iterator, please file a PR to rust-lang/rust if you have an implementation or file an issue over there if you don't and want to hear from the library team. My gut feeling is that .scan(..).flatten() is sufficient tho.