Question about batching strategy #66
Description
Hi, I'm wondering whether lookahead decoding is orthogonal with any batching strategies. Batch size is set to 1 in the evaluation part.
Whether lookahead decoding framework supports a higher batch size, for example, 10 ?
