proposal: bufio: Scanner.{Text,Bytes}Seq

[pkierski]

Proposal Details

Once iterators are introduced in 1.23 there is growing number of libraries based on this feature. Reading data by lines or words are quite common task and could be accomplished with for ... range loop. It requires just two simple methods for bufio.Scanner:

func (s *Scanner) TextSeq() iter.Seq[string] {
	return func(yield func(string) bool) {
		for s.Scan() {
			if !yield(s.Text()) {
				break
			}
		}
	}
}

func (s *Scanner) BytesSeq() iter.Seq[[]byte] {
	return func(yield func([]byte) bool) {
		for s.Scan() {
			if !yield(s.Bytes()) {
				break
			}
		}
	}
}

Reading whole file as collection of lines could be like:

	f, err := os.Open("file.txt")
	if err != nil {
		panic(err)
	}
	defer f.Close()
	scanner := bufio.NewScanner(f)
	
	// read all lines as slice of strings
	lines := slices.Collect(scanner.TextSeq())

	// instead of:
	// lines := make([]string, 0)
	// for scanner.Scan() {
	// 	lines = append(lines, scanner.Text())
	// }

[gabyhelp]

Related Issues

• bufio: reading lines is too cumbersome #4802 (closed)
• bufio: can improve bufio.Scanner and bufio.Reader for bytes.Reader and strings.Reader #11655 (closed)

Related Documentation

• Go 1.1 Release Notes > Changes to the standard library > bufio.Scanner

Related Discussions

• proposal: new interface for scanning lines in bufio

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

this doesn't address how errors should be surfaced, see #70084 for related discussion

[mateusz834]

Also why global functions, instead of methods?

[pkierski]

Also why global functions, instead of methods?

My bad, I've copied code from my experiments. I've thought about method ofc.

[mateusz834]

this doesn't address how errors should be surfaced, see #70084 for related discussion

And #70631

[earthboundkid]

I think the naming convention is BytesSeq, like strings.SplitSeq etc. in 1.24.

[pkierski]

I think the naming convention is BytesSeq, like strings.SplitSeq etc. in 1.24.

Thank you, changed according to this suggestion.

[adonovan]

The need to remember to call Scanner.Err after iterating, even with the traditional API, is already a problem; iterators may make it even easier to forget to do so. I think we should put this proposal aside until we have a consensus on how to deal with iterable sequences with the potential for errors. In the meantime, you can always write this function yourself.