Windows-style line endings break rendering

[muesli]

Windows-style line endings \r\n break chroma's syntax highlighting. I've seen it break in different ways, but most commonly it will then render everything as spaces instead of text.

Providing a copyable text is difficult, as the GitHub interface would cleanup the line endings. GitHub also doesn't let me attach such a file to this issue.

Replacing \r\n and \r with \n before passing the source to chroma works around the problem.

[chmike]

I confirm the problem.

Text obtain from a textarea html form has its lines terminated by \r\n. A filtering is required to replace \r\n with \n.

[alecthomas]

Can someone try #336? Given there's no data to verify that this fixes the issue, I'd prefer someone to verify it before merging.

[muesli]

@alecthomas This is pretty much the workaround I use locally, at the moment. I'm actually wondering if it needs to be a separate option or if this isn't a sanitizer we always want to run?

[satotake]

I am wondering, too. @muesli

Theoretically, there can be lexers which do not handle \n as EOL and accept only \r or \r\n.
I had suspected some windows-driven files might be so.
Fortunately, it was not the case of the lexers which chorma is using.
At least, they are so rare.

[alecthomas]

Fix merged please verify.

[muesli]

@alecthomas Verified as fixed. Thanks everyone!