CodePoints uncons? Deprecate drop?

[jamesdbrock]

I just noticed that instance stringStringLike uses CodeUnits for uncons.

purescript-parsing/src/Text/Parsing/Parser/String.purs

Lines 27 to 29 in 3d7976e

	instance stringLikeString :: StringLike String where
	uncons = SCU.uncons
	drop = S.drop

Doesn't that mean that anyChar will be wrong for astral characters?

purescript-parsing/src/Text/Parsing/Parser/String.purs

Lines 54 to 56 in 3d7976e

	anyChar = do
	input <- gets \(ParseState input _ _) -> input
	case uncons input of

Also, the drop member of StringLike is now unused?

[drathier]

I just hit the anyChar issue you found above, but with the purerl backend.

[jamesdbrock]

I'm looking through the code for this package and basically the whole thing is completely oriented to UTF-16 codeunits...

[jamesdbrock]

You might try using this instead @drathier https://pursuit.purescript.org/packages/purescript-string-parsers

[drathier]

The problem is that https://pursuit.purescript.org/packages/purescript-string-parsers/6.0.1/docs/Text.Parsing.StringParser.CodePoints#v:anyChar puts the result into a Char so it isn't parsing code points after all. we'll be writing our own anyCodePoint and anyGrapheme later today.

[jamesdbrock]

( Reference https://pursuit.purescript.org/packages/purescript-strings/5.0.0/docs/Data.String.CodePoints#t:CodePoint )

[jamesdbrock]

@drathier I'd like to see your anyGrapheme parser. Link please? Do you think that this library should include anyGrapheme?

[jamesdbrock]

@drathier Would PR #119 solve your problem? That PR deletes the StringLike class. Would you like to rely on that class for purerl? Or have your implemented the Data.String API for purerl?

[jamesdbrock]

More from @drathier :
https://discourse.purescript.org/t/make-char-represent-an-opaque-code-point-instead-of-utf-16-code-unit/2630