Skip to content

Utf8 encoding issues #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ws-garcia merged 2 commits into from
Jun 27, 2025
Merged

Utf8 encoding issues #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cceeb96
Update CSVinterface.cls
ws-garcia Jun 27, 2025
388c294
Update CSVTextStream.cls
ws-garcia Jun 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/CSVTextStream.cls
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,7 @@ Private Function IsValidUTF8(byteData() As Byte) As Boolean
Dim i As Long
Dim ANSIfile As Boolean

On Error GoTo err_handler
i = 0
ANSIfile = True
While i <= UBound(byteData)
Expand Down Expand Up @@ -417,6 +418,9 @@ Private Function IsValidUTF8(byteData() As Byte) As Boolean
End If
Wend
IsValidUTF8 = Not ANSIfile
Exit Function
err_handler:
IsValidUTF8 = False
End Function
Private Sub NormalizeLineBreaks()
Do While InStrB(1, P_TEXT, vbCrLf, vbBinaryCompare)
Expand Down
3 changes: 2 additions & 1 deletion src/CSVinterface.cls
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -793,7 +793,7 @@ Public Function GetDataFromCSV(csvPathAndFilename As String) As String
On Error GoTo ErrHandler_GetDataFromCSV
FileHandled = FreeFile
Open csvPathAndFilename For Binary As #FileHandled
GetDataFromCSV = SPACE$(LOF(FileHandled))
GetDataFromCSV = Space$(LOF(FileHandled))
Get #FileHandled, , GetDataFromCSV
Close #FileHandled
ErrHandler_GetDataFromCSV:
Expand Down Expand Up @@ -3573,6 +3573,7 @@ Private Sub StreamParseCSV(configObj As CSVparserConfig, _
CSVstream.linebreakMatchingBehavior = EndLineMatchingBehavior.OnlyBackwardSense
CSVstream.unifiedLFOutput = .multiEndOfLineCSV
CSVstream.utf8EncodedFile = .utf8EncodedFile
CSVstream.autoDetectEncoding = .autoDetectEncoding
CSVstream.OpenStream .path
commToken = AscW(.commentsToken)
dynamicType = .dynamicTyping
Expand Down