Skip to content

Commit e1796ea

Browse files
committed
doc: document limitations with PCRE2_MATCH_INVALID_UTF
Closes: #187
1 parent 0746b3d commit e1796ea

File tree

2 files changed

+7
-0
lines changed

2 files changed

+7
-0
lines changed

doc/pcre2api.3

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1615,6 +1615,7 @@ PCRE2_EXTRA_MATCH_WORD are also supported. Any other options cause an error.
16151615
.sp
16161616
This option forces PCRE2_UTF (see below) and also enables support for matching
16171617
by \fBpcre2_match()\fP in subject strings that contain invalid UTF sequences.
1618+
(Assuming the matching text is suitably aligned).
16181619
This facility is not supported for DFA matching. For details, see the
16191620
.\" HREF
16201621
\fBpcre2unicode\fP

doc/pcre2unicode.3

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -442,6 +442,12 @@ would match an instance of WORD that is surrounded by invalid UTF code units.
442442
Using PCRE2_MATCH_INVALID_UTF, an application can run matches on arbitrary
443443
data, knowing that any matched strings that are returned are valid UTF. This
444444
can be useful when searching for UTF text in executable or other binary files.
445+
.P
446+
The test for validity of Unicode characters is done for whole code units, so
447+
if the valid Unicode string is not aligned to a multiple of the code point size
448+
it won't be identified as valid. If that is a possibility on your data and
449+
unaligned access is not a problem in your system, you might want to try the
450+
matches with different starting byte offsets as a work around.
445451
.
446452
.
447453
.SH AUTHOR

0 commit comments

Comments
 (0)