Add StringUtils.truncateToByteLength

[kiddos]

We sometimes need to store Unicode text in a fixed space (e.g., in a database column of type CHARACTER(32)). It's acceptable for the text to be truncated, but because we're dealing with Unicode, we can't simply treat the text as raw bytes and truncate it at 16 bytes — that might split a character in the middle. The function StringUtils.truncateToByteLength(String str, int maxBytes, Charset charset) helps handle this by safely truncating the string based on byte length while preserving valid character boundaries.

[ecki]

Agree, very useful when dealing with UTF8 databases. Wonder if it should have a utf8 variant, where it does not have to re truncate, it can just look at the byte patterns at the border.

The current version does not deal with UTF16 code units properly. (Substring might cut them in half)

[garydgregory]

Hello all,

I think you'll want tests that cover grapheme clusters to avoid problems like https://issues.apache.org/jira/browse/LANG-1770

[kiddos]

I added some test cases for emoji characters 🚀✨🎉
I did some testing and found that current implementation the escape characters worked ("\uD83D\uDE80\u2728\uD83C\uDF89")
but "🚀✨🎉" doesn't

After adding <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> in pom.xml, "🚀✨🎉" seems to work.

[garydgregory]

@kiddos
Please see my previous comment.

[kiddos]

Oh, right.
it's just tricky to handle grapheme cluster.
the codePoint solution you mention does seems to work.
I'll add more tests using grapheme clusters.

[garydgregory]

I'm not requesting support for grapheme cluster in the runtime, but we should set expectations in unit tests, whether they are supported or not. This is a larger discussion, which I raised in https://issues.apache.org/jira/browse/LANG-1770