More information about use cases

[littledan]

The comparison function here is not linguistically appropriate, so I'm trying to understand its use cases. In the README, you discuss how other programming languages and databases have code point order as the default for sorting. Can you share more about the use cases for the default search in those systems? I'm not sure why this is something we want to align with.

[mhofman]

Not all data is meant to be presented to the user. When processing data we need a deterministic way of sorting it that is portable. Sorting strings by codepoint is the only portable order that makes sense (UTF-16 code units are only a feature of languages that were created before we had modern Unicode, like Java and JavaScript).

More specifically our system implements "Virtual Collections" that are transparently backed by a SQLite DB. These collections do not use insertion order for iteration but instead use a well defined sort based on data types. We also have compatible collection implementation that keep data in JS heap, where we need to sort keys the same as the DB would, and we need to implement a custom sort function that compares stings by codepoints to be compatible. We also need to access these virtual collections from other systems.

This is one example. In general sorting linguistically is for humans. Sorting for computer systems should rely on a portable sort order.

Please note I am not asking to change the default sort and comparison order of strings in JavaScript. Just that a compare function be provided by the standard library to facilitate opt-in sort and comparison using code point order.