Are supported by the wxWidgets wxString, which provide support to and from wide character variants and locale variants. (We don’t want locale variants, they are obsolete. The whole world is switching to UTF, but our software and operating environments lag)
Locales still matter in case insensitive compare, collation order, canonicalization of utf-8 strings, and a rats nest of issues, which linux and sqlite avoids by doing binary compares, and if it cannot avoid capitalization issues, only considering A-Z to be capitals.
If you tell sqlite to incorporate the ICU library, sqlite will attempt to do case lowering and collation for all of utf-8 - which strikes me as something that cannot really be done, and I am not at all sure how it will interact with wxWidgets attempting to do the same thing.
What happens is that operations become locale dependent. It will have a different view of what characters are equivalent in different places. And changing the locale on a database will break an index or table that has a non binary collation order. Which probably will not matter much because we are likely to have few entries that only differ in capitalization. The sql results will be wrong, but the database will not crash, and when we have a lot of entires that affected by non latin capitalization rules, it is probably going to be viewed only in that locale. But any collation order that is global to all parties on the blockchain has to be latin or binary.
wxWidgets does not include the full unicode library, so cannot do this
stuff. But sqlite provides some C string functions that are guaranteed to
do whatever it does, and if you include the ICU library it attempts
to handle capitalization on the entire unicode set
int sqlite3_stricmp(const char *, const char *);
sqlite3_strlike(P,X,E)
The ICU library also provides a real regex function on unicode
(sqlite3_strlike being the C equivalent of the SQL LIKE,
providing a rather truncated fragment of regex capability)
Pretty sure the wxWidgets regex does something unwanted on unicode
wString::ToUTF8() and wString::FromUTF8() do what you would expect.
wxString::c_str() does something too clever by half.
On visual studio, need to set your source files to have bom, so that Visual
Studio knows that they are UTF‑8, need to set the compiler environment in
Visual Studio to UTF‑8 with /Zc:__cplusplus /utf-8 %(AdditionalOptions)
And you need to set the run time environment of the program to UTF‑8 with a manifest. Not at all sure how codelite will handle manifests, but there is a codelite build that does handle utf-8, presumably with a manifest. Does not do it in the standard build on windows.
You will need to place all UTF‑8 string literals and string constants in a resource file, which you will use for translated versions.
If you fail to set the compilation and run time environment to UTF‑8 then for extra confusion, your debugger and compiler will look as if they are handling UTF‑8 characters correctly as single byte characters, while at least wxString alerts you that something bad is happening by run time translating to the null string.
Automatic string conversion in wxWidgets is not UTF‑8, and if you have any unusual symbols in your string, you get a run time error and the empty string. So wxString automagic conversions will rape you in the ass at runtime, and for double the confusion, your correctly translated UTF‑8 strings will look like errors. Hence the need to make sure that the whole environment from source code to run time execution is consistently UTF‑8, which has to be separately ensured in three separate place.
When wxWidgets is compiled using #define wxUSE_UNICODE_UTF8 1,
it provides UTF‑8 iterators and caches a character index, so that accessing
a character by index near a recently used character is fast. The usual
iterators wx.begin(), wx.end(), const and reverse iterators are available.
I assume something bad happens if you advance a reverse iterator after
writing to it.
wxWidgets compiled with #define wxUSE_UNICODE_UTF8 1 is the
way of the future, but not the way of the present. Still a work in progress
Does not build under Windows. Windows now provide UTF8 entries to all
its system functions, which should make it easy.
A powerful library for handling UTF‑8. This somewhat duplicates the
facilities provided by wxWidgets with wxUSE_UNICODE_UTF8==1
For most purposes, wxString should suffice, when it actually works with UTF8. Which it does not yet on windows. We shall see. wxWidgets recommends not using wxString except to communicate with wxWidgets, and not using it as general UTF‑8 system. Which is certainly the current state of play with wxWidgets.
For regex to work correctly, probably need to do it on wxString’s native
UTF‑16 (windows) or UTF‑32 (unix), but it supposedly works on UTF8,
assuming you can successfully compile it, which you cannot.
reaction.la gpg key 154588427F2709CD9D7146B01C99BB982002C39F
This work is licensed under the Creative Commons Attribution 4.0 International License.