the vast majority of our supra-BMP output on windows/msys is fubar

[dankamongmen]

I was kinda hoping this would resolve itself without my intervention, but it appears not. Run some notcurses code. Note that most UTF-8 doesn't show up, or is otherwise fucked up. I refuse to believe that Microsoft Windows in 2021 cannot display most unicode. I mean, we don't even seem to have quadrants, and yet I think I've seen quadrants generated by other programs.

Figure out what we're doing wrong, and rectify it.

[dankamongmen]

@j4james, how would you describe Unicode capabilities in Windows Terminal?

[j4james]

Here's a screenshot from the notcurses uniblock demo running under WSL. Most of the BMP blocks should be showing content like this, so if most UTF-8 is not working for you, then something is seriously wrong.

Once you get to the higher planes things become a little more messy, mostly in terms of the width calculations being wrong, so you'll have text "leaking" out of the area you were expecting it to fit and screwing up the rest of the page. I think this is because Windows stores Unicode in 16-bit "wide characters", and code points from the supplementary planes require surrogate pairs, so they don't fit in a single cell. Don't quote me on that, though. I just know that it's a known issue that they're still working on.

It's also worth mentioning that there are additional limitations in the old conhost console. It uses a GDI renderer that doesn't do font fallback so if your selected font doesn't support a particular code point, you're just going to get a � (or something of the sort). I think the GDI renderer also ignored anything outside the BMP, so none of the supplementary planes are likely to work. Both those issues have been worked on recently, but unless you're building from source, you won't have those fixes.

Then if you're testing on Mintty (or any other third-party terminal), there could be a number of other issues coming into play, so I'd suggest you stick to Windows Terminal to start with, until you've got something working reasonably.

[dankamongmen]

if you could run notcurses-info.exe for me, and let me know if it's mostly empty on your side, i'd appreciate it. that looks great, holy crap!

[dankamongmen]

i'm testing on both. right now windows terminal is getting close to usable. mintty i would not say so, due to input problems detailed in #2116. in neither do i get much in the way of good notcurses-info.exe output. here's a pretty ideal output (kitty):

[j4james]

OK, there are quite a few gaps on the notcurses-info page. So if that's what's worrying you, that's to be expected.

[j4james]

Just looking at a couple of the missing code points at random, they seem to be outside the BMP, but are meant to be narrow width, so this could be the surrogate pair issue I mentioned above. Otherwise it may just be that my system doesn't have an appropriate font for all of those code points, although then I would expect a bunch of � replacement characters instead. Whatever it is, I don't think it's your problem.

[j4james]

Now I'm beginning to have some doubts. I've just installed a font that it is supposed to have the segmented digit characters. That didn't seem to help with font fallback, but if I select it as my primary font, I can display those characters in the shell, like this:

When I run notcurse-info, though, I'm still seeing those code points missing. I wouldn't have expected it to work perfectly, because I know the width calculations are going to be wrong, but I am surprised it's not showing anything at all.

[dankamongmen]

oh hey you seem to be seeing a good bit more than i am.

[dankamongmen]

[dankamongmen]

you have uniblock running in a Windows Terminal? it exits with failure immediately for me =[

[dankamongmen]

ohhhhhhhhhhhh you have WSL there not Windows Terminal. that's a whole different thing, no? that's using UNIX interfaces.