Skip to content

UTF-16 console support for Windows#15051

Merged
bors merged 1 commit intorust-lang:masterfrom
retep998:master
Jul 4, 2014
Merged

UTF-16 console support for Windows#15051
bors merged 1 commit intorust-lang:masterfrom
retep998:master

Conversation

@retep998
Copy link
Contributor

This implementation does have the minor issue of not handling things correctly when a codepoint is split across multiple writes or reads, but its better than not having unicode support at all.

Adds a Windows specific struct WindowsTTY in libnative and make tty_open create that struct on Windows. Adds needed functions and constants to c_win32.rs.

Libuv still needs to be updated before #15028 can be closed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're trying to keep things out of liblibc nowadays, could you add these to c_win32.rs instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(inside of libnative)

@alexcrichton
Copy link
Member

It would be nice to update libuv at the same time. If you open a PR against the rust-lang/libuv repo with the relevant libuv commit I'll push it so you can update the submodule.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a windows-specific implementation, it should probably use the windows-specific function GetConsoleMode

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetConsoleMode merely tells us the set of flags that we've set for the console, not whether it is or isn't a TTY. isatty does the job fine at telling whether we're attached to a TTY or just redirected to a file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it suffices for libuv, and I tend to not trust posix functions with file descriptors on windows, but if it's working well I guess it's working well.

@retep998
Copy link
Contributor Author

No, setting the code page to UTF-8 does not work. I have already tested that option extensively in C++. The Windows console only supports narrow byte encodings and UTF-16.

@retep998
Copy link
Contributor Author

Ah, so output with UTF-8 does indeed work, but I've used both SetConsoleCP and SetConsoleOutputCP and input with UTF-8 simply does not work. It just ignores the entire line of text I entered if there's any unicode in it.

@retep998
Copy link
Contributor Author

Even still, when using UTF-8, Windows will often act buggy. For some examples:

http://social.msdn.microsoft.com/Forums/vstudio/en-US/e4b91f49-6f60-4ffe-887a-e18e39250905/possible-bugs-in-writefile-and-crt-unicode-issues?forum=vcgeneral

It is probably safer to simply use UTF-16 because that is what Windows is built around.

@retep998
Copy link
Contributor Author

Added another commit to take into account comments by @alexcrichton

@retep998 retep998 changed the title Crude implementation of Unicode console support for Windows. Implementation of UTF-16 console support for Windows Jun 21, 2014
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not happen here. It's the responsibility of the caller to fall back to a file descriptor. This should have the same error path as the unix equivalent.

@retep998 retep998 changed the title Implementation of UTF-16 console support for Windows UTF-16 console support for Windows Jun 22, 2014
@alexcrichton
Copy link
Member

Oh dear, sorry for being so late to this! Feel free to ping a PR whenever you update this because sadly github doesn't send out notifications when that happens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants