lib9: no support for non-ascii chars in path on Windows #3754
Labels
Milestone
Comments
Each Windows api have 2 version: ansi and utf16. We always use utf16 versions. Then we convert from utf16 into utf8 and back. If some api calls do not follow that path, we should change them. @minix: I do not know how your "multibyte characters" fit into ansi/utf16 world. Perhaps, you should tell us more about the problem you are having. Alex |
re #3, sorry, i don't mean go side, i mean the C side (gc, ld, etc); they incorrectly assume paths will be coded in utf8, but in fact, file paths might contain multibyte characters (for example, gbk encoding of chinese characters), esp. if the path come from program argument. in other words, our C based commands directly use msvcrt provided _open(2) function, which uses the ansi version of the underlying win32 api. |
if you can create a directory with non-ASCII characters (e.g. with character code > 0x80), say A, put some.go in it, and then go tool 8g A/some.go will fail because 8g can't create A/some.8. it's not strictly multibyte character that is troublesome, any character with code > 0x80 and not utf-8 encoded in a path will cause the problem. |
there are certainly people complaining this (I didn't discover this issue myself). as I mentioned in original report, when your user name contains non-ascii characters, cmd/go won't work at all (because $WORK will contain non-ascii characters) for example, this thread reports this bug in 'go test' (in Chinese): https://groups.google.com/group/golang-china/browse_thread/thread/84059760815f527a |
|
Issue #7327 has been merged into this issue. |
minux
added
accepted
Suggested
|
Fixed by the transition of toolchain components to Go in Go 1.5. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
multibyte characters in Windows path is not encoded in utf8, but all our command line programs implicitly assume utf8 encoding of path by using lib9/fmt (even smprint("%s", path) can't correctly handle multibyte paths). the worst case is: if your user name contains multibyte characters, then you won't be able to use cmd/go at all, because your os.TempDir() contains multibyte characters; and you don't have any easy workarounds (os.TempDir() uses syscall.GetTempPath, and you can't override result of GetTempPath easily). Could we make os.TempDir() look for %TEMP% or %TMP% first before calling GetTempPath() so that the user at least could change the temporary directory to workaround the aforementioned problem?The text was updated successfully, but these errors were encountered: