path/filepath: Need a way to canonicalize paths

[sschuberth]

What version of Go are you using (go version)?

go version go1.5.3 windows/amd64

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=
set GORACE=
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GO15VENDOREXPERIMENT=
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1

What did you do?

I was looking for a way to canonize a path as Abs "is not guaranteed to be unique". Calling Abs on Windows for example might return a path containing DOS-style 8.3 short names. On Linux, Abs might return a path with symlinks.

What did you expect to see?

I was expecting to find a function to canonize a path, so paths can be safely compared.

What did you see instead?

I did not find a way canonize a path / to create a unique absolute path.

[minux]

How about Abs(EvalSymlinks(path)) on Linux? However, if you just want to uniquely identify a file, it's better to use a file specific attribute rather than its path (as file could be moved even if it's open on Unix) https://golang.org/pkg/os/#SameFile might help here.

[sschuberth]

How about Abs(EvalSymlinks(path)) on Linux?

That looks like a viable implementation for Linux of a function to canonize paths.

Thanks for pointing out os.SameFile() which indeed works for my use-case. However, I still feel having a function to canonize paths could be useful. Feel free to close this if you disagree.

[alexbrainman]

I think EvalSymlinks does what you want on windows.

Alex

[sschuberth]

I've just confirmed that something like

filepath.EvalSymlinks("C:\\Users\\sebastian\\DOWNLO~1")

will indeed return

C:\Users\sebastian\Download

on Windows, which is what I wanted. So probably EvalSymlinks()'s documentation just needs some improvement in this regard.

Thanks!

[sschuberth]

I've created a small change to clarify the docs: https://go-review.googlesource.com/29055

[sschuberth]

Reopening because @alexbrainman stated that we do not want to document EvalSymlinks() to work that way. So we indeed would need a separate function to canonize paths which is documented to also convert Windows short filenames to long filenames.

[bradfitz]

Something in golang.org/x/sys/windows perhaps?

[sschuberth]

I was more thinking about an OS-independent place. golang.org/x/sys/windows seems to be more about direct 1-to-1 mappings to the Windows API. Interestingly, the Windows API has a PathCanonicalize function, but that does not do the right thing.

Basically, I'm suggesting something like Java's getCanonicalPath, implemented for all OSes.

[robpike]

I'm not sure this is even possible in general, or at least I don't know how to define a portable definition of a canonical file name. Perhaps it's a Windows-specific issue.

[griesemer]

We do have path/filepath.ToSlash which is half of the story.

[minux]

I think the requirement itself is problematic. Trying to use a path to determine two files are the same portably is inherently impossible. Given that, what's the definition for a canonical path? Every Clean'ed and EvalSymlink'ed path is as canonical as any others.

[alexbrainman]

So we indeed would need a separate function to canonize paths which is documented to also convert Windows short filenames to long filenames.

Why would you need such function?

You can use os.SameFile to test if 2 files are the same. You can even use filepath.EvalSymlinks to convert real file paths in different format to the same string. For example this https://play.golang.org/p/1FnLrOOXoq outputs:

C:\Documents and Settings\brainman\.gitconfig
C:\Documents and Settings\brainman\.gitconfig
C:\Documents and Settings\brainman\.gitconfig
C:\Documents and Settings\brainman\.gitconfig
C:\Documents and Settings\brainman\.gitconfig
C:\Documents and Settings\brainman\.gitconfig

on my computer.

filepath.EvalSymlinks started as "canonical" way to express file path https://codereview.appspot.com/5713043/#msg5 And we tried to keep filepath.EvalSymlinks working this way. But I don't think there is any definition of "canonical" in Windows API. There is no Windows API that would provide that functionality. What we do in filepath.EvalSymlinks is mish and mash.

Something in golang.org/x/sys/windows perhaps?

Lets see what exactly @sschuberth wants.

We do have path/filepath.ToSlash which is half of the story.

filepath.ToSlash just converts \ into / in a string. filepath.EvalSymlinks works on real files / directories and use Windows help to discover "real" file names. As you can see from https://play.golang.org/p/1FnLrOOXoq Windows accepts many different ways to name a file, and filepath.EvalSymlinks converts all different variations into a single form.

... what's the definition for a canonical path? Every Clean'ed and
EvalSymlink'ed path is as canonical as any others.

I agree. Putting the word "canonical" in the documentation might just confuse things even more.

Maybe we should let Russ decide wha to do here. He started it https://codereview.appspot.com/5713043/#msg5 :-)

Alex