SameFile opens file to discover identifier and volume serial
number that uniquely identify the file. SameFile uses Windows
CreateFile API to open the file, and that works well for files
and directories. But CreateFile always follows symlinks, so
SameFile always opens symlink target instead of symlink itself.
This CL uses FILE_FLAG_OPEN_REPARSE_POINT flag to adjust
CreateFile behavior when handling symlinks.
As per https://docs.microsoft.com/en-us/windows/desktop/FileIO/symbolic-link-effects-on-file-systems-functions#createfile-and-createfiletransacted
"... If FILE_FLAG_OPEN_REPARSE_POINT is specified and:
If an existing file is opened and it is a symbolic link, the handle
returned is a handle to the symbolic link. ...".
I also added new tests for both issue #21854 and #27225.
Issue #27225 is still to be fixed, so skipping the test on
windows for the moment.
Fixes#21854
Updates #27225
Change-Id: I8aaa13ad66ce3b4074991bb50994d2aeeeaa7c95
Reviewed-on: https://go-review.googlesource.com/134195
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Also add more tests to test both nul and NUL on windows.
Fixes#24482
Change-Id: I3dfe68ec8de7f90ca869c1096dde0054df3c5cf6
Reviewed-on: https://go-review.googlesource.com/102457
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
os.Stat implementation uses instructions described at
https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
to distinguish symlinks. In particular, it calls
GetFileAttributesEx or FindFirstFile and checks
either WIN32_FILE_ATTRIBUTE_DATA.dwFileAttributes
or WIN32_FIND_DATA.dwFileAttributes to see if
FILE_ATTRIBUTES_REPARSE_POINT flag is set.
And that seems to worked fine so far.
But now we discovered that OneDrive root folder
is determined as directory:
c:\>dir C:\Users\Alex | grep OneDrive
30/11/2017 07:25 PM <DIR> OneDrive
c:\>
while Go identified it as symlink.
But we did not follow Microsoft's advice to the letter - we never
checked WIN32_FIND_DATA.Reserved0. And adding that extra check
makes Go treat OneDrive as symlink. So use FindFirstFile and
WIN32_FIND_DATA.Reserved0 to determine symlinks.
Fixes#22579
Change-Id: I0cb88929eb8b47b1d24efaf1907ad5a0e20de83f
Reviewed-on: https://go-review.googlesource.com/86556
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In general, there are no guarantee that `go` command exist on $PATH.
This CL tries to get `go` command from $GOROOT/bin instead.
There are three kinds of code we should handle:
For normal code, the CL implements goCmd() or goCmdName().
For unit tests, the CL uses testenv.GoTool() or testenv.GoToolPath().
For integration tests, the CL sets PATH=$GOROOT/bin:$PATH in cmd/dist.
Note that make.bash sets PATH=$GOROOT/bin:$PATH in the build process.
So this change is only useful when we use toolchain manually.
Updates #21875
Change-Id: I963b9f22ea732dd735363ececde4cf94a5db5ca2
Reviewed-on: https://go-review.googlesource.com/64650
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently windows Stat uses combination of Lstat and Readlink to
walk symlinks until it reaches file or directory. Windows Readlink
is implemented via Windows DeviceIoControl(FSCTL_GET_REPARSE_POINT, ...)
call, but that call does not work on network shares or inside of
Docker container (see issues #18555 ad #19922 for details).
But Raymond Chen suggests different approach:
https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
- he suggests to use Windows I/O manager to dereferences the
symbolic link.
This appears to work for all normal symlinks, but also for network
shares and inside of Docker container.
This CL implements described procedure.
I also had to adjust TestStatSymlinkLoop, because the test is
expecting Stat to return syscall.ELOOP for symlink with a loop.
But new Stat returns Windows error of ERROR_CANT_RESOLVE_FILENAME
= 1921 instead. I could map ERROR_CANT_RESOLVE_FILENAME into
syscall.ELOOP, but I suspect the former is broader than later.
And ERROR_CANT_RESOLVE_FILENAME message text of "The name of
the file cannot be resolved by the system." sounds fine to me.
Fixes#10935Fixes#18555Fixes#19922
Change-Id: I979636064cdbdb9c7c840cf8ae73fe2c24499879
Reviewed-on: https://go-review.googlesource.com/41834
Reviewed-by: Harshavardhana <hrshvardhana@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When using Lstat against symlinks that point to a directory,
the function returns FileInfo with both ModeDir and ModeSymlink set.
Change that to never set ModeDir if ModeSymlink is set.
Fixes#10424Fixes#17540Fixes#17541
Change-Id: Iba280888aad108360b8c1f18180a24493fe7ad2b
Reviewed-on: https://go-review.googlesource.com/41830
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Use an original name instead of a symlink's target path.
Fixes#20064
Change-Id: I9be3837a156bdcda0e9e065abbb425d535b27be3
Reviewed-on: https://go-review.googlesource.com/41310
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Go uses CommandLineToArgV from shell32.dll to parse command
line parameters. But shell32.dll is slow to load. Implement
Windows command line parsing in Go. This should make starting
Go programs faster.
I can see these speed ups for runtime.BenchmarkRunningGoProgram
on my Windows 7 amd64:
name old time/op new time/op delta
RunningGoProgram-2 11.2ms ± 1% 10.4ms ± 2% -6.63% (p=0.000 n=9+10)
on my Windows XP 386:
name old time/op new time/op delta
RunningGoProgram-2 19.0ms ± 3% 12.1ms ± 1% -36.20% (p=0.000 n=10+10)
on @egonelbre Windows 10 amd64:
name old time/op new time/op delta
RunningGoProgram-8 17.0ms ± 1% 15.3ms ± 2% -9.71% (p=0.000 n=10+10)
This CL is based on CL 22932 by John Starks.
Fixes#15588.
Change-Id: Ib14be0206544d0d4492ca1f0d91fac968be52241
Reviewed-on: https://go-review.googlesource.com/37915
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This changes the os package to use the runtime poller for file I/O
where possible. When a system call blocks on a pollable descriptor,
the goroutine will be blocked on the poller but the thread will be
released to run other goroutines. When using a non-pollable
descriptor, the os package will continue to use thread-blocking system
calls as before.
For example, on GNU/Linux, the runtime poller uses epoll. epoll does
not support ordinary disk files, so they will continue to use blocking
I/O as before. The poller will be used for pipes.
Since this means that the poller is used for many more programs, this
modifies the runtime to only block waiting for the poller if there is
some goroutine that is waiting on the poller. Otherwise, there is no
point, as the poller will never make any goroutine ready. This
preserves the runtime's current simple deadlock detection.
This seems to crash FreeBSD systems, so it is disabled on FreeBSD.
This is issue 19093.
Using the poller on Windows requires opening the file with
FILE_FLAG_OVERLAPPED. We should only do that if we can remove that
flag if the program calls the Fd method. This is issue 19098.
Update #6817.
Update #7903.
Update #15021.
Update #18507.
Update #19093.
Update #19098.
Change-Id: Ia5197dcefa7c6fbcca97d19a6f8621b2abcbb1fe
Reviewed-on: https://go-review.googlesource.com/36800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Go 1.5 worked with Unicode console input but not ^Z.
Go 1.6 did not work with Unicode console input but did handle one ^Z case.
Go 1.7 did not work with Unicode console input but did handle one ^Z case.
The intent of this CL is for Go 1.8 to work with Unicode console input
and also handle all ^Z cases.
Here's a simple test program for reading from the console.
It prints a "> " prompt, calls read, prints what it gets, and repeats.
package main
import (
"fmt"
"os"
)
func main() {
p := make([]byte, 100)
fmt.Printf("> ")
for {
n, err := os.Stdin.Read(p)
fmt.Printf("[%d %q %v]\n> ", n, p[:n], err)
}
}
On Unix, typing a ^D produces a break in the input stream.
If the ^D is at the beginning of a line, then the 0 bytes returned
appear as an io.EOF:
$ go run /tmp/x.go
> hello
[6 "hello\n" <nil>]
> hello^D[5 "hello" <nil>]
> ^D[0 "" EOF]
> ^D[0 "" EOF]
> hello^Dworld
[5 "hello" <nil>]
> [6 "world\n" <nil>]
>
On Windows, the EOF character is ^Z, not ^D, and there has
been a long-standing problem that in Go programs, ^Z on Windows
does not behave in the expected way, namely like ^D on Unix.
Instead, the ^Z come through as literal ^Z characters:
C:\>c:\go1.5.4\bin\go run x.go
> ^Z
[3 "\x1a\r\n" <nil>]
> hello^Zworld
[13 "hello\x1aworld\r\n" <nil>]
>
CL 4310 attempted to fix this bug, then known as #6303,
by changing the use of ReadConsole to ReadFile.
This CL was released as part of Go 1.6 and did fix the case
of a ^Z by itself, but not as part of a larger input:
C:\>c:\go1.6.3\bin\go run x.go
> ^Z
[0 "" EOF]
> hello^Zworld
[13 "hello\x1aworld\r\n" <nil>]
>
So the fix was incomplete.
Worse, the fix broke Unicode console input.
ReadFile does not handle Unicode console input correctly.
To handle Unicode correctly, programs must use ReadConsole.
Early versions of Go used ReadFile to read the console,
leading to incorrect Unicode handling, which was filed as #4760
and fixed in CL 7312053, which switched to ReadConsole
and was released as part of Go 1.1 and still worked as of Go 1.5:
C:\>c:\go1.5.4\bin\go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[16 "hello world™\r\n" <nil>]
>
But in Go 1.6:
C:\>c:\go1.6.3\bin\go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[0 "" EOF]
>
That is, changing back to ReadFile in Go 1.6 reintroduced #4760,
which has been refiled as #17097. (We have no automated test
for this because we don't know how to simulate console input
in a test: it appears that one must actually type at a keyboard
to use the real APIs. This CL at least adds a comment warning
not to reintroduce ReadFile again.)
CL 29493 attempted to fix#17097, but it was not a complete fix:
the hello world™ example above still fails, as does Shift-JIS input,
which was filed as #17939.
CL 29493 also broke ^Z handling, which was filed as #17427.
This CL attempts the never before successfully performed trick
of simultaneously fixing Unicode console input and ^Z handling.
It changes the console input to use ReadConsole again,
as in Go 1.5, which seemed to work for all known Unicode input.
Then it adds explicit handling of ^Z in the input stream.
(In the case where standard input is a redirected file, ^Z processing
should not happen, and it does not, because this code path is only
invoked when standard input is the console.)
With this CL:
C:\>go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[16 "hello world™\r\n" <nil>]
> ^Z
[0 "" EOF]
> [2 "\r\n" <nil>]
> hello^Zworld
[5 "hello" <nil>]
> [0 "" EOF]
> [7 "world\r\n" <nil>]
This almost matches Unix:
$ go run /tmp/x.go
> hello
[6 "hello\n" <nil>]
> hello world™
[15 "hello world™\n" <nil>]
> ^D
[0 "" EOF]
> [1 "\n" <nil>]
> hello^Dworld
[5 "hello" <nil>]
> [6 "world\n" <nil>]
>
The difference is in the handling of hello^Dworld / hello^Zworld.
On Unix, hello^Dworld terminates the read of hello but does not
result in a zero-length read between reading hello and world.
This is dictated by the tty driver, not any special Go code.
On Windows, in this CL, hello^Zworld inserts a zero length read
result between hello and world, which is treated as an interior EOF.
This is implemented by the Go code in this CL, but it matches the
handling of ^Z on the console in other programs:
C:\>copy con x.txt
hello^Zworld
1 file(s) copied.
C:\>type x.txt
hello
C:\>
A natural question is how to test all this. As noted above, we don't
know how to write automated tests using the actual Windows console.
CL 29493 introduced the idea of substituting a different syscall.ReadFile
implementation for testing; this CL continues that idea but substituting
for syscall.ReadConsole instead. To avoid the regression of putting
ReadFile back, this CL adds a comment warning against that.
Fixes#17427.
Fixes#17939.
Change-Id: Ibaabd0ceb2d7af501d44ac66d53f64aba3944142
Reviewed-on: https://go-review.googlesource.com/33451
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Quentin Smith <quentin@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TestReadStdin always fill up buffer provided by ReadFile caller full.
But we do not know if real ReadFile does the same. Add tests where
buffer is only filled with limited data.
Change-Id: I0fc776325c2b1fe60511126c439f4b0560e9d653
Reviewed-on: https://go-review.googlesource.com/33030
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Current implementation of syscall.Readlink mistakenly calculates
the end offset of the PrintName field.
Also, there are some cases that the PrintName field is empty.
Instead, the CL uses SubstituteName with correct calculation.
Fixes#15978Fixes#16145
Change-Id: If3257137141129ac1c552d003726d5b9c08bb754
Reviewed-on: https://go-review.googlesource.com/31118
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL introduces first test for readConsole. And new test
discovered couple of problems with readConsole.
Console characters consist of multiple bytes each, but byte blocks
returned by syscall.ReadFile have no character boundaries. Some
multi-byte characters might start at the end of one block, and end
at the start of next block. readConsole feeds these blocks to
syscall.MultiByteToWideChar to convert them into utf16, but if some
multi-byte characters have no ending or starting bytes, the
syscall.MultiByteToWideChar might get confused. Current version of
syscall.MultiByteToWideChar call will make
syscall.MultiByteToWideChar ignore all these not complete
multi-byte characters.
The CL solves this issue by changing processing from "randomly
sized block of bytes at a time" to "one multi-byte character at a
time". New readConsole code calls syscall.ReadFile to get 1 byte
first. Then it feeds this byte to syscall.MultiByteToWideChar.
The new syscall.MultiByteToWideChar call uses MB_ERR_INVALID_CHARS
flag to make syscall.MultiByteToWideChar return error if input is
not complete character. If syscall.MultiByteToWideChar returns
correspondent error, we read another byte and pass 2 byte buffer
into syscall.MultiByteToWideChar, and so on until success.
Old readConsole code would also sometimes return no data if user
buffer was smaller then uint16 size, which would confuse callers
that supply 1 byte buffer. This CL fixes that problem too.
Fixes#17097
Change-Id: I88136cdf6a7bf3aed5fbb9ad2c759b6c0304ce30
Reviewed-on: https://go-review.googlesource.com/29493
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
https://go-review.googlesource.com/#/c/27580 added the test.
However the test use syscall.ELOOP which is not defined on plan9.
Move test code from "os_test.go" to "os_windows_test.go" to prevent
build error.
Change-Id: Ie7f05bfb9ab229e06a8e82a4b3b8a7ca82d4663b
Reviewed-on: https://go-review.googlesource.com/27833
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Include integration test. Confirmed that without the fix, the test case
TestDeleteReadOnly fails.
This permits to revert "cmd/go: reset read-only flag during TestIssue10952"
This reverts commit 3b7841b3aff9204f054ffabbe4dd39d3e3dd3e91.
Fixes#9606
Change-Id: Ib55c151a8cf1a1da02ab18c34a9b58f615c34254
Reviewed-on: https://go-review.googlesource.com/18235
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Open(`C:`) currently opens root directory on C:. Change that to open
current directory on C:. Just like cmd.exe's "dir C:" command does.
Just like FindFirstFile("C:*") Windows API does. It is also consistent
with what filepath.Join("C:", "a") currently does.
Fixes#13763
Change-Id: I60b6e7d80215d110bbbb6265c9f32717401638c6
Reviewed-on: https://go-review.googlesource.com/18184
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Otherwise IsNotExist does not account for not existent servers and shares.
Fixes#12374
Change-Id: I37f6850198f91dcb02a4a917b793339d7e30e934
Reviewed-on: https://go-review.googlesource.com/14579
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
