Most architectures currently already implement Pipe using the pipe2
syscall. Only 386, amd64 and mips{,le} still use the pipe syscall.
However, some systems (e.g. Android seccomp policies) block that
syscall, see #40828 for an example.
The pipe2 syscall was added in Linux kernel version 2.6.27. The minimum
required Linux kernel version for Go 1.18 will be changed to 2.6.32
per #45964 so it is possible to unify the implementation of Pipe using
the pipe2 syscall.
For #45964
Change-Id: I8ed6a391300c95f3107b4ec6b27d320e42fb535b
Reviewed-on: https://go-review.googlesource.com/c/go/+/350530
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This allows the use of CLONE_VFORK and CLONE_VM for fork/exec, preventing
'fork/exec ...: cannot allocate memory' failures from occuring when attempting
to execute commands from a Go process that has a large memory footprint.
Additionally, this should reduce the latency of fork/exec on these platforms.
Fixes#31936
Change-Id: I4e28cf0763173145cacaa5340680dca9ff449305
Reviewed-on: https://go-review.googlesource.com/c/go/+/295849
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Make all our package sources use Go 1.17 gofmt format
(adding //go:build lines).
Part of //go:build change (#41184).
See https://golang.org/design/draft-gobuild
Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/294430
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change fixes two failng tests on linux-ppc64x:
- TestAllThreadsSyscall() exposed a real bug in the ppc64x support:
- It turns out that the r2 syscall return value is not defined
on all architectures. Notably linux-ppc64x so address that by
introducing a private architectural constant in the syscall
package, archHonorsR2: true if r2 has a determanistic value.
- TestSetuidEtc() was sensitive to /proc/<PID>/status content:
- The amount of padding space has changed with kernel vintage.
- Stress testing revealed a race with /proc files disappearing.
Fixes#42178
Change-Id: Ie6fc0b8f2f94a409ac0e5756e73bfce113274709
Reviewed-on: https://go-review.googlesource.com/c/go/+/266202
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
This change adds two new methods for invoking system calls
under Linux: syscall.AllThreadsSyscall() and
syscall.AllThreadsSyscall6().
These system call wrappers ensure that all OSThreads mirror
a common system call. The wrappers serialize execution of the
runtime to ensure no race conditions where any Go code observes
a non-atomic OS state change. As such, the syscalls have
higher runtime overhead than regular system calls, and only
need to be used where such thread (or 'm' in the parlance
of the runtime sources) consistency is required.
The new support is used to enable these functions under Linux:
syscall.Setegid(), syscall.Seteuid(), syscall.Setgroups(),
syscall.Setgid(), syscall.Setregid(), syscall.Setreuid(),
syscall.Setresgid(), syscall.Setresuid() and syscall.Setuid().
They work identically to their glibc counterparts.
Extensive discussion of the background issue addressed in this
patch can be found here:
https://github.com/golang/go/issues/1435
In the case where cgo is used, the C runtime can launch pthreads that
are not managed by the Go runtime. As such, the added
syscall.AllThreadsSyscall*() return ENOTSUP when cgo is enabled.
However, for the 9 syscall.Set*() functions listed above, when cgo is
active, these functions redirect to invoke their C.set*() equivalents
in glibc, which wraps the raw system calls with a nptl:setxid fixup
mechanism. This achieves POSIX semantics for these functions in the
combined Go and C runtime.
As a side note, the glibc/nptl:setxid support (2019-11-30) does not
extend to all security related system calls under Linux so using
native Go (CGO_ENABLED=0) and these AllThreadsSyscall*()s, where
needed, will yield more well defined/consistent behavior over all
threads of a Go program. That is, using the
syscall.AllThreadsSyscall*() wrappers for things like setting state
through SYS_PRCTL and SYS_CAPSET etc.
Fixes#1435
Change-Id: Ib1a3e16b9180f64223196a32fc0f9dce14d9105c
Reviewed-on: https://go-review.googlesource.com/c/go/+/210639
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Android O and newer blocks the dup2 syscall.
Change-Id: Ibca01fc72ef114deeef6c0450a8b81a556ed0530
Reviewed-on: https://go-review.googlesource.com/c/go/+/235537
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reimplement syscall wrappers for linux/arm64 in terms of supported
syscalls (or in case of Ustat make it return ENOSYS) and remove the
manually added SYS_* consts for the deprecated syscalls. Adapted from
golang.org/x/sys/unix where this is already done since CL 119655.
Change-Id: I94ab48a4645924df3822497d0575f1a1573d509f
Reviewed-on: https://go-review.googlesource.com/c/go/+/199140
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
On linux/riscv64, the renameat syscall no longer exists and has been
superseded by renameat2. Thus we'll have to use Renameat2 to implement
Renameat on linux/riscv64 for #27532. Prepare for this by moving the
Renameat definition to the GOARCH specific files.
Follow CL 157899 which did the same for golang.org/x/sys/unix
Change-Id: I9670213cc3987df48fee962ddee36915a7785560
Reviewed-on: https://go-review.googlesource.com/c/go/+/192077
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The Linux kernel faccessat system call does not take a flags parameter.
The flag parameter to the C library faccessat function is implemented in C.
The syscall.Faccessat function takes a flags parameter. In older releases
we have passed the flags parameter to the kernel, which ignored it.
In CL 120015 we started returning an error if any flags were set.
That seems clearly better than ignoring them, but it turns out that some
code was using the flags. The code was previously subtly broken.
Now it is obviously broken. That is better, but we can do better still:
we can implement the flags as the C library does. That is what this CL does.
Change-Id: I259bd6f240c3951e939b81c3032dead3d9c567b4
Reviewed-on: https://go-review.googlesource.com/126415
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The getdents64 syscall is only available for mips64/mips64le starting
with Linux kernel 3.10. Since mips64le requires at least 4.8 according
to [1] (regarding #16848) using it should be fine.
[1] https://golang.org/wiki/MinimumRequirements
This CL changes the binary layout of type Dirent for mips64/mips64le,
but not the public API. But since the currently used layout doesn't
match the struct linux_dirent returned by the getdents syscall this
should be fine as well.
Fixes#23624
Change-Id: Iaa7306fa6e4442ad2fed41c60b37627a7314f117
Reviewed-on: https://go-review.googlesource.com/91055
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
syscall.Select uses SYS_PSELECT6 on arm64 and mipx64x, however this
syscall expects its 5th argument to be of type Timespec (with seconds
and nanoseconds) instead of type Timeval (with seconds and microseconds)
This leads to the timeout being too short by a factor of 1000.
This CL fixes this by adjusting the timeout argument accordingly,
similarly to how glibc does it for architectures where neither
SYS_SELECT nor SYS__NEWSELECT are available.
Fixes#22246
Change-Id: I33a183b0b87c2dae4a77a2d00f8615169fad48dd
Reviewed-on: https://go-review.googlesource.com/70590
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This greatly improves the latency of starting a child process when
the Go process is using a lot of memory. Even though the kernel uses
copy-on-write, preparation for that can take up to several 100ms under
certain conditions. All other goroutines are suspended while starting
a subprocess so this latency directly affects total throughput.
With CLONE_VM the child process shares the same memory with the parent
process. On its own this would lead to conflicting use of the same
memory, so CLONE_VFORK is used to suspend the parent process until the
child releases the memory when switching to to the new program binary
via the exec syscall. When the parent process continues to run, one
has to consider the changes to memory that the child process did,
namely the return address of the syscall function needs to be restored
from a register.
A simple benchmark has shown a difference in latency of 16ms vs. 0.5ms
at 10GB memory usage. However, much higher latencies of several 100ms
have been observed in real world scenarios. For more information see
comments on #5838.
Fixes#5838
Change-Id: I6377d7bd8dcd00c85ca0c52b6683e70ce2174ba6
Reviewed-on: https://go-review.googlesource.com/37439
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
All the implementations of NsecToTimespec and NsecToTimeval were the
same other than types. Write a single version that uses
GOARCH/GOOS-specific setTimespec and setTimeval functions to handle the
types.
The logic in NsecToTimespec and NsecToTimeval caused times before 1970
to have a negative usec/nsec. The Linux kernel requires that usec
contain a positive number; for consistency, we do this for both
NsecToTimespec and NsecToTimeval.
Change-Id: I525eaba2e7cdb00cb57fa00182dabf19fec298ae
Reviewed-on: https://go-review.googlesource.com/30826
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
All implementations of these functions are identical.
Change-Id: I7cbea53c02bb0cee75e30beed19d29ba0a7ef657
Reviewed-on: https://go-review.googlesource.com/30819
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
syscall.Getpagesize currently returns hard-coded page sizes on all
architectures (some of which are probably always wrong, and some of
which are definitely not always right). The runtime now has this
information, queried from the OS during runtime init, so make
syscall.Getpagesize return the page size that the runtime knows.
Updates #10180.
Change-Id: I4daa6fbc61a2193eb8fa9e7878960971205ac346
Reviewed-on: https://go-review.googlesource.com/25051
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Syscall getdents64 is relatively new in linux/mips64, only since kernel
version 3.10. To support older kernel, syscall getdents is used for
mips64.
Change-Id: I892b05dff7d93e7ddb0d700abd6a5e6d4084ab4c
Reviewed-on: https://go-review.googlesource.com/14449
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
