Compare commits

...

70 Commits

Author SHA1 Message Date
Gopher Robot
d04e3cbc92 [release-branch.go1.23] go1.23.5
Change-Id: I849328cf93adf24c223f103f5b834635970ea652
Reviewed-on: https://go-review.googlesource.com/c/go/+/643137
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-01-16 12:07:56 -08:00
Damien Neil
bb8230f805 [release-branch.go1.23] net/http: persist header stripping across repeated redirects
When an HTTP redirect changes the host of a request, we drop
sensitive headers such as Authorization from the redirected request.
Fix a bug where a chain of redirects could result in sensitive
headers being sent to the wrong host:

  1. request to a.tld with Authorization header
  2. a.tld redirects to b.tld
  3. request to b.tld with no Authorization header
  4. b.tld redirects to b.tld
  3. request to b.tld with Authorization header restored

Thanks to Kyle Seely for reporting this issue.

For #70530
Fixes ##71211
Fixes CVE-2024-45336

Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1641
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Change-Id: I326544358de71ff892d9e9fe338252a5dd04001f
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1764
Reviewed-on: https://go-review.googlesource.com/c/go/+/643104
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2025-01-16 11:00:39 -08:00
Roland Shoemaker
fdb8413fe5 [release-branch.go1.23] crypto/x509: properly check for IPv6 hosts in URIs
When checking URI constraints, use netip.ParseAddr, which understands
zones, unlike net.ParseIP which chokes on them. This prevents zone IDs
from mistakenly satisfying URI constraints.

Thanks to Juho Forsén of Mattermost for reporting this issue.

For #71156
Fixes #71208
Fixes CVE-2024-45341

Change-Id: Iecac2529f3605382d257996e0fb6d6983547e400
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1700
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
(cherry picked from commit 22ca55d396ba801e6ae9b2bd67a059fcb30562fd)
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1762
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/643103
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2025-01-16 11:00:37 -08:00
Michael Anthony Knyszek
1dde0b4844 [release-branch.go1.23] runtime: hold traceAcquire across casgstatus in injectglist
Currently injectglist emits all the trace events before actually calling
casgstatus on each goroutine. This is a problem, since tracing can
observe an inconsistent state (gstatus does not match tracer's 'emitted
an event' state).

This change fixes the problem by having injectglist do what every other
scheduler function does, and that's wrap each call to casgstatus in
traceAcquire/traceRelease.

For #70883.
Fixes #71147.

Change-Id: I857e96cec01688013597e8efc0c4c3d0b72d3a70
Reviewed-on: https://go-review.googlesource.com/c/go/+/638558
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit f025d19e7b3f0c66242760c213cc2b54cb100f69)
Reviewed-on: https://go-review.googlesource.com/c/go/+/641378
Auto-Submit: Michael Pratt <mpratt@google.com>
2025-01-08 10:02:57 -08:00
Filippo Valsorda
3417000c69 [release-branch.go1.23] crypto/tls: fix Config.Time in tests using expired certificates
Updates #71077
Fixes #71104

Change-Id: I6a6a465685f3bd50a5bb35a160f87b59b74fa6af
Reviewed-on: https://go-review.googlesource.com/c/go/+/639655
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/640315
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2025-01-08 09:33:16 -08:00
Ian Lance Taylor
1576793c51 [release-branch.go1.23] net: don't use sendfile for irregular files; handle EOPNOTSUPP/ENOTSUP
This is not a cherry pick, because the code has changed on tip.

For #70763
Fixes #70789

Change-Id: If9fcfee17e86a746cf8c72293dc34f80b430f6e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/635397
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-12-18 09:42:15 -08:00
Cherry Mui
59b7d40774 [release-branch.go1.23] runtime: update and restore g0 stack bounds at cgocallback
Currently, at a cgo callback where there is already a Go frame on
the stack (i.e. C->Go->C->Go), we require that at the inner Go
callback the SP is within the g0's stack bounds set by a previous
callback. This is to prevent that the C code switches stack while
having a Go frame on the stack, which we don't really support. But
this could also happen when we cannot get accurate stack bounds,
e.g. when pthread_getattr_np is not available. Since the stack
bounds are just estimates based on the current SP, if there are
multiple C->Go callbacks with various stack depth, it is possible
that the SP of a later callback falls out of a previous call's
estimate. This leads to runtime throw in a seemingly reasonable
program.

This CL changes it to save the old g0 stack bounds at cgocallback,
update the bounds, and restore the old bounds at return. So each
callback will get its own stack bounds based on the current SP,
and when it returns, the outer callback has the its old stack
bounds restored.

Also, at a cgo callback when there is no Go frame on the stack,
we currently always get new stack bounds. We do this because if
we can only get estimated bounds based on the SP, and the stack
depth varies a lot between two C->Go calls, the previous
estimates may be off and we fall out or nearly fall out of the
previous bounds. But this causes a performance problem: the
pthread API to get accurate stack bounds (pthread_getattr_np) is
very slow when called on the main thread. Getting the stack bounds
every time significantly slows down repeated C->Go calls on the
main thread.

This CL fixes it by "caching" the stack bounds if they are
accurate. I.e. at the second time Go calls into C, if the previous
stack bounds are accurate, and the current SP is in bounds, we can
be sure it is the same stack and we don't need to update the bounds.
This avoids the repeated calls to pthread_getattr_np. If we cannot
get the accurate bounds, we continue to update the stack bounds
based on the SP, and that operation is very cheap.

On a Linux/AMD64 machine with glibc:

name                     old time/op  new time/op  delta
CgoCallbackMainThread-8  96.4µs ± 3%   0.1µs ± 2%  -99.92%  (p=0.000 n=10+9)

Updates #68285.
Updates #68587.
Fixes #69988.

Change-Id: I3422badd5ad8ff63e1a733152d05fb7a44d5d435
Reviewed-on: https://go-review.googlesource.com/c/go/+/600296
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit 76a8409eb81eda553363783dcdd9d6224368ae0e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/635775
2024-12-13 11:18:11 -08:00
Tim King
69c8cfe29b [release-branch.go1.23] cmd/compile/internal/importer: enable aliases
Flips the pkgReader.enableAlias flag to true when reading unified IR.
This was disabled while resolving #66873. This resolves the TODO to
flip it back to true.

Fixes #70394
Fixes #70517
Updates #66873

Change-Id: Ifd52b0f9510d6bcf151de1c9a18d71ab548c14e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/604099
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
(cherry picked from commit 209ed1a9c75d17046285c416b74a14bb89799757)
Reviewed-on: https://go-review.googlesource.com/c/go/+/631855
Commit-Queue: Tim King <taking@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
2024-12-11 11:17:27 -08:00
Gopher Robot
194de8fbfa [release-branch.go1.23] go1.23.4
Change-Id: I8d26e5231e868476949390ec900f0273c816d807
Reviewed-on: https://go-review.googlesource.com/c/go/+/633217
Reviewed-by: Veronica Silina <veronicasilina@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-12-03 18:27:46 +00:00
Felix Geisendörfer
5164a865e3 [release-branch.go1.23] cmd/trace: also show end stack traces
Fix a regression that appeared in 1.23 when it comes to the stack traces
shown in the trace viewer. In 1.22 and earlier, the viewer was always
showing end stack traces. In 1.23 and later the viewer started to
exclusively show start stack traces.

Showing only the start stack traces made it impossible to see the last
stack trace produced by a goroutine. It also made it hard to understand
why a goroutine went off-cpu, as one had to hunt down the next running
slice of the same goroutine.

Emit end stack traces in addition to start stack traces to fix the
issue.

Fixes #70592

Change-Id: Ib22ea61388c1d94cdbc99fae2d207c4dce011a59
Reviewed-on: https://go-review.googlesource.com/c/go/+/631895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit 6405e60ca6be798c1f8c1d0365fd63b89b524df5)
Reviewed-on: https://go-review.googlesource.com/c/go/+/632075
Reviewed-by: Veronica Silina <veronicasilina@google.com>
Auto-Submit: Veronica Silina <veronicasilina@google.com>
2024-11-27 20:24:20 +00:00
Russ Cox
25f042daec [release-branch.go1.23] runtime: reserve 4kB for system stack on windows-386
The failures in #70288 are consistent with and strongly imply
stack corruption during fault handling, and debug prints show
that the Go code run during fault handling is running about
300 bytes above the bottom of the goroutine stack.
That should be okay, but that implies the DLL code that called
Go's handler was running near the bottom of the stack too,
and maybe it called other deeper things before or after the
Go handler and smashed the stack that way.

stackSystem is already 4096 bytes on amd64;
making it match that on 386 makes the flaky failures go away.
It's a little unsatisfying not to be able to say exactly what is
overflowing the stack, but the circumstantial evidence is
very strong that it's Windows.

For #70288.
Fixes #70475.

Change-Id: Ife89385873d5e5062a71629dbfee40825edefa49
Reviewed-on: https://go-review.googlesource.com/c/go/+/627375
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 7eeb0a188eb644486da9f77bae0375d91433d0bf)
Reviewed-on: https://go-review.googlesource.com/c/go/+/632196
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Veronica Silina <veronicasilina@google.com>
2024-11-27 19:14:27 +00:00
Michael Anthony Knyszek
be062b7f61 [release-branch.go1.23] runtime: explicitly keep handle alive during getOrAddWeakHandle
getOrAddWeakHandle is very careful about keeping its input alive across
the operation, but not very careful about keeping the heap-allocated
handle it creates alive. In fact, there's a window in this function
where it is *only* visible via the special. Specifically, the window of
time between when the handle is stored in the special and when the
special actually becomes visible to the GC.

(If we fail to add the special because it already exists, that case is
fine. We don't even use the same handle value, but the one we obtain
from the attached GC-visible special, *and* we return that value, so it
remains live.)

For #70455.
Fixes #70469.

Change-Id: Iadaff0cfb93bcaf61ba2b05be7fa0519c481de82
Reviewed-on: https://go-review.googlesource.com/c/go/+/630316
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-20 19:59:34 +00:00
Michael Anthony Knyszek
d8adc6c4c7 [release-branch.go1.23] runtime: prevent weak->strong conversions during mark termination
Currently it's possible for weak->strong conversions to create more GC
work during mark termination. When a weak->strong conversion happens
during the mark phase, we need to mark the newly-strong pointer, since
it may now be the only pointer to that object. In other words, the
object could be white.

But queueing new white objects creates GC work, and if this happens
during mark termination, we could end up violating mark termination
invariants. In the parlance of the mark termination algorithm, the
weak->strong conversion is a non-monotonic source of GC work, unlike the
write barriers (which will eventually only see black objects).

This change fixes the problem by forcing weak->strong conversions to
block during mark termination. We can do this efficiently by setting a
global flag before the ragged barrier that is checked at each
weak->strong conversion. If the flag is set, then the conversions block.
The ragged barrier ensures that all Ps have observed the flag and that
any weak->strong conversions which completed before the ragged barrier
have their newly-minted strong pointers visible in GC work queues if
necessary. We later unset the flag and wake all the blocked goroutines
during the mark termination STW.

There are a few subtleties that we need to account for. For one, it's
possible that a goroutine which blocked in a weak->strong conversion
wakes up only to find it's mark termination time again, so we need to
recheck the global flag on wake. We should also stay non-preemptible
while performing the check, so that if the check *does* appear as true,
it cannot switch back to false while we're actively trying to block. If
it switches to false while we try to block, then we'll be stuck in the
queue until the following GC.

All-in-all, this CL is more complicated than I would have liked, but
it's the only idea so far that is clearly correct to me at a high level.

This change adds a test which is somewhat invasive as it manipulates
mark termination, but hopefully that infrastructure will be useful for
debugging, fixing, and regression testing mark termination whenever we
do fix it.

For #69803.
Fixes #70323.

Change-Id: Ie314e6fd357c9e2a07a9be21f217f75f7aba8c4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/623615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
(cherry picked from commit 80d306da50aef6334bcb65fb02f5728cb9513691)
Reviewed-on: https://go-review.googlesource.com/c/go/+/627615
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2024-11-20 19:08:32 +00:00
qmuntal
847cb6f9ca [release-branch.go1.23] syscall: mark SyscallN as noescape
syscall.SyscallN is implemented by runtime.syscall_syscalln, which makes
sure that the variadic argument doesn't escape.

There is no need to worry about the lifetime of the elements of the
variadic argument, as the compiler will keep them live until the
function returns.

For #70197
Fixes #70202

Change-Id: I12991f0be12062eea68f2b103fa0a794c1b527eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/625297
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 7fff741016c8157e107cce8013ee3ca621725384)
Reviewed-on: https://go-review.googlesource.com/c/go/+/630196
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
2024-11-20 18:07:04 +00:00
Dmitri Shuralyov
777f43ab27 [release-branch.go1.23]time: accept "+01" in TestLoadFixed on OpenBSD
This stops the test from failing with a known failure mode, and
creates time to look into what the next steps should be, if any.

For #69840
Fixes #70239

Change-Id: I060903d256ed65c5dfcd70ae76eb361cab63186f
Reviewed-on: https://go-review.googlesource.com/c/go/+/625197
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Eric Grosse <grosse@gmail.com>
(cherry picked from commit bea9b91f0f4be730c880edbe496ab25c9b742cad)
Reviewed-on: https://go-review.googlesource.com/c/go/+/627575
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-11-19 18:04:02 +00:00
David Chase
3726f07c46 [release-branch.go1.23] cmd/compile: use a non-fragile test for "does f contain closure c?"
The old test relied on naming conventions.  The new test
uses an explicit parent pointer chain initialized when the
closures are created (in the same place that the names
used in the older fragile test were assigned).

Fixes #70198.

Change-Id: Ie834103c7096e4505faaff3bed1fc6e918a21211
Reviewed-on: https://go-review.googlesource.com/c/go/+/622656
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/625535
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2024-11-19 17:52:28 +00:00
Gopher Robot
c390a1c22e [release-branch.go1.23] go1.23.3
Change-Id: I065005a4a18f801d09ad3ebc886e90a6dd1df69a
Reviewed-on: https://go-review.googlesource.com/c/go/+/626137
Reviewed-by: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2024-11-06 22:48:16 +00:00
qmuntal
1207de4f6c [release-branch.go1.23] runtime: reduce syscall.SyscallX stack usage
syscall.SyscallX consumes a lot of stack space, which is a problem
because they are nosplit functions. They used to use less stack space,
but CL 563315, that landed in Go 1.23, increased the stack usage by a
lot.

This CL reduces the stack usage back to the previous level.

Fixes #69848
Updates #69813

Change-Id: Iddedd28b693c66a258da687389768055c493fc2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/618497
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit fa7343aca326aad061ab877c1a4cebb96c4355c1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/623516
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-10-30 17:06:14 +00:00
Felix Geisendörfer
a0d15cb9c8 [release-branch.go1.23] runtime: fix MutexProfile missing root frames
Fix a regression introduced in CL 598515 causing runtime.MutexProfile
stack traces to omit their root frames.

In most cases this was merely causing the `runtime.goexit` frame to go
missing. But in the case of runtime._LostContendedRuntimeLock, an empty
stack trace was being produced.

Add a test that catches this regression by checking for a stack trace
with the `runtime.goexit` frame.

Also fix a separate problem in expandFrame that could cause
out-of-bounds panics when profstackdepth is set to a value below 32.
There is no test for this fix because profstackdepth can't be changed at
runtime right now.

Fixes #69865

Change-Id: I1600fe62548ea84981df0916d25072c3ddf1ea1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/611615
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit c64ca8c6ef13723b9f25f4b5e1c7b6986b958d2e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/621276
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-10-30 17:05:06 +00:00
Andy Pan
958f3a0309 [release-branch.go1.23] internal/poll: handle the special case of sendfile(2) sending the full chunk
CL 622235 would fix #70000 while resulting in one extra sendfile(2) system
call when sendfile(2) returns (>0, EAGAIN).
That's also why I left sendfile_bsd.go behind, and didn't make it line up
with other two implementations: sendfile_linux.go and sendfile_solaris.go.

Unlike sendfile(2)'s on Linux and Solaris that always return (0, EAGAIN),
sendfile(2)'s on *BSD and macOS may return (>0, EAGAIN) when using a socket
marked for non-blocking I/O. In that case, the current code will try to re-call
sendfile(2) immediately, which will most likely get us a (0, EAGAIN).
After that, it goes to `dstFD.pd.waitWrite(dstFD.isFile)` below,
which should have been done in the first place.

Thus, the real problem that leads to #70000 is that the old code doesn't handle
the special case of sendfile(2) sending the exact number of bytes the caller requested.

Fixes #70000
Fixes #70020

Change-Id: I6073d6b9feb58b3d7e114ec21e4e80d9727bca66
Reviewed-on: https://go-review.googlesource.com/c/go/+/622255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/622697
2024-10-30 17:04:03 +00:00
Damien Neil
6ba3a8a6ba [release-branch.go1.23] internal/poll: keep copying after successful Sendfile return on BSD
The BSD implementation of poll.SendFile incorrectly halted
copying after succesfully writing one full chunk of data.
Adjust the copy loop to match the Linux and Solaris
implementations.

In testing, empirically macOS appears to sometimes return
EAGAIN from sendfile after successfully copying a full
chunk. Add a check to all implementations to return nil
after successfully copying all data if the last sendfile
call returns EAGAIN.

For #70000
For #70020

Change-Id: I57ba649491fc078c7330310b23e1cfd85135c8ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/622235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
(cherry picked from commit bd388c0216bcb33d7325b0ad9722a3be8155a289)
Reviewed-on: https://go-review.googlesource.com/c/go/+/622696
2024-10-30 17:03:55 +00:00
Cherry Mui
5472853843 [release-branch.go1.23] cmd/link: generate Mach-O UUID when -B flag is specified
Currently, on Mach-O, the Go linker doesn't generate LC_UUID in
internal linking mode. This causes some macOS system tools unable
to track the binary, as well as in some cases the binary unable
to access local network on macOS 15.

This CL makes the linker start generate LC_UUID. Currently, the
UUID is generated if the -B flag is specified. And we'll make it
generate UUID by default in a later CL. The -B flag is currently
for generating GNU build ID on ELF, which is a similar concept to
Mach-O's UUID. Instead of introducing another flag, we just use
the same flag and the same setting. Specifically, "-B gobuildid"
will generate a UUID based on the Go build ID.

Updates #68678.
Fixes #69992.

Cq-Include-Trybots: luci.golang.try:go1.23-darwin-amd64_14,go1.23-darwin-arm64_13
Change-Id: I90089a78ba144110bf06c1c6836daf2d737ff10a
Reviewed-on: https://go-review.googlesource.com/c/go/+/618595
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ingo Oeser <nightlyone@googlemail.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 20ed60311848ca40e51cb430fa602dd83a9c726f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/622595
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-10-30 17:03:35 +00:00
Michael Anthony Knyszek
cfe0ae0b70 [release-branch.go1.23] runtime: uphold goroutine profile invariants in coroswitch
Goroutine profiles require checking in with the profiler before any
goroutine starts running. coroswitch is a place where a goroutine may
start running, but where we do not check in with the profiler, which
leads to crashes. Fix this by checking in with the profiler the same way
execute does.

For #69998.
Fixes #70001.

Change-Id: Idef6dd31b70a73dd1c967b56c307c7a46a26ba73
Reviewed-on: https://go-review.googlesource.com/c/go/+/622016
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 2a98a1849f059ffa94ab23a1ab7d8fa0fd0b48dd)
Reviewed-on: https://go-review.googlesource.com/c/go/+/622375
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-10-30 16:49:25 +00:00
Michael Anthony Knyszek
58babf6e0b [release-branch.go1.23] runtime,time: use atomic.Int32 for isSending
This change switches isSending to be an atomic.Int32 instead of an
atomic.Uint8. The Int32 version is managed as a counter, which is
something that we couldn't do with Uint8 without adding a new intrinsic
which may not be available on all architectures.

That is, instead of only being able to support 8 concurrent timer
firings on the same timer because we only have 8 independent bits to set
for each concurrent timer firing, we can now have 2^31-1 concurrent
timer firings before running into any issues. Like the fact that each
bit-set was matched with a clear, here we match increments with
decrements to indicate that we're in the "sending on a channel" critical
section in the timer code, so we can report the correct result back on
Stop or Reset.

We choose an Int32 instead of a Uint32 because it's easier to check for
obviously bad values (negative values are always bad) and 2^31-1
concurrent timer firings should be enough for anyone.

Previously, we avoided anything bigger than a Uint8 because we could
pack it into some padding in the runtime.timer struct. But it turns out
that the type that actually matters, runtime.timeTimer, is exactly 96
bytes in size. This means its in the next size class up in the 112 byte
size class because of an allocation header. We thus have some free space
to work with. This change increases the size of this struct from 96
bytes to 104 bytes.

(I'm not sure if runtime.timer is often allocated directly, but if it
is, we get lucky in the same way too. It's exactly 80 bytes in size,
which means its in the 96-byte size class, leaving us with some space to
work with.)

Fixes #69978
For #69969.
Related to #69880 and #69312 and #69882.

Change-Id: I9fd59cb6a69365c62971d1f225490a65c58f3e77
Cq-Include-Trybots: luci.golang.try:go1.23-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/621616
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 6a49f81edc7aa8aa12e26a1a0ed8819a3e5c7b5e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/621856
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-10-25 20:34:25 +00:00
Ian Lance Taylor
8d79bf799b [release-branch.go1.23] runtime: don't frob isSending for tickers
The Ticker Stop and Reset methods don't report a value,
so we don't need to track whether they are interrupting a send.

This includes a test that used to fail about 2% of the time on
my laptop when run under x/tools/cmd/stress.

For #69880
Fixes #69882

Change-Id: Ic6d14b344594149dd3c24b37bbe4e42e83f9a9ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/620136
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit 48849e0866f64a40d04a9151e44e5a73acdfc17b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/620137
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-10-23 22:07:00 +00:00
Felix Geisendörfer
35c010ad6d [release-branch.go1.23] runtime: fix GoroutineProfile stacks not getting null terminated
Fix a regression introduced in CL 572396 causing goroutine stacks not
getting null terminated.

This bug impacts callers that reuse the []StackRecord slice for multiple
calls to GoroutineProfile. See https://github.com/felixge/fgprof/issues/33
for an example of the problem.

Add a test case to prevent similar regressions in the future. Use null
padding instead of null termination to be consistent with other profile
types and because it's less code to implement. Also fix the
ThreadCreateProfile code path.

Fixes #69258

Change-Id: I0b9414f6c694c304bc03a5682586f619e9bf0588
Reviewed-on: https://go-review.googlesource.com/c/go/+/609815
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit 49e542aa85b7c2d9f6cf50de00843b455bc1e635)
Reviewed-on: https://go-review.googlesource.com/c/go/+/621277
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-10-21 18:37:43 +00:00
Michael Pratt
6495ce0495 [release-branch.go1.23] syscall: use SYS_EXIT_GROUP in CLONE_PIDFD feature check child
Inside Google we have seen issues with QEMU user mode failing to wake a
parent waitid when this child exits with SYS_EXIT. This bug appears to
not affect SYS_EXIT_GROUP.

It is currently unclear if this is a general QEMU or specific to
Google's configuration, but SYS_EXIT and SYS_EXIT_GROUP are semantically
equivalent here, so we can use the latter here in case this is a general
QEMU bug.

For #68976.
For #69259.

Change-Id: I34e51088c9a6b7493a060e2a719a3cc4a3d54aa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/617417
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 47a99359206f0dd41228deda0aa31f1e769cc156)
Reviewed-on: https://go-review.googlesource.com/c/go/+/617716
2024-10-11 17:02:01 +00:00
Michael Pratt
7fc8312673 [release-branch.go1.23] os: add clone(CLONE_PIDFD) check to pidfd feature check
clone(CLONE_PIDFD) was added in Linux 5.2 and pidfd_open was added in
Linux 5.3. Thus our feature check for pidfd_open should be sufficient to
ensure that clone(CLONE_PIDFD) works.

Unfortuantely, some alternative Linux implementations may not follow
this strict ordering. For example, QEMU 7.2 (Dec 2022) added pidfd_open,
but clone(CLONE_PIDFD) was only added in QEMU 8.0 (Apr 2023).

Debian bookworm provides QEMU 7.2 by default.

For #68976.
Fixes #69259.

Change-Id: Ie3f3dc51f0cd76944871bf98690abf59f68fd7bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/592078
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
(cherry picked from commit 7a5fc9b34deb8d9fe22c9d060a5839827344fcc2)
Reviewed-on: https://go-review.googlesource.com/c/go/+/612218
2024-10-11 17:01:46 +00:00
Ian Lance Taylor
cc16cdf48f [release-branch.go1.23] runtime: clear isSending bit earlier
I've done some more testing of the new isSending field.
I'm not able to get more than 2 bits set. That said,
with this change it's significantly less likely to have even
2 bits set. The idea here is to clear the bit before possibly
locking the channel we are sending the value on, thus avoiding
some delay and some serialization.

For #69312
For #69333

Change-Id: I8b5f167f162bbcbcbf7ea47305967f349b62b0f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/617596
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-10-08 22:28:41 +00:00
cions
9563300f6e [release-branch.go1.23] os: ignore SIGSYS in checkPidfd
In Android version 11 and earlier, pidfd-related system calls
are not allowed by the seccomp policy, which causes crashes due
to SIGSYS signals.

For #69065
Fixes #69640

Change-Id: Ib29631639a5cf221ac11b4d82390cb79436b8657
GitHub-Last-Rev: aad6b3b32c81795f86bc4a9e81aad94899daf520
GitHub-Pull-Request: golang/go#69543
Reviewed-on: https://go-review.googlesource.com/c/go/+/614277
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
(cherry picked from commit a3a05ed04cb53c53bdacded2d16f0f3e5facdbb0)
Reviewed-on: https://go-review.googlesource.com/c/go/+/616077
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-10-02 16:52:24 +00:00
Shulhan
f8080edefd [release-branch.go1.23] runtime: fix TestGdbAutotmpTypes on gdb version 15
On Arch Linux with gdb version 15.1, the test for TestGdbAutotmpTypes print
the following output,

----
~/src/go/src/runtime
$ go test -run=TestGdbAutotmpTypes -v
=== RUN   TestGdbAutotmpTypes
=== PAUSE TestGdbAutotmpTypes
=== CONT  TestGdbAutotmpTypes
    runtime-gdb_test.go:78: gdb version 15.1
    runtime-gdb_test.go:570: gdb output:
        Loading Go Runtime support.
        Target 'exec' cannot support this command.
        Breakpoint 1 at 0x46e416: file /tmp/TestGdbAutotmpTypes750485513/001/main.go, line 8.

        This GDB supports auto-downloading debuginfo from the following URLs:
          <https://debuginfod.archlinux.org>
        Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
        Debuginfod has been disabled.
        To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
        [New LWP 355373]
        [New LWP 355374]
        [New LWP 355375]
        [New LWP 355376]

        Thread 1 "a.exe" hit Breakpoint 1, main.main () at /tmp/TestGdbAutotmpTypes750485513/001/main.go:8
        8       func main() {
        9               var iface interface{} = map[string]astruct{}
        All types matching regular expression "astruct":

        File runtime:
                []main.astruct
                bucket<string,main.astruct>
                hash<string,main.astruct>
                main.astruct
                typedef hash<string,main.astruct> * map[string]main.astruct;
                typedef noalg.[8]main.astruct noalg.[8]main.astruct;
                noalg.map.bucket[string]main.astruct
    runtime-gdb_test.go:587: could not find []main.astruct; in 'info typrs astruct' output
!!! FAIL
exit status 1
FAIL    runtime 0.273s
$
----

In the back trace for "File runtime", each output lines does not end with
";" anymore, while in test we check the string with it.

While at it, print the expected string with "%q" instead of "%s" for
better error message.

For #67089
Fixes #69746

Change-Id: If6019ee68c0d8e495c920f98568741462c7d0fd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/598135
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Meng Zhuo <mengzhuo1203@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
(cherry picked from commit ff695ca2e3ea37dcb688d470e86ed64849c61f2e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/617455
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-10-02 16:23:59 +00:00
Gopher Robot
ed07b321ae [release-branch.go1.23] go1.23.2
Change-Id: I904d2e951796dd4142d6e9de4a55af07852bca51
Reviewed-on: https://go-review.googlesource.com/c/go/+/617019
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2024-10-01 17:24:29 +00:00
Ian Lance Taylor
3b2e846e11 [release-branch.go1.23] runtime: if stop/reset races with running timer, return correct result
The timer code is careful to ensure that if stop/reset is called
while a timer is being run, we cancel the run. However, the code
failed to ensure that in that case stop/reset returned true,
meaning that the timer had been stopped. In the racing case
stop/reset could see that t.when had been set to zero,
and return false, even though the timer had not and never would fire.

Fix this by tracking whether a timer run is in progress,
and using that to reliably detect that the run was cancelled,
meaning that stop/reset should return true.

For #69312
Fixes #69333

Change-Id: I78e870063eb96650638f12c056e32c931417c84a
Reviewed-on: https://go-review.googlesource.com/c/go/+/611496
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 2ebaff4890596ed6064e2dcbbe5e68bc93bed882)
Reviewed-on: https://go-review.googlesource.com/c/go/+/616096
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-09-27 18:23:47 +00:00
Cuong Manh Le
fbddfae62f [release-branch.go1.23] cmd/compile: fix wrong esacpe analysis for rangefunc
CL 584596 "-range<N>" suffix to the name of closure generated for a
rangefunc loop body. However, this breaks the condition that escape
analysis uses for checking whether a closure contains within function,
which is "F.funcN" for outer function "F" and closure "funcN".

Fixing this by adding new "-rangeN" to the condition.

Updates #69434
Fixes #69511

Change-Id: I411de8f63b69a6514a9e9504d49d62e00ce4115d
Reviewed-on: https://go-review.googlesource.com/c/go/+/614096
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/614195
2024-09-25 16:22:40 +00:00
Ian Lance Taylor
c8c6f9abfb [release-branch.go1.23] syscall: on exec failure, close pidfd
For #69284
Fixes #69402

Change-Id: I6350209302778ba5e44fa03d0b9e680d2b4ec192
Reviewed-on: https://go-review.googlesource.com/c/go/+/611495
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: roger peppe <rogpeppe@gmail.com>
Reviewed-by: Tim King <taking@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
(cherry picked from commit 8926ca9c5ec3ea0b51e413e87f737aeb1422ea48)
Reviewed-on: https://go-review.googlesource.com/c/go/+/613616
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2024-09-18 19:41:05 +00:00
Michael Anthony Knyszek
a74951c5af [release-branch.go1.23] unique: don't retain uncloned input as key
Currently the unique package tries to clone strings that get stored in
its internal map to avoid retaining large strings.

However, this falls over entirely due to the fact that the original
string is *still* stored in the map as a key. Whoops. Fix this by
storing the cloned value in the map instead.

This change also adds a test which fails without this change.

For #69370.
Fixes #69383.

Change-Id: I1a6bb68ed79b869ea12ab6be061a5ae4b4377ddb
Reviewed-on: https://go-review.googlesource.com/c/go/+/610738
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 21ac23a96f204dfb558a8d3071380c1d105a93ba)
Reviewed-on: https://go-review.googlesource.com/c/go/+/612295
Auto-Submit: Tim King <taking@google.com>
2024-09-11 18:11:20 +00:00
Wei Fu
e6598e7baa [release-branch.go1.23] os: dup pidfd if caller sets PidFD manually
For #68984.
Fixes #69119.

Change-Id: I16d25777cb38a337cd4204a8147eaf866c3df9e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/607695
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 239666cd7343d46c40a5b929c8bec8b532dbe83f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/611415
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@golang.org>
2024-09-06 16:06:09 +00:00
Michael Anthony Knyszek
82575f76b8 [release-branch.go1.23] internal/weak: shade pointer in weak-to-strong conversion
There's a bug in the weak-to-strong conversion in that creating the
*only* strong pointer to some weakly-held object during the mark phase
may result in that object not being properly marked.

The exact mechanism for this is that the new strong pointer will always
point to a white object (because it was only weakly referenced up until
this point) and it can then be stored in a blackened stack, hiding it
from the garbage collector.

This "hide a white pointer in the stack" problem is pretty much exactly
what the Yuasa part of the hybrid write barrier is trying to catch, so
we need to do the same thing the write barrier would do: shade the
pointer.

Added a test and confirmed that it fails with high probability if the
pointer shading is missing.

For #69210.
Fixes #69240.

Change-Id: Iaae64ae95ea7e975c2f2c3d4d1960e74e1bd1c3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/610396
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
(cherry picked from commit 79fd633632cdbaf9ca38f7559e5abb5c07fbbd9d)
Reviewed-on: https://go-review.googlesource.com/c/go/+/610696
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2024-09-06 14:34:16 +00:00
Keith Randall
a886959aa2 [release-branch.go1.23] runtime: size maps.Clone destination bucket array safely
In rare situations, like during same-sized grows, the source map for
maps.Clone may be overloaded (has more than 6.5 entries per
bucket). This causes the runtime to allocate a larger bucket array for
the destination map than for the source map. The maps.Clone code
walks off the end of the source array if it is smaller than the
destination array.

This is a pretty simple fix, ensuring that the destination bucket
array is never longer than the source bucket array. Maybe a better fix
is to make the Clone code handle shorter source arrays correctly, but
this fix is deliberately simple to reduce the risk of backporting this
fix.

Fixes #69156

Change-Id: I824c93d1db690999f25a3c43b2816fc28ace7509
Reviewed-on: https://go-review.googlesource.com/c/go/+/610377
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@google.com>
2024-09-06 14:32:57 +00:00
Ian Lance Taylor
80ff7cd35a [release-branch.go1.23] cmd/cgo: correct padding required by alignment
If the aligned offset isn't sufficient for the field offset,
we were padding based on the aligned offset. We need to pad
based on the original offset instead.

Also set the Go alignment correctly for int128. We were defaulting
to the maximum alignment, but since we translate int128 into an
array of uint8 the correct Go alignment is 1.

For #69086
Fixes #69219

Change-Id: I23ce583335c81beac2ac51f7f9336ac97ccebf09
Reviewed-on: https://go-review.googlesource.com/c/go/+/608815
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit c2098929056481d0dc09f5f42b8959f4db8878f2)
Reviewed-on: https://go-review.googlesource.com/c/go/+/611296
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
2024-09-05 21:57:44 +00:00
Gopher Robot
69234ded30 [release-branch.go1.23] go1.23.1
Change-Id: I1f2dab5560d3214c8934074a53f7750d8d431936
Reviewed-on: https://go-review.googlesource.com/c/go/+/611196
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Commit-Queue: Gopher Robot <gobot@golang.org>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
2024-09-05 15:20:23 +00:00
Roland Shoemaker
032ac075c2 [release-branch.go1.23] go/build/constraint: add parsing limits
Limit the size of build constraints that we will parse. This prevents a
number of stack exhaustions that can be hit when parsing overly complex
constraints. The imposed limits are unlikely to ever be hit in real
world usage.

Updates #69141
Fixes #69149
Fixes CVE-2024-34158

Change-Id: I38b614bf04caa36eefc6a4350d848588c4cef3c4
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1540
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Russ Cox <rsc@google.com>
(cherry picked from commit 0c74dc9e0da0cf1e12494b514d822b5bebbc9f04)
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1562
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/611177
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2024-09-05 14:55:17 +00:00
Roland Shoemaker
fa8ff1a46d [release-branch.go1.23] encoding/gob: cover missed cases when checking ignore depth
This change makes sure that we are properly checking the ignored field
recursion depth in decIgnoreOpFor consistently. This prevents stack
exhaustion when attempting to decode a message that contains an
extremely deeply nested struct which is ignored.

Thanks to Md Sakib Anwar of The Ohio State University (anwar.40@osu.edu)
for reporting this issue.

Updates #69139
Fixes #69145
Fixes CVE-2024-34156

Change-Id: Iacce06be95a5892b3064f1c40fcba2e2567862d6
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1440
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
(cherry picked from commit 9f2ea73c5f2a7056b7da5d579a485a7216f4b20a)
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1581
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/611176
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
2024-09-05 14:55:14 +00:00
Roland Shoemaker
53487e5477 [release-branch.go1.23] go/parser: track depth in nested element lists
Prevents stack exhaustion with extremely deeply nested literal values,
i.e. field values in structs.

Updates #69138
Fixes #69143
Fixes CVE-2024-34155

Change-Id: I2e8e33b44105cc169d7ed1ae83fb56df0c10f1ee
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1520
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Russ Cox <rsc@google.com>
(cherry picked from commit eb1b038c0d01761694e7a735ef87ac9164c6568e)
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1560
Commit-Queue: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/611175
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-09-05 14:55:11 +00:00
Michael Pratt
3d1f1f27cf [release-branch.go1.23] cmd: vendor golang.org/x/telemetry@internal-branch.go1.23-vendor
Update x/telemetry to fix #68976 and #68946.

Commands run:
  go get golang.org/x/telemetry@internal-branch.go1.23-vendor
  go mod tidy
  go mod vendor

Fixes #68994.
Fixes #68995.

Change-Id: I63b892ad4c313aa92f21fbd4f519a0b19d725849
Reviewed-on: https://go-review.googlesource.com/c/go/+/609355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
2024-08-29 00:09:27 +00:00
Nic Klaassen
6de5a7180c [release-branch.go1.23] database/sql: fix panic with concurrent Conn and Close
The current implementation has a panic when the database is closed
concurrently with a new connection attempt.

connRequestSet.CloseAndRemoveAll sets connRequestSet.s to a nil slice.
If this happens between calls to connRequestSet.Add and
connRequestSet.Delete, there is a panic when trying to write to the nil
slice. This is sequence is likely to occur in DB.conn, where the mutex
is released between calls to db.connRequests.Add and
db.connRequests.Delete

This change updates connRequestSet.CloseAndRemoveAll to set the curIdx
to -1 for all pending requests before setting its internal slice to nil.
CloseAndRemoveAll already iterates the full slice to close all the request
channels. It seems appropriate to set curIdx to -1 before deleting the
slice for 3 reasons:
1. connRequestSet.deleteIndex also sets curIdx to -1
2. curIdx will not be relevant to anything after the slice is set to nil
3. connRequestSet.Delete already checks for negative indices

For #68949
Fixes #69041

Change-Id: I6b7ebc5a71b67322908271d13865fa12f2469b87
GitHub-Last-Rev: 7d2669155b24043dd9d276f915689511572f2e49
GitHub-Pull-Request: golang/go#68953
Reviewed-on: https://go-review.googlesource.com/c/go/+/607238
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Commit-Queue: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
(cherry picked from commit 08707d66c350927560faa11b0c195d37d281ab89)
Reviewed-on: https://go-review.googlesource.com/c/go/+/609255
2024-08-28 18:37:00 +00:00
Tim King
9625a7faae [release-branch.go1.23] go/types, types2: unalias tilde terms in underIs
Unalias the ~T terms during underIs. Before, if T was an alias
of U, it may pass T to the iteration function. The iterator
function expects an underlying type, under(U), to be passed.
This caused several bugs where underIs is used without
eventually taking the underlying type.

Fixes #68905

Change-Id: Ie8691d8dddaea00e1dcba94d17c0f1b021fc49a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/606075
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
(cherry picked from commit 1a90dcdaaf46d9dd0ee61781dcb9b6e05b80d926)
Reviewed-on: https://go-review.googlesource.com/c/go/+/607635
2024-08-28 18:10:23 +00:00
Michael Anthony Knyszek
9c939a1e60 [release-branch.go1.23] runtime: store bp on cgocallback as unsafe.Pointer
As of CL 580255, the runtime tracks the frame pointer (or base pointer,
bp) when entering syscalls, so that we can use fpTracebackPCs on
goroutines that are sitting in syscalls. That CL mostly got things
right, but missed one very subtle detail.

When calling from Go->C->Go, the goroutine stack performing the calls
when returning to Go is free to move around in memory due to growth,
shrinking, etc. But upon returning back to C, it needs to restore
gp.syscall*, including gp.syscallsp and gp.syscallbp. The way syscallsp
currently gets updated is automagically: it's stored as an
unsafe.Pointer on the stack so that it shows up in a stack map. If the
stack ever moves, it'll get updated correctly. But gp.syscallbp isn't
saved to the stack as an unsafe.Pointer, but rather as a uintptr, so it
never gets updated! As a result, in rare circumstances, fpTracebackPCs
can correctly try to use gp.syscallbp as the starting point for the
traceback, but the value is stale.

This change fixes the problem by just storing gp.syscallbp to the stack
on cgocallback as an unsafe.Pointer, like gp.syscallsp. It also adds a
comment documenting this subtlety; the lack of explanation for the
unsafe.Pointer type on syscallsp meant this detail was missed -- let's
not miss it again in the future.

Now, we have a fix, what about a test? Unfortunately, testing this is
going to be incredibly annoying because the circumstances under which
gp.syscallbp are actually used for traceback are non-deterministic and
hard to arrange, especially from within testprogcgo where we don't have
export_test.go and can't reach into the runtime.

So, instead, add a gp.syscallbp check to reentersyscall and
entersyscallblock that mirrors the gp.syscallbp consistency check. This
probably causes some miniscule slowdown to the syscall path, but it'll
catch the issue without having to actually perform a traceback.

For #69085.
Fixes #69087.

Change-Id: Iaf771758f1666024b854f5fbe2b2c63cbe35b201
Reviewed-on: https://go-review.googlesource.com/c/go/+/608775
Reviewed-by: Nick Ripley <nick.ripley@datadoghq.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
(cherry picked from commit 54fe0fd43fcf8609666c16ae6d15ed92873b1564)
Reviewed-on: https://go-review.googlesource.com/c/go/+/608835
2024-08-28 17:42:20 +00:00
Michael Matloob
7afe17bbdb [release-branch.go1.23] go/types, types2: use max(fileVersion, go1.21) if fileVersion present
Change the rules for how //go:build "file versions" are applied: instead
of considering whether a file version is an upgrade or downgrade from
the -lang version, always use max(fileVersion, go1.21). This prevents
file versions from downgrading the version below go1.21.  Before Go 1.21
the //go:build version did not have the meaning of setting the file's
langage version.

This fixes an issue that was appearing in GOPATH builds: Go 1.23.0
started providing -lang versions to the compiler in GOPATH mode (among
other places) which it wasn't doing before, and it set -lang to the
toolchain version (1.23). Because the -lang version was greater than
go1.21, language version used to compile the file would be set to the
//go:build file version. //go:build file versions below 1.21 could cause
files that could previously build to stop building.

For example, take a Go file with a //go:build line specifying go1.10.
If that file used a 1.18 feature, that use would compile fine with a Go
1.22 toolchain. But it would produce an error when compiling with the
1.23.0 toolchain because it set the language version to 1.10 and
disallowed the 1.18 feature. This breaks backwards compatibility: when
the build tag was added, it did not have the meaning of restricting the
language version.

For #68658
Fixes #69094

Change-Id: I6cedda81a55bcccffaa3501eef9e2be6541b6ece
Reviewed-on: https://go-review.googlesource.com/c/go/+/607955
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
(cherry picked from commit aeac0b6cbfb42bc9c9301913a191bb09454d316a)
Reviewed-on: https://go-review.googlesource.com/c/go/+/608935
2024-08-28 15:43:17 +00:00
Paul E. Murphy
8002845759 [release-branch.go1.23] runtime: on AIX, fix call to _cgo_sys_thread_create in _rt0_ppc64_aix_lib
The AIX ABI requires allocating parameter save space when calling
a function, even if the arguments are passed via registers.

gcc sometimes uses this space. In the case of the cgo c-archive
tests, it clobbered the storage space of argc/argv which prevented
the test program from running the expected test.

Fixes #68973

Change-Id: I8a267b463b1abb2b37ac85231f6c328f406b7515
Reviewed-on: https://go-review.googlesource.com/c/go/+/606895
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/607195
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-08-21 20:26:47 +00:00
Robert Griesemer
9166d2feec [release-branch.go1.23] go/types, types2: Named.cleanup must also handle *Alias types
Named.cleanup is called at the end of type-checking to ensure that
a named type is fully set up; specifically that it's underlying
field is not (still) a Named type. Now it can also be an *Alias
type. Add this case to the respective type switch.

Fixes #68894.

Change-Id: I29bc0024ac9d8b0152a3d97c82dd28d09d5dbd66
Reviewed-on: https://go-review.googlesource.com/c/go/+/605977
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/606656
2024-08-21 17:42:47 +00:00
Michael Anthony Knyszek
76346b3543 [release-branch.go1.23] unique: use TypeFor instead of TypeOf to get type in Make
Currently the first thing Make does it get the abi.Type of its argument,
and uses abi.TypeOf to do it. However, this has a problem for interface
types, since the type of the value stored in the interface value will
bleed through. This is a classic reflection mistake.

Fix this by implementing and using a generic TypeFor which matches
reflect.TypeFor. This gets the type of the type parameter, which is far
less ambiguous and error-prone.

For #68990.
Fixes #68992.

Change-Id: Idd8d9a1095ef017e9cd7c7779314f7d4034f01a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/607355
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 755c18ecdfe64df060be91fb669ca1a68527830b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/607435
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-08-21 17:42:03 +00:00
Andy Pan
3c9340557c [release-branch.go1.23] os: use O_EXCL instead of O_TRUNC in CopyFS to disallow rewriting existing files does not exist
On Linux, a call to creat() is equivalent to calling open() with flags
equal to O_CREAT|O_WRONLY|O_TRUNC, which applies to other platforms
as well in a similar manner. Thus, to force CopyFS's behavior to
comply with the function comment, we need to replace O_TRUNC with O_EXCL.

Fixes #68907

Change-Id: I3e2ab153609d3c8cf20ce5969d6f3ef593833cd1
Reviewed-on: https://go-review.googlesource.com/c/go/+/606095
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
(cherry picked from commit aa5d672a00f5bf64865d0e821623ed29bc416405)
Reviewed-on: https://go-review.googlesource.com/c/go/+/606415
2024-08-21 17:05:16 +00:00
Kir Kolyshkin
dbecb416d1 [release-branch.go1.23] os: fix Chtimes test flakes
It appears that some builders (notably, linux-arm) have some additional
security software installed, which apparently reads the files created by
tests. As a result, test file atime is changed, making the test fail
like these:

=== RUN   TestChtimesOmit
    ...
    os_test.go:1475: atime mismatch, got: "2024-07-30 18:42:03.450932494 +0000 UTC", want: "2024-07-30 18:42:02.450932494 +0000 UTC"

=== RUN   TestChtimes
    ...
    os_test.go:1539: AccessTime didn't go backwards; was=2024-07-31 20:45:53.390326147 +0000 UTC, after=2024-07-31 20:45:53.394326118 +0000 UTC

According to inode(7), atime is changed when more than 0 bytes are read
from the file. So, one possible solution to these flakes is to make the
test files empty, so no one can read more than 0 bytes from them.

For #68687
For #68663
Fixes #68812

Change-Id: Ib9234567883ef7b16ff8811e3360cd26c2d6bdab
Reviewed-on: https://go-review.googlesource.com/c/go/+/604315
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
(cherry picked from commit 84266e1469cfa6fa8e1b41518528a96950db7562)
Reviewed-on: https://go-review.googlesource.com/c/go/+/604196
2024-08-14 17:44:07 +00:00
Gopher Robot
6885bad7dd [release-branch.go1.23] go1.23.0
Change-Id: I2b0514157b85ca61f9f6b8931df6ac874598a045
Reviewed-on: https://go-review.googlesource.com/c/go/+/605215
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Gopher Robot <gobot@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-08-13 16:16:34 +00:00
Carlos Amedee
ec7d6094e6 [release-branch.go1.23] revert "go/types, types2: only use fileVersion if 1.21 or greater"
This reverts commit CL 604935.

Reason for revert: The team has decided that this change will be added to a point release.

Change-Id: I1c1032b881c3a98312a4753b9767cb7c8eed9e09
Reviewed-on: https://go-review.googlesource.com/c/go/+/605096
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2024-08-13 14:45:01 +00:00
Michael Matloob
63b0f805cd [release-branch.go1.23] go/types, types2: only use fileVersion if 1.21 or greater
Only honor //go:build language version downgrades if the version
specified is 1.21 or greater. Before 1.21 the version in //go:build
lines didn't have the meaning of setting the file's language version.

This fixes an issue that was appearing in GOPATH builds: Go 1.23 started
providing -lang versions to the compiler in GOPATH mode (among other
places) which it wasn't doing before.

For example, take a go file with a //go:build line specifying go1.10.
If that file used a 1.18 feature, that use would compile fine with a Go
1.22 toolchain. But, before this change, it would produce an error when
compiling with the 1.23 toolchain because it set the language version to
1.10 and disallowed the 1.18 feature. This breaks backwards
compatibility: when the build tag was added, it did not have the meaning
of restricting the language version.

Fixes #68658

Change-Id: I4ac2b45a981cd019183d52ba324ba8f0fed93a8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/603895
Reviewed-by: Robert Griesemer <gri@google.com>
Commit-Queue: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Matloob <matloob@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/604935
2024-08-12 20:21:53 +00:00
Filippo Valsorda
7adb012205 [release-branch.go1.23] crypto/tls: fix testHandshake close flakes
The flakes were introduced by me in CL 586655. It's unclear why only
FreeBSD seems affected, maybe other TCP stacks handle sending on a
half-closed connection differently, or aren't as quick to propagate the
RST over localhost.

Updates #68155

Change-Id: I32a1b474a7d6531dbab93910c23568b867629e8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/602635
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2024-08-02 19:55:18 +00:00
Cuong Manh Le
c9940fe2a9 [release-branch.go1.23] types2, go/types: fix instantiation of named type with generic alias
The typechecker is assuming that alias instances cannot be reached from
a named type. However, when type parameters on aliases are permited, it
can happen.

This CL changes the typechecker to propagate the correct named instance
is being expanded.

Updates #46477
Fixes #68580

Change-Id: Id0879021f4640c0fefe277701d5096c649413811
Reviewed-on: https://go-review.googlesource.com/c/go/+/601115
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/601116
2024-07-26 16:35:29 +00:00
Robert Griesemer
3509415eca [release-branch.go1.23] cmd/compile: more informative panic when importing generic type alias
When GOEXPERIMENT=aliastypeparams is set, type aliases may have type
parameters. The compiler export data doesn't export that type parameter
information yet, which leads to an index-out-of-bounds panic when a
client package imports a package with a general type alias and then
refers to one of the missing type parameters.

This CL detects this specific case and panics with a more informative
panic message explaining the shortcoming. The change is only in effect
if the respective GOEXPERIMENT is enabled.

Manually tested. No test addded since this is just a temporary fix
(Go 1.24 will have a complete implementation), and because the existing
testing framework doesn't easily support testing that a compilation
panics.

Together with @taking and input from @rfindley.

For #68526.

Change-Id: I24737b153a7e2f9b705cd29a5b70b2b9e808dffc
Reviewed-on: https://go-review.googlesource.com/c/go/+/601035
Reviewed-by: Tim King <taking@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Griesemer <gri@google.com>
2024-07-26 03:55:54 +00:00
Damien Neil
559c77592f [release-branch.go1.23] os: document CopyFS behavior for symlinks in destination
Also clarify the permissions of created files,
and note that CopyFS will not overwrite files.

Update a few places in documentation to use 0oXXX for octal consts.

For #62484

Change-Id: I208ed2bde250304bc7fac2b93963ba57037e791e
Reviewed-on: https://go-review.googlesource.com/c/go/+/600775
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
(cherry picked from commit 910e6b5fae7cbf84e4a3fcfa6739e20239080bcd)
Reviewed-on: https://go-review.googlesource.com/c/go/+/600815
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-07-24 18:55:12 +00:00
Dmitri Shuralyov
f5e4e45ef7 [release-branch.go1.23] all: merge master (5e8a731) into release-branch.go1.23
Merge List:

+ 2024-07-22 5e8a731665 README: fix CC BY license name
+ 2024-07-22 a799fa5189 slices: explicitly document nil and empty slice equality
+ 2024-07-22 f0de94ff12 cmd/compile: don't inline runtime functions in -d=checkptr build
+ 2024-07-19 3959d54c0b runtime: mark lockWithRankMayAcquire as nosplit
+ 2024-07-17 70491a8111 maps: document handling of non-reflexive keys
+ 2024-07-17 7321aa91c6 cmd: vendor golang.org/x/telemetry@0b706e1
+ 2024-07-17 420037b16d os: don't try to signal PID -1 on Unix
+ 2024-07-17 87abb4afb6 runtime: avoid multiple records with identical stacks from MutexProfile
+ 2024-07-17 8c88f0c736 cmd/cgo/internal/testcarchive: remove 1-minute timeout
+ 2024-07-17 fc51e5023e math/big: fix comment typo in natdiv.go
+ 2024-07-17 90c6558b6a internal/bytealg: extend memchr result correctly on wasm
+ 2024-07-16 355711821e cmd/internal/cov: close counter data files eagerly
+ 2024-07-16 f2bcab5fb3 regexp: more cross-references in docstrings
+ 2024-07-16 9915b87059 bytes: more cross-references in docstrings
+ 2024-07-16 97ccc224f1 math/big: use lists in docstrings
+ 2024-07-16 66e940b6f8 math/big: more cross-references in docstrings
+ 2024-07-16 0dae393a26 encoding/binary: use list format in docstrings
+ 2024-07-16 451a284d80 strings,bytes,regexp: use lists in Split* docstrings
+ 2024-07-16 b4a92f56ff crypto/tls: add exclude tls flags to bogo_shim_test
+ 2024-07-16 3bfbfa821f encoding/json: document compact json output in Encoder.Encode
+ 2024-07-16 d5479e197d net: document ParseIP always returns IPv6 addresses
+ 2024-07-16 5c7f541386 archive/zip: document handling of duplicate names in Writer.Create
+ 2024-07-16 6b97448132 sort: add example for Find
+ 2024-07-16 8b48290895 cmd/compile: fix recursive generic interface instantiation
+ 2024-07-15 959b3fd426 flag: add FlagSet example

Change-Id: I0e7cac2f1cef42ccf3d392c57dad793ca3b08b11
2024-07-22 14:12:36 -04:00
Gopher Robot
30b6fd60a6 [release-branch.go1.23] go1.23rc2
Change-Id: I73a3f2e680a84aa698c6f64b1e924bb1b9a85a89
Reviewed-on: https://go-review.googlesource.com/c/go/+/598555
TryBot-Bypass: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Auto-Submit: Gopher Robot <gobot@golang.org>
2024-07-16 15:33:13 +00:00
Cherry Mui
7e4d6c2bcb [release-branch.go1.23] all: merge master (6948b4d) into release-branch.go1.23
Merge List:

+ 2024-07-15 6948b4df8c Revert "runtime: avoid multiple records with identical stacks from MutexProfile"

Change-Id: I94d1f445b0304eb4355ef6e91bf9f8585abfe0f8
2024-07-15 16:40:36 -04:00
Cherry Mui
8bd4ed6cbb [release-branch.go1.23] all: merge master (8f1ec59) into release-branch.go1.23
Merge List:

+ 2024-07-15 8f1ec59bdb strings: re-introduce noescape wrapper
+ 2024-07-15 5d36bc18d5 net/http: document io.Seeker requirement for fs.FS arguments
+ 2024-07-12 071b8d51c1 cmd: vendor golang.org/x/telemetry@268b4a8ec2d7
+ 2024-07-12 4e77872d16 go/types: fix assertion failure when range over int is not permitted
+ 2024-07-12 8bc32ab6b1 os: clean up after TestIssue60181
+ 2024-07-11 b31e9a63a4 unsafe: say "functions like syscall.Syscall", not only Syscall
+ 2024-07-11 a71bb570d0 all: make struct comments match struct names
+ 2024-07-11 611f18c4e9 strings: more cross-references in docstrings
+ 2024-07-11 08a6e080ca database/sql/driver: fix name in comment
+ 2024-07-11 dfaaa91f05 os: clarify that IsNotExist, IsExist, IsPermission and IsTimeout work with nil errors
+ 2024-07-10 5881d857c5 crypto/tls: add support for -expect-no-hrr to bogo_shim_test
+ 2024-07-10 b3040679ad math: remove riscv64 assembly implementations of rounding
+ 2024-07-10 70e453b436 context: handle nil values for valueCtx.String()
+ 2024-07-09 183a40db6d runtime: avoid multiple records with identical stacks from MutexProfile
+ 2024-07-09 e89e880eac crypto/tls: add support for -reject-alpn and -decline-alpn flags to bogo_shim_test
+ 2024-07-09 73186ba002 crypto/internal/cryptotest: add common tests for the hash.Hash interface
+ 2024-07-08 87ec2c959c testing: remove call to os.Exit in documentation for TestMain
+ 2024-07-08 6d89b38ed8 unsafe: clarify when String bytes can be modified
+ 2024-07-07 5565462a86 cmd/dist: remove iter,slices,maps test on GOEXPERIMENT=rangefunc
+ 2024-07-07 b43d6c57de io: add test for Pipe constructor allocations
+ 2024-07-07 d0146bd85b os/exec: only use cachedLookExtensions if Cmd.Path is unmodified
+ 2024-07-05 ad77cefeb2 cmd/compile: correct RewriteMultiValueCall fatal message
+ 2024-07-05 be152920b9 cmd/compile: fix ICE when compiling global a, b = f()
+ 2024-07-03 82c14346d8 cmd/link: don't disable memory profiling when pprof.WriteHeapProfile is used
+ 2024-07-03 7d19d508a9 cmd/cgo: read CGO_LDFLAGS environment variable
+ 2024-07-03 5f50b1e3bf cmd/compile: fix mis-compilation when switching over channels
+ 2024-07-03 71f9dbb1e4 cmd/compile: emit error message on mismatch import path
+ 2024-07-03 148755a27b cmd/link: document -checklinkname option
+ 2024-07-02 f12ac5be70 time: fix time zone parsing when format includes time zone seconds
+ 2024-07-02 09aeb6e33a os: add TODO about removing test exception
+ 2024-07-01 94982a0782 cmd/go/internal/workcmd: remove a potentially confusing statement
+ 2024-07-01 f71c00b616 cmd/link: align .reloc block starts by 32 bits for PE target
+ 2024-07-01 d3c93f2f00 cmd/go: update go clean help message
+ 2024-07-01 cdbf5f2f7e sync: refer to Locker interface methods in RWMutex.RLocker doc
+ 2024-07-01 c33144c47c runtime: fix nil pointer in TestGoroutineParallelism2 when offline
+ 2024-06-28 82c371a307 cmd/compile: drop internal range-over-func vars from DWARF output
+ 2024-06-28 773767def0 net/http: avoid appending an existing trailing slash to path again
+ 2024-06-28 7f90b960a9 cmd/compile: don't elide zero extension on top of signed values
+ 2024-06-27 ea537cca31 cmd/go/internal/help: add documentation for language version downgrading
+ 2024-06-27 b0927fdd49 slices: update docs for All, Backward, Values
+ 2024-06-26 5a18e79687 cmd/link: don't skip code sign even if dsymutil didn't generate a file
+ 2024-06-26 5f319b7507 cmd/link: don't let dsymutil delete our temp directory
+ 2024-06-26 a2e90be996 os: rewrite TestChtimesWithZeroTimes
+ 2024-06-25 90bcc552c0 crypto/tls: apply QUIC session event flag to QUICResumeSession events
+ 2024-06-25 b1fd047508 cmd/internal/obj/arm64: fix return with register
+ 2024-06-25 b3b4556c24 cmd/compile: update README to reflect dead code elimination changes
+ 2024-06-24 68315bc8ce cmd: run go mod tidy after CL 593684
+ 2024-06-24 f214a76075 cmd/vendor: vendor x/telemetry@38a4430
+ 2024-06-24 29b1a6765f net/http: document that Request.Clone does not deep copy Body
+ 2024-06-24 cf54a3d114 crypto/tls: replay test recordings without network
+ 2024-06-24 b98803e8e5 os: TestChtimes: separate hasNoatime
+ 2024-06-24 0def9d5c02 cmd/internal/obj/arm64: Enable arm64 assembler tests for cross-compiler builds
+ 2024-06-24 085cf0fcdc net/netip: add test that Compare and reflect.DeepEqual match
+ 2024-06-24 740043f516 net/netip: unexport fields of addrDetail
+ 2024-06-23 e8ee1dc4f9 cmd/link/internal/ld: handle "\r" in MinGW "--print-prog-name" output
+ 2024-06-22 44f1870666 cmd/link: handle dynamic import variables on Darwin in plugin mode
+ 2024-06-21 0af2148fdc cmd: vendor golang.org/x/telemetry@a740542
+ 2024-06-21 cb3b34349b doc/next: delete
+ 2024-06-21 d79c350916 cmd/internal: separate counter package from telemetry package
+ 2024-06-21 52ce25b44e cmd/vendor: pull in golang.org/x/telemetry@b4de734
+ 2024-06-21 fed2c11d67 iter: minor doc comment updates
+ 2024-06-21 d73a8a206a cmd/cgo: fail on v, err := C.fn when fn is a builtin function
+ 2024-06-21 1b4f1dc95d os: improve newFile, rm newDir
+ 2024-06-21 72e2220b50 encoding/json: clarify the map's key type for Unmarshal
+ 2024-06-21 e9a306e004 types2, go/types: correct NewTypeParam documentation
+ 2024-06-21 6fea409424 text/template/parse: fix handling of assignment/declaration in PipeNode.String
+ 2024-06-21 d67839f58a crypto/tls: add support for -expect-version to bogo_shim_test
+ 2024-06-21 201129414f sync/atomic: correct result names for Or methods
+ 2024-06-21 20b79fd577 time: provide non-default metric for asynctimerchan
+ 2024-06-20 9d33956503 internal/godebugs: fix old value for httpservecontentkeepheaders
+ 2024-06-20 477ad7dd51 cmd/compile: support generic alias type
+ 2024-06-18 4f77a83589 internal/syscall/unix: fix UTIME_OMIT for dragonfly

Change-Id: I3864b03b8c377e8fe82014eee96dc7b77aea64e2
2024-07-15 13:23:34 -04:00
Gopher Robot
7dff7439dc [release-branch.go1.23] go1.23rc1
Change-Id: Ied4bb63f49d13bd7d421cf9cb269220974641b89
Reviewed-on: https://go-review.googlesource.com/c/go/+/593897
Auto-Submit: David Chase <drchase@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
2024-06-21 16:32:19 +00:00
Damien Neil
62c3a6350b [release-branch.go1.23] internal/godebugs: fix old value for httpservecontentkeepheaders
The pre-Go 1.23 behavior is httpservecontentkeepheaders=1.

For #66343

Change-Id: If6f92853b38522f19a8908ff11ac49b12f3dc3e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/593775
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/593795
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-06-20 19:14:07 +00:00
Cuong Manh Le
eba9e08766 [release-branch.go1.23] cmd/compile: support generic alias type
Type parameters on aliases are now allowed after #46477 accepted.

Updates #46477
Fixes #68054

Change-Id: Ic2e3b6f960a898163f47666e3a6bfe43b8cc22e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/593715
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/593797
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-06-20 19:13:52 +00:00
Kir Kolyshkin
f3bdcda88a [release-branch.go1.23] internal/syscall/unix: fix UTIME_OMIT for dragonfly
CL 219638 added UTIME_OMIT values for various systems. The value for
DragonFly BSD appears to be incorrect.

The correct value is -2 (see references below), while -1 is used for
UTIME_NOW. As a result, timestamp is changed to the current time instead
of not touching. This should have been caught by the accompanying test
case, TestChtimesWithZeroTimes, but its failures are essentially skipped
on dragonfly (this is being fixed separately in a followup CL 591535).

Improve formatting while at it.

References:
 - https://github.com/DragonFlyBSD/DragonFlyBSD/blob/965b380e9609/sys/sys/stat.h#L284
 - https://go.googlesource.com/sys/+/refs/tags/v0.20.0/unix/zerrors_dragonfly_amd64.go#1421

Change-Id: I432360ca982c84b7cd70d0cf01d860af9ff985fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/589496
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/593796
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-06-20 19:13:34 +00:00
David Chase
362f22d2d2 [release-branch.go1.23] update codereview.cfg for release-branch.go1.23
Change-Id: Ib335bc903e2b8d5b7be6a158a6debe5db48e79a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/593535
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: David Chase <drchase@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-06-18 21:25:28 +00:00
132 changed files with 3117 additions and 763 deletions

2
VERSION Normal file
View File

@ -0,0 +1,2 @@
go1.23.5
time 2025-01-10T16:44:59Z

View File

@ -1 +1,2 @@
branch: master
branch: release-branch.go1.23
parent-branch: master

View File

@ -2579,6 +2579,11 @@ func (c *typeConv) loadType(dtype dwarf.Type, pos token.Pos, parent string) *Typ
if dt.BitSize > 0 {
fatalf("%s: unexpected: %d-bit int type - %s", lineno(pos), dt.BitSize, dtype)
}
if t.Align = t.Size; t.Align >= c.ptrSize {
t.Align = c.ptrSize
}
switch t.Size {
default:
fatalf("%s: unexpected: %d-byte int type - %s", lineno(pos), t.Size, dtype)
@ -2595,9 +2600,8 @@ func (c *typeConv) loadType(dtype dwarf.Type, pos token.Pos, parent string) *Typ
Len: c.intExpr(t.Size),
Elt: c.uint8,
}
}
if t.Align = t.Size; t.Align >= c.ptrSize {
t.Align = c.ptrSize
// t.Align is the alignment of the Go type.
t.Align = 1
}
case *dwarf.PtrType:
@ -2826,6 +2830,11 @@ func (c *typeConv) loadType(dtype dwarf.Type, pos token.Pos, parent string) *Typ
if dt.BitSize > 0 {
fatalf("%s: unexpected: %d-bit uint type - %s", lineno(pos), dt.BitSize, dtype)
}
if t.Align = t.Size; t.Align >= c.ptrSize {
t.Align = c.ptrSize
}
switch t.Size {
default:
fatalf("%s: unexpected: %d-byte uint type - %s", lineno(pos), t.Size, dtype)
@ -2842,9 +2851,8 @@ func (c *typeConv) loadType(dtype dwarf.Type, pos token.Pos, parent string) *Typ
Len: c.intExpr(t.Size),
Elt: c.uint8,
}
}
if t.Align = t.Size; t.Align >= c.ptrSize {
t.Align = c.ptrSize
// t.Align is the alignment of the Go type.
t.Align = 1
}
case *dwarf.VoidType:
@ -3110,10 +3118,11 @@ func (c *typeConv) Struct(dt *dwarf.StructType, pos token.Pos) (expr *ast.Struct
}
// Round off up to talign, assumed to be a power of 2.
origOff := off
off = (off + talign - 1) &^ (talign - 1)
if f.ByteOffset > off {
fld, sizes = c.pad(fld, sizes, f.ByteOffset-off)
fld, sizes = c.pad(fld, sizes, f.ByteOffset-origOff)
off = f.ByteOffset
}
if f.ByteOffset < off {

View File

@ -70,6 +70,7 @@ func Test31891(t *testing.T) { test31891(t) }
func Test42018(t *testing.T) { test42018(t) }
func Test45451(t *testing.T) { test45451(t) }
func Test49633(t *testing.T) { test49633(t) }
func Test69086(t *testing.T) { test69086(t) }
func TestAlign(t *testing.T) { testAlign(t) }
func TestAtol(t *testing.T) { testAtol(t) }
func TestBlocking(t *testing.T) { testBlocking(t) }

View File

@ -940,6 +940,19 @@ typedef struct {
} issue67517struct;
static void issue67517(issue67517struct* p) {}
// Issue 69086.
// GCC added the __int128 type in GCC 4.6, released in 2011.
typedef struct {
int a;
#ifdef __SIZEOF_INT128__
unsigned __int128 b;
#else
uint64_t b;
#endif
unsigned char c;
} issue69086struct;
static int issue690861(issue69086struct* p) { p->b = 1234; return p->c; }
static int issue690862(unsigned long ul1, unsigned long ul2, unsigned int u, issue69086struct s) { return (int)(s.b); }
*/
import "C"
@ -2349,3 +2362,24 @@ func issue67517() {
b: nil,
})
}
// Issue 69086.
func test69086(t *testing.T) {
var s C.issue69086struct
typ := reflect.TypeOf(s)
for i := 0; i < typ.NumField(); i++ {
f := typ.Field(i)
t.Logf("field %d: name %s size %d align %d offset %d", i, f.Name, f.Type.Size(), f.Type.Align(), f.Offset)
}
s.c = 1
got := C.issue690861(&s)
if got != 1 {
t.Errorf("field: got %d, want 1", got)
}
got = C.issue690862(1, 2, 3, s)
if got != 1234 {
t.Errorf("call: got %d, want 1234", got)
}
}

View File

@ -33,7 +33,7 @@ import (
"unicode"
)
var globalSkip = func(t *testing.T) {}
var globalSkip = func(t testing.TB) {}
// Program to run.
var bin []string
@ -59,12 +59,12 @@ func TestMain(m *testing.M) {
func testMain(m *testing.M) int {
if testing.Short() && os.Getenv("GO_BUILDER_NAME") == "" {
globalSkip = func(t *testing.T) { t.Skip("short mode and $GO_BUILDER_NAME not set") }
globalSkip = func(t testing.TB) { t.Skip("short mode and $GO_BUILDER_NAME not set") }
return m.Run()
}
if runtime.GOOS == "linux" {
if _, err := os.Stat("/etc/alpine-release"); err == nil {
globalSkip = func(t *testing.T) { t.Skip("skipping failing test on alpine - go.dev/issue/19938") }
globalSkip = func(t testing.TB) { t.Skip("skipping failing test on alpine - go.dev/issue/19938") }
return m.Run()
}
}
@ -1291,8 +1291,8 @@ func TestPreemption(t *testing.T) {
}
}
// Issue 59294. Test calling Go function from C after using some
// stack space.
// Issue 59294 and 68285. Test calling Go function from C after with
// various stack space.
func TestDeepStack(t *testing.T) {
globalSkip(t)
testenv.MustHaveGoBuild(t)
@ -1350,6 +1350,53 @@ func TestDeepStack(t *testing.T) {
}
}
func BenchmarkCgoCallbackMainThread(b *testing.B) {
// Benchmark for calling into Go fron C main thread.
// See issue #68587.
//
// It uses a subprocess, which is a C binary that calls
// Go on the main thread b.N times. There is some overhead
// for launching the subprocess. It is probably fine when
// b.N is large.
globalSkip(b)
testenv.MustHaveGoBuild(b)
testenv.MustHaveCGO(b)
testenv.MustHaveBuildMode(b, "c-archive")
if !testWork {
defer func() {
os.Remove("testp10" + exeSuffix)
os.Remove("libgo10.a")
os.Remove("libgo10.h")
}()
}
cmd := exec.Command("go", "build", "-buildmode=c-archive", "-o", "libgo10.a", "./libgo10")
out, err := cmd.CombinedOutput()
b.Logf("%v\n%s", cmd.Args, out)
if err != nil {
b.Fatal(err)
}
ccArgs := append(cc, "-o", "testp10"+exeSuffix, "main10.c", "libgo10.a")
out, err = exec.Command(ccArgs[0], ccArgs[1:]...).CombinedOutput()
b.Logf("%v\n%s", ccArgs, out)
if err != nil {
b.Fatal(err)
}
argv := cmdToRun("./testp10")
argv = append(argv, fmt.Sprint(b.N))
cmd = exec.Command(argv[0], argv[1:]...)
b.ResetTimer()
err = cmd.Run()
if err != nil {
b.Fatal(err)
}
}
func TestSharedObject(t *testing.T) {
// Test that we can put a Go c-archive into a C shared object.
globalSkip(t)

View File

@ -0,0 +1,12 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import "C"
//export GoF
func GoF() {}
func main() {}

View File

@ -6,9 +6,29 @@ package main
import "runtime"
// extern void callGoWithVariousStack(int);
import "C"
func main() {}
//export GoF
func GoF() { runtime.GC() }
func GoF(p int32) {
runtime.GC()
if p != 0 {
panic("panic")
}
}
//export callGoWithVariousStackAndGoFrame
func callGoWithVariousStackAndGoFrame(p int32) {
if p != 0 {
defer func() {
e := recover()
if e == nil {
panic("did not panic")
}
runtime.GC()
}()
}
C.callGoWithVariousStack(C.int(p));
}

View File

@ -0,0 +1,22 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
#include <stdio.h>
#include <stdlib.h>
#include "libgo10.h"
int main(int argc, char **argv) {
int n, i;
if (argc != 2) {
perror("wrong arg");
return 2;
}
n = atoi(argv[1]);
for (i = 0; i < n; i++)
GoF();
return 0;
}

View File

@ -6,19 +6,27 @@
void use(int *x) { (*x)++; }
void callGoFWithDeepStack() {
void callGoFWithDeepStack(int p) {
int x[10000];
use(&x[0]);
use(&x[9999]);
GoF();
GoF(p);
use(&x[0]);
use(&x[9999]);
}
int main() {
GoF(); // call GoF without using much stack
callGoFWithDeepStack(); // call GoF with a deep stack
void callGoWithVariousStack(int p) {
GoF(0); // call GoF without using much stack
callGoFWithDeepStack(p); // call GoF with a deep stack
GoF(0); // again on a shallow stack
}
int main() {
callGoWithVariousStack(0);
callGoWithVariousStackAndGoFrame(0); // normal execution
callGoWithVariousStackAndGoFrame(1); // panic and recover
}

View File

@ -318,9 +318,10 @@ func containsClosure(f, c *ir.Func) bool {
return false
}
// Closures within function Foo are named like "Foo.funcN..."
// TODO(mdempsky): Better way to recognize this.
fn := f.Sym().Name
cn := c.Sym().Name
return len(cn) > len(fn) && cn[:len(fn)] == fn && cn[len(fn)] == '.'
for p := c.ClosureParent; p != nil; p = p.ClosureParent {
if p == f {
return true
}
}
return false
}

View File

@ -582,6 +582,23 @@ func TestIssue25596(t *testing.T) {
compileAndImportPkg(t, "issue25596")
}
func TestIssue70394(t *testing.T) {
testenv.MustHaveGoBuild(t)
// This package only handles gc export data.
if runtime.Compiler != "gc" {
t.Skipf("gc-built packages not available (compiler = %s)", runtime.Compiler)
}
pkg := compileAndImportPkg(t, "alias")
obj := lookupObj(t, pkg.Scope(), "A")
typ := obj.Type()
if _, ok := typ.(*types2.Alias); !ok {
t.Fatalf("type of %s is %s, wanted an alias", obj, typ)
}
}
func importPkg(t *testing.T, path, srcDir string) *types2.Package {
pkg, err := Import(make(map[string]*types2.Package), path, srcDir, nil)
if err != nil {

View File

@ -0,0 +1,7 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package testdata
type A = int32

View File

@ -9,6 +9,7 @@ import (
"cmd/compile/internal/syntax"
"cmd/compile/internal/types2"
"cmd/internal/src"
"internal/buildcfg"
"internal/pkgbits"
)
@ -28,11 +29,9 @@ func ReadPackage(ctxt *types2.Context, imports map[string]*types2.Package, input
pr := pkgReader{
PkgDecoder: input,
ctxt: ctxt,
imports: imports,
// Currently, the compiler panics when using Alias types.
// TODO(gri) set to true once this is fixed (issue #66873)
enableAlias: false,
ctxt: ctxt,
imports: imports,
enableAlias: true,
posBases: make([]*syntax.PosBase, input.NumElems(pkgbits.RelocPosBase)),
pkgs: make([]*types2.Package, input.NumElems(pkgbits.RelocPkg)),
@ -411,6 +410,14 @@ func (pr *pkgReader) objIdx(idx pkgbits.Index) (*types2.Package, string) {
panic("weird")
case pkgbits.ObjAlias:
if buildcfg.Experiment.AliasTypeParams && len(r.dict.bounds) > 0 {
// Temporary work-around for issue #68526: rather than panicking
// with an non-descriptive index-out-of-bounds panic when trying
// to access a missing type parameter, instead panic with a more
// descriptive error. Only needed for Go 1.23; Go 1.24 will have
// the correct implementation.
panic("importing generic type aliases is not supported in Go 1.23 (see issue #68526)")
}
pos := r.pos()
typ := r.typ()
return newAliasTypeName(pr.enableAlias, pos, objPkg, objName, typ)

View File

@ -51,6 +51,8 @@ import (
// the generated ODCLFUNC, but there is no
// pointer from the Func back to the OMETHVALUE.
type Func struct {
// if you add or remove a field, don't forget to update sizeof_test.go
miniNode
Body Nodes
@ -76,6 +78,9 @@ type Func struct {
// Populated during walk.
Closures []*Func
// Parent of a closure
ClosureParent *Func
// Parents records the parent scope of each scope within a
// function. The root scope (0) has no parent, so the i'th
// scope's parent is stored at Parents[i-1].
@ -512,6 +517,7 @@ func NewClosureFunc(fpos, cpos src.XPos, why Op, typ *types.Type, outerfn *Func,
fn.Nname.Defn = fn
pkg.Funcs = append(pkg.Funcs, fn)
fn.ClosureParent = outerfn
return fn
}

View File

@ -20,7 +20,7 @@ func TestSizeof(t *testing.T) {
_32bit uintptr // size on 32bit platforms
_64bit uintptr // size on 64bit platforms
}{
{Func{}, 176, 296},
{Func{}, 180, 304},
{Name{}, 96, 168},
}

View File

@ -2099,3 +2099,27 @@ func TestTwoLevelReturnCheck(t *testing.T) {
t.Errorf("Expected y=3, got y=%d\n", y)
}
}
func Bug70035(s1, s2, s3 []string) string {
var c1 string
for v1 := range slices.Values(s1) {
var c2 string
for v2 := range slices.Values(s2) {
var c3 string
for v3 := range slices.Values(s3) {
c3 = c3 + v3
}
c2 = c2 + v2 + c3
}
c1 = c1 + v1 + c2
}
return c1
}
func Test70035(t *testing.T) {
got := Bug70035([]string{"1", "2", "3"}, []string{"a", "b", "c"}, []string{"A", "B", "C"})
want := "1aABCbABCcABC2aABCbABCcABC3aABCbABCcABC"
if got != want {
t.Errorf("got %v, want %v", got, want)
}
}

View File

@ -134,10 +134,10 @@ func (check *Checker) newAlias(obj *TypeName, rhs Type) *Alias {
// newAliasInstance creates a new alias instance for the given origin and type
// arguments, recording pos as the position of its synthetic object (for error
// reporting).
func (check *Checker) newAliasInstance(pos syntax.Pos, orig *Alias, targs []Type, ctxt *Context) *Alias {
func (check *Checker) newAliasInstance(pos syntax.Pos, orig *Alias, targs []Type, expanding *Named, ctxt *Context) *Alias {
assert(len(targs) > 0)
obj := NewTypeName(pos, orig.obj.pkg, orig.obj.name, nil)
rhs := check.subst(pos, orig.fromRHS, makeSubstMap(orig.TypeParams().list(), targs), nil, ctxt)
rhs := check.subst(pos, orig.fromRHS, makeSubstMap(orig.TypeParams().list(), targs), expanding, ctxt)
res := check.newAlias(obj, rhs)
res.orig = orig
res.tparams = orig.tparams

View File

@ -2898,22 +2898,48 @@ func TestFileVersions(t *testing.T) {
fileVersion string
wantVersion string
}{
{"", "", ""}, // no versions specified
{"go1.19", "", "go1.19"}, // module version specified
{"", "go1.20", ""}, // file upgrade ignored
{"go1.19", "go1.20", "go1.20"}, // file upgrade permitted
{"go1.20", "go1.19", "go1.20"}, // file downgrade not permitted
{"go1.21", "go1.19", "go1.19"}, // file downgrade permitted (module version is >= go1.21)
{"", "", ""}, // no versions specified
{"go1.19", "", "go1.19"}, // module version specified
{"", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1", "", "go1"}, // no file version specified
{"go1", "goo1.22", "go1"}, // invalid file version specified
{"go1", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.19", "", "go1.19"}, // no file version specified
{"go1.19", "goo1.22", "go1.19"}, // invalid file version specified
{"go1.19", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.19", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.19", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.20", "", "go1.20"}, // no file version specified
{"go1.20", "goo1.22", "go1.20"}, // invalid file version specified
{"go1.20", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1.20", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.20", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.20", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.21", "", "go1.21"}, // no file version specified
{"go1.21", "goo1.22", "go1.21"}, // invalid file version specified
{"go1.21", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1.21", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.21", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.21", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.22", "", "go1.22"}, // no file version specified
{"go1.22", "goo1.22", "go1.22"}, // invalid file version specified
{"go1.22", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1.22", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.22", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.22", "go1.22", "go1.22"}, // file version specified above 1.21
// versions containing release numbers
// (file versions containing release numbers are considered invalid)
{"go1.19.0", "", "go1.19.0"}, // no file version specified
{"go1.20", "go1.20.1", "go1.20"}, // file upgrade ignored
{"go1.20.1", "go1.20", "go1.20.1"}, // file upgrade ignored
{"go1.20.1", "go1.21", "go1.21"}, // file upgrade permitted
{"go1.20.1", "go1.19", "go1.20.1"}, // file downgrade not permitted
{"go1.21.1", "go1.19.1", "go1.21.1"}, // file downgrade not permitted (invalid file version)
{"go1.21.1", "go1.19", "go1.19"}, // file downgrade permitted (module version is >= go1.21)
{"go1.20.1", "go1.19.1", "go1.20.1"}, // invalid file version
{"go1.20.1", "go1.21.1", "go1.20.1"}, // invalid file version
{"go1.21.1", "go1.19.1", "go1.21.1"}, // invalid file version
{"go1.21.1", "go1.21.1", "go1.21.1"}, // invalid file version
{"go1.22.1", "go1.19.1", "go1.22.1"}, // invalid file version
{"go1.22.1", "go1.21.1", "go1.22.1"}, // invalid file version
} {
var src string
if test.fileVersion != "" {

View File

@ -327,7 +327,6 @@ func (check *Checker) initFiles(files []*syntax.File) {
check.errorf(files[0], TooNew, "package requires newer Go version %v (application built with %v)",
check.version, go_current)
}
downgradeOk := check.version.cmp(go1_21) >= 0
// determine Go version for each file
for _, file := range check.files {
@ -336,33 +335,18 @@ func (check *Checker) initFiles(files []*syntax.File) {
// unlike file versions which are Go language versions only, if valid.)
v := check.conf.GoVersion
fileVersion := asGoVersion(file.GoVersion)
if fileVersion.isValid() {
// use the file version, if applicable
// (file versions are either the empty string or of the form go1.dd)
if pkgVersionOk {
cmp := fileVersion.cmp(check.version)
// Go 1.21 introduced the feature of setting the go.mod
// go line to an early version of Go and allowing //go:build lines
// to “upgrade” (cmp > 0) the Go version in a given file.
// We can do that backwards compatibly.
//
// Go 1.21 also introduced the feature of allowing //go:build lines
// to “downgrade” (cmp < 0) the Go version in a given file.
// That can't be done compatibly in general, since before the
// build lines were ignored and code got the module's Go version.
// To work around this, downgrades are only allowed when the
// module's Go version is Go 1.21 or later.
//
// If there is no valid check.version, then we don't really know what
// Go version to apply.
// Legacy tools may do this, and they historically have accepted everything.
// Preserve that behavior by ignoring //go:build constraints entirely in that
// case (!pkgVersionOk).
if cmp > 0 || cmp < 0 && downgradeOk {
v = file.GoVersion
}
}
// If the file specifies a version, use max(fileVersion, go1.21).
if fileVersion := asGoVersion(file.GoVersion); fileVersion.isValid() {
// Go 1.21 introduced the feature of allowing //go:build lines
// to sometimes set the Go version in a given file. Versions Go 1.21 and later
// can be set backwards compatibly as that was the first version
// files with go1.21 or later build tags could be built with.
//
// Set the version to max(fileVersion, go1.21): That will allow a
// downgrade to a version before go1.22, where the for loop semantics
// change was made, while being backwards compatible with versions of
// go before the new //go:build semantics were introduced.
v = string(versionMax(fileVersion, go1_21))
// Report a specific error for each tagged file that's too new.
// (Normally the build system will have filtered files by version,
@ -377,6 +361,13 @@ func (check *Checker) initFiles(files []*syntax.File) {
}
}
func versionMax(a, b goVersion) goVersion {
if a.cmp(b) > 0 {
return a
}
return b
}
// A bailout panic is used for early termination.
type bailout struct{}

View File

@ -11,6 +11,7 @@ import (
"cmd/compile/internal/syntax"
"errors"
"fmt"
"internal/buildcfg"
. "internal/types/errors"
)
@ -126,8 +127,9 @@ func (check *Checker) instance(pos syntax.Pos, orig genericType, targs []Type, e
res = check.newNamedInstance(pos, orig, targs, expanding) // substituted lazily
case *Alias:
// TODO(gri) is this correct?
assert(expanding == nil) // Alias instances cannot be reached from Named types
if !buildcfg.Experiment.AliasTypeParams {
assert(expanding == nil) // Alias instances cannot be reached from Named types
}
tparams := orig.TypeParams()
// TODO(gri) investigate if this is needed (type argument and parameter count seem to be correct here)
@ -138,7 +140,7 @@ func (check *Checker) instance(pos syntax.Pos, orig genericType, targs []Type, e
return orig // nothing to do (minor optimization)
}
return check.newAliasInstance(pos, orig, targs, ctxt)
return check.newAliasInstance(pos, orig, targs, expanding, ctxt)
case *Signature:
assert(expanding == nil) // function instances cannot be reached from Named types

View File

@ -1121,3 +1121,23 @@ func f(x int) {
t.Errorf("got: %s want: %s", got, want)
}
}
func TestIssue68877(t *testing.T) {
const src = `
package p
type (
S struct{}
A = S
T A
)`
conf := Config{EnableAlias: true}
pkg := mustTypecheck(src, &conf, nil)
T := pkg.Scope().Lookup("T").(*TypeName)
got := T.String() // this must not panic (was issue)
const want = "type p.T struct{}"
if got != want {
t.Errorf("got %s, want %s", got, want)
}
}

View File

@ -282,7 +282,7 @@ func (t *Named) cleanup() {
if t.TypeArgs().Len() == 0 {
panic("nil underlying")
}
case *Named:
case *Named, *Alias:
t.under() // t.under may add entries to check.cleaners
}
t.check = nil

View File

@ -115,7 +115,7 @@ func (subst *subster) typ(typ Type) Type {
// that has a type argument for it.
targs, updated := subst.typeList(t.TypeArgs().list())
if updated {
return subst.check.newAliasInstance(subst.pos, t.orig, targs, subst.ctxt)
return subst.check.newAliasInstance(subst.pos, t.orig, targs, subst.expanding, subst.ctxt)
}
case *Array:

View File

@ -131,8 +131,8 @@ func (s *_TypeSet) underIs(f func(Type) bool) bool {
}
for _, t := range s.terms {
assert(t.typ != nil)
// x == under(x) for ~x terms
u := t.typ
// Unalias(x) == under(x) for ~x terms
u := Unalias(t.typ)
if !t.tilde {
u = under(u)
}

View File

@ -9,7 +9,7 @@ require (
golang.org/x/mod v0.19.0
golang.org/x/sync v0.7.0
golang.org/x/sys v0.22.0
golang.org/x/telemetry v0.0.0-20240717194752-0b706e19b701
golang.org/x/telemetry v0.0.0-20240828213427-40b6b7fe7147
golang.org/x/term v0.20.0
golang.org/x/tools v0.22.1-0.20240618181713-f2d2ebe43e72
)

View File

@ -16,8 +16,8 @@ golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/telemetry v0.0.0-20240717194752-0b706e19b701 h1:+bltxAtk8YFEQ61B/lcYQM8e+7XjLwSDbpspVaVYkz8=
golang.org/x/telemetry v0.0.0-20240717194752-0b706e19b701/go.mod h1:amNmu/SBSm2GAF3X+9U2C0epLocdh+r5Z+7oMYO5cLM=
golang.org/x/telemetry v0.0.0-20240828213427-40b6b7fe7147 h1:Lj8KbuZmoFUbI6pQ28G3Diz/5bRYD2UY5vfAmhrLZWo=
golang.org/x/telemetry v0.0.0-20240828213427-40b6b7fe7147/go.mod h1:amNmu/SBSm2GAF3X+9U2C0epLocdh+r5Z+7oMYO5cLM=
golang.org/x/term v0.20.0 h1:VnkxpohqXaOBYJtBmEppKUG6mXpi+4O6purfc2+sMhw=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=

View File

@ -805,13 +805,19 @@ func elfwritefreebsdsig(out *OutBuf) int {
return int(sh.Size)
}
func addbuildinfo(val string) {
func addbuildinfo(ctxt *Link) {
val := *flagHostBuildid
if val == "gobuildid" {
buildID := *flagBuildid
if buildID == "" {
Exitf("-B gobuildid requires a Go build ID supplied via -buildid")
}
if ctxt.IsDarwin() {
buildinfo = uuidFromGoBuildId(buildID)
return
}
hashedBuildID := notsha256.Sum256([]byte(buildID))
buildinfo = hashedBuildID[:20]
@ -821,11 +827,13 @@ func addbuildinfo(val string) {
if !strings.HasPrefix(val, "0x") {
Exitf("-B argument must start with 0x: %s", val)
}
ov := val
val = val[2:]
const maxLen = 32
maxLen := 32
if ctxt.IsDarwin() {
maxLen = 16
}
if hex.DecodedLen(len(val)) > maxLen {
Exitf("-B option too long (max %d digits): %s", maxLen, ov)
}

View File

@ -297,6 +297,8 @@ func getMachoHdr() *MachoHdr {
return &machohdr
}
// Create a new Mach-O load command. ndata is the number of 32-bit words for
// the data (not including the load command header).
func newMachoLoad(arch *sys.Arch, type_ uint32, ndata uint32) *MachoLoad {
if arch.PtrSize == 8 && (ndata&1 != 0) {
ndata++
@ -849,6 +851,20 @@ func asmbMacho(ctxt *Link) {
}
}
if ctxt.IsInternal() && len(buildinfo) > 0 {
ml := newMachoLoad(ctxt.Arch, LC_UUID, 4)
// Mach-O UUID is 16 bytes
if len(buildinfo) < 16 {
buildinfo = append(buildinfo, make([]byte, 16)...)
}
// By default, buildinfo is already in UUIDv3 format
// (see uuidFromGoBuildId).
ml.data[0] = ctxt.Arch.ByteOrder.Uint32(buildinfo)
ml.data[1] = ctxt.Arch.ByteOrder.Uint32(buildinfo[4:])
ml.data[2] = ctxt.Arch.ByteOrder.Uint32(buildinfo[8:])
ml.data[3] = ctxt.Arch.ByteOrder.Uint32(buildinfo[12:])
}
if ctxt.IsInternal() && ctxt.NeedCodeSign() {
ml := newMachoLoad(ctxt.Arch, LC_CODE_SIGNATURE, 2)
ml.data[0] = uint32(codesigOff)

View File

@ -42,7 +42,7 @@ func uuidFromGoBuildId(buildID string) []byte {
// to use this UUID flavor than any of the others. This is similar
// to how other linkers handle this (for example this code in lld:
// https://github.com/llvm/llvm-project/blob/2a3a79ce4c2149d7787d56f9841b66cacc9061d0/lld/MachO/Writer.cpp#L524).
rv[6] &= 0xcf
rv[6] &= 0x0f
rv[6] |= 0x30
rv[8] &= 0x3f
rv[8] |= 0xc0

View File

@ -95,6 +95,7 @@ var (
flagN = flag.Bool("n", false, "no-op (deprecated)")
FlagS = flag.Bool("s", false, "disable symbol table")
flag8 bool // use 64-bit addresses in symbol table
flagHostBuildid = flag.String("B", "", "set ELF NT_GNU_BUILD_ID `note` or Mach-O UUID; use \"gobuildid\" to generate it from the Go build ID")
flagInterpreter = flag.String("I", "", "use `linker` as ELF dynamic linker")
flagCheckLinkname = flag.Bool("checklinkname", true, "check linkname symbol references")
FlagDebugTramp = flag.Int("debugtramp", 0, "debug trampolines")
@ -196,7 +197,6 @@ func Main(arch *sys.Arch, theArch Arch) {
flag.Var(&ctxt.LinkMode, "linkmode", "set link `mode`")
flag.Var(&ctxt.BuildMode, "buildmode", "set build `mode`")
flag.BoolVar(&ctxt.compressDWARF, "compressdwarf", true, "compress DWARF if possible")
objabi.Flagfn1("B", "add an ELF NT_GNU_BUILD_ID `note` when using ELF; use \"gobuildid\" to generate it from the Go build ID", addbuildinfo)
objabi.Flagfn1("L", "add specified `directory` to library path", func(a string) { Lflag(ctxt, a) })
objabi.AddVersionFlag() // -V
objabi.Flagfn1("X", "add string value `definition` of the form importpath.name=value", func(s string) { addstrdata1(ctxt, s) })
@ -294,6 +294,10 @@ func Main(arch *sys.Arch, theArch Arch) {
*flagBuildid = "go-openbsd"
}
if *flagHostBuildid != "" {
addbuildinfo(ctxt)
}
// enable benchmarking
var bench *benchmark.Metrics
if len(*benchmarkFlag) != 0 {

View File

@ -257,6 +257,10 @@ func (gs *gState[R]) stop(ts trace.Time, stack trace.Stack, ctx *traceContext) {
if gs.lastStopStack != trace.NoStack {
stk = ctx.Stack(viewerFrames(gs.lastStopStack))
}
var endStk int
if stack != trace.NoStack {
endStk = ctx.Stack(viewerFrames(stack))
}
// Check invariants.
if gs.startRunningTime == 0 {
panic("silently broken trace or generator invariant (startRunningTime != 0) not held")
@ -270,6 +274,7 @@ func (gs *gState[R]) stop(ts trace.Time, stack trace.Stack, ctx *traceContext) {
Dur: ts.Sub(gs.startRunningTime),
Resource: uint64(gs.executing),
Stack: stk,
EndStack: endStk,
})
// Flush completed ranges.

View File

@ -16,6 +16,7 @@ import (
"os"
"os/exec"
"path/filepath"
"sync/atomic"
"golang.org/x/telemetry/internal/telemetry"
)
@ -29,12 +30,22 @@ const (
// creation flag.
var needNoConsole = func(cmd *exec.Cmd) {}
var downloads int64
// Downloads reports, for testing purposes, the number of times [Download] has
// been called.
func Downloads() int64 {
return atomic.LoadInt64(&downloads)
}
// Download fetches the requested telemetry UploadConfig using "go mod
// download". If envOverlay is provided, it is appended to the environment used
// for invoking the go command.
//
// The second result is the canonical version of the requested configuration.
func Download(version string, envOverlay []string) (*telemetry.UploadConfig, string, error) {
atomic.AddInt64(&downloads, 1)
if version == "" {
version = "latest"
}

View File

@ -21,12 +21,12 @@ import (
"golang.org/x/telemetry/internal/counter"
)
// Supported reports whether the runtime supports [runtime.SetCrashOutput].
// Supported reports whether the runtime supports [runtime/debug.SetCrashOutput].
//
// TODO(adonovan): eliminate once go1.23+ is assured.
func Supported() bool { return setCrashOutput != nil }
var setCrashOutput func(*os.File) error // = runtime.SetCrashOutput on go1.23+
var setCrashOutput func(*os.File) error // = runtime/debug.SetCrashOutput on go1.23+
// Parent sets up the parent side of the crashmonitor. It requires
// exclusive use of a writable pipe connected to the child process's stdin.

View File

@ -112,9 +112,24 @@ func newUploader(rcfg RunConfig) (*uploader, error) {
logger := log.New(logWriter, "", log.Ltime|log.Lmicroseconds|log.Lshortfile)
// Fetch the upload config, if it is not provided.
config, configVersion, err := configstore.Download("latest", rcfg.Env)
if err != nil {
return nil, err
var (
config *telemetry.UploadConfig
configVersion string
)
if mode, _ := dir.Mode(); mode == "on" {
// golang/go#68946: only download the upload config if it will be used.
//
// TODO(rfindley): This is a narrow change aimed at minimally fixing the
// associated bug. In the future, we should read the mode only once during
// the upload process.
config, configVersion, err = configstore.Download("latest", rcfg.Env)
if err != nil {
return nil, err
}
} else {
config = &telemetry.UploadConfig{}
configVersion = "v0.0.0-0"
}
// Set the start time, if it is not provided.

View File

@ -206,7 +206,8 @@ func startChild(reportCrashes, upload bool, result *StartResult) {
fd, err := os.Stat(telemetry.Default.DebugDir())
if err != nil {
if !os.IsNotExist(err) {
log.Fatalf("failed to stat debug directory: %v", err)
log.Printf("failed to stat debug directory: %v", err)
return
}
} else if fd.IsDir() {
// local/debug exists and is a directory. Set stderr to a log file path
@ -214,23 +215,31 @@ func startChild(reportCrashes, upload bool, result *StartResult) {
childLogPath := filepath.Join(telemetry.Default.DebugDir(), "sidecar.log")
childLog, err := os.OpenFile(childLogPath, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0600)
if err != nil {
log.Fatalf("opening sidecar log file for child: %v", err)
log.Printf("opening sidecar log file for child: %v", err)
return
}
defer childLog.Close()
cmd.Stderr = childLog
}
var crashOutputFile *os.File
if reportCrashes {
pipe, err := cmd.StdinPipe()
if err != nil {
log.Fatalf("StdinPipe: %v", err)
log.Printf("StdinPipe: %v", err)
return
}
crashmonitor.Parent(pipe.(*os.File)) // (this conversion is safe)
crashOutputFile = pipe.(*os.File) // (this conversion is safe)
}
if err := cmd.Start(); err != nil {
log.Fatalf("can't start telemetry child process: %v", err)
// The child couldn't be started. Log the failure.
log.Printf("can't start telemetry child process: %v", err)
return
}
if reportCrashes {
crashmonitor.Parent(crashOutputFile)
}
result.wg.Add(1)
go func() {

View File

@ -45,7 +45,7 @@ golang.org/x/sync/semaphore
golang.org/x/sys/plan9
golang.org/x/sys/unix
golang.org/x/sys/windows
# golang.org/x/telemetry v0.0.0-20240717194752-0b706e19b701
# golang.org/x/telemetry v0.0.0-20240828213427-40b6b7fe7147
## explicit; go 1.20
golang.org/x/telemetry
golang.org/x/telemetry/counter

View File

@ -852,6 +852,7 @@ func testResumption(t *testing.T, version uint16) {
MaxVersion: version,
CipherSuites: []uint16{TLS_RSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA},
Certificates: testConfig.Certificates,
Time: testTime,
}
issuer, err := x509.ParseCertificate(testRSACertificateIssuer)
@ -868,6 +869,7 @@ func testResumption(t *testing.T, version uint16) {
ClientSessionCache: NewLRUClientSessionCache(32),
RootCAs: rootCAs,
ServerName: "example.golang",
Time: testTime,
}
testResumeState := func(test string, didResume bool) {
@ -914,7 +916,7 @@ func testResumption(t *testing.T, version uint16) {
// An old session ticket is replaced with a ticket encrypted with a fresh key.
ticket = getTicket()
serverConfig.Time = func() time.Time { return time.Now().Add(24*time.Hour + time.Minute) }
serverConfig.Time = func() time.Time { return testTime().Add(24*time.Hour + time.Minute) }
testResumeState("ResumeWithOldTicket", true)
if bytes.Equal(ticket, getTicket()) {
t.Fatal("old first ticket matches the fresh one")
@ -922,13 +924,13 @@ func testResumption(t *testing.T, version uint16) {
// Once the session master secret is expired, a full handshake should occur.
ticket = getTicket()
serverConfig.Time = func() time.Time { return time.Now().Add(24*8*time.Hour + time.Minute) }
serverConfig.Time = func() time.Time { return testTime().Add(24*8*time.Hour + time.Minute) }
testResumeState("ResumeWithExpiredTicket", false)
if bytes.Equal(ticket, getTicket()) {
t.Fatal("expired first ticket matches the fresh one")
}
serverConfig.Time = func() time.Time { return time.Now() } // reset the time back
serverConfig.Time = testTime // reset the time back
key1 := randomKey()
serverConfig.SetSessionTicketKeys([][32]byte{key1})
@ -945,11 +947,11 @@ func testResumption(t *testing.T, version uint16) {
testResumeState("KeyChangeFinish", true)
// Age the session ticket a bit, but not yet expired.
serverConfig.Time = func() time.Time { return time.Now().Add(24*time.Hour + time.Minute) }
serverConfig.Time = func() time.Time { return testTime().Add(24*time.Hour + time.Minute) }
testResumeState("OldSessionTicket", true)
ticket = getTicket()
// Expire the session ticket, which would force a full handshake.
serverConfig.Time = func() time.Time { return time.Now().Add(24*8*time.Hour + time.Minute) }
serverConfig.Time = func() time.Time { return testTime().Add(24*8*time.Hour + 2*time.Minute) }
testResumeState("ExpiredSessionTicket", false)
if bytes.Equal(ticket, getTicket()) {
t.Fatal("new ticket wasn't provided after old ticket expired")
@ -957,7 +959,7 @@ func testResumption(t *testing.T, version uint16) {
// Age the session ticket a bit at a time, but don't expire it.
d := 0 * time.Hour
serverConfig.Time = func() time.Time { return time.Now().Add(d) }
serverConfig.Time = func() time.Time { return testTime().Add(d) }
deleteTicket()
testResumeState("GetFreshSessionTicket", false)
for i := 0; i < 13; i++ {
@ -968,7 +970,7 @@ func testResumption(t *testing.T, version uint16) {
// handshake occurs for TLS 1.2. Resumption should still occur for
// TLS 1.3 since the client should be using a fresh ticket sent over
// by the server.
d += 12 * time.Hour
d += 12*time.Hour + time.Minute
if version == VersionTLS13 {
testResumeState("ExpiredSessionTicket", true)
} else {
@ -984,6 +986,7 @@ func testResumption(t *testing.T, version uint16) {
MaxVersion: version,
CipherSuites: []uint16{TLS_RSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA},
Certificates: testConfig.Certificates,
Time: testTime,
}
serverConfig.SetSessionTicketKeys([][32]byte{key2})
@ -1009,6 +1012,7 @@ func testResumption(t *testing.T, version uint16) {
CurvePreferences: []CurveID{CurveP521, CurveP384, CurveP256},
MaxVersion: version,
Certificates: testConfig.Certificates,
Time: testTime,
}
testResumeState("InitialHandshake", false)
testResumeState("WithHelloRetryRequest", true)
@ -1018,6 +1022,7 @@ func testResumption(t *testing.T, version uint16) {
MaxVersion: version,
CipherSuites: []uint16{TLS_RSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA},
Certificates: testConfig.Certificates,
Time: testTime,
}
}
@ -1736,6 +1741,7 @@ func testVerifyConnection(t *testing.T, version uint16) {
serverConfig := &Config{
MaxVersion: version,
Certificates: []Certificate{testConfig.Certificates[0]},
Time: testTime,
ClientCAs: rootCAs,
NextProtos: []string{"protocol1"},
}
@ -1749,6 +1755,7 @@ func testVerifyConnection(t *testing.T, version uint16) {
RootCAs: rootCAs,
ServerName: "example.golang",
Certificates: []Certificate{testConfig.Certificates[0]},
Time: testTime,
NextProtos: []string{"protocol1"},
}
test.configureClient(clientConfig, &clientCalled)
@ -1791,8 +1798,6 @@ func testVerifyPeerCertificate(t *testing.T, version uint16) {
rootCAs := x509.NewCertPool()
rootCAs.AddCert(issuer)
now := func() time.Time { return time.Unix(1476984729, 0) }
sentinelErr := errors.New("TestVerifyPeerCertificate")
verifyPeerCertificateCallback := func(called *bool, rawCerts [][]byte, validatedChains [][]*x509.Certificate) error {
@ -2038,7 +2043,7 @@ func testVerifyPeerCertificate(t *testing.T, version uint16) {
config.ServerName = "example.golang"
config.ClientAuth = RequireAndVerifyClientCert
config.ClientCAs = rootCAs
config.Time = now
config.Time = testTime
config.MaxVersion = version
config.Certificates = make([]Certificate, 1)
config.Certificates[0].Certificate = [][]byte{testRSACertificate}
@ -2055,7 +2060,7 @@ func testVerifyPeerCertificate(t *testing.T, version uint16) {
config := testConfig.Clone()
config.ServerName = "example.golang"
config.RootCAs = rootCAs
config.Time = now
config.Time = testTime
config.MaxVersion = version
test.configureClient(config, &clientCalled)
clientErr := Client(c, config).Handshake()
@ -2368,7 +2373,7 @@ func testGetClientCertificate(t *testing.T, version uint16) {
serverConfig.RootCAs = x509.NewCertPool()
serverConfig.RootCAs.AddCert(issuer)
serverConfig.ClientCAs = serverConfig.RootCAs
serverConfig.Time = func() time.Time { return time.Unix(1476984729, 0) }
serverConfig.Time = testTime
serverConfig.MaxVersion = version
clientConfig := testConfig.Clone()
@ -2539,6 +2544,7 @@ func testResumptionKeepsOCSPAndSCT(t *testing.T, ver uint16) {
ClientSessionCache: NewLRUClientSessionCache(32),
ServerName: "example.golang",
RootCAs: roots,
Time: testTime,
}
serverConfig := testConfig.Clone()
serverConfig.MaxVersion = ver

View File

@ -501,6 +501,7 @@ func testCrossVersionResume(t *testing.T, version uint16) {
serverConfig := &Config{
CipherSuites: []uint16{TLS_RSA_WITH_AES_128_CBC_SHA},
Certificates: testConfig.Certificates,
Time: testTime,
}
clientConfig := &Config{
CipherSuites: []uint16{TLS_RSA_WITH_AES_128_CBC_SHA},
@ -508,6 +509,7 @@ func testCrossVersionResume(t *testing.T, version uint16) {
ClientSessionCache: NewLRUClientSessionCache(1),
ServerName: "servername",
MinVersion: VersionTLS12,
Time: testTime,
}
// Establish a session at TLS 1.3.

View File

@ -491,9 +491,10 @@ func testHandshake(t *testing.T, clientConfig, serverConfig *Config) (serverStat
if got := string(buf); got != sentinel {
t.Errorf("read %q from TLS connection, but expected %q", got, sentinel)
}
if err := cli.Close(); err != nil {
t.Errorf("failed to call cli.Close: %v", err)
}
// We discard the error because after ReadAll returns the server must
// have already closed the connection. Sending data (the closeNotify
// alert) can cause a reset, that will make Close return an error.
cli.Close()
}()
server := Server(s, serverConfig)
err = server.Handshake()
@ -518,6 +519,11 @@ func fromHex(s string) []byte {
return b
}
// testTime is 2016-10-20T17:32:09.000Z, which is within the validity period of
// [testRSACertificate], [testRSACertificateIssuer], [testRSA2048Certificate],
// [testRSA2048CertificateIssuer], and [testECDSACertificate].
var testTime = func() time.Time { return time.Unix(1476984729, 0) }
var testRSACertificate = fromHex("3082024b308201b4a003020102020900e8f09d3fe25beaa6300d06092a864886f70d01010b0500301f310b3009060355040a1302476f3110300e06035504031307476f20526f6f74301e170d3136303130313030303030305a170d3235303130313030303030305a301a310b3009060355040a1302476f310b300906035504031302476f30819f300d06092a864886f70d010101050003818d0030818902818100db467d932e12270648bc062821ab7ec4b6a25dfe1e5245887a3647a5080d92425bc281c0be97799840fb4f6d14fd2b138bc2a52e67d8d4099ed62238b74a0b74732bc234f1d193e596d9747bf3589f6c613cc0b041d4d92b2b2423775b1c3bbd755dce2054cfa163871d1e24c4f31d1a508baab61443ed97a77562f414c852d70203010001a38193308190300e0603551d0f0101ff0404030205a0301d0603551d250416301406082b0601050507030106082b06010505070302300c0603551d130101ff0402300030190603551d0e041204109f91161f43433e49a6de6db680d79f60301b0603551d230414301280104813494d137e1631bba301d5acab6e7b30190603551d1104123010820e6578616d706c652e676f6c616e67300d06092a864886f70d01010b0500038181009d30cc402b5b50a061cbbae55358e1ed8328a9581aa938a495a1ac315a1a84663d43d32dd90bf297dfd320643892243a00bccf9c7db74020015faad3166109a276fd13c3cce10c5ceeb18782f16c04ed73bbb343778d0c1cf10fa1d8408361c94c722b9daedb4606064df4c1b33ec0d1bd42d4dbfe3d1360845c21d33be9fae7")
var testRSACertificateIssuer = fromHex("3082021930820182a003020102020900ca5e4e811a965964300d06092a864886f70d01010b0500301f310b3009060355040a1302476f3110300e06035504031307476f20526f6f74301e170d3136303130313030303030305a170d3235303130313030303030305a301f310b3009060355040a1302476f3110300e06035504031307476f20526f6f7430819f300d06092a864886f70d010101050003818d0030818902818100d667b378bb22f34143b6cd2008236abefaf2852adf3ab05e01329e2c14834f5105df3f3073f99dab5442d45ee5f8f57b0111c8cb682fbb719a86944eebfffef3406206d898b8c1b1887797c9c5006547bb8f00e694b7a063f10839f269f2c34fff7a1f4b21fbcd6bfdfb13ac792d1d11f277b5c5b48600992203059f2a8f8cc50203010001a35d305b300e0603551d0f0101ff040403020204301d0603551d250416301406082b0601050507030106082b06010505070302300f0603551d130101ff040530030101ff30190603551d0e041204104813494d137e1631bba301d5acab6e7b300d06092a864886f70d01010b050003818100c1154b4bab5266221f293766ae4138899bd4c5e36b13cee670ceeaa4cbdf4f6679017e2fe649765af545749fe4249418a56bd38a04b81e261f5ce86b8d5c65413156a50d12449554748c59a30c515bc36a59d38bddf51173e899820b282e40aa78c806526fd184fb6b4cf186ec728edffa585440d2b3225325f7ab580e87dd76")

View File

@ -1112,8 +1112,6 @@ func TestConnectionState(t *testing.T) {
rootCAs := x509.NewCertPool()
rootCAs.AddCert(issuer)
now := func() time.Time { return time.Unix(1476984729, 0) }
const alpnProtocol = "golang"
const serverName = "example.golang"
var scts = [][]byte{[]byte("dummy sct 1"), []byte("dummy sct 2")}
@ -1129,7 +1127,7 @@ func TestConnectionState(t *testing.T) {
}
t.Run(name, func(t *testing.T) {
config := &Config{
Time: now,
Time: testTime,
Rand: zeroSource{},
Certificates: make([]Certificate, 1),
MaxVersion: v,
@ -1760,7 +1758,7 @@ func testVerifyCertificates(t *testing.T, version uint16) {
var serverVerifyPeerCertificates, clientVerifyPeerCertificates bool
clientConfig := testConfig.Clone()
clientConfig.Time = func() time.Time { return time.Unix(1476984729, 0) }
clientConfig.Time = testTime
clientConfig.MaxVersion = version
clientConfig.MinVersion = version
clientConfig.RootCAs = rootCAs

View File

@ -1607,6 +1607,23 @@ var nameConstraintsTests = []nameConstraintsTest{
leaf: leafSpec{sans: []string{"dns:.example.com"}},
expectedError: "cannot parse dnsName \".example.com\"",
},
// #86: URIs with IPv6 addresses with zones and ports are rejected
{
roots: []constraintsSpec{
{
ok: []string{"uri:example.com"},
},
},
intermediates: [][]constraintsSpec{
{
{},
},
},
leaf: leafSpec{
sans: []string{"uri:http://[2006:abcd::1%25.example.com]:16/"},
},
expectedError: "URI with IP",
},
}
func makeConstraintsCACert(constraints constraintsSpec, name string, key *ecdsa.PrivateKey, parent *Certificate, parentKey *ecdsa.PrivateKey) (*Certificate, error) {

View File

@ -11,6 +11,7 @@ import (
"errors"
"fmt"
"net"
"net/netip"
"net/url"
"reflect"
"runtime"
@ -434,8 +435,10 @@ func matchURIConstraint(uri *url.URL, constraint string) (bool, error) {
}
}
if strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]") ||
net.ParseIP(host) != nil {
// netip.ParseAddr will reject the URI IPv6 literal form "[...]", so we
// check if _either_ the string parses as an IP, or if it is enclosed in
// square brackets.
if _, err := netip.ParseAddr(host); err == nil || (strings.HasPrefix(host, "[") && strings.HasSuffix(host, "]")) {
return false, fmt.Errorf("URI with IP (%q) cannot be matched against constraints", uri.String())
}

View File

@ -1368,8 +1368,8 @@ func (db *DB) conn(ctx context.Context, strategy connReuseStrategy) (*driverConn
db.waitDuration.Add(int64(time.Since(waitStart)))
// If we failed to delete it, that means something else
// grabbed it and is about to send on it.
// If we failed to delete it, that means either the DB was closed or
// something else grabbed it and is about to send on it.
if !deleted {
// TODO(bradfitz): rather than this best effort select, we
// should probably start a goroutine to read from req. This best
@ -3594,6 +3594,7 @@ type connRequestAndIndex struct {
// and clears the set.
func (s *connRequestSet) CloseAndRemoveAll() {
for _, v := range s.s {
*v.curIdx = -1
close(v.req)
}
s.s = nil

View File

@ -4920,6 +4920,17 @@ func TestConnRequestSet(t *testing.T) {
t.Error("wasn't random")
}
})
t.Run("close-delete", func(t *testing.T) {
reset()
ch := make(chan connRequest)
dh := s.Add(ch)
wantLen(1)
s.CloseAndRemoveAll()
wantLen(0)
if s.Delete(dh) {
t.Error("unexpected delete after CloseAndRemoveAll")
}
})
}
func BenchmarkConnRequestSet(b *testing.B) {

View File

@ -911,8 +911,11 @@ func (dec *Decoder) decOpFor(wireId typeId, rt reflect.Type, name string, inProg
var maxIgnoreNestingDepth = 10000
// decIgnoreOpFor returns the decoding op for a field that has no destination.
func (dec *Decoder) decIgnoreOpFor(wireId typeId, inProgress map[typeId]*decOp, depth int) *decOp {
if depth > maxIgnoreNestingDepth {
func (dec *Decoder) decIgnoreOpFor(wireId typeId, inProgress map[typeId]*decOp) *decOp {
// Track how deep we've recursed trying to skip nested ignored fields.
dec.ignoreDepth++
defer func() { dec.ignoreDepth-- }()
if dec.ignoreDepth > maxIgnoreNestingDepth {
error_(errors.New("invalid nesting depth"))
}
// If this type is already in progress, it's a recursive type (e.g. map[string]*T).
@ -938,7 +941,7 @@ func (dec *Decoder) decIgnoreOpFor(wireId typeId, inProgress map[typeId]*decOp,
errorf("bad data: undefined type %s", wireId.string())
case wire.ArrayT != nil:
elemId := wire.ArrayT.Elem
elemOp := dec.decIgnoreOpFor(elemId, inProgress, depth+1)
elemOp := dec.decIgnoreOpFor(elemId, inProgress)
op = func(i *decInstr, state *decoderState, value reflect.Value) {
state.dec.ignoreArray(state, *elemOp, wire.ArrayT.Len)
}
@ -946,15 +949,15 @@ func (dec *Decoder) decIgnoreOpFor(wireId typeId, inProgress map[typeId]*decOp,
case wire.MapT != nil:
keyId := dec.wireType[wireId].MapT.Key
elemId := dec.wireType[wireId].MapT.Elem
keyOp := dec.decIgnoreOpFor(keyId, inProgress, depth+1)
elemOp := dec.decIgnoreOpFor(elemId, inProgress, depth+1)
keyOp := dec.decIgnoreOpFor(keyId, inProgress)
elemOp := dec.decIgnoreOpFor(elemId, inProgress)
op = func(i *decInstr, state *decoderState, value reflect.Value) {
state.dec.ignoreMap(state, *keyOp, *elemOp)
}
case wire.SliceT != nil:
elemId := wire.SliceT.Elem
elemOp := dec.decIgnoreOpFor(elemId, inProgress, depth+1)
elemOp := dec.decIgnoreOpFor(elemId, inProgress)
op = func(i *decInstr, state *decoderState, value reflect.Value) {
state.dec.ignoreSlice(state, *elemOp)
}
@ -1115,7 +1118,7 @@ func (dec *Decoder) compileSingle(remoteId typeId, ut *userTypeInfo) (engine *de
func (dec *Decoder) compileIgnoreSingle(remoteId typeId) *decEngine {
engine := new(decEngine)
engine.instr = make([]decInstr, 1) // one item
op := dec.decIgnoreOpFor(remoteId, make(map[typeId]*decOp), 0)
op := dec.decIgnoreOpFor(remoteId, make(map[typeId]*decOp))
ovfl := overflow(dec.typeString(remoteId))
engine.instr[0] = decInstr{*op, 0, nil, ovfl}
engine.numInstr = 1
@ -1160,7 +1163,7 @@ func (dec *Decoder) compileDec(remoteId typeId, ut *userTypeInfo) (engine *decEn
localField, present := srt.FieldByName(wireField.Name)
// TODO(r): anonymous names
if !present || !isExported(wireField.Name) {
op := dec.decIgnoreOpFor(wireField.Id, make(map[typeId]*decOp), 0)
op := dec.decIgnoreOpFor(wireField.Id, make(map[typeId]*decOp))
engine.instr[fieldnum] = decInstr{*op, fieldnum, nil, ovfl}
continue
}

View File

@ -35,6 +35,8 @@ type Decoder struct {
freeList *decoderState // list of free decoderStates; avoids reallocation
countBuf []byte // used for decoding integers while parsing messages
err error
// ignoreDepth tracks the depth of recursively parsed ignored fields
ignoreDepth int
}
// NewDecoder returns a new decoder that reads from the [io.Reader].

View File

@ -806,6 +806,8 @@ func TestIgnoreDepthLimit(t *testing.T) {
defer func() { maxIgnoreNestingDepth = oldNestingDepth }()
b := new(bytes.Buffer)
enc := NewEncoder(b)
// Nested slice
typ := reflect.TypeFor[int]()
nested := reflect.ArrayOf(1, typ)
for i := 0; i < 100; i++ {
@ -819,4 +821,16 @@ func TestIgnoreDepthLimit(t *testing.T) {
if err := dec.Decode(&output); err == nil || err.Error() != expectedErr {
t.Errorf("Decode didn't fail with depth limit of 100: want %q, got %q", expectedErr, err)
}
// Nested struct
nested = reflect.StructOf([]reflect.StructField{{Name: "F", Type: typ}})
for i := 0; i < 100; i++ {
nested = reflect.StructOf([]reflect.StructField{{Name: "F", Type: nested}})
}
badStruct = reflect.New(reflect.StructOf([]reflect.StructField{{Name: "F", Type: nested}}))
enc.Encode(badStruct.Interface())
dec = NewDecoder(b)
if err := dec.Decode(&output); err == nil || err.Error() != expectedErr {
t.Errorf("Decode didn't fail with depth limit of 100: want %q, got %q", expectedErr, err)
}
}

View File

@ -16,6 +16,10 @@ import (
"unicode/utf8"
)
// maxSize is a limit used to control the complexity of expressions, in order
// to prevent stack exhaustion issues due to recursion.
const maxSize = 1000
// An Expr is a build tag constraint expression.
// The underlying concrete type is *[AndExpr], *[OrExpr], *[NotExpr], or *[TagExpr].
type Expr interface {
@ -151,7 +155,7 @@ func Parse(line string) (Expr, error) {
return parseExpr(text)
}
if text, ok := splitPlusBuild(line); ok {
return parsePlusBuildExpr(text), nil
return parsePlusBuildExpr(text)
}
return nil, errNotConstraint
}
@ -201,6 +205,8 @@ type exprParser struct {
tok string // last token read
isTag bool
pos int // position (start) of last token
size int
}
// parseExpr parses a boolean build tag expression.
@ -249,6 +255,10 @@ func (p *exprParser) and() Expr {
// On entry, the next input token has not yet been lexed.
// On exit, the next input token has been lexed and is in p.tok.
func (p *exprParser) not() Expr {
p.size++
if p.size > maxSize {
panic(&SyntaxError{Offset: p.pos, Err: "build expression too large"})
}
p.lex()
if p.tok == "!" {
p.lex()
@ -388,7 +398,13 @@ func splitPlusBuild(line string) (expr string, ok bool) {
}
// parsePlusBuildExpr parses a legacy build tag expression (as used with “// +build”).
func parsePlusBuildExpr(text string) Expr {
func parsePlusBuildExpr(text string) (Expr, error) {
// Only allow up to 100 AND/OR operators for "old" syntax.
// This is much less than the limit for "new" syntax,
// but uses of old syntax were always very simple.
const maxOldSize = 100
size := 0
var x Expr
for _, clause := range strings.Fields(text) {
var y Expr
@ -414,19 +430,25 @@ func parsePlusBuildExpr(text string) Expr {
if y == nil {
y = z
} else {
if size++; size > maxOldSize {
return nil, errComplex
}
y = and(y, z)
}
}
if x == nil {
x = y
} else {
if size++; size > maxOldSize {
return nil, errComplex
}
x = or(x, y)
}
}
if x == nil {
x = tag("ignore")
}
return x
return x, nil
}
// isValidTag reports whether the word is a valid build tag.

View File

@ -222,7 +222,7 @@ var parsePlusBuildExprTests = []struct {
func TestParsePlusBuildExpr(t *testing.T) {
for i, tt := range parsePlusBuildExprTests {
t.Run(fmt.Sprint(i), func(t *testing.T) {
x := parsePlusBuildExpr(tt.in)
x, _ := parsePlusBuildExpr(tt.in)
if x.String() != tt.x.String() {
t.Errorf("parsePlusBuildExpr(%q):\nhave %v\nwant %v", tt.in, x, tt.x)
}
@ -319,3 +319,66 @@ func TestPlusBuildLines(t *testing.T) {
})
}
}
func TestSizeLimits(t *testing.T) {
for _, tc := range []struct {
name string
expr string
}{
{
name: "go:build or limit",
expr: "//go:build " + strings.Repeat("a || ", maxSize+2),
},
{
name: "go:build and limit",
expr: "//go:build " + strings.Repeat("a && ", maxSize+2),
},
{
name: "go:build and depth limit",
expr: "//go:build " + strings.Repeat("(a &&", maxSize+2),
},
{
name: "go:build or depth limit",
expr: "//go:build " + strings.Repeat("(a ||", maxSize+2),
},
} {
t.Run(tc.name, func(t *testing.T) {
_, err := Parse(tc.expr)
if err == nil {
t.Error("expression did not trigger limit")
} else if syntaxErr, ok := err.(*SyntaxError); !ok || syntaxErr.Err != "build expression too large" {
if !ok {
t.Errorf("unexpected error: %v", err)
} else {
t.Errorf("unexpected syntax error: %s", syntaxErr.Err)
}
}
})
}
}
func TestPlusSizeLimits(t *testing.T) {
maxOldSize := 100
for _, tc := range []struct {
name string
expr string
}{
{
name: "+build or limit",
expr: "// +build " + strings.Repeat("a ", maxOldSize+2),
},
{
name: "+build and limit",
expr: "// +build " + strings.Repeat("a,", maxOldSize+2),
},
} {
t.Run(tc.name, func(t *testing.T) {
_, err := Parse(tc.expr)
if err == nil {
t.Error("expression did not trigger limit")
} else if err != errComplex {
t.Errorf("unexpected error: got %q, want %q", err, errComplex)
}
})
}
}

View File

@ -1676,6 +1676,8 @@ func (p *parser) parseElementList() (list []ast.Expr) {
}
func (p *parser) parseLiteralValue(typ ast.Expr) ast.Expr {
defer decNestLev(incNestLev(p))
if p.trace {
defer un(trace(p, "LiteralValue"))
}

View File

@ -598,10 +598,11 @@ var parseDepthTests = []struct {
{name: "chan2", format: "package main; var x «<-chan »int"},
{name: "interface", format: "package main; var x «interface { M() «int» }»", scope: true, scopeMultiplier: 2}, // Scopes: InterfaceType, FuncType
{name: "map", format: "package main; var x «map[int]»int"},
{name: "slicelit", format: "package main; var x = «[]any{«»}»", parseMultiplier: 2}, // Parser nodes: UnaryExpr, CompositeLit
{name: "arraylit", format: "package main; var x = «[1]any{«nil»}»", parseMultiplier: 2}, // Parser nodes: UnaryExpr, CompositeLit
{name: "structlit", format: "package main; var x = «struct{x any}{«nil»}»", parseMultiplier: 2}, // Parser nodes: UnaryExpr, CompositeLit
{name: "maplit", format: "package main; var x = «map[int]any{1:«nil»}»", parseMultiplier: 2}, // Parser nodes: CompositeLit, KeyValueExpr
{name: "slicelit", format: "package main; var x = []any{«[]any{«»}»}", parseMultiplier: 3}, // Parser nodes: UnaryExpr, CompositeLit
{name: "arraylit", format: "package main; var x = «[1]any{«nil»}»", parseMultiplier: 3}, // Parser nodes: UnaryExpr, CompositeLit
{name: "structlit", format: "package main; var x = «struct{x any}{«nil»}»", parseMultiplier: 3}, // Parser nodes: UnaryExpr, CompositeLit
{name: "maplit", format: "package main; var x = «map[int]any{1:«nil»}»", parseMultiplier: 3}, // Parser nodes: CompositeLit, KeyValueExpr
{name: "element", format: "package main; var x = struct{x any}{x: «{«»}»}"},
{name: "dot", format: "package main; var x = «x.»x"},
{name: "index", format: "package main; var x = x«[1]»"},
{name: "slice", format: "package main; var x = x«[1:2]»"},

View File

@ -137,10 +137,10 @@ func (check *Checker) newAlias(obj *TypeName, rhs Type) *Alias {
// newAliasInstance creates a new alias instance for the given origin and type
// arguments, recording pos as the position of its synthetic object (for error
// reporting).
func (check *Checker) newAliasInstance(pos token.Pos, orig *Alias, targs []Type, ctxt *Context) *Alias {
func (check *Checker) newAliasInstance(pos token.Pos, orig *Alias, targs []Type, expanding *Named, ctxt *Context) *Alias {
assert(len(targs) > 0)
obj := NewTypeName(pos, orig.obj.pkg, orig.obj.name, nil)
rhs := check.subst(pos, orig.fromRHS, makeSubstMap(orig.TypeParams().list(), targs), nil, ctxt)
rhs := check.subst(pos, orig.fromRHS, makeSubstMap(orig.TypeParams().list(), targs), expanding, ctxt)
res := check.newAlias(obj, rhs)
res.orig = orig
res.tparams = orig.tparams

View File

@ -2904,22 +2904,48 @@ func TestFileVersions(t *testing.T) {
fileVersion string
wantVersion string
}{
{"", "", ""}, // no versions specified
{"go1.19", "", "go1.19"}, // module version specified
{"", "go1.20", ""}, // file upgrade ignored
{"go1.19", "go1.20", "go1.20"}, // file upgrade permitted
{"go1.20", "go1.19", "go1.20"}, // file downgrade not permitted
{"go1.21", "go1.19", "go1.19"}, // file downgrade permitted (module version is >= go1.21)
{"", "", ""}, // no versions specified
{"go1.19", "", "go1.19"}, // module version specified
{"", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1", "", "go1"}, // no file version specified
{"go1", "goo1.22", "go1"}, // invalid file version specified
{"go1", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.19", "", "go1.19"}, // no file version specified
{"go1.19", "goo1.22", "go1.19"}, // invalid file version specified
{"go1.19", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.19", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.19", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.20", "", "go1.20"}, // no file version specified
{"go1.20", "goo1.22", "go1.20"}, // invalid file version specified
{"go1.20", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1.20", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.20", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.20", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.21", "", "go1.21"}, // no file version specified
{"go1.21", "goo1.22", "go1.21"}, // invalid file version specified
{"go1.21", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1.21", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.21", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.21", "go1.22", "go1.22"}, // file version specified above 1.21
{"go1.22", "", "go1.22"}, // no file version specified
{"go1.22", "goo1.22", "go1.22"}, // invalid file version specified
{"go1.22", "go1.19", "go1.21"}, // file version specified below minimum of 1.21
{"go1.22", "go1.20", "go1.21"}, // file version specified below minimum of 1.21
{"go1.22", "go1.21", "go1.21"}, // file version specified at 1.21
{"go1.22", "go1.22", "go1.22"}, // file version specified above 1.21
// versions containing release numbers
// (file versions containing release numbers are considered invalid)
{"go1.19.0", "", "go1.19.0"}, // no file version specified
{"go1.20", "go1.20.1", "go1.20"}, // file upgrade ignored
{"go1.20.1", "go1.20", "go1.20.1"}, // file upgrade ignored
{"go1.20.1", "go1.21", "go1.21"}, // file upgrade permitted
{"go1.20.1", "go1.19", "go1.20.1"}, // file downgrade not permitted
{"go1.21.1", "go1.19.1", "go1.21.1"}, // file downgrade not permitted (invalid file version)
{"go1.21.1", "go1.19", "go1.19"}, // file downgrade permitted (module version is >= go1.21)
{"go1.20.1", "go1.19.1", "go1.20.1"}, // invalid file version
{"go1.20.1", "go1.21.1", "go1.20.1"}, // invalid file version
{"go1.21.1", "go1.19.1", "go1.21.1"}, // invalid file version
{"go1.21.1", "go1.21.1", "go1.21.1"}, // invalid file version
{"go1.22.1", "go1.19.1", "go1.22.1"}, // invalid file version
{"go1.22.1", "go1.21.1", "go1.22.1"}, // invalid file version
} {
var src string
if test.fileVersion != "" {

View File

@ -349,7 +349,6 @@ func (check *Checker) initFiles(files []*ast.File) {
check.errorf(files[0], TooNew, "package requires newer Go version %v (application built with %v)",
check.version, go_current)
}
downgradeOk := check.version.cmp(go1_21) >= 0
// determine Go version for each file
for _, file := range check.files {
@ -358,33 +357,19 @@ func (check *Checker) initFiles(files []*ast.File) {
// unlike file versions which are Go language versions only, if valid.)
v := check.conf.GoVersion
fileVersion := asGoVersion(file.GoVersion)
if fileVersion.isValid() {
// use the file version, if applicable
// (file versions are either the empty string or of the form go1.dd)
if pkgVersionOk {
cmp := fileVersion.cmp(check.version)
// Go 1.21 introduced the feature of setting the go.mod
// go line to an early version of Go and allowing //go:build lines
// to “upgrade” (cmp > 0) the Go version in a given file.
// We can do that backwards compatibly.
//
// Go 1.21 also introduced the feature of allowing //go:build lines
// to “downgrade” (cmp < 0) the Go version in a given file.
// That can't be done compatibly in general, since before the
// build lines were ignored and code got the module's Go version.
// To work around this, downgrades are only allowed when the
// module's Go version is Go 1.21 or later.
//
// If there is no valid check.version, then we don't really know what
// Go version to apply.
// Legacy tools may do this, and they historically have accepted everything.
// Preserve that behavior by ignoring //go:build constraints entirely in that
// case (!pkgVersionOk).
if cmp > 0 || cmp < 0 && downgradeOk {
v = file.GoVersion
}
}
// If the file specifies a version, use max(fileVersion, go1.21).
if fileVersion := asGoVersion(file.GoVersion); fileVersion.isValid() {
// Go 1.21 introduced the feature of setting the go.mod
// go line to an early version of Go and allowing //go:build lines
// to set the Go version in a given file. Versions Go 1.21 and later
// can be set backwards compatibly as that was the first version
// files with go1.21 or later build tags could be built with.
//
// Set the version to max(fileVersion, go1.21): That will allow a
// downgrade to a version before go1.22, where the for loop semantics
// change was made, while being backwards compatible with versions of
// go before the new //go:build semantics were introduced.
v = string(versionMax(fileVersion, go1_21))
// Report a specific error for each tagged file that's too new.
// (Normally the build system will have filtered files by version,
@ -399,6 +384,13 @@ func (check *Checker) initFiles(files []*ast.File) {
}
}
func versionMax(a, b goVersion) goVersion {
if a.cmp(b) < 0 {
return b
}
return a
}
// A bailout panic is used for early termination.
type bailout struct{}

View File

@ -14,6 +14,7 @@ import (
"errors"
"fmt"
"go/token"
"internal/buildcfg"
. "internal/types/errors"
)
@ -129,8 +130,9 @@ func (check *Checker) instance(pos token.Pos, orig genericType, targs []Type, ex
res = check.newNamedInstance(pos, orig, targs, expanding) // substituted lazily
case *Alias:
// TODO(gri) is this correct?
assert(expanding == nil) // Alias instances cannot be reached from Named types
if !buildcfg.Experiment.AliasTypeParams {
assert(expanding == nil) // Alias instances cannot be reached from Named types
}
tparams := orig.TypeParams()
// TODO(gri) investigate if this is needed (type argument and parameter count seem to be correct here)
@ -141,7 +143,7 @@ func (check *Checker) instance(pos token.Pos, orig genericType, targs []Type, ex
return orig // nothing to do (minor optimization)
}
return check.newAliasInstance(pos, orig, targs, ctxt)
return check.newAliasInstance(pos, orig, targs, expanding, ctxt)
case *Signature:
assert(expanding == nil) // function instances cannot be reached from Named types

View File

@ -1131,3 +1131,23 @@ func f(x int) {
t.Errorf("got: %s want: %s", got, want)
}
}
func TestIssue68877(t *testing.T) {
const src = `
package p
type (
S struct{}
A = S
T A
)`
t.Setenv("GODEBUG", "gotypesalias=1")
pkg := mustTypecheck(src, nil, nil)
T := pkg.Scope().Lookup("T").(*TypeName)
got := T.String() // this must not panic (was issue)
const want = "type p.T struct{}"
if got != want {
t.Errorf("got %s, want %s", got, want)
}
}

View File

@ -285,7 +285,7 @@ func (t *Named) cleanup() {
if t.TypeArgs().Len() == 0 {
panic("nil underlying")
}
case *Named:
case *Named, *Alias:
t.under() // t.under may add entries to check.cleaners
}
t.check = nil

View File

@ -118,7 +118,7 @@ func (subst *subster) typ(typ Type) Type {
// that has a type argument for it.
targs, updated := subst.typeList(t.TypeArgs().list())
if updated {
return subst.check.newAliasInstance(subst.pos, t.orig, targs, subst.ctxt)
return subst.check.newAliasInstance(subst.pos, t.orig, targs, subst.expanding, subst.ctxt)
}
case *Array:

View File

@ -134,8 +134,8 @@ func (s *_TypeSet) underIs(f func(Type) bool) bool {
}
for _, t := range s.terms {
assert(t.typ != nil)
// x == under(x) for ~x terms
u := t.typ
// Unalias(x) == under(x) for ~x terms
u := Unalias(t.typ)
if !t.tilde {
u = under(u)
}

View File

@ -177,6 +177,15 @@ func TypeOf(a any) *Type {
return (*Type)(NoEscape(unsafe.Pointer(eface.Type)))
}
// TypeFor returns the abi.Type for a type parameter.
func TypeFor[T any]() *Type {
var v T
if t := TypeOf(v); t != nil {
return t // optimize for T being a non-interface kind
}
return TypeOf((*T)(nil)).Elem() // only for an interface kind
}
func (t *Type) Kind() Kind { return t.Kind_ & KindMask }
func (t *Type) HasName() bool {

View File

@ -32,28 +32,46 @@ func SendFile(dstFD *FD, src int, pos, remain int64) (written int64, err error,
if int64(n) > remain {
n = int(remain)
}
m := n
pos1 := pos
n, err = syscall.Sendfile(dst, src, &pos1, n)
if n > 0 {
pos += int64(n)
written += int64(n)
remain -= int64(n)
// (n, nil) indicates that sendfile(2) has transferred
// the exact number of bytes we requested, or some unretryable
// error have occurred with partial bytes sent. Either way, we
// don't need to go through the following logic to check EINTR
// or fell into dstFD.pd.waitWrite, just continue to send the
// next chunk or break the loop.
if n == m {
continue
} else if err != syscall.EAGAIN &&
err != syscall.EINTR &&
err != syscall.EBUSY {
// Particularly, EPIPE. Errors like that would normally lead
// the subsequent sendfile(2) call to (-1, EBADF).
break
}
} else if err != syscall.EAGAIN && err != syscall.EINTR {
// This includes syscall.ENOSYS (no kernel
// support) and syscall.EINVAL (fd types which
// don't implement sendfile), and other errors.
// We should end the loop when there is no error
// returned from sendfile(2) or it is not a retryable error.
break
}
if err == syscall.EINTR {
continue
}
// This includes syscall.ENOSYS (no kernel
// support) and syscall.EINVAL (fd types which
// don't implement sendfile), and other errors.
// We should end the loop when there is no error
// returned from sendfile(2) or it is not a retryable error.
if err != syscall.EAGAIN {
break
}
if err = dstFD.pd.waitWrite(dstFD.isFile); err != nil {
break
}
}
handled = written != 0 || (err != syscall.ENOSYS && err != syscall.EINVAL)
if err == syscall.EAGAIN {
err = nil
}
handled = written != 0 || (err != syscall.ENOSYS && err != syscall.EINVAL && err != syscall.EOPNOTSUPP && err != syscall.ENOTSUP)
return
}

View File

@ -50,6 +50,9 @@ func SendFile(dstFD *FD, src int, remain int64) (written int64, err error, handl
break
}
}
if err == syscall.EAGAIN {
err = nil
}
handled = written != 0 || (err != syscall.ENOSYS && err != syscall.EINVAL)
return
}

View File

@ -61,6 +61,9 @@ func SendFile(dstFD *FD, src int, pos, remain int64) (written int64, err error,
break
}
}
if err == syscall.EAGAIN {
err = nil
}
handled = written != 0 || (err != syscall.ENOSYS && err != syscall.EINVAL)
return
}

View File

@ -14,4 +14,4 @@ type Slice []byte
type Array [8]byte
var s Slice
var p = (Array)(s /* ok because Go 1.20 ignored the //go:build go1.19 */)
var p = (Array)(s /* ok because file versions below go1.21 set the langage version to go1.21 */)

View File

@ -14,4 +14,4 @@ type Slice []byte
type Array [8]byte
var s Slice
var p = (Array)(s /* ERROR "requires go1.20 or later" */)
var p = (Array)(s /* ok because file versions below go1.21 set the langage version to go1.21 */)

View File

@ -0,0 +1,16 @@
// -lang=go1.21
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Check Go language version-specific errors.
//go:build go1.22
package p
func f() {
for _ = range /* ok because of upgrade to 1.22 */ 10 {
}
}

View File

@ -0,0 +1,16 @@
// -lang=go1.22
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Check Go language version-specific errors.
//go:build go1.21
package p
func f() {
for _ = range 10 /* ERROR "requires go1.22 or later" */ {
}
}

View File

@ -1,14 +1,9 @@
// -lang=go1.21
// -lang=go1.13
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Note: Downgrading to go1.13 requires at least go1.21,
// hence the need for -lang=go1.21 at the top.
//go:build go1.13
package p
import "io"

View File

@ -0,0 +1,24 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package p
type A = [4]int
type B = map[string]interface{}
func _[T ~A](x T) {
_ = len(x)
}
func _[U ~A](x U) {
_ = cap(x)
}
func _[V ~A]() {
_ = V{}
}
func _[W ~B](a interface{}) {
_ = a.(W)["key"]
}

View File

@ -0,0 +1,26 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package p
type A = struct {
F string
G int
}
func Make[T ~A]() T {
return T{
F: "blah",
G: 1234,
}
}
type N struct {
F string
G int
}
func _() {
_ = Make[N]()
}

View File

@ -5,9 +5,12 @@
package weak_test
import (
"context"
"internal/weak"
"runtime"
"sync"
"testing"
"time"
)
type T struct {
@ -128,3 +131,82 @@ func TestPointerFinalizer(t *testing.T) {
t.Errorf("weak pointer is non-nil even after finalization: %v", wt)
}
}
// Regression test for issue 69210.
//
// Weak-to-strong conversions must shade the new strong pointer, otherwise
// that might be creating the only strong pointer to a white object which
// is hidden in a blackened stack.
//
// Never fails if correct, fails with some high probability if incorrect.
func TestIssue69210(t *testing.T) {
if testing.Short() {
t.Skip("this is a stress test that takes seconds to run on its own")
}
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
// What we're trying to do is manufacture the conditions under which this
// bug happens. Specifically, we want:
//
// 1. To create a whole bunch of objects that are only weakly-pointed-to,
// 2. To call Strong while the GC is in the mark phase,
// 3. The new strong pointer to be missed by the GC,
// 4. The following GC cycle to mark a free object.
//
// Unfortunately, (2) and (3) are hard to control, but we can increase
// the likelihood by having several goroutines do (1) at once while
// another goroutine constantly keeps us in the GC with runtime.GC.
// Like throwing darts at a dart board until they land just right.
// We can increase the likelihood of (4) by adding some delay after
// creating the strong pointer, but only if it's non-nil. If it's nil,
// that means it was already collected in which case there's no chance
// of triggering the bug, so we want to retry as fast as possible.
// Our heap here is tiny, so the GCs will go by fast.
//
// As of 2024-09-03, removing the line that shades pointers during
// the weak-to-strong conversion causes this test to fail about 50%
// of the time.
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for {
runtime.GC()
select {
case <-ctx.Done():
return
default:
}
}
}()
for range max(runtime.GOMAXPROCS(-1)-1, 1) {
wg.Add(1)
go func() {
defer wg.Done()
for {
for range 5 {
bt := new(T)
wt := weak.Make(bt)
bt = nil
time.Sleep(1 * time.Millisecond)
bt = wt.Strong()
if bt != nil {
time.Sleep(4 * time.Millisecond)
bt.t = bt
bt.a = 12
}
runtime.KeepAlive(bt)
}
select {
case <-ctx.Done():
return
default:
}
}
}()
}
wg.Wait()
}

View File

@ -613,8 +613,9 @@ func (c *Client) do(req *Request) (retres *Response, reterr error) {
reqBodyClosed = false // have we closed the current req.Body?
// Redirect behavior:
redirectMethod string
includeBody bool
redirectMethod string
includeBody = true
stripSensitiveHeaders = false
)
uerr := func(err error) error {
// the body may have been closed already by c.send()
@ -681,7 +682,12 @@ func (c *Client) do(req *Request) (retres *Response, reterr error) {
// in case the user set Referer on their first request.
// If they really want to override, they can do it in
// their CheckRedirect func.
copyHeaders(req)
if !stripSensitiveHeaders && reqs[0].URL.Host != req.URL.Host {
if !shouldCopyHeaderOnRedirect(reqs[0].URL, req.URL) {
stripSensitiveHeaders = true
}
}
copyHeaders(req, stripSensitiveHeaders)
// Add the Referer header from the most recent
// request URL to the new one, if it's not https->http:
@ -744,7 +750,7 @@ func (c *Client) do(req *Request) (retres *Response, reterr error) {
// makeHeadersCopier makes a function that copies headers from the
// initial Request, ireq. For every redirect, this function must be called
// so that it can copy headers into the upcoming Request.
func (c *Client) makeHeadersCopier(ireq *Request) func(*Request) {
func (c *Client) makeHeadersCopier(ireq *Request) func(req *Request, stripSensitiveHeaders bool) {
// The headers to copy are from the very initial request.
// We use a closured callback to keep a reference to these original headers.
var (
@ -758,8 +764,7 @@ func (c *Client) makeHeadersCopier(ireq *Request) func(*Request) {
}
}
preq := ireq // The previous request
return func(req *Request) {
return func(req *Request, stripSensitiveHeaders bool) {
// If Jar is present and there was some initial cookies provided
// via the request header, then we may need to alter the initial
// cookies as we follow redirects since each redirect may end up
@ -796,12 +801,15 @@ func (c *Client) makeHeadersCopier(ireq *Request) func(*Request) {
// Copy the initial request's Header values
// (at least the safe ones).
for k, vv := range ireqhdr {
if shouldCopyHeaderOnRedirect(k, preq.URL, req.URL) {
sensitive := false
switch CanonicalHeaderKey(k) {
case "Authorization", "Www-Authenticate", "Cookie", "Cookie2":
sensitive = true
}
if !(sensitive && stripSensitiveHeaders) {
req.Header[k] = vv
}
}
preq = req // Update previous Request with the current request
}
}
@ -977,28 +985,23 @@ func (b *cancelTimerBody) Close() error {
return err
}
func shouldCopyHeaderOnRedirect(headerKey string, initial, dest *url.URL) bool {
switch CanonicalHeaderKey(headerKey) {
case "Authorization", "Www-Authenticate", "Cookie", "Cookie2":
// Permit sending auth/cookie headers from "foo.com"
// to "sub.foo.com".
func shouldCopyHeaderOnRedirect(initial, dest *url.URL) bool {
// Permit sending auth/cookie headers from "foo.com"
// to "sub.foo.com".
// Note that we don't send all cookies to subdomains
// automatically. This function is only used for
// Cookies set explicitly on the initial outgoing
// client request. Cookies automatically added via the
// CookieJar mechanism continue to follow each
// cookie's scope as set by Set-Cookie. But for
// outgoing requests with the Cookie header set
// directly, we don't know their scope, so we assume
// it's for *.domain.com.
// Note that we don't send all cookies to subdomains
// automatically. This function is only used for
// Cookies set explicitly on the initial outgoing
// client request. Cookies automatically added via the
// CookieJar mechanism continue to follow each
// cookie's scope as set by Set-Cookie. But for
// outgoing requests with the Cookie header set
// directly, we don't know their scope, so we assume
// it's for *.domain.com.
ihost := idnaASCIIFromURL(initial)
dhost := idnaASCIIFromURL(dest)
return isDomainOrSubdomain(dhost, ihost)
}
// All other headers are copied:
return true
ihost := idnaASCIIFromURL(initial)
dhost := idnaASCIIFromURL(dest)
return isDomainOrSubdomain(dhost, ihost)
}
// isDomainOrSubdomain reports whether sub is a subdomain (or exact

View File

@ -1536,6 +1536,55 @@ func testClientCopyHeadersOnRedirect(t *testing.T, mode testMode) {
}
}
// Issue #70530: Once we strip a header on a redirect to a different host,
// the header should stay stripped across any further redirects.
func TestClientStripHeadersOnRepeatedRedirect(t *testing.T) {
run(t, testClientStripHeadersOnRepeatedRedirect)
}
func testClientStripHeadersOnRepeatedRedirect(t *testing.T, mode testMode) {
var proto string
ts := newClientServerTest(t, mode, HandlerFunc(func(w ResponseWriter, r *Request) {
if r.Host+r.URL.Path != "a.example.com/" {
if h := r.Header.Get("Authorization"); h != "" {
t.Errorf("on request to %v%v, Authorization=%q, want no header", r.Host, r.URL.Path, h)
}
}
// Follow a chain of redirects from a to b and back to a.
// The Authorization header is stripped on the first redirect to b,
// and stays stripped even if we're sent back to a.
switch r.Host + r.URL.Path {
case "a.example.com/":
Redirect(w, r, proto+"://b.example.com/", StatusFound)
case "b.example.com/":
Redirect(w, r, proto+"://b.example.com/redirect", StatusFound)
case "b.example.com/redirect":
Redirect(w, r, proto+"://a.example.com/redirect", StatusFound)
case "a.example.com/redirect":
w.Header().Set("X-Done", "true")
default:
t.Errorf("unexpected request to %v", r.URL)
}
})).ts
proto, _, _ = strings.Cut(ts.URL, ":")
c := ts.Client()
c.Transport.(*Transport).Dial = func(_ string, _ string) (net.Conn, error) {
return net.Dial("tcp", ts.Listener.Addr().String())
}
req, _ := NewRequest("GET", proto+"://a.example.com/", nil)
req.Header.Add("Cookie", "foo=bar")
req.Header.Add("Authorization", "secretpassword")
res, err := c.Do(req)
if err != nil {
t.Fatal(err)
}
defer res.Body.Close()
if res.Header.Get("X-Done") != "true" {
t.Fatalf("response missing expected header: X-Done=true")
}
}
// Issue 22233: copy host when Client follows a relative redirect.
func TestClientCopyHostOnRedirect(t *testing.T) { run(t, testClientCopyHostOnRedirect) }
func testClientCopyHostOnRedirect(t *testing.T, mode testMode) {
@ -1702,43 +1751,39 @@ func testClientAltersCookiesOnRedirect(t *testing.T, mode testMode) {
// Part of Issue 4800
func TestShouldCopyHeaderOnRedirect(t *testing.T) {
tests := []struct {
header string
initialURL string
destURL string
want bool
}{
{"User-Agent", "http://foo.com/", "http://bar.com/", true},
{"X-Foo", "http://foo.com/", "http://bar.com/", true},
// Sensitive headers:
{"cookie", "http://foo.com/", "http://bar.com/", false},
{"cookie2", "http://foo.com/", "http://bar.com/", false},
{"authorization", "http://foo.com/", "http://bar.com/", false},
{"authorization", "http://foo.com/", "https://foo.com/", true},
{"authorization", "http://foo.com:1234/", "http://foo.com:4321/", true},
{"www-authenticate", "http://foo.com/", "http://bar.com/", false},
{"authorization", "http://foo.com/", "http://[::1%25.foo.com]/", false},
{"http://foo.com/", "http://bar.com/", false},
{"http://foo.com/", "http://bar.com/", false},
{"http://foo.com/", "http://bar.com/", false},
{"http://foo.com/", "https://foo.com/", true},
{"http://foo.com:1234/", "http://foo.com:4321/", true},
{"http://foo.com/", "http://bar.com/", false},
{"http://foo.com/", "http://[::1%25.foo.com]/", false},
// But subdomains should work:
{"www-authenticate", "http://foo.com/", "http://foo.com/", true},
{"www-authenticate", "http://foo.com/", "http://sub.foo.com/", true},
{"www-authenticate", "http://foo.com/", "http://notfoo.com/", false},
{"www-authenticate", "http://foo.com/", "https://foo.com/", true},
{"www-authenticate", "http://foo.com:80/", "http://foo.com/", true},
{"www-authenticate", "http://foo.com:80/", "http://sub.foo.com/", true},
{"www-authenticate", "http://foo.com:443/", "https://foo.com/", true},
{"www-authenticate", "http://foo.com:443/", "https://sub.foo.com/", true},
{"www-authenticate", "http://foo.com:1234/", "http://foo.com/", true},
{"http://foo.com/", "http://foo.com/", true},
{"http://foo.com/", "http://sub.foo.com/", true},
{"http://foo.com/", "http://notfoo.com/", false},
{"http://foo.com/", "https://foo.com/", true},
{"http://foo.com:80/", "http://foo.com/", true},
{"http://foo.com:80/", "http://sub.foo.com/", true},
{"http://foo.com:443/", "https://foo.com/", true},
{"http://foo.com:443/", "https://sub.foo.com/", true},
{"http://foo.com:1234/", "http://foo.com/", true},
{"authorization", "http://foo.com/", "http://foo.com/", true},
{"authorization", "http://foo.com/", "http://sub.foo.com/", true},
{"authorization", "http://foo.com/", "http://notfoo.com/", false},
{"authorization", "http://foo.com/", "https://foo.com/", true},
{"authorization", "http://foo.com:80/", "http://foo.com/", true},
{"authorization", "http://foo.com:80/", "http://sub.foo.com/", true},
{"authorization", "http://foo.com:443/", "https://foo.com/", true},
{"authorization", "http://foo.com:443/", "https://sub.foo.com/", true},
{"authorization", "http://foo.com:1234/", "http://foo.com/", true},
{"http://foo.com/", "http://foo.com/", true},
{"http://foo.com/", "http://sub.foo.com/", true},
{"http://foo.com/", "http://notfoo.com/", false},
{"http://foo.com/", "https://foo.com/", true},
{"http://foo.com:80/", "http://foo.com/", true},
{"http://foo.com:80/", "http://sub.foo.com/", true},
{"http://foo.com:443/", "https://foo.com/", true},
{"http://foo.com:443/", "https://sub.foo.com/", true},
{"http://foo.com:1234/", "http://foo.com/", true},
}
for i, tt := range tests {
u0, err := url.Parse(tt.initialURL)
@ -1751,10 +1796,10 @@ func TestShouldCopyHeaderOnRedirect(t *testing.T) {
t.Errorf("%d. dest URL %q parse error: %v", i, tt.destURL, err)
continue
}
got := Export_shouldCopyHeaderOnRedirect(tt.header, u0, u1)
got := Export_shouldCopyHeaderOnRedirect(u0, u1)
if got != tt.want {
t.Errorf("%d. shouldCopyHeaderOnRedirect(%q, %q => %q) = %v; want %v",
i, tt.header, tt.initialURL, tt.destURL, got, tt.want)
t.Errorf("%d. shouldCopyHeaderOnRedirect(%q => %q) = %v; want %v",
i, tt.initialURL, tt.destURL, got, tt.want)
}
}
}

View File

@ -10,56 +10,56 @@ import "strings"
// LocalhostCert is a PEM-encoded TLS cert with SAN IPs
// "127.0.0.1" and "[::1]", expiring at Jan 29 16:00:00 2084 GMT.
// generated from src/crypto/tls:
// go run generate_cert.go --rsa-bits 2048 --host 127.0.0.1,::1,example.com --ca --start-date "Jan 1 00:00:00 1970" --duration=1000000h
// go run generate_cert.go --rsa-bits 2048 --host 127.0.0.1,::1,example.com,*.example.com --ca --start-date "Jan 1 00:00:00 1970" --duration=1000000h
var LocalhostCert = []byte(`-----BEGIN CERTIFICATE-----
MIIDOTCCAiGgAwIBAgIQSRJrEpBGFc7tNb1fb5pKFzANBgkqhkiG9w0BAQsFADAS
MIIDSDCCAjCgAwIBAgIQEP/md970HysdBTpuzDOf0DANBgkqhkiG9w0BAQsFADAS
MRAwDgYDVQQKEwdBY21lIENvMCAXDTcwMDEwMTAwMDAwMFoYDzIwODQwMTI5MTYw
MDAwWjASMRAwDgYDVQQKEwdBY21lIENvMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
MIIBCgKCAQEA6Gba5tHV1dAKouAaXO3/ebDUU4rvwCUg/CNaJ2PT5xLD4N1Vcb8r
bFSW2HXKq+MPfVdwIKR/1DczEoAGf/JWQTW7EgzlXrCd3rlajEX2D73faWJekD0U
aUgz5vtrTXZ90BQL7WvRICd7FlEZ6FPOcPlumiyNmzUqtwGhO+9ad1W5BqJaRI6P
YfouNkwR6Na4TzSj5BrqUfP0FwDizKSJ0XXmh8g8G9mtwxOSN3Ru1QFc61Xyeluk
POGKBV/q6RBNklTNe0gI8usUMlYyoC7ytppNMW7X2vodAelSu25jgx2anj9fDVZu
h7AXF5+4nJS4AAt0n1lNY7nGSsdZas8PbQIDAQABo4GIMIGFMA4GA1UdDwEB/wQE
MIIBCgKCAQEAxcl69ROJdxjN+MJZnbFrYxyQooADCsJ6VDkuMyNQIix/Hk15Nk/u
FyBX1Me++aEpGmY3RIY4fUvELqT/srvAHsTXwVVSttMcY8pcAFmXSqo3x4MuUTG/
jCX3Vftj0r3EM5M8ImY1rzA/jqTTLJg00rD+DmuDABcqQvoXw/RV8w1yTRi5BPoH
DFD/AWTt/YgMvk1l2Yq/xI8VbMUIpjBoGXxWsSevQ5i2s1mk9/yZzu0Ysp1tTlzD
qOPa4ysFjBitdXiwfxjxtv5nXqOCP5rheKO0sWLk0fetMp1OV5JSJMAJw6c2ZMkl
U2WMqAEpRjdE/vHfIuNg+yGaRRqI07NZRQIDAQABo4GXMIGUMA4GA1UdDwEB/wQE
AwICpDATBgNVHSUEDDAKBggrBgEFBQcDATAPBgNVHRMBAf8EBTADAQH/MB0GA1Ud
DgQWBBStsdjh3/JCXXYlQryOrL4Sh7BW5TAuBgNVHREEJzAlggtleGFtcGxlLmNv
bYcEfwAAAYcQAAAAAAAAAAAAAAAAAAAAATANBgkqhkiG9w0BAQsFAAOCAQEAxWGI
5NhpF3nwwy/4yB4i/CwwSpLrWUa70NyhvprUBC50PxiXav1TeDzwzLx/o5HyNwsv
cxv3HdkLW59i/0SlJSrNnWdfZ19oTcS+6PtLoVyISgtyN6DpkKpdG1cOkW3Cy2P2
+tK/tKHRP1Y/Ra0RiDpOAmqn0gCOFGz8+lqDIor/T7MTpibL3IxqWfPrvfVRHL3B
grw/ZQTTIVjjh4JBSW3WyWgNo/ikC1lrVxzl4iPUGptxT36Cr7Zk2Bsg0XqwbOvK
5d+NTDREkSnUbie4GeutujmX3Dsx88UiV6UY/4lHJa6I5leHUNOHahRbpbWeOfs/
WkBKOclmOV2xlTVuPw==
DgQWBBQR5QIzmacmw78ZI1C4MXw7Q0wJ1jA9BgNVHREENjA0ggtleGFtcGxlLmNv
bYINKi5leGFtcGxlLmNvbYcEfwAAAYcQAAAAAAAAAAAAAAAAAAAAATANBgkqhkiG
9w0BAQsFAAOCAQEACrRNgiioUDzxQftd0fwOa6iRRcPampZRDtuaF68yNHoNWbOu
LUwc05eOWxRq3iABGSk2xg+FXM3DDeW4HhAhCFptq7jbVZ+4Jj6HeJG9mYRatAxR
Y/dEpa0D0EHhDxxVg6UzKOXB355n0IetGE/aWvyTV9SiDs6QsaC57Q9qq1/mitx5
2GFBoapol9L5FxCc77bztzK8CpLujkBi25Vk6GAFbl27opLfpyxkM+rX/T6MXCPO
6/YBacNZ7ff1/57Etg4i5mNA6ubCpuc4Gi9oYqCNNohftr2lkJr7REdDR6OW0lsL
rF7r4gUnKeC7mYIH1zypY7laskopiLFAfe96Kg==
-----END CERTIFICATE-----`)
// LocalhostKey is the private key for LocalhostCert.
var LocalhostKey = []byte(testingKey(`-----BEGIN RSA TESTING KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDoZtrm0dXV0Aqi
4Bpc7f95sNRTiu/AJSD8I1onY9PnEsPg3VVxvytsVJbYdcqr4w99V3AgpH/UNzMS
gAZ/8lZBNbsSDOVesJ3euVqMRfYPvd9pYl6QPRRpSDPm+2tNdn3QFAvta9EgJ3sW
URnoU85w+W6aLI2bNSq3AaE771p3VbkGolpEjo9h+i42TBHo1rhPNKPkGupR8/QX
AOLMpInRdeaHyDwb2a3DE5I3dG7VAVzrVfJ6W6Q84YoFX+rpEE2SVM17SAjy6xQy
VjKgLvK2mk0xbtfa+h0B6VK7bmODHZqeP18NVm6HsBcXn7iclLgAC3SfWU1jucZK
x1lqzw9tAgMBAAECggEABWzxS1Y2wckblnXY57Z+sl6YdmLV+gxj2r8Qib7g4ZIk
lIlWR1OJNfw7kU4eryib4fc6nOh6O4AWZyYqAK6tqNQSS/eVG0LQTLTTEldHyVJL
dvBe+MsUQOj4nTndZW+QvFzbcm2D8lY5n2nBSxU5ypVoKZ1EqQzytFcLZpTN7d89
EPj0qDyrV4NZlWAwL1AygCwnlwhMQjXEalVF1ylXwU3QzyZ/6MgvF6d3SSUlh+sq
XefuyigXw484cQQgbzopv6niMOmGP3of+yV4JQqUSb3IDmmT68XjGd2Dkxl4iPki
6ZwXf3CCi+c+i/zVEcufgZ3SLf8D99kUGE7v7fZ6AQKBgQD1ZX3RAla9hIhxCf+O
3D+I1j2LMrdjAh0ZKKqwMR4JnHX3mjQI6LwqIctPWTU8wYFECSh9klEclSdCa64s
uI/GNpcqPXejd0cAAdqHEEeG5sHMDt0oFSurL4lyud0GtZvwlzLuwEweuDtvT9cJ
Wfvl86uyO36IW8JdvUprYDctrQKBgQDycZ697qutBieZlGkHpnYWUAeImVA878sJ
w44NuXHvMxBPz+lbJGAg8Cn8fcxNAPqHIraK+kx3po8cZGQywKHUWsxi23ozHoxo
+bGqeQb9U661TnfdDspIXia+xilZt3mm5BPzOUuRqlh4Y9SOBpSWRmEhyw76w4ZP
OPxjWYAgwQKBgA/FehSYxeJgRjSdo+MWnK66tjHgDJE8bYpUZsP0JC4R9DL5oiaA
brd2fI6Y+SbyeNBallObt8LSgzdtnEAbjIH8uDJqyOmknNePRvAvR6mP4xyuR+Bv
m+Lgp0DMWTw5J9CKpydZDItc49T/mJ5tPhdFVd+am0NAQnmr1MCZ6nHxAoGABS3Y
LkaC9FdFUUqSU8+Chkd/YbOkuyiENdkvl6t2e52jo5DVc1T7mLiIrRQi4SI8N9bN
/3oJWCT+uaSLX2ouCtNFunblzWHBrhxnZzTeqVq4SLc8aESAnbslKL4i8/+vYZlN
s8xtiNcSvL+lMsOBORSXzpj/4Ot8WwTkn1qyGgECgYBKNTypzAHeLE6yVadFp3nQ
Ckq9yzvP/ib05rvgbvrne00YeOxqJ9gtTrzgh7koqJyX1L4NwdkEza4ilDWpucn0
xiUZS4SoaJq6ZvcBYS62Yr1t8n09iG47YL8ibgtmH3L+svaotvpVxVK+d7BLevA/
ZboOWVe3icTy64BT3OQhmg==
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDFyXr1E4l3GM34
wlmdsWtjHJCigAMKwnpUOS4zI1AiLH8eTXk2T+4XIFfUx775oSkaZjdEhjh9S8Qu
pP+yu8AexNfBVVK20xxjylwAWZdKqjfHgy5RMb+MJfdV+2PSvcQzkzwiZjWvMD+O
pNMsmDTSsP4Oa4MAFypC+hfD9FXzDXJNGLkE+gcMUP8BZO39iAy+TWXZir/EjxVs
xQimMGgZfFaxJ69DmLazWaT3/JnO7RiynW1OXMOo49rjKwWMGK11eLB/GPG2/mde
o4I/muF4o7SxYuTR960ynU5XklIkwAnDpzZkySVTZYyoASlGN0T+8d8i42D7IZpF
GojTs1lFAgMBAAECggEAIYthUi1lFBDd5gG4Rzlu+BlBIn5JhcqkCqLEBiJIFfOr
/4yuMRrvS3bNzqWt6xJ9MSAC4ZlN/VobRLnxL/QNymoiGYUKCT3Ww8nvPpPzR9OE
sE68TUL9tJw/zZJcRMKwgvrGqSLimfq53MxxkE+kLdOc0v9C8YH8Re26mB5ZcWYa
7YFyZQpKsQYnsmu/05cMbpOQrQWhtmIqRoyn8mG/par2s3NzjtpSE9NINyz26uFc
k/3ovFJQIHkUmTS7KHD3BgY5vuCqP98HramYnOysJ0WoYgvSDNCWw3037s5CCwJT
gCKuM+Ow6liFrj83RrdKBpm5QUGjfNpYP31o+QNP4QKBgQDSrUQ2XdgtAnibAV7u
7kbxOxro0EhIKso0Y/6LbDQgcXgxLqltkmeqZgG8nC3Z793lhlSasz2snhzzooV5
5fTy1y8ikXqjhG0nNkInFyOhsI0auE28CFoDowaQd+5cmCatpN4Grqo5PNRXxm1w
HktfPEgoP11NNCFHvvN5fEKbbQKBgQDwVlOaV20IvW3IPq7cXZyiyabouFF9eTRo
VJka1Uv+JtyvL2P0NKkjYHOdN8gRblWqxQtJoTNk020rVA4UP1heiXALy50gvj/p
hMcybPTLYSPOhAGx838KIcvGR5oskP1aUCmFbFQzGELxhJ9diVVjxUtbG2DuwPKd
tD9TLxT2OQKBgQCcdlHSjp+dzdgERmBa0ludjGfPv9/uuNizUBAbO6D690psPFtY
JQMYaemgSd1DngEOFVWADt4e9M5Lose+YCoqr+UxpxmNlyv5kzJOFcFAs/4XeglB
PHKdgNW/NVKxMc6H54l9LPr+x05sYdGlEtqnP/3W5jhEvhJ5Vjc8YiyVgQKBgQCl
zwjyrGo+42GACy7cPYE5FeIfIDqoVByB9guC5bD98JXEDu/opQQjsgFRcBCJZhOY
M0UsURiB8ROaFu13rpQq9KrmmF0ZH+g8FSzQbzcbsTLg4VXCDXmR5esOKowFPypr
Sm667BfTAGP++D5ya7MLmCv6+RKQ5XD8uEQQAaV2kQKBgAD8qeJuWIXZT0VKkQrn
nIhgtzGERF/6sZdQGW2LxTbUDWG74AfFkkEbeBfwEkCZXY/xmnYqYABhvlSex8jU
supU6Eea21esIxIub2zv/Np0ojUb6rlqTPS4Ox1E27D787EJ3VOXpriSD10vyNnZ
jel6uj2FOP9g54s+GzlSVg/T
-----END RSA TESTING KEY-----`))
func testingKey(s string) string { return strings.ReplaceAll(s, "TESTING KEY", "PRIVATE KEY") }

View File

@ -53,6 +53,9 @@ func sendFile(c *netFD, r io.Reader) (written int64, err error, handled bool) {
if err != nil {
return 0, err, false
}
if fi.Mode()&(fs.ModeSymlink|fs.ModeDevice|fs.ModeCharDevice|fs.ModeIrregular) != 0 {
return 0, nil, false
}
remain = fi.Size()
}

View File

@ -0,0 +1,86 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build unix
package net
import (
"internal/testpty"
"io"
"os"
"sync"
"syscall"
"testing"
)
// Issue 70763: test that we don't fail on sendfile from a tty.
func TestCopyFromTTY(t *testing.T) {
pty, ttyName, err := testpty.Open()
if err != nil {
t.Skipf("skipping test because pty open failed: %v", err)
}
defer pty.Close()
// Use syscall.Open so that the tty is blocking.
ttyFD, err := syscall.Open(ttyName, syscall.O_RDWR, 0)
if err != nil {
t.Skipf("skipping test because tty open failed: %v", err)
}
defer syscall.Close(ttyFD)
tty := os.NewFile(uintptr(ttyFD), "tty")
defer tty.Close()
ln := newLocalListener(t, "tcp")
defer ln.Close()
ch := make(chan bool)
const data = "data\n"
var wg sync.WaitGroup
defer wg.Wait()
wg.Add(1)
go func() {
defer wg.Done()
conn, err := ln.Accept()
if err != nil {
t.Error(err)
return
}
defer conn.Close()
buf := make([]byte, len(data))
if _, err := io.ReadFull(conn, buf); err != nil {
t.Error(err)
}
ch <- true
}()
conn, err := Dial("tcp", ln.Addr().String())
if err != nil {
t.Fatal(err)
}
defer conn.Close()
wg.Add(1)
go func() {
defer wg.Done()
if _, err := pty.Write([]byte(data)); err != nil {
t.Error(err)
}
<-ch
if err := pty.Close(); err != nil {
t.Error(err)
}
}()
lr := io.LimitReader(tty, int64(len(data)))
if _, err := io.Copy(conn, lr); err != nil {
t.Error(err)
}
}

154
src/os/copy_test.go Normal file
View File

@ -0,0 +1,154 @@
// Copyright 2024 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package os_test
import (
"bytes"
"errors"
"io"
"math/rand/v2"
"net"
"os"
"runtime"
"sync"
"testing"
"golang.org/x/net/nettest"
)
// Exercise sendfile/splice fast paths with a moderately large file.
//
// https://go.dev/issue/70000
func TestLargeCopyViaNetwork(t *testing.T) {
const size = 10 * 1024 * 1024
dir := t.TempDir()
src, err := os.Create(dir + "/src")
if err != nil {
t.Fatal(err)
}
defer src.Close()
if _, err := io.CopyN(src, newRandReader(), size); err != nil {
t.Fatal(err)
}
if _, err := src.Seek(0, 0); err != nil {
t.Fatal(err)
}
dst, err := os.Create(dir + "/dst")
if err != nil {
t.Fatal(err)
}
defer dst.Close()
client, server := createSocketPair(t, "tcp")
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
if n, err := io.Copy(dst, server); n != size || err != nil {
t.Errorf("copy to destination = %v, %v; want %v, nil", n, err, size)
}
}()
go func() {
defer wg.Done()
defer client.Close()
if n, err := io.Copy(client, src); n != size || err != nil {
t.Errorf("copy from source = %v, %v; want %v, nil", n, err, size)
}
}()
wg.Wait()
if _, err := dst.Seek(0, 0); err != nil {
t.Fatal(err)
}
if err := compareReaders(dst, io.LimitReader(newRandReader(), size)); err != nil {
t.Fatal(err)
}
}
func compareReaders(a, b io.Reader) error {
bufa := make([]byte, 4096)
bufb := make([]byte, 4096)
for {
na, erra := io.ReadFull(a, bufa)
if erra != nil && erra != io.EOF {
return erra
}
nb, errb := io.ReadFull(b, bufb)
if errb != nil && errb != io.EOF {
return errb
}
if !bytes.Equal(bufa[:na], bufb[:nb]) {
return errors.New("contents mismatch")
}
if erra == io.EOF && errb == io.EOF {
break
}
}
return nil
}
type randReader struct {
rand *rand.Rand
}
func newRandReader() *randReader {
return &randReader{rand.New(rand.NewPCG(0, 0))}
}
func (r *randReader) Read(p []byte) (int, error) {
var v uint64
var n int
for i := range p {
if n == 0 {
v = r.rand.Uint64()
n = 8
}
p[i] = byte(v & 0xff)
v >>= 8
n--
}
return len(p), nil
}
func createSocketPair(t *testing.T, proto string) (client, server net.Conn) {
t.Helper()
if !nettest.TestableNetwork(proto) {
t.Skipf("%s does not support %q", runtime.GOOS, proto)
}
ln, err := nettest.NewLocalListener(proto)
if err != nil {
t.Fatalf("NewLocalListener error: %v", err)
}
t.Cleanup(func() {
if ln != nil {
ln.Close()
}
if client != nil {
client.Close()
}
if server != nil {
server.Close()
}
})
ch := make(chan struct{})
go func() {
var err error
server, err = ln.Accept()
if err != nil {
t.Errorf("Accept new connection error: %v", err)
}
ch <- struct{}{}
}()
client, err = net.Dial(proto, ln.Addr().String())
<-ch
if err != nil {
t.Fatalf("Dial new connection error: %v", err)
}
return client, server
}

View File

@ -132,15 +132,18 @@ func ReadDir(name string) ([]DirEntry, error) {
// CopyFS copies the file system fsys into the directory dir,
// creating dir if necessary.
//
// Newly created directories and files have their default modes
// where any bits from the file in fsys that are not part of the
// standard read, write, and execute permissions will be zeroed
// out, and standard read and write permissions are set for owner,
// group, and others while retaining any existing execute bits from
// the file in fsys.
// Files are created with mode 0o666 plus any execute permissions
// from the source, and directories are created with mode 0o777
// (before umask).
//
// Symbolic links in fsys are not supported, a *PathError with Err set
// to ErrInvalid is returned on symlink.
// CopyFS will not overwrite existing files. If a file name in fsys
// already exists in the destination, CopyFS will return an error
// such that errors.Is(err, fs.ErrExist) will be true.
//
// Symbolic links in fsys are not supported. A *PathError with Err set
// to ErrInvalid is returned when copying from a symbolic link.
//
// Symbolic links in dir are followed.
//
// Copying stops at and returns the first error encountered.
func CopyFS(dir string, fsys fs.FS) error {
@ -174,7 +177,7 @@ func CopyFS(dir string, fsys fs.FS) error {
if err != nil {
return err
}
w, err := OpenFile(newPath, O_CREATE|O_TRUNC|O_WRONLY, 0666|info.Mode()&0777)
w, err := OpenFile(newPath, O_CREATE|O_EXCL|O_WRONLY, 0666|info.Mode()&0777)
if err != nil {
return err
}

View File

@ -61,7 +61,7 @@ func ExampleFileMode() {
log.Fatal(err)
}
fmt.Printf("permissions: %#o\n", fi.Mode().Perm()) // 0400, 0777, etc.
fmt.Printf("permissions: %#o\n", fi.Mode().Perm()) // 0o400, 0o777, etc.
switch mode := fi.Mode(); {
case mode.IsRegular():
fmt.Println("regular file")

View File

@ -35,10 +35,11 @@ func startProcess(name string, argv []string, attr *ProcAttr) (p *Process, err e
}
}
attrSys, shouldDupPidfd := ensurePidfd(attr.Sys)
sysattr := &syscall.ProcAttr{
Dir: attr.Dir,
Env: attr.Env,
Sys: ensurePidfd(attr.Sys),
Sys: attrSys,
}
if sysattr.Env == nil {
sysattr.Env, err = execenv.Default(sysattr.Sys)
@ -63,7 +64,7 @@ func startProcess(name string, argv []string, attr *ProcAttr) (p *Process, err e
// For Windows, syscall.StartProcess above already returned a process handle.
if runtime.GOOS != "windows" {
var ok bool
h, ok = getPidfd(sysattr.Sys)
h, ok = getPidfd(sysattr.Sys, shouldDupPidfd)
if !ok {
return newPIDProcess(pid), nil
}

View File

@ -366,7 +366,7 @@ func Open(name string) (*File, error) {
}
// Create creates or truncates the named file. If the file already exists,
// it is truncated. If the file does not exist, it is created with mode 0666
// it is truncated. If the file does not exist, it is created with mode 0o666
// (before umask). If successful, methods on the returned File can
// be used for I/O; the associated file descriptor has mode O_RDWR.
// If there is an error, it will be of type *PathError.
@ -602,11 +602,11 @@ func UserHomeDir() (string, error) {
// On Unix, the mode's permission bits, ModeSetuid, ModeSetgid, and
// ModeSticky are used.
//
// On Windows, only the 0200 bit (owner writable) of mode is used; it
// On Windows, only the 0o200 bit (owner writable) of mode is used; it
// controls whether the file's read-only attribute is set or cleared.
// The other bits are currently unused. For compatibility with Go 1.12
// and earlier, use a non-zero mode. Use mode 0400 for a read-only
// file and 0600 for a readable+writable file.
// and earlier, use a non-zero mode. Use mode 0o400 for a read-only
// file and 0o600 for a readable+writable file.
//
// On Plan 9, the mode's permission bits, ModeAppend, ModeExclusive,
// and ModeTemporary are used.

View File

@ -1376,8 +1376,7 @@ func TestChtimes(t *testing.T) {
t.Parallel()
f := newFile(t)
f.Write([]byte("hello, world\n"))
// This should be an empty file (see #68687, #68663).
f.Close()
testChtimes(t, f.Name())
@ -1395,12 +1394,9 @@ func TestChtimesOmit(t *testing.T) {
func testChtimesOmit(t *testing.T, omitAt, omitMt bool) {
t.Logf("omit atime: %v, mtime: %v", omitAt, omitMt)
file := newFile(t)
_, err := file.Write([]byte("hello, world\n"))
if err != nil {
t.Fatal(err)
}
// This should be an empty file (see #68687, #68663).
name := file.Name()
err = file.Close()
err := file.Close()
if err != nil {
t.Error(err)
}
@ -3358,6 +3354,14 @@ func TestCopyFS(t *testing.T) {
t.Fatal("comparing two directories:", err)
}
// Test whether CopyFS disallows copying for disk filesystem when there is any
// existing file in the destination directory.
if err := CopyFS(tmpDir, fsys); !errors.Is(err, fs.ErrExist) {
t.Errorf("CopyFS should have failed and returned error when there is"+
"any existing file in the destination directory (in disk filesystem), "+
"got: %v, expected any error that indicates <file exists>", err)
}
// Test with memory filesystem.
fsys = fstest.MapFS{
"william": {Data: []byte("Shakespeare\n")},
@ -3395,6 +3399,14 @@ func TestCopyFS(t *testing.T) {
}); err != nil {
t.Fatal("comparing two directories:", err)
}
// Test whether CopyFS disallows copying for memory filesystem when there is any
// existing file in the destination directory.
if err := CopyFS(tmpDir, fsys); !errors.Is(err, fs.ErrExist) {
t.Errorf("CopyFS should have failed and returned error when there is"+
"any existing file in the destination directory (in memory filesystem), "+
"got: %v, expected any error that indicates <file exists>", err)
}
}
func TestCopyFSWithSymlinks(t *testing.T) {

View File

@ -8,20 +8,28 @@
// v5.3: pidfd_open syscall, clone3 syscall;
// v5.4: P_PIDFD idtype support for waitid syscall;
// v5.6: pidfd_getfd syscall.
//
// N.B. Alternative Linux implementations may not follow this ordering. e.g.,
// QEMU user mode 7.2 added pidfd_open, but CLONE_PIDFD was not added until
// 8.0.
package os
import (
"errors"
"internal/syscall/unix"
"runtime"
"sync"
"syscall"
"unsafe"
)
func ensurePidfd(sysAttr *syscall.SysProcAttr) *syscall.SysProcAttr {
// ensurePidfd initializes the PidFD field in sysAttr if it is not already set.
// It returns the original or modified SysProcAttr struct and a flag indicating
// whether the PidFD should be duplicated before using.
func ensurePidfd(sysAttr *syscall.SysProcAttr) (*syscall.SysProcAttr, bool) {
if !pidfdWorks() {
return sysAttr
return sysAttr, false
}
var pidfd int
@ -29,23 +37,33 @@ func ensurePidfd(sysAttr *syscall.SysProcAttr) *syscall.SysProcAttr {
if sysAttr == nil {
return &syscall.SysProcAttr{
PidFD: &pidfd,
}
}, false
}
if sysAttr.PidFD == nil {
newSys := *sysAttr // copy
newSys.PidFD = &pidfd
return &newSys
return &newSys, false
}
return sysAttr
return sysAttr, true
}
func getPidfd(sysAttr *syscall.SysProcAttr) (uintptr, bool) {
// getPidfd returns the value of sysAttr.PidFD (or its duplicate if needDup is
// set) and a flag indicating whether the value can be used.
func getPidfd(sysAttr *syscall.SysProcAttr, needDup bool) (uintptr, bool) {
if !pidfdWorks() {
return 0, false
}
return uintptr(*sysAttr.PidFD), true
h := *sysAttr.PidFD
if needDup {
dupH, e := unix.Fcntl(h, syscall.F_DUPFD_CLOEXEC, 0)
if e != nil {
return 0, false
}
h = dupH
}
return uintptr(h), true
}
func pidfdFind(pid int) (uintptr, error) {
@ -126,14 +144,21 @@ func pidfdWorks() bool {
var checkPidfdOnce = sync.OnceValue(checkPidfd)
// checkPidfd checks whether all required pidfd-related syscalls work.
// This consists of pidfd_open and pidfd_send_signal syscalls, and waitid
// syscall with idtype of P_PIDFD.
// checkPidfd checks whether all required pidfd-related syscalls work. This
// consists of pidfd_open and pidfd_send_signal syscalls, waitid syscall with
// idtype of P_PIDFD, and clone(CLONE_PIDFD).
//
// Reasons for non-working pidfd syscalls include an older kernel and an
// execution environment in which the above system calls are restricted by
// seccomp or a similar technology.
func checkPidfd() error {
// In Android version < 12, pidfd-related system calls are not allowed
// by seccomp and trigger the SIGSYS signal. See issue #69065.
if runtime.GOOS == "android" {
ignoreSIGSYS()
defer restoreSIGSYS()
}
// Get a pidfd of the current process (opening of "/proc/self" won't
// work for waitid).
fd, err := unix.PidFDOpen(syscall.Getpid(), 0)
@ -159,5 +184,27 @@ func checkPidfd() error {
return NewSyscallError("pidfd_send_signal", err)
}
// Verify that clone(CLONE_PIDFD) works.
//
// This shouldn't be necessary since pidfd_open was added in Linux 5.3,
// after CLONE_PIDFD in Linux 5.2, but some alternative Linux
// implementations may not adhere to this ordering.
if err := checkClonePidfd(); err != nil {
return err
}
return nil
}
// Provided by syscall.
//
//go:linkname checkClonePidfd
func checkClonePidfd() error
// Provided by runtime.
//
//go:linkname ignoreSIGSYS
func ignoreSIGSYS()
//go:linkname restoreSIGSYS
func restoreSIGSYS()

View File

@ -6,8 +6,10 @@ package os_test
import (
"errors"
"internal/syscall/unix"
"internal/testenv"
"os"
"os/exec"
"syscall"
"testing"
)
@ -57,3 +59,93 @@ func TestFindProcessViaPidfd(t *testing.T) {
t.Fatalf("Release: got %v, want <nil>", err)
}
}
func TestStartProcessWithPidfd(t *testing.T) {
testenv.MustHaveGoBuild(t)
t.Parallel()
if err := os.CheckPidfdOnce(); err != nil {
// Non-pidfd code paths tested in exec_unix_test.go.
t.Skipf("skipping: pidfd not available: %v", err)
}
var pidfd int
p, err := os.StartProcess(testenv.GoToolPath(t), []string{"go"}, &os.ProcAttr{
Sys: &syscall.SysProcAttr{
PidFD: &pidfd,
},
})
if err != nil {
t.Fatalf("starting test process: %v", err)
}
defer syscall.Close(pidfd)
if _, err := p.Wait(); err != nil {
t.Fatalf("Wait: got %v, want <nil>", err)
}
// Check the pidfd is still valid
err = unix.PidFDSendSignal(uintptr(pidfd), syscall.Signal(0))
if !errors.Is(err, syscall.ESRCH) {
t.Errorf("SendSignal: got %v, want %v", err, syscall.ESRCH)
}
}
// Issue #69284
func TestPidfdLeak(t *testing.T) {
testenv.MustHaveExec(t)
exe, err := os.Executable()
if err != nil {
t.Fatal(err)
}
// Find the next 10 descriptors.
// We need to get more than one descriptor in practice;
// the pidfd winds up not being the next descriptor.
const count = 10
want := make([]int, count)
for i := range count {
var err error
want[i], err = syscall.Open(exe, syscall.O_RDONLY, 0)
if err != nil {
t.Fatal(err)
}
}
// Close the descriptors.
for _, d := range want {
syscall.Close(d)
}
// Start a process 10 times.
for range 10 {
// For testing purposes this has to be an absolute path.
// Otherwise we will fail finding the executable
// and won't start a process at all.
cmd := exec.Command("/noSuchExecutable")
cmd.Run()
}
// Open the next 10 descriptors again.
got := make([]int, count)
for i := range count {
var err error
got[i], err = syscall.Open(exe, syscall.O_RDONLY, 0)
if err != nil {
t.Fatal(err)
}
}
// Close the descriptors
for _, d := range got {
syscall.Close(d)
}
t.Logf("got %v", got)
t.Logf("want %v", want)
// Allow some slack for runtime epoll descriptors and the like.
if got[count-1] > want[count-1]+5 {
t.Errorf("got descriptor %d, want %d", got[count-1], want[count-1])
}
}

View File

@ -8,11 +8,11 @@ package os
import "syscall"
func ensurePidfd(sysAttr *syscall.SysProcAttr) *syscall.SysProcAttr {
return sysAttr
func ensurePidfd(sysAttr *syscall.SysProcAttr) (*syscall.SysProcAttr, bool) {
return sysAttr, false
}
func getPidfd(_ *syscall.SysProcAttr) (uintptr, bool) {
func getPidfd(_ *syscall.SysProcAttr, _ bool) (uintptr, bool) {
return 0, false
}

View File

@ -14,15 +14,12 @@ import (
"net"
. "os"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"syscall"
"testing"
"time"
"golang.org/x/net/nettest"
)
func TestCopyFileRange(t *testing.T) {
@ -784,41 +781,3 @@ func testGetPollFDAndNetwork(t *testing.T, proto string) {
t.Fatalf("server Control error: %v", err)
}
}
func createSocketPair(t *testing.T, proto string) (client, server net.Conn) {
t.Helper()
if !nettest.TestableNetwork(proto) {
t.Skipf("%s does not support %q", runtime.GOOS, proto)
}
ln, err := nettest.NewLocalListener(proto)
if err != nil {
t.Fatalf("NewLocalListener error: %v", err)
}
t.Cleanup(func() {
if ln != nil {
ln.Close()
}
if client != nil {
client.Close()
}
if server != nil {
server.Close()
}
})
ch := make(chan struct{})
go func() {
var err error
server, err = ln.Accept()
if err != nil {
t.Errorf("Accept new connection error: %v", err)
}
ch <- struct{}{}
}()
client, err = net.Dial(proto, ln.Addr().String())
<-ch
if err != nil {
t.Fatalf("Dial new connection error: %v", err)
}
return client, server
}

View File

@ -31,10 +31,11 @@ x_cgo_getstackbound(uintptr bounds[2])
pthread_attr_get_np(pthread_self(), &attr);
pthread_attr_getstack(&attr, &addr, &size); // low address
#else
// We don't know how to get the current stacks, so assume they are the
// same as the default stack bounds.
pthread_attr_getstacksize(&attr, &size);
addr = __builtin_frame_address(0) + 4096 - size;
// We don't know how to get the current stacks, leave it as
// 0 and the caller will use an estimate based on the current
// SP.
addr = 0;
size = 0;
#endif
pthread_attr_destroy(&attr);

View File

@ -231,34 +231,6 @@ func cgocall(fn, arg unsafe.Pointer) int32 {
func callbackUpdateSystemStack(mp *m, sp uintptr, signal bool) {
g0 := mp.g0
inBound := sp > g0.stack.lo && sp <= g0.stack.hi
if mp.ncgo > 0 && !inBound {
// ncgo > 0 indicates that this M was in Go further up the stack
// (it called C and is now receiving a callback).
//
// !inBound indicates that we were called with SP outside the
// expected system stack bounds (C changed the stack out from
// under us between the cgocall and cgocallback?).
//
// It is not safe for the C call to change the stack out from
// under us, so throw.
// Note that this case isn't possible for signal == true, as
// that is always passing a new M from needm.
// Stack is bogus, but reset the bounds anyway so we can print.
hi := g0.stack.hi
lo := g0.stack.lo
g0.stack.hi = sp + 1024
g0.stack.lo = sp - 32*1024
g0.stackguard0 = g0.stack.lo + stackGuard
g0.stackguard1 = g0.stackguard0
print("M ", mp.id, " procid ", mp.procid, " runtime: cgocallback with sp=", hex(sp), " out of bounds [", hex(lo), ", ", hex(hi), "]")
print("\n")
exit(2)
}
if !mp.isextra {
// We allocated the stack for standard Ms. Don't replace the
// stack bounds with estimated ones when we already initialized
@ -266,26 +238,37 @@ func callbackUpdateSystemStack(mp *m, sp uintptr, signal bool) {
return
}
// This M does not have Go further up the stack. However, it may have
// previously called into Go, initializing the stack bounds. Between
// that call returning and now the stack may have changed (perhaps the
// C thread is running a coroutine library). We need to update the
// stack bounds for this case.
inBound := sp > g0.stack.lo && sp <= g0.stack.hi
if inBound && mp.g0StackAccurate {
// This M has called into Go before and has the stack bounds
// initialized. We have the accurate stack bounds, and the SP
// is in bounds. We expect it continues to run within the same
// bounds.
return
}
// We don't have an accurate stack bounds (either it never calls
// into Go before, or we couldn't get the accurate bounds), or the
// current SP is not within the previous bounds (the stack may have
// changed between calls). We need to update the stack bounds.
//
// N.B. we need to update the stack bounds even if SP appears to
// already be in bounds. Our "bounds" may actually be estimated dummy
// bounds (below). The actual stack bounds could have shifted but still
// have partial overlap with our dummy bounds. If we failed to update
// in that case, we could find ourselves seemingly called near the
// bottom of the stack bounds, where we quickly run out of space.
// already be in bounds, if our bounds are estimated dummy bounds
// (below). We may be in a different region within the same actual
// stack bounds, but our estimates were not accurate. Or the actual
// stack bounds could have shifted but still have partial overlap with
// our dummy bounds. If we failed to update in that case, we could find
// ourselves seemingly called near the bottom of the stack bounds, where
// we quickly run out of space.
// Set the stack bounds to match the current stack. If we don't
// actually know how big the stack is, like we don't know how big any
// scheduling stack is, but we assume there's at least 32 kB. If we
// can get a more accurate stack bound from pthread, use that, provided
// it actually contains SP..
// it actually contains SP.
g0.stack.hi = sp + 1024
g0.stack.lo = sp - 32*1024
mp.g0StackAccurate = false
if !signal && _cgo_getstackbound != nil {
// Don't adjust if called from the signal handler.
// We are on the signal stack, not the pthread stack.
@ -296,12 +279,16 @@ func callbackUpdateSystemStack(mp *m, sp uintptr, signal bool) {
asmcgocall(_cgo_getstackbound, unsafe.Pointer(&bounds))
// getstackbound is an unsupported no-op on Windows.
//
// On Unix systems, if the API to get accurate stack bounds is
// not available, it returns zeros.
//
// Don't use these bounds if they don't contain SP. Perhaps we
// were called by something not using the standard thread
// stack.
if bounds[0] != 0 && sp > bounds[0] && sp <= bounds[1] {
g0.stack.lo = bounds[0]
g0.stack.hi = bounds[1]
mp.g0StackAccurate = true
}
}
g0.stackguard0 = g0.stack.lo + stackGuard
@ -319,6 +306,8 @@ func cgocallbackg(fn, frame unsafe.Pointer, ctxt uintptr) {
}
sp := gp.m.g0.sched.sp // system sp saved by cgocallback.
oldStack := gp.m.g0.stack
oldAccurate := gp.m.g0StackAccurate
callbackUpdateSystemStack(gp.m, sp, false)
// The call from C is on gp.m's g0 stack, so we must ensure
@ -338,9 +327,14 @@ func cgocallbackg(fn, frame unsafe.Pointer, ctxt uintptr) {
// stack. However, since we're returning to an earlier stack frame and
// need to pair with the entersyscall() call made by cgocall, we must
// save syscall* and let reentersyscall restore them.
//
// Note: savedsp and savedbp MUST be held in locals as an unsafe.Pointer.
// When we call into Go, the stack is free to be moved. If these locals
// aren't visible in the stack maps, they won't get updated properly,
// and will end up being stale when restored by reentersyscall.
savedsp := unsafe.Pointer(gp.syscallsp)
savedpc := gp.syscallpc
savedbp := gp.syscallbp
savedbp := unsafe.Pointer(gp.syscallbp)
exitsyscall() // coming out of cgo call
gp.m.incgo = false
if gp.m.isextra {
@ -372,9 +366,15 @@ func cgocallbackg(fn, frame unsafe.Pointer, ctxt uintptr) {
osPreemptExtEnter(gp.m)
// going back to cgo call
reentersyscall(savedpc, uintptr(savedsp), savedbp)
reentersyscall(savedpc, uintptr(savedsp), uintptr(savedbp))
gp.m.winsyscall = winsyscall
// Restore the old g0 stack bounds
gp.m.g0.stack = oldStack
gp.m.g0.stackguard0 = oldStack.lo + stackGuard
gp.m.g0.stackguard1 = gp.m.g0.stackguard0
gp.m.g0StackAccurate = oldAccurate
}
func cgocallbackg1(fn, frame unsafe.Pointer, ctxt uintptr) {

View File

@ -208,6 +208,18 @@ func coroswitch_m(gp *g) {
// directly if possible.
setGNoWB(&mp.curg, gnext)
setMNoWB(&gnext.m, mp)
// Synchronize with any out-standing goroutine profile. We're about to start
// executing, and an invariant of the profiler is that we tryRecordGoroutineProfile
// whenever a goroutine is about to start running.
//
// N.B. We must do this before transitioning to _Grunning but after installing gnext
// in curg, so that we have a valid curg for allocation (tryRecordGoroutineProfile
// may allocate).
if goroutineProfile.active {
tryRecordGoroutineProfile(gnext, nil, osyield)
}
if !gnext.atomicstatus.CompareAndSwap(_Gwaiting, _Grunning) {
// The CAS failed: use casgstatus, which will take care of
// coordinating with the garbage collector about the state change.

View File

@ -1886,3 +1886,30 @@ func (m *TraceMap) PutString(s string) (uint64, bool) {
func (m *TraceMap) Reset() {
m.traceMap.reset()
}
func SetSpinInGCMarkDone(spin bool) {
gcDebugMarkDone.spinAfterRaggedBarrier.Store(spin)
}
func GCMarkDoneRestarted() bool {
// Only read this outside of the GC. If we're running during a GC, just report false.
mp := acquirem()
if gcphase != _GCoff {
releasem(mp)
return false
}
restarted := gcDebugMarkDone.restartedDueTo27993
releasem(mp)
return restarted
}
func GCMarkDoneResetRestartFlag() {
mp := acquirem()
for gcphase != _GCoff {
releasem(mp)
Gosched()
mp = acquirem()
}
gcDebugMarkDone.restartedDueTo27993 = false
releasem(mp)
}

View File

@ -6,6 +6,8 @@ package runtime_test
import (
"fmt"
"internal/testenv"
"internal/weak"
"math/bits"
"math/rand"
"os"
@ -787,3 +789,78 @@ func TestMemoryLimitNoGCPercent(t *testing.T) {
func TestMyGenericFunc(t *testing.T) {
runtime.MyGenericFunc[int]()
}
func TestWeakToStrongMarkTermination(t *testing.T) {
testenv.MustHaveParallelism(t)
type T struct {
a *int
b int
}
defer runtime.GOMAXPROCS(runtime.GOMAXPROCS(2))
defer debug.SetGCPercent(debug.SetGCPercent(-1))
w := make([]weak.Pointer[T], 2048)
// Make sure there's no out-standing GC from a previous test.
runtime.GC()
// Create many objects with a weak pointers to them.
for i := range w {
x := new(T)
x.a = new(int)
w[i] = weak.Make(x)
}
// Reset the restart flag.
runtime.GCMarkDoneResetRestartFlag()
// Prevent mark termination from completing.
runtime.SetSpinInGCMarkDone(true)
// Start a GC, and wait a little bit to get something spinning in mark termination.
// Simultaneously, fire off another goroutine to disable spinning. If everything's
// working correctly, then weak.Strong will block, so we need to make sure something
// prevents the GC from continuing to spin.
done := make(chan struct{})
go func() {
runtime.GC()
done <- struct{}{}
}()
go func() {
time.Sleep(100 * time.Millisecond)
// Let mark termination continue.
runtime.SetSpinInGCMarkDone(false)
}()
time.Sleep(10 * time.Millisecond)
// Perform many weak->strong conversions in the critical window.
var wg sync.WaitGroup
for _, wp := range w {
wg.Add(1)
go func() {
defer wg.Done()
wp.Strong()
}()
}
// Make sure the GC completes.
<-done
// Make sure all the weak->strong conversions finish.
wg.Wait()
// The bug is triggered if there's still mark work after gcMarkDone stops the world.
//
// This can manifest in one of two ways today:
// - An exceedingly rare crash in mark termination.
// - gcMarkDone restarts, as if issue #27993 is at play.
//
// Check for the latter. This is a fairly controlled environment, so #27993 is very
// unlikely to happen (it's already rare to begin with) but we'll always _appear_ to
// trigger the same bug if weak->strong conversions aren't properly coordinated with
// mark termination.
if runtime.GCMarkDoneRestarted() {
t.Errorf("gcMarkDone restarted")
}
}

View File

@ -17,6 +17,7 @@ const (
lockRankDefer
lockRankSweepWaiters
lockRankAssistQueue
lockRankStrongFromWeakQueue
lockRankSweep
lockRankTestR
lockRankTestW
@ -84,64 +85,65 @@ const lockRankLeafRank lockRank = 1000
// lockNames gives the names associated with each of the above ranks.
var lockNames = []string{
lockRankSysmon: "sysmon",
lockRankScavenge: "scavenge",
lockRankForcegc: "forcegc",
lockRankDefer: "defer",
lockRankSweepWaiters: "sweepWaiters",
lockRankAssistQueue: "assistQueue",
lockRankSweep: "sweep",
lockRankTestR: "testR",
lockRankTestW: "testW",
lockRankTimerSend: "timerSend",
lockRankAllocmW: "allocmW",
lockRankExecW: "execW",
lockRankCpuprof: "cpuprof",
lockRankPollCache: "pollCache",
lockRankPollDesc: "pollDesc",
lockRankWakeableSleep: "wakeableSleep",
lockRankHchan: "hchan",
lockRankAllocmR: "allocmR",
lockRankExecR: "execR",
lockRankSched: "sched",
lockRankAllg: "allg",
lockRankAllp: "allp",
lockRankNotifyList: "notifyList",
lockRankSudog: "sudog",
lockRankTimers: "timers",
lockRankTimer: "timer",
lockRankNetpollInit: "netpollInit",
lockRankRoot: "root",
lockRankItab: "itab",
lockRankReflectOffs: "reflectOffs",
lockRankUserArenaState: "userArenaState",
lockRankTraceBuf: "traceBuf",
lockRankTraceStrings: "traceStrings",
lockRankFin: "fin",
lockRankSpanSetSpine: "spanSetSpine",
lockRankMspanSpecial: "mspanSpecial",
lockRankTraceTypeTab: "traceTypeTab",
lockRankGcBitsArenas: "gcBitsArenas",
lockRankProfInsert: "profInsert",
lockRankProfBlock: "profBlock",
lockRankProfMemActive: "profMemActive",
lockRankProfMemFuture: "profMemFuture",
lockRankGscan: "gscan",
lockRankStackpool: "stackpool",
lockRankStackLarge: "stackLarge",
lockRankHchanLeaf: "hchanLeaf",
lockRankWbufSpans: "wbufSpans",
lockRankMheap: "mheap",
lockRankMheapSpecial: "mheapSpecial",
lockRankGlobalAlloc: "globalAlloc",
lockRankTrace: "trace",
lockRankTraceStackTab: "traceStackTab",
lockRankPanic: "panic",
lockRankDeadlock: "deadlock",
lockRankRaceFini: "raceFini",
lockRankAllocmRInternal: "allocmRInternal",
lockRankExecRInternal: "execRInternal",
lockRankTestRInternal: "testRInternal",
lockRankSysmon: "sysmon",
lockRankScavenge: "scavenge",
lockRankForcegc: "forcegc",
lockRankDefer: "defer",
lockRankSweepWaiters: "sweepWaiters",
lockRankAssistQueue: "assistQueue",
lockRankStrongFromWeakQueue: "strongFromWeakQueue",
lockRankSweep: "sweep",
lockRankTestR: "testR",
lockRankTestW: "testW",
lockRankTimerSend: "timerSend",
lockRankAllocmW: "allocmW",
lockRankExecW: "execW",
lockRankCpuprof: "cpuprof",
lockRankPollCache: "pollCache",
lockRankPollDesc: "pollDesc",
lockRankWakeableSleep: "wakeableSleep",
lockRankHchan: "hchan",
lockRankAllocmR: "allocmR",
lockRankExecR: "execR",
lockRankSched: "sched",
lockRankAllg: "allg",
lockRankAllp: "allp",
lockRankNotifyList: "notifyList",
lockRankSudog: "sudog",
lockRankTimers: "timers",
lockRankTimer: "timer",
lockRankNetpollInit: "netpollInit",
lockRankRoot: "root",
lockRankItab: "itab",
lockRankReflectOffs: "reflectOffs",
lockRankUserArenaState: "userArenaState",
lockRankTraceBuf: "traceBuf",
lockRankTraceStrings: "traceStrings",
lockRankFin: "fin",
lockRankSpanSetSpine: "spanSetSpine",
lockRankMspanSpecial: "mspanSpecial",
lockRankTraceTypeTab: "traceTypeTab",
lockRankGcBitsArenas: "gcBitsArenas",
lockRankProfInsert: "profInsert",
lockRankProfBlock: "profBlock",
lockRankProfMemActive: "profMemActive",
lockRankProfMemFuture: "profMemFuture",
lockRankGscan: "gscan",
lockRankStackpool: "stackpool",
lockRankStackLarge: "stackLarge",
lockRankHchanLeaf: "hchanLeaf",
lockRankWbufSpans: "wbufSpans",
lockRankMheap: "mheap",
lockRankMheapSpecial: "mheapSpecial",
lockRankGlobalAlloc: "globalAlloc",
lockRankTrace: "trace",
lockRankTraceStackTab: "traceStackTab",
lockRankPanic: "panic",
lockRankDeadlock: "deadlock",
lockRankRaceFini: "raceFini",
lockRankAllocmRInternal: "allocmRInternal",
lockRankExecRInternal: "execRInternal",
lockRankTestRInternal: "testRInternal",
}
func (rank lockRank) String() string {
@ -163,62 +165,63 @@ func (rank lockRank) String() string {
//
// Lock ranks that allow self-cycles list themselves.
var lockPartialOrder [][]lockRank = [][]lockRank{
lockRankSysmon: {},
lockRankScavenge: {lockRankSysmon},
lockRankForcegc: {lockRankSysmon},
lockRankDefer: {},
lockRankSweepWaiters: {},
lockRankAssistQueue: {},
lockRankSweep: {},
lockRankTestR: {},
lockRankTestW: {},
lockRankTimerSend: {},
lockRankAllocmW: {},
lockRankExecW: {},
lockRankCpuprof: {},
lockRankPollCache: {},
lockRankPollDesc: {},
lockRankWakeableSleep: {},
lockRankHchan: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankWakeableSleep, lockRankHchan},
lockRankAllocmR: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan},
lockRankExecR: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan},
lockRankSched: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR},
lockRankAllg: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched},
lockRankAllp: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched},
lockRankNotifyList: {},
lockRankSudog: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankWakeableSleep, lockRankHchan, lockRankNotifyList},
lockRankTimers: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankTimers},
lockRankTimer: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankTimers},
lockRankNetpollInit: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankTimers, lockRankTimer},
lockRankRoot: {},
lockRankItab: {},
lockRankReflectOffs: {lockRankItab},
lockRankUserArenaState: {},
lockRankTraceBuf: {lockRankSysmon, lockRankScavenge},
lockRankTraceStrings: {lockRankSysmon, lockRankScavenge, lockRankTraceBuf},
lockRankFin: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankSpanSetSpine: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankMspanSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankTraceTypeTab: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankGcBitsArenas: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankMspanSpecial},
lockRankProfInsert: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankProfBlock: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankProfMemActive: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankProfMemFuture: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankProfMemActive},
lockRankGscan: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture},
lockRankStackpool: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan},
lockRankStackLarge: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan},
lockRankHchanLeaf: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankHchanLeaf},
lockRankWbufSpans: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan},
lockRankMheap: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans},
lockRankMheapSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap},
lockRankGlobalAlloc: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankMheapSpecial},
lockRankTrace: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap},
lockRankTraceStackTab: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankTrace},
lockRankPanic: {},
lockRankDeadlock: {lockRankPanic, lockRankDeadlock},
lockRankRaceFini: {lockRankPanic},
lockRankAllocmRInternal: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankAllocmW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR},
lockRankExecRInternal: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankExecR},
lockRankTestRInternal: {lockRankTestR, lockRankTestW},
lockRankSysmon: {},
lockRankScavenge: {lockRankSysmon},
lockRankForcegc: {lockRankSysmon},
lockRankDefer: {},
lockRankSweepWaiters: {},
lockRankAssistQueue: {},
lockRankStrongFromWeakQueue: {},
lockRankSweep: {},
lockRankTestR: {},
lockRankTestW: {},
lockRankTimerSend: {},
lockRankAllocmW: {},
lockRankExecW: {},
lockRankCpuprof: {},
lockRankPollCache: {},
lockRankPollDesc: {},
lockRankWakeableSleep: {},
lockRankHchan: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankWakeableSleep, lockRankHchan},
lockRankAllocmR: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan},
lockRankExecR: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan},
lockRankSched: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR},
lockRankAllg: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched},
lockRankAllp: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched},
lockRankNotifyList: {},
lockRankSudog: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankWakeableSleep, lockRankHchan, lockRankNotifyList},
lockRankTimers: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankTimers},
lockRankTimer: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankTimers},
lockRankNetpollInit: {lockRankSysmon, lockRankScavenge, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankTimers, lockRankTimer},
lockRankRoot: {},
lockRankItab: {},
lockRankReflectOffs: {lockRankItab},
lockRankUserArenaState: {},
lockRankTraceBuf: {lockRankSysmon, lockRankScavenge},
lockRankTraceStrings: {lockRankSysmon, lockRankScavenge, lockRankTraceBuf},
lockRankFin: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankSpanSetSpine: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankMspanSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankTraceTypeTab: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankGcBitsArenas: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankMspanSpecial},
lockRankProfInsert: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankProfBlock: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankProfMemActive: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings},
lockRankProfMemFuture: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankProfMemActive},
lockRankGscan: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture},
lockRankStackpool: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan},
lockRankStackLarge: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan},
lockRankHchanLeaf: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankHchanLeaf},
lockRankWbufSpans: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan},
lockRankMheap: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans},
lockRankMheapSpecial: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap},
lockRankGlobalAlloc: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankMheapSpecial},
lockRankTrace: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap},
lockRankTraceStackTab: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankDefer, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollCache, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR, lockRankExecR, lockRankSched, lockRankAllg, lockRankAllp, lockRankNotifyList, lockRankSudog, lockRankTimers, lockRankTimer, lockRankNetpollInit, lockRankRoot, lockRankItab, lockRankReflectOffs, lockRankUserArenaState, lockRankTraceBuf, lockRankTraceStrings, lockRankFin, lockRankSpanSetSpine, lockRankMspanSpecial, lockRankGcBitsArenas, lockRankProfInsert, lockRankProfBlock, lockRankProfMemActive, lockRankProfMemFuture, lockRankGscan, lockRankStackpool, lockRankStackLarge, lockRankWbufSpans, lockRankMheap, lockRankTrace},
lockRankPanic: {},
lockRankDeadlock: {lockRankPanic, lockRankDeadlock},
lockRankRaceFini: {lockRankPanic},
lockRankAllocmRInternal: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankAllocmW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankAllocmR},
lockRankExecRInternal: {lockRankSysmon, lockRankScavenge, lockRankForcegc, lockRankSweepWaiters, lockRankAssistQueue, lockRankStrongFromWeakQueue, lockRankSweep, lockRankTestR, lockRankTimerSend, lockRankExecW, lockRankCpuprof, lockRankPollDesc, lockRankWakeableSleep, lockRankHchan, lockRankExecR},
lockRankTestRInternal: {lockRankTestR, lockRankTestW},
}

View File

@ -1209,6 +1209,11 @@ func (h *hmap) sameSizeGrow() bool {
return h.flags&sameSizeGrow != 0
}
//go:linkname sameSizeGrowForIssue69110Test
func sameSizeGrowForIssue69110Test(h *hmap) bool {
return h.sameSizeGrow()
}
// noldbuckets calculates the number of buckets prior to the current map growth.
func (h *hmap) noldbuckets() uintptr {
oldB := h.B
@ -1668,7 +1673,16 @@ func moveToBmap(t *maptype, h *hmap, dst *bmap, pos int, src *bmap) (*bmap, int)
}
func mapclone2(t *maptype, src *hmap) *hmap {
dst := makemap(t, src.count, nil)
hint := src.count
if overLoadFactor(hint, src.B) {
// Note: in rare cases (e.g. during a same-sized grow) the map
// can be overloaded. Make sure we don't allocate a destination
// bucket array larger than the source bucket array.
// This will cause the cloned map to be overloaded also,
// but that's better than crashing. See issue 69110.
hint = int(loadFactorNum * (bucketShift(src.B) / loadFactorDen))
}
dst := makemap(t, hint, nil)
dst.hash0 = src.hash0
dst.nevacuate = 0
// flags do not need to be copied here, just like a new map has no flags.

View File

@ -190,6 +190,7 @@ func gcinit() {
work.markDoneSema = 1
lockInit(&work.sweepWaiters.lock, lockRankSweepWaiters)
lockInit(&work.assistQueue.lock, lockRankAssistQueue)
lockInit(&work.strongFromWeak.lock, lockRankStrongFromWeakQueue)
lockInit(&work.wbufSpans.lock, lockRankWbufSpans)
}
@ -418,6 +419,26 @@ type workType struct {
list gList
}
// strongFromWeak controls how the GC interacts with weak->strong
// pointer conversions.
strongFromWeak struct {
// block is a flag set during mark termination that prevents
// new weak->strong conversions from executing by blocking the
// goroutine and enqueuing it onto q.
//
// Mutated only by one goroutine at a time in gcMarkDone,
// with globally-synchronizing events like forEachP and
// stopTheWorld.
block bool
// q is a queue of goroutines that attempted to perform a
// weak->strong conversion during mark termination.
//
// Protected by lock.
lock mutex
q gQueue
}
// cycles is the number of completed GC cycles, where a GC
// cycle is sweep termination, mark, mark termination, and
// sweep. This differs from memstats.numgc, which is
@ -800,6 +821,19 @@ func gcStart(trigger gcTrigger) {
// This is protected by markDoneSema.
var gcMarkDoneFlushed uint32
// gcDebugMarkDone contains fields used to debug/test mark termination.
var gcDebugMarkDone struct {
// spinAfterRaggedBarrier forces gcMarkDone to spin after it executes
// the ragged barrier.
spinAfterRaggedBarrier atomic.Bool
// restartedDueTo27993 indicates that we restarted mark termination
// due to the bug described in issue #27993.
//
// Protected by worldsema.
restartedDueTo27993 bool
}
// gcMarkDone transitions the GC from mark to mark termination if all
// reachable objects have been marked (that is, there are no grey
// objects and can be no more in the future). Otherwise, it flushes
@ -842,6 +876,10 @@ top:
// stop the world later, so acquire worldsema now.
semacquire(&worldsema)
// Prevent weak->strong conversions from generating additional
// GC work. forEachP will guarantee that it is observed globally.
work.strongFromWeak.block = true
// Flush all local buffers and collect flushedWork flags.
gcMarkDoneFlushed = 0
forEachP(waitReasonGCMarkTermination, func(pp *p) {
@ -872,6 +910,10 @@ top:
goto top
}
// For debugging/testing.
for gcDebugMarkDone.spinAfterRaggedBarrier.Load() {
}
// There was no global work, no local work, and no Ps
// communicated work since we took markDoneSema. Therefore
// there are no grey objects and no more objects can be
@ -910,6 +952,8 @@ top:
}
})
if restart {
gcDebugMarkDone.restartedDueTo27993 = true
getg().m.preemptoff = ""
systemstack(func() {
// Accumulate the time we were stopped before we had to start again.
@ -936,6 +980,11 @@ top:
// start the world again.
gcWakeAllAssists()
// Wake all blocked weak->strong conversions. These will run
// when we start the world again.
work.strongFromWeak.block = false
gcWakeAllStrongFromWeak()
// Likewise, release the transition lock. Blocked
// workers and assists will run when we start the
// world again.

View File

@ -2049,8 +2049,19 @@ func internal_weak_runtime_registerWeakPointer(p unsafe.Pointer) unsafe.Pointer
func internal_weak_runtime_makeStrongFromWeak(u unsafe.Pointer) unsafe.Pointer {
handle := (*atomic.Uintptr)(u)
// Prevent preemption. We want to make sure that another GC cycle can't start.
// Prevent preemption. We want to make sure that another GC cycle can't start
// and that work.strongFromWeak.block can't change out from under us.
mp := acquirem()
// Yield to the GC if necessary.
if work.strongFromWeak.block {
releasem(mp)
// Try to park and wait for mark termination.
// N.B. gcParkStrongFromWeak calls acquirem before returning.
mp = gcParkStrongFromWeak()
}
p := handle.Load()
if p == 0 {
releasem(mp)
@ -2073,14 +2084,67 @@ func internal_weak_runtime_makeStrongFromWeak(u unsafe.Pointer) unsafe.Pointer {
// Even if we just swept some random span that doesn't contain this object, because
// this object is long dead and its memory has since been reused, we'll just observe nil.
ptr := unsafe.Pointer(handle.Load())
// This is responsible for maintaining the same GC-related
// invariants as the Yuasa part of the write barrier. During
// the mark phase, it's possible that we just created the only
// valid pointer to the object pointed to by ptr. If it's only
// ever referenced from our stack, and our stack is blackened
// already, we could fail to mark it. So, mark it now.
if gcphase != _GCoff {
shade(uintptr(ptr))
}
releasem(mp)
// Explicitly keep ptr alive. This seems unnecessary since we return ptr,
// but let's be explicit since it's important we keep ptr alive across the
// call to shade.
KeepAlive(ptr)
return ptr
}
// gcParkStrongFromWeak puts the current goroutine on the weak->strong queue and parks.
func gcParkStrongFromWeak() *m {
// Prevent preemption as we check strongFromWeak, so it can't change out from under us.
mp := acquirem()
for work.strongFromWeak.block {
lock(&work.strongFromWeak.lock)
releasem(mp) // N.B. Holding the lock prevents preemption.
// Queue ourselves up.
work.strongFromWeak.q.pushBack(getg())
// Park.
goparkunlock(&work.strongFromWeak.lock, waitReasonGCWeakToStrongWait, traceBlockGCWeakToStrongWait, 2)
// Re-acquire the current M since we're going to check the condition again.
mp = acquirem()
// Re-check condition. We may have awoken in the next GC's mark termination phase.
}
return mp
}
// gcWakeAllStrongFromWeak wakes all currently blocked weak->strong
// conversions. This is used at the end of a GC cycle.
//
// work.strongFromWeak.block must be false to prevent woken goroutines
// from immediately going back to sleep.
func gcWakeAllStrongFromWeak() {
lock(&work.strongFromWeak.lock)
list := work.strongFromWeak.q.popList()
injectglist(&list)
unlock(&work.strongFromWeak.lock)
}
// Retrieves or creates a weak pointer handle for the object p.
func getOrAddWeakHandle(p unsafe.Pointer) *atomic.Uintptr {
// First try to retrieve without allocating.
if handle := getWeakHandle(p); handle != nil {
// Keep p alive for the duration of the function to ensure
// that it cannot die while we're trying to do this.
KeepAlive(p)
return handle
}
@ -2105,7 +2169,17 @@ func getOrAddWeakHandle(p unsafe.Pointer) *atomic.Uintptr {
scanblock(uintptr(unsafe.Pointer(&s.handle)), goarch.PtrSize, &oneptrmask[0], gcw, nil)
releasem(mp)
}
return s.handle
// Keep p alive for the duration of the function to ensure
// that it cannot die while we're trying to do this.
//
// Same for handle, which is only stored in the special.
// There's a window where it might die if we don't keep it
// alive explicitly. Returning it here is probably good enough,
// but let's be defensive and explicit. See #70455.
KeepAlive(p)
KeepAlive(handle)
return handle
}
// There was an existing handle. Free the special
@ -2124,8 +2198,11 @@ func getOrAddWeakHandle(p unsafe.Pointer) *atomic.Uintptr {
}
// Keep p alive for the duration of the function to ensure
// that it cannot die while we're trying to this.
// that it cannot die while we're trying to do this.
//
// Same for handle, just to be defensive.
KeepAlive(p)
KeepAlive(handle)
return handle
}
@ -2154,6 +2231,9 @@ func getWeakHandle(p unsafe.Pointer) *atomic.Uintptr {
unlock(&span.speciallock)
releasem(mp)
// Keep p alive for the duration of the function to ensure
// that it cannot die while we're trying to do this.
KeepAlive(p)
return handle
}

View File

@ -50,6 +50,7 @@ NONE < defer;
NONE <
sweepWaiters,
assistQueue,
strongFromWeakQueue,
sweep;
# Test only
@ -66,6 +67,7 @@ assistQueue,
hchan,
pollDesc, # pollDesc can interact with timers, which can lock sched.
scavenge,
strongFromWeakQueue,
sweep,
sweepWaiters,
testR,

View File

@ -1136,11 +1136,12 @@ func expandFrames(p []BlockProfileRecord) {
for i := range p {
cf := CallersFrames(p[i].Stack())
j := 0
for ; j < len(expandedStack); j++ {
for j < len(expandedStack) {
f, more := cf.Next()
// f.PC is a "call PC", but later consumers will expect
// "return PCs"
expandedStack[j] = f.PC + 1
j++
if !more {
break
}
@ -1270,7 +1271,8 @@ func pprof_mutexProfileInternal(p []profilerecord.BlockProfileRecord) (n int, ok
// of calling ThreadCreateProfile directly.
func ThreadCreateProfile(p []StackRecord) (n int, ok bool) {
return threadCreateProfileInternal(len(p), func(r profilerecord.StackRecord) {
copy(p[0].Stack0[:], r.Stack)
i := copy(p[0].Stack0[:], r.Stack)
clear(p[0].Stack0[i:])
p = p[1:]
})
}
@ -1649,7 +1651,8 @@ func GoroutineProfile(p []StackRecord) (n int, ok bool) {
return
}
for i, mr := range records[0:n] {
copy(p[i].Stack0[:], mr.Stack)
l := copy(p[i].Stack0[:], mr.Stack)
clear(p[i].Stack0[l:])
}
return
}

View File

@ -879,8 +879,9 @@ func runPerThreadSyscall() {
}
const (
_SI_USER = 0
_SI_TKILL = -6
_SI_USER = 0
_SI_TKILL = -6
_SYS_SECCOMP = 1
)
// sigFromUser reports whether the signal was sent because of a call
@ -892,6 +893,14 @@ func (c *sigctxt) sigFromUser() bool {
return code == _SI_USER || code == _SI_TKILL
}
// sigFromSeccomp reports whether the signal was sent from seccomp.
//
//go:nosplit
func (c *sigctxt) sigFromSeccomp() bool {
code := int32(c.sigcode())
return code == _SYS_SECCOMP
}
//go:nosplit
func mprotect(addr unsafe.Pointer, n uintptr, prot int32) (ret int32, errno int32) {
r, _, err := syscall.Syscall6(syscall.SYS_MPROTECT, uintptr(addr), n, uintptr(prot), 0, 0, 0)

View File

@ -13,3 +13,10 @@ package runtime
func (c *sigctxt) sigFromUser() bool {
return c.sigcode() == _SI_USER
}
// sigFromSeccomp reports whether the signal was sent from seccomp.
//
//go:nosplit
func (c *sigctxt) sigFromSeccomp() bool {
return false
}

Some files were not shown because too many files have changed in this diff Show More