As described here:
https://github.com/golang/go/issues/31636#issuecomment-493271830
"Find the lexically earliest package that is not initialized yet,
but has had all its dependencies initialized, initialize that package,
and repeat."
Simplify the runtime a bit, by just computing the ordering required
in the linker and giving a list to the runtime.
Update #31636Fixes#57411
RELNOTE=yes
Change-Id: I1e4d3878ebe6e8953527aedb730824971d722cac
Reviewed-on: https://go-review.googlesource.com/c/go/+/462035
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
The external linker will need to support the new PC relative relocations
when they are generated by Go in a future patch. If it does not, many
unhelpful relocation errors will be generated by the external linker.
Use the -mcpu=power10 option as a surrogate for -mpcrel. It most cases,
it should indicate whether the underlying linker has support for
resolving PC relative relocations.
Change-Id: I84b151ce04512ccaeb17835aaf44105a5f6b515b
Reviewed-on: https://go-review.googlesource.com/c/go/+/469576
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Archana Ravindar <aravind5@in.ibm.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Previously, when a program depends on cgo packages in the standard
library (e.g. net, os/user) but not otherwise use cgo, we default
to internal linking mode. As we shipped pre-built cgo-using packages
in Go distributions, we don't require a C compiler to build those
packages. Then, by using internal linking we can link programs
using those packages without requiring a C toolchain.
As of Go 1.20, we stopped shipping those pre-built packages. If a
user doesn't have a C toolchain, they will use the non-cgo version
of the package. If they have a C toolchain, they can get cgo-using
packages but they can link with the external linker as well. So
there is no strong need to be able to link the cgo version of the
packages without a C toolchain. This CL makes it default to
external linking mode.
Fixes#58619.
Fixes#58620.
Change-Id: I62d3744c2b82ce734813c0e303e417d85dd29868
Reviewed-on: https://go-review.googlesource.com/c/go/+/470298
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The bug in GNU ld appears to have been fixed in GNU binutils 2.28 by
GNU binutils revision 5522f910cb539905d6adfdceab208ddfa5e84557.
(This may have been accidental as the ChangeLog for the fix makes
no reference to it; the fix is from
https://sourceware.org/bugzilla/show_bug.cgi?id=19962).
Continue using gold on arm64, at least for now, because as reported in
issue #22040 GNU ld still fails there.
For #15696
For #22040
Change-Id: I5534bb8b5680daf536a7941aba5c701e8a4138ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/468655
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Update the code that tries to satisfy unresolved references to
"__stack_chk_fail_local" to look for "libssp_nonshared.a" in addition
to "libc_nonshared.a" (the former archive is the correct place on
Alpine).
Updates #52919.
Updates #54313.
Updates #57261.
Fixes#58385.
Change-Id: Id6cd3ebb4d5388df50a838e6efa5e5b683545b01
Reviewed-on: https://go-review.googlesource.com/c/go/+/466936
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
In external linking mode, when the external linker fails to handle
something in a host object file, it usually reports the name of
the host object which is a copied file named 000NNN.o. This is
often not helpful to determine what file it is. Add some debug
print so at least in -v mode it is more helpful.
Change-Id: Ibe762bff6a25640d16ee0dc082736ba5161b7522
Reviewed-on: https://go-review.googlesource.com/c/go/+/458735
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
While we're here rename setupplt to setupPLT.
This is a pure naming change with no semantic change.
Change-Id: Ib0312fb6568475b620dab7632438b4d25e4d9cc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/463744
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
For internal linking, at the point where we finish reading libgcc.a,
if the symbol "__stack_chk_local" is still undefined, then read
in the host archive libc_nonshared.a as well.
Updates #57261.
Change-Id: I0b1e485aa50aa7940db8cabcb3b9a7959bf99ce7
Reviewed-on: https://go-review.googlesource.com/c/go/+/456856
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This is a pure cleanup to bring the ELF hooks together.
Change-Id: I01d5227c70f30e4a659dcd7904e7c247266e95b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/463981
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
Since this may add a large number of --export-dynamic-symbol options,
use a response file if the command line gets large.
Fixes#53579
Change-Id: Ic226bf372bf1e177a3dae886d1c48f4ce3569c0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/414654
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The importcycles method has not been useful since April 2016 when a large code deletion was performed.
The compiler itself provides some protection against import cycles, and the linker does import cycle detection in linksetup -> postorder.
For #57400
Change-Id: I3095bdb3f16a82ba25681bf4a20ceaa3c9613921
GitHub-Last-Rev: 87a46153b136db67675874ffe46e5881b9c756ce
GitHub-Pull-Request: golang/go#57462
Reviewed-on: https://go-review.googlesource.com/c/go/+/459475
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Pass -Wl,--no-insert-timestamp to the external linker on windows, so
as to suppress generation of the PE file header data/time stamp. This
is in order to make it possible to get reproducible CGO builds on
windows (note that we already zero the timestamp field in question for
internal linkage).
Updates #35006.
Change-Id: I3d69cf1fd32e099bd9bb4b0431a4c5f43e4b08f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/455535
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This patch reworks the handling of DLL import symbols in the PE host
object loader to ensure that the Go linker can deal with them properly
during internal linking.
Prior to this point the strategy was to immediately treat an import
symbol reference of the form "__imp__XXX" as if it were a reference to
the corresponding DYNIMPORT symbol XXX, except for certain special
cases. This worked for the most part, but ran into problems in
situations where the target ("XXX") wasn't a previously created
DYNIMPORT symbol (and when these problems happened, the root cause was
not always easy to see).
The new strategy is to not do any renaming or forwarding immediately,
but to delay handling until host object loading is complete. At that
point we make a scan through the newly introduced text+data sections
looking at the relocations that target import symbols, forwarding
the references to the corresponding DYNIMPORT sym where appropriate
and where there are direct refs to the DYNIMPORT syms, tagging them
for stub generation later on.
Updates #35006.
Updates #53540.
Change-Id: I2d42b39141ae150a9f82ecc334001749ae8a3b4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/451738
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Add a new debugging flag "-capturehostobjs" that instructs the linker
to capture copies of all object files loaded in during the host object
loading portion of CGO internal linking. The intent is to make it
easier to analyze the objects after the fact (as opposed to having to
dig around inside archives, which can be a "find needle in haystack"
exercise).
Change-Id: I7023a5b72b1b899ea9b3bd6501f069d1f21bbaf0
Reviewed-on: https://go-review.googlesource.com/c/go/+/451737
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
When ctxt.Debugvlog > 1, produce additional trace output to describe
which object files are being pulled out of host archive libraries and
why they were pulled (e.g. which symbol had a reference to something
in a library). Intended to make it easier to debug problems with cgo
internal linking.
Change-Id: Icd64aff244b9145162a00cb51642ef32f26adfba
Reviewed-on: https://go-review.googlesource.com/c/go/+/451736
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Many stages in the linker assume some runtime symbols exist.
Error out if the runtime package cannot be found.
Fixes#56685.
Change-Id: I39b2663f6d7419688e30e6d8650e5eb8fc43a2b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/449638
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
If no external linker was passed with -extld, link currently assumes
that it is "gcc" which is not correct for platforms that use clang
toolchain. Return "clang" for platforms that use it, this fixes dir
tests on freebsd/riscv64.
For #53466
Change-Id: Ie3bce1b9581839d0b3b2129908355cd30ae9a713
Reviewed-on: https://go-review.googlesource.com/c/go/+/432756
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Mikaël Urankar <mikael.urankar@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Joedian Reid <joedian@golang.org>
CL 424034 introduced two new packages that trigger external
linking mode where internal linking mode is sufficient.
Change-Id: I81583210331fe4151d631b5efd7fc5d3c8f11f39
Reviewed-on: https://go-review.googlesource.com/c/go/+/435256
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
For #45557
Change-Id: I56824135d86452603dd4ed4bab0e24c201bb0683
Reviewed-on: https://go-review.googlesource.com/c/go/+/426257
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Apparently the new darwin linker starts to emit a warning about
-no_pie deprecation. Maybe we want to switch to PIE by default.
For now, suppress the warning. This also makes it easier for
backporting to previous releases.
For #54482.
Change-Id: I1a3b74c237a9d00ec3b030fc3a9940a31e5cd37e
Reviewed-on: https://go-review.googlesource.com/c/go/+/430937
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
We added -pagezero_size in CL 72730, where it was intented for iOS.
The current code passes it only on macOS/AMD64 instead. It is not
really necessary there. Also, the new darwin linker starts to emit
a warning about deprecation of the flag. Stop passing it.
For #54482.
Change-Id: If9db7a1645c37d4284e48f075856912df8d8c1a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/430936
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
If the entry symbol cannot be found (for example, a new port is being brought
up and no rt0 code has been provided), the linker will currently panic. Rather
than panicing, generate an error that aids in debugging:
missing entry symbol "_rt0_arm64_openbsd"
Change-Id: I9cc38eaab48f730d596ca7fa9e9e3d68250ae4d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/430555
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
As it can't appear in user package paths.
There is a hack for handling "go:buildid" and "type:*" on windows/386.
Previously, windows/386 requires underscore prefix on external symbols,
but that's only applied for SHOSTOBJ/SUNDEFEXT or cgo export symbols.
"go.buildid" is STEXT, "type.*" is STYPE, thus they are not prefixed
with underscore.
In external linking mode, the external linker can't resolve them as
external symbols. But we are lucky that they have "." in their name,
so the external linker see them as Forwarder RVA exports. See:
- https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#export-address-table
- https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=ld/pe-dll.c;h=e7b82ba6ffadf74dc1b9ee71dc13d48336941e51;hb=HEAD#l972)
This CL changes "." to ":" in symbols name, so theses symbols can not be
found by external linker anymore. So a hacky way is adding the
underscore prefix for these 2 symbols. I don't have enough knowledge to
verify whether adding the underscore for all STEXT/STYPE symbols are
fine, even if it could be, that would be done in future CL.
Fixes#37762
Change-Id: I92eaaf24c0820926a36e0530fdb07b07af1fcc35
Reviewed-on: https://go-review.googlesource.com/c/go/+/317917
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Doing the test at link time lets us distribute one Linux toolchain
that works on both glibc-based and musl-based Linux systems.
The old way built a toolchain that only ran on one or the other.
Fixes#54197.
Change-Id: Iaae8c274c78e1091eee828a720b49646be9bfffe
Reviewed-on: https://go-review.googlesource.com/c/go/+/420774
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Turn off PIE explicitly for windows/amd64 when -race is in effect,
since at the moment the race detector runtime doesn't seem to handle
PIE binaries correctly. Note that newer C compilers on windows
produce PIE binaries by default, so the Go linker needs to explicitly
turn off PIE when invoking the external linker in this case.
Updates #53539.
Change-Id: Ib990621f22cf61a5fa383584bab81d3dfd7552e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/415676
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
As of LLVM rev 41cb504b7c4b18ac15830107431a0c1eec73a6b2, the
race detector runtime now refers to things in the windows
synchronization library, hence when doing windows internal
linking, at that library to the list of host archives that
we visit. The tsan code that makes the reference is here:
41cb504b7c/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp (L48)41cb504b7c/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp (L834)
Note that libsynchronization.a is not guaranteed to be available on
all windows systems, so in the external linking case, check for its
existence before adding "-lsynchronization" to the external linker
args.
Updates #53539.
Change-Id: I433c95c869915693d59e9c1082d5b8a11da1fc8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/413817
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
For a package that uses cgo, the file _cgo_import.go is created to
record information required for internal linking: the non-Go dynamic
symbols and libraries that the package depends on. Generating this
information sometimes fails, because it can require recreating all the
dependencies of all transitively imported packages. And the
information is rarely needed, since by default we use external linking
when there are packages outside of the standard library that use cgo.
With this CL, if generating _cgo_import.go fails, we don't report an
error. Instead, we mark the package as requiring external linking, by
adding an empty file named "dynimportfail" into the generated archive.
If the linker sees a file with that name, it rejects an attempt to use
internal linking.
Fixes#52863
Change-Id: Ie586e6753a5b67e49bb14533cd7603d9defcf0ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/413460
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Both GNU and LLVM linkers de facto accept `-zPARAM`, and Go sometimes
does it. Inconsistently: there are more uses of `-z PARAM` than
`-zPARAM`:
$ git grep -E -- '-Wl,-z[^,]' master | wc -l
4
$ git grep -E -- '-Wl,-z,' master | wc -l
7
However, not adding a space between `-z` and the param is not
documented:
llvm-13:
$ man ld.lld-13 | grep -E -A1 -w -- "^ +-z"
-z option
Linker option extensions.
gnu ld:
$ man ld | grep -E -A1 -w -- "^ +-z"
-z keyword
The recognized keywords are:
--
-z defs
Report unresolved symbol references from regular object files. This is done even if the linker is creating a non-symbolic
--
-z muldefs
Normally when a symbol is defined multiple times, the linker will report a fatal error. These options allow multiple definitions
--
-z
--imagic
... and thus should be avoided.
`zig cc`, when used as the C compiler (`CC="zig cc" go build ...`), will
bark, because `zig cc` accepts only `-z PARAM`, as documented.
Closesziglang/zig#11669
Change-Id: I758054ecaa3ce01a72600bf65d7f7b5c3ec46d09
GitHub-Last-Rev: e068e007da9f2b0441ee0aa8b198a7ba3cd93ed3
GitHub-Pull-Request: golang/go#53030
Reviewed-on: https://go-review.googlesource.com/c/go/+/407834
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
The basic arch-specific hooks are implemented, which
are used for internal and external linker.
Contributors to the loong64 port are:
Weining Lu <luweining@loongson.cn>
Lei Wang <wanglei@loongson.cn>
Lingqin Gong <gonglingqin@loongson.cn>
Xiaolin Zhao <zhaoxiaolin@loongson.cn>
Meidan Li <limeidan@loongson.cn>
Xiaojuan Zhai <zhaixiaojuan@loongson.cn>
Qiyuan Pu <puqiyuan@loongson.cn>
Guoqi Chen <chenguoqi@loongson.cn>
This port has been updated to Go 1.15.6:
https://github.com/loongson/go
Updates #46229
Change-Id: I4680eb0635dd0fa3d6ea8348a2488da9c7e33d3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/349514
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
We added internal/execabs back in January 2021 in order to fix
a security problem caused by os/exec's handling of the current
directory. Now that os/exec has that code, internal/execabs is
superfluous and can be deleted.
This commit rewrites all the imports back to os/exec and
deletes internal/execabs.
For #43724.
Change-Id: Ib9736baf978be2afd42a1225e2ab3fd5d33d19df
Reviewed-on: https://go-review.googlesource.com/c/go/+/381375
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
A plain make.bash in this tree will produce a working,
standard Go toolchain, not a BoringCrypto-enabled one.
The BoringCrypto-enabled one will be created with:
GOEXPERIMENT=boringcrypto ./make.bash
For #51940.
Change-Id: Ia9102ed993242eb1cb7f9b93eca97e81986a27b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/395881
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When we add GOEXPERIMENT=boringcrypto, the bootstrap process
will not converge if the compiler itself depends on the boringcrypto
cgo-based implementations of sha1 and sha256.
Using notsha256 avoids boringcrypto and makes bootstrap converge.
Removing md5 is not strictly necessary but it seemed worthwhile to
be consistent.
For #51940.
Change-Id: Iba649507e0964d1a49a1d16e463dd23c4e348f14
Reviewed-on: https://go-review.googlesource.com/c/go/+/402595
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
The linker performs a global analysis of all nosplit call chains to
check they fit in the stack space ensured by splittable functions.
That analysis has two problems right now:
1. It's inefficient. It performs a top-down analysis, starting with
every nosplit function and the nosplit stack limit and walking *down*
the call graph to compute how much stack remains at every call. As a
result, it visits the same functions over and over, often with
different remaining stack depths. This approach is historical: this
check was originally written in C and this approach avoided the need
for any interesting data structures.
2. If some call chain is over the limit, it only reports a single call
chain. As a result, if the check does fail, you often wind up playing
whack-a-mole by guessing where the problem is in the one chain, trying
to reduce the stack size, and then seeing if the link works or reports
a different path.
This CL completely rewrites the nosplit stack check. It now uses a
bottom-up analysis, computing the maximum stack height required by
every function's call tree. This visits every function exactly once,
making it much more efficient. It uses slightly more heap space for
intermediate storage, but still very little in the scheme of the
overall link. For example, when linking cmd/go, the new algorithm
virtually eliminates the time spent in this pass, and reduces overall
link time:
│ before │ after │
│ sec/op │ sec/op vs base │
Dostkcheck 7.926m ± 4% 1.831m ± 6% -76.90% (p=0.000 n=20)
TotalTime 301.3m ± 1% 296.4m ± 3% -1.62% (p=0.040 n=20)
│ before │ after │
│ B/op │ B/op vs base │
Dostkcheck 40.00Ki ± 0% 212.15Ki ± 0% +430.37% (p=0.000 n=20)
Most of this time is spent analyzing the runtime, so for larger
binaries, the total time saved is roughly the same, and proportionally
less of the overall link.
If the new implementation finds an error, it redoes the analysis,
switching to preferring quality of error reporting over performance.
For error reporting, it computes stack depths top-down (like the old
algorithm), and reports *all* paths that are over the stack limit,
presented as a tree for compactness. For example, this is the output
from a simple test case from test/nosplit with two over-limit paths
from f1:
main.f1: nosplit stack overflow
main.f1
grows 768 bytes, calls main.f2
grows 56 bytes, calls main.f4
grows 48 bytes
80 bytes over limit
grows 768 bytes, calls main.f3
grows 104 bytes
80 bytes over limit
While we're here, we do a few nice cleanups:
- We add a debug output flag, which will be useful for understanding
what our nosplit chains look like and which ones are close to
running over.
- We move the implementation out of the fog of lib.go to its own file.
- The implementation is generally more Go-like and less C-like.
Change-Id: If1ab31197f5215475559b93695c44a01bd16e276
Reviewed-on: https://go-review.googlesource.com/c/go/+/398176
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Storing this information in the Arch eliminates some code duplication
between the compiler and linker. This information is entirely
determined by the Arch, so the current approach of attaching it to an
entire Ctxt is a little silly. This will also make it easier to use
this information from tests.
The next CL will be a rote refactoring to eliminate the
Ctxt.FixedFrameSize methods.
Change-Id: I315c524fa66a0ea99f63ae5a2a6fdc367d843bad
Reviewed-on: https://go-review.googlesource.com/c/go/+/400818
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
When doing external linking on windows, the existing Go linker code
assumed that the external linker defaulted to "--no-dynamicbase" (if
no explicit option was given). This assumption doesn't hold for LLD,
which turns on "--dynamicbase" by default for 64-bit apps. Change the
linker to detect whether a more modern toolchain is in use and to
explicitly pass "--dynamicbase" either way , so as to take the
external linker default out of the equation. This also applies to the
"--high-entropy-va" option as well.
Updates #35006.
Change-Id: I3e12cf6d331c9d003e3d2bd566d45de5710588b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/384156
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
When doing an internal link on Windows, it's possible to see
unresolved references to the symbols "__CTOR_LIST__" and/or
"__DTOR_LIST__" (which are needed in some circumstances). If these are
still unresolved at the point where we're done reading host objects,
then synthesize dummy versions of them.
Updates #35006.
Change-Id: I408bf18499bba05752710cf5a41621123bd84a3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/383836
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
For Windows internal linking with CGO, when using more modern
LLVM-based compilers, we may need to read in the object file "crt2.o"
so as to have a definition of "atexit" (for example when linking the
runtime/cgo test), and we also need to allow for the possibility that
a given host archive might have to be looked at more than once. The goal
here is to get all.bash working on Windows when using an up to date
mingw C compiler (including those based on clang + LLD).
This patch also adds a new "hostObject" helper routine, similar to
"hostArchive" but specific to individual object files. There is also a
change to hostArchive to modify the pseudo-package name assigned when
reading archive elements: up until this point, a package name of
"libgcc" was used (even when reading a host archive like
"libmingex.a"), which led to very confusing errors messages if symbols
were missing or there were duplicate definitions.
Updates #35006.
Change-Id: I19c17dea9cfffa9e79030fc23064c7c63a612097
Reviewed-on: https://go-review.googlesource.com/c/go/+/382838
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
In the beginning the Go compiler was in C, and C had a function
'getgoroot' that returned GOROOT from either the environment or a
generated constant. 'getgoroot' was mechanically converted to Go
(as obj.Getgoroot) in CL 3046.
obj.Getgoroot begat obj.GOROOT. obj.GOROOT begat objabi.GOROOT,
which begat buildcfg.GOROOT.
As far as I can tell, today's buildcfg.GOROOT is functionally
identical to runtime.GOROOT(). Let's reduce some complexity by
defining it in those terms.
While we're thinking about buildcfg.GOROOT, also check whether it is
non-empty: if the toolchain is built with -trimpath, the value of
GOROOT might not be valid or meaningful if the user invokes
cmd/compile or cmd/link directly, or via a build tool other than
cmd/go that doesn't care as much about GOROOT. (As of CL 390024,
runtime.GOROOT will return the empty string instead of a bogus one
when built with -trimpath.)
For #51461.
Change-Id: I9fec020d5fa65d4aff0dd39b805f5ca93f86c36e
Reviewed-on: https://go-review.googlesource.com/c/go/+/393155
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
When building/using plugins on darwin, we need to use flat
namespace so the same symbol from the main executable and the
plugin can be resolved to the same address. Apparently, when using
flat namespace the dynamic linker can hang at forkExec when
resolving a lazy binding. Work around it by forcing early bindings.
Fixes#38824.
Change-Id: I983aa0a0960b15bf3f7871382e8231ee244655f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/372798
Trust: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
When the link exits on error it currently calls Out.Close, which
will munmap the output buffer and close the file. This may be
called in concurrent phase where other goroutines may be writing
to the output buffer. The munmap can race with the write, causing
it to write to unmapped memory and crash. This CL changes it to
just close the file without unmapping. We're exiting on error
anyway so no need to unmap.
Fixes#47816.
Change-Id: I0e89aca991bdada3d017b7d5c8efc29e46308c03
Reviewed-on: https://go-review.googlesource.com/c/go/+/363357
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Since GCC version 11, the 64-bit version of GCC starting files are
now suffixed by "_64" instead of being stored without suffix under
"ppc64" multilib directory.
Change-Id: Ibe53521ed24d36e5f6282e3574849b9ae11a1e9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/362594
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
