131 Commits

Author SHA1 Message Date
Austin Clements
8af7fac5a1 internal/sys: add LR and fixed frame size to sys.Arch
Storing this information in the Arch eliminates some code duplication
between the compiler and linker. This information is entirely
determined by the Arch, so the current approach of attaching it to an
entire Ctxt is a little silly. This will also make it easier to use
this information from tests.

The next CL will be a rote refactoring to eliminate the
Ctxt.FixedFrameSize methods.

Change-Id: I315c524fa66a0ea99f63ae5a2a6fdc367d843bad
Reviewed-on: https://go-review.googlesource.com/c/go/+/400818
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-19 15:59:19 +00:00
Josh Bleecher Snyder
901ac46df8 cmd/link: document generator symbols more
And make the existing docs easier to parse.

In particular, document the interaction with relocations.

Change-Id: I97579f88d9248c12e64af723a946959b88d4b674
Reviewed-on: https://go-review.googlesource.com/c/go/+/351112
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-09-21 16:34:40 +00:00
Austin Clements
48531da9e7 cmd/link: build dynexp symbol list directly
Currently, setCgoAttr populates the cgo_export_{static,dynamic} maps
with symbol names of exported symbols, which are then re-looked-up by
deadcode and setupdynexp, which in turn puts the re-looked-up symbols
in ctxt.dynexp. setCgoAttr already looked up the Syms, so simplify all
of this by making setCgoAttr populate ctxt.dynexp directly and
eliminating the cgo_export_{static,dynamic} maps. Recording Syms
directly also sets us up to use correct symbol versions for these
exports, rather than just assuming version 0 for all lookups.

Since setupdynexp doesn't really do any "setting up" of dynexp any
more with this change, we fold the remaining logic from setupdynexp
directly into addexport, where it has better context anyway. This also
eliminates a sorting step, since we no longer do a non-deterministic
map iteration to build the dynexp slice.

For #40724.

Change-Id: I3e1a65165268da8c2bf50d7485f2624133433260
Reviewed-on: https://go-review.googlesource.com/c/go/+/309340
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-04-13 20:07:46 +00:00
Jeremy Faller
9ae8f71c94 [dev.link] cmd/link: stop renumbering files for pclntab generation
Creates two new symbols: runtime.cutab, and runtime.filetab, and strips
the filenames out of runtime.pclntab_old.

All stats are for cmd/compile.

Time:
Pclntab_GC                   48.2ms ± 3%    45.5ms ± 9%     -5.47%  (p=0.004 n=9+9)

Alloc/op:
Pclntab_GC                   30.0MB ± 0%    29.5MB ± 0%     -1.88%  (p=0.000 n=10+10)

Allocs/op:
Pclntab_GC                    90.4k ± 0%     73.1k ± 0%    -19.11%  (p=0.000 n=10+10)

live-B:
Pclntab_GC                    29.1M ± 0%     29.2M ± 0%     +0.10%  (p=0.000 n=10+10)

binary sizes:
NEW: 18565600
OLD: 18532768

The size differences in the binary are caused by the increased size of
the Func objects, and (less likely) some extra alignment padding needed
as a result. This is probably the maximum increase in size we'll size
from the pclntab reworking.

Change-Id: Idd95a9b159fea46f7701cfe6506813b88257fbea
Reviewed-on: https://go-review.googlesource.com/c/go/+/246497
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-08-12 17:14:51 +00:00
Cherry Zhang
ee8541e5b8 [dev.link] cmd/link: remove OutData
OutData was used for a symbol to point to its data in the output
buffer, in order to apply relocations. Now we fold relocation
application to Asmb next to symbol data writing. We can just pass
the output data as a local variable.

Linking cmd/compile,

name         old time/op    new time/op    delta
Asmb_GC        19.0ms ±10%    16.6ms ± 9%  -12.50%  (p=0.032 n=5+5)

name         old alloc/op   new alloc/op   delta
Asmb_GC        3.78MB ± 0%    0.14MB ± 1%  -96.41%  (p=0.008 n=5+5)

name         old live-B     new live-B     delta
Asmb_GC         27.5M ± 0%     23.9M ± 0%  -13.24%  (p=0.008 n=5+5)

Change-Id: Id870a10dce2a0a7447a05029c6d0ab39b47d0a12
Reviewed-on: https://go-review.googlesource.com/c/go/+/244017
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-07-21 21:23:44 +00:00
Jeremy Faller
f033e23414 [dev.link] cmd/link: add generator symbols
Create a new class of symbols internal to the linker. These symbols live
in the Loader, and are real smybols, but have no data, only size. After
symbols are allocated in the binary in asmb() a function is called that
is responsible for filling in the data.

This allows the linker to create large symbols, but not pay the price on
the heap memory.

Change-Id: Ib4291fc6e578478057ed2ec163d7b27426f1d5ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/239280
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-06-25 18:42:27 +00:00
Cherry Zhang
6bf51c3406 [dev.link] all: merge branch 'master' into dev.link
Merge conflicts are mostly recently changed nm/objdump output
format and its tests. Resolved easily (mostly just using the
format on master branch).

Change-Id: I99d8410a9a02947ecf027d9cae5762861562baf5
2020-06-05 16:55:43 -04:00
Tobias Klauser
bffb8818e7 all: fix dead links to inferno-os bitbucket repository
Generated using:

  perl -i -npe 's#inferno-os/src/default#inferno-os/src/master#' $(git grep -l "inferno-os/src/default" | grep -v vendor)

Change-Id: I4b6443bd09a8ea4c8aaeb40a1c73520d1f7ca648
Reviewed-on: https://go-review.googlesource.com/c/go/+/235821
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-06-04 07:25:06 +00:00
Cherry Zhang
de1f07d56d [dev.link] cmd/link: delete sym.Symbols
Now the only thing it does is to track versions. Move it to ctxt.
And delete sym.Symbols.

Change-Id: Ie6b974f9bf79c4f33ace02213dcb89463eadd26a
Reviewed-on: https://go-review.googlesource.com/c/go/+/234884
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-05-21 21:19:17 +00:00
Cherry Zhang
1ccd585e94 [dev.link] cmd/link: remove "2" from names
Change-Id: I203caaf9cbe7136cf2060de7dc91c28f6ced1ee2
Reviewed-on: https://go-review.googlesource.com/c/go/+/234038
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-05-18 15:14:38 +00:00
Cherry Zhang
7f872be02f [dev.link] cmd/link: delete some deadcode
Change-Id: Ic4ca89ced19bf91e3d3ee10b7522d7ebe6b7c7fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/234116
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-05-18 15:06:22 +00:00
Cherry Zhang
b97212d1f1 [dev.link] cmd/link: delete sym.Symbol and sym.Reloc
This deletes all sym.Symbol and sym.Reloc references. This is
certainly not complete, and there are more cleanups to do. But I
feel this makes a good first round.

Change-Id: I7621d016957f7ef114be5f0606fcb3ad6aee71c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/234097
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-05-14 21:42:03 +00:00
Cherry Zhang
8e7ddac52d [dev.link] cmd/link: fix accidental shadowing
ctxt.Tlsg2 is supposed to be the embedded ArchSyms.Tlsg2.

Change-Id: I4f9711f83999d4a98bcf6d99c24fab756c580905
Reviewed-on: https://go-review.googlesource.com/c/go/+/230379
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-04-28 14:06:52 +00:00
Cherry Zhang
027055240f [dev.link] cmd/link: check fingerprint for index consistency
Previous CL introduced index fingerprint in the object files.
This CL implements the second part: checking fingerprint
consistency in the linker when packages are loaded.

Change-Id: I05dd4c4045a65adfd95e77b625d6c75a7a70e4f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/229618
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-24 17:48:07 +00:00
Than McIntosh
941de9760b [dev.link] cmd/link: begin converting dodata() to loader APIs
This patch begins the work of converting the linker's dodata phase to
work with loader APIs. Passes all.bash on linux/amd64, but hasn't been
tested on anything else (more arch-specific code needs to be written).
Use of the new dodata() phase is currently gated by a temporary
command line flag ("-newdodata"), and there is code in the linker's
main routine to insure that we only use the new version for the right
GOOS/GOARCH (currently restricted to ELF + AMD64).

Change-Id: Ied3966677d2a450bc3e0990e0f519b3fceaab806
Reviewed-on: https://go-review.googlesource.com/c/go/+/229706
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-24 14:09:38 +00:00
Cherry Zhang
245a2f5780 [dev.link] cmd/link: delete ctxt.Reachparent
It is no longer needed as we have converted the fieldtrack pass
to using the loader.

Also free loader.Reachparent after we are done with it.

Change-Id: Ibc4b29f282e1e4aea363a1b549755e31f84b0295
Reviewed-on: https://go-review.googlesource.com/c/go/+/229322
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-22 14:57:26 +00:00
Jeremy Faller
e77c99ce4c [dev.link] cmd/link: remove some globals from symtab.go
Change-Id: Ia2540779c1bf01248591568e1ddef1eef6edc20e
Reviewed-on: https://go-review.googlesource.com/c/go/+/227917
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-04-14 13:54:31 +00:00
Than McIntosh
db48c404cf [dev.link] cmd/link: convert pcln linker phase to use loader APIs
Rework the linker's pcln phase to work with the new loader. As part of
this set of changes the handling of "go.file..." symbols has been
revised somewhat -- previously they were treated as always live in the
loader, and now we no longer do this.

The original plan had been to have the new implementation generate
nameless "inltree" symbols, however the plan now is to keep them
named for now and convert them to nameless in a subsequent patch.

Change-Id: If71c93ff1f146dbb63b6ee2546308acdc94b643c
Reviewed-on: https://go-review.googlesource.com/c/go/+/227759
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-10 16:35:02 +00:00
Cherry Zhang
850c602ed6 [dev.link] cmd/link: convert text address assignment to new style
Implement text address assignment and trampoline generation using
the loader.

Note: the trampoline insertion part doesn't actually work. It
also needs to propagate Aux symbols for external symbols in
LoadFull.  But it won't be needed after converting pclntab
generation, so I'll leave it out for now. This could break
linking large binaries on PPC64 and ARM.

Change-Id: Ie46a35b25d7c027983dd877207cfa8f67c32530b
Reviewed-on: https://go-review.googlesource.com/c/go/+/227482
Reviewed-by: Than McIntosh <thanm@google.com>
2020-04-08 15:32:39 +00:00
Jeremy Faller
c46632a2e0 [dev.link] cmd/link: parallelize asmb on amd64
Introduces a parallel OutBuf implementation, and POC on amd64. Due to
some of the weird behaviors I saw on MacOS (SIGBUS while calling msync),
I will wait for feedback to port to other architectures.

On my mac, sped up Asmb by ~78% for cmd/compile (below). Will likely
have an appreciable speedup on kubelet benchmark.

Asmb                      39.1ms ±11%     8.5ms ±10%     -78.17%  (p=0.000 n=10+9)
TotalTime                  596ms ± 2%     577ms ± 8%      -3.07%  (p=0.034 n=8+10)

Change-Id: Id2a2577c3f4da155d8dccc862897f43b941877ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/223742
Run-TryBot: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-30 21:05:36 +00:00
Than McIntosh
e14efb6865 [dev.link] cmd/link: move setupdynexp before loadlibfull
Rewrite setupdynexp to work with loader.Sym, and move the call to it
before the body of loadlibfull. After loadlibfull is complete,
construct the old *sym.Symbol version of dynexp, since not all all
clients that access this list are converted to the loader APIs.

Change-Id: I347d24958e2f3e2332fbe33f2eb6ec25cc126bdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/224382
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-25 12:47:14 +00:00
Jeremy Faller
b328ab1d1e [dev.link] cmd/link: remove global datap, moved to *Link
This change moves datap from global space into the link context. Rather
than having it exist in context, we could have it returned from dodata,
and pass it as a parameter, but it is used in awkward places in the
Arch functions. Easiest for now is just keeping it in the context, until
we more formally move it to slices of loader.Sym.

This is a largely non-functional change.

Change-Id: Ica93bd857c39913ad470a61c63bc8d21704d6308
Reviewed-on: https://go-review.googlesource.com/c/go/+/222664
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-24 16:01:15 +00:00
Jeremy Faller
7d8aef689d [dev.link] cmd/link: remove *Link from relocsym
This is the last step requried before relocsym can go parallel.

Change-Id: Id1c1c530c2b9277917208c3767eeb29e02c17a9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/220841
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-04 20:36:34 +00:00
Jeremy Faller
4ab7ab54b0 [dev.link] cmd/link: remove *Link from architecture relocers
Also correct a few places I hadn't switched over to Target from *Link
context.

Change-Id: I6de6b39723762de3fa5f8d12800085421aca76fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/220840
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-04 18:55:08 +00:00
Jeremy Faller
d8b03c4678 [dev.link] cmd/link: remove some dependence on *Link
In an effort to make relocation application thread-safe remove another
dependence on context.

Change-Id: Ic53ea122cce72117fcebe56e386b710755f6eb68
Reviewed-on: https://go-review.googlesource.com/c/go/+/220838
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-28 21:24:32 +00:00
Jeremy Faller
e9056a6a73 [dev.link] cmd/link: create Target in order to make relocsym threadsafe
In its current form, relocsym requires the Link context -- largely in a
readonly state. Rather than passing around such heavy-weight object to
the function, this CL separates out the link target. From here, the
number of touchpoints (and thread unsafe portions) of relocing symbols
can be mentally reasoned about.

(NB: My personal intent is to make this more universal -- passing only
the necessary variables/state to reloc functions. This is a taste to see
if we like this form.)

Change-Id: Id9177d03267fedf0bb572a9d61bc07b64309c3b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/220837
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-28 21:04:36 +00:00
Jeremy Faller
76d0977745 [dev.link] cmd/link: remove flag FlagNewLdElf
No real reason to keep the old code around as the new stuff is working
just fine.

Change-Id: I5e886f8274344738663a0ead181c5d58d9f5a45f
Reviewed-on: https://go-review.googlesource.com/c/go/+/215997
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-28 16:26:14 +00:00
Than McIntosh
5279166402 [dev.link] cmd/link: convert Link.linksetup method to work with loader
Switch the linker's Link.linksetup method to work with loader.Sym
instead of sym.Symbols. Currently enabled when the new ELF host object
loader is turned on.

Change-Id: I336cc9f36166767baac574455531e195b6f1ac57
Reviewed-on: https://go-review.googlesource.com/c/go/+/213423
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-10 12:36:22 +00:00
Than McIntosh
fa284af720 [dev.link] cmd/link: convert ELF host object loading to new loader
This is a rewrite of the ELF host object loader to use just the Loader
interfaces for symbol creation, without constructing sym.Symbols. At
the moment this is gated under the temporary linker command line
option "-newldelf".  This version is able to get through all.bash
on linux/amd64.

Change-Id: I99f41368f75b0df9e35ef3c2cf2a702b732540c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/210779
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-09 20:48:48 +00:00
Than McIntosh
186e783730 [dev.link] cmd/link: keep loader symbol info in sym.CompilationUnit
In sym.Library and sym.CompilationUnit there are slices of *sym.Symbol
pointer that hold text symbols contained in the unit lib. To support
DWARF generation with new loader, add equivalent slices that hold
loader.Sym values for functions in scope. This will be needed if at
some point we push the sym.Symbol creation "wavefront" beyond dwarf
gen.

This patch also insures that live host object symbols are added to the
context Textp2 slice, since they would not make it on otherwise.
[NB: not sure if this is the best way to do this.]

Change-Id: I4f440e12cebc525b1e37082ad39cf7338aeb6b99
Reviewed-on: https://go-review.googlesource.com/c/go/+/208231
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-01-06 18:46:13 +00:00
Than McIntosh
c9470b0483 [dev.link] cmd/link/internal/objfile: relocate loader to new package
Third change of several to update the loader API to reflect the final
consensus version of the loader API as described in Cherry's doc.
This piece:

   - move objfile.Loader into its own separate package, and update
     clients accordingly.

This includes a few minor cleanups, including converting a couple
of loader-related functions to methods, and privatizing some of the
loader methods such as ToGlobal/ToLocal.

Change-Id: Iae20585751a45491d8b19dcffc096aadae6bbfc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/200998
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-16 13:24:55 +00:00
Cherry Zhang
cab29ebd84 [dev.link] cmd/link: create sym.Symbols after deadcode in newobj mode
With the new object files, now we can run the deadcode pass on
indices instead of Symbol structs, so we can delay creating
Symbols after the deadcode pass. Then we only need to create
reachable symbols.

Not create Symbols in LoadNew and LoadRefs, and recombine
LoadReloc into LoadFull.

Split loadcgo into two parts: the first finds root symbols, the
second create Symbols and sets attributes. The first runs before
the deadcode pass, while the second runs after.

TODO: currently there are still symbols that are not marked
reachable but still used. This includes DWARF symbols, file
symbols, and type symbols that are referenced by DWARF symbols.
We still need to create them (conservatively).

Change-Id: I695779c9312be9d49ab1683957ac3e72e1f65a1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/199643
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-15 18:55:32 +00:00
Cherry Zhang
2495095275 [dev.link] cmd/link: load full symbol contents after deadcode pass
If the new object file format is used, now we load full symbol
contents after the deadcode pass, for reachable symbols only.
We still load some informations early, like relocations and the
contents of type symbols, which are used in the deadcode pass.
If we rewrite deadcode to use index directly, we could delay more
of the loading (to sym.Symbol), and perhaps delay the creation of
sym.Symbol.

TODO: internal linking with host objects doesn't work yet.

Change-Id: I7d4880e8f150e8709ffac277e62191623440e4cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/197258
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-09 16:05:40 +00:00
Cherry Zhang
6051fd0ad9 [dev.link] cmd/link: support new object file format
Parse new object file format in the linker. At least we can link
a hello-world program.

Add a basic "loader", which handles symbol references in the
object file.
- mapping between local and global indices
- resolve by-name references
  (TODO: the overwrite logic isn't implemented yet)

Currently we still create sym.Symbol rather early, and, after all
the object files are loaded and indexed references are resolved,
add all symbols to sym.Symbols.

The code here is probably not going in the final version. This
is basically only for debugging purposes -- to make sure the
writer and the reader work as expected.

Change-Id: I895aeea68326fabdb7e5aa1371b8cac7211a09dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/196032
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-04 15:21:40 +00:00
Jeremy Faller
78a3734714 cmd/link: add notion of multiple compilation units per package
As we move the debug_line generation into the compiler, we need to
upgrade the notion of compilationUnit to not just be on a per package
basis.  That won't be the case as it will be impossible for all
compilationUnits to have the same set of files names used to build the
debug_lines table. (For example, assembled files in a package don't know
about any files but themselves, so the debug_lines table could only
reference themseves. As such, we need to break the 1:1 relationship
between compUnit and package.)

Change-Id: I2e517bb6c01de0115bbf777af828a2fe59c09ce8
Reviewed-on: https://go-review.googlesource.com/c/go/+/189618
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-24 21:33:10 +00:00
Than McIntosh
b1a20253fe cmd/link: memoize/cache whether plugin.Open symbol available
Perform a single lookup of "plugin.Open" at the point where we set the
loaded flag for the context, then cache whether the result is nil, so
that we can consult this cached value later on (instead of having to
look up the symbol each time). This helps speed up the DynLinkingGo()
context method, which is called from within some very hot loops in the
linker (when linking 'hyperkube' from kubernetes, reduces total calls
to "sym.(*Symbols).ROLookup" from 6.5M to 4.3M)

Change-Id: I92a2ea2b21d24f67aec0a7afeef4acc77c095adf
Reviewed-on: https://go-review.googlesource.com/c/go/+/193260
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-06 12:16:32 +00:00
Cherry Zhang
248444d5eb cmd/link: apply DWARF relocations while doing compression
We are preparing for applying relocations to the output buffer.
However, for DWARF compression, relocations need to be applied
before compression, but we don't have an output buffer at that
time. We also cannot delay DWARF compression to when we mmap the
output file, because we need the size of the DWARF sections to
compute the file size.

Instead of applying all the relocations together, we apply
relocations in DWARF sections one symbol at a time, right before
it is writing out for compression. As the symbol content may be
in read-only memory (in the future), we use a temporary buffer
for applying the relocations, and immediately write it out.

If compression is not used, relocations are still applied all
together.

This is in preparation for mmap'ing input files read-only.

Change-Id: Iae6d2dd71313897d5054bcc458d3bb78075b30c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/171397
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-04-19 18:25:20 +00:00
Josh Bleecher Snyder
6d64dd731b cmd/link: make Pciter more idiomatic
Rename it to PCIter and convert it to use methods.

Set pcscale once, during construction, to make call sites clearer.

Change some ints to bools.

Use a simple iteration termination condition,
instead of the cap comparison from the c2go translation.

Instead of requiring a Pcdata, which requires one caller
to synthesize a fake Pcdata, just ask for a byte slice.

Passes toolstash-check.

Change-Id: I811da0e929cf4a806bd6d70357ccf2911cd0c737
Reviewed-on: https://go-review.googlesource.com/c/go/+/171770
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-12 21:41:09 +00:00
Austin Clements
66065c3115 cmd/link: fix confusing error on unresolved symbol
Currently, if an assembly file includes a static reference to an
undefined symbol, and another package also has an undefined reference
to that symbol, the linker can report an error like:

  x: relocation target zero not defined for ABI0 (but is defined for ABI0)

Since the symbol is referenced in another package, the code in
ErrorUnresolved that looks for alternative ABI symbols finds that
symbol in the symbol table, but doesn't check that it's actually
defined, which is where the "but is defined for ABI0" comes from. The
"not defined for ABI0" is because ErrorUnresolved failed to turn the
static symbol's version back into an ABI, and it happened to print the
zero value for an ABI.

This CL fixes both of these problems. It explicitly maps the
relocation version back to an ABI and detects if it can't be mapped
back (e.g., because it's a static reference). Then, if it finds a
symbol with a different ABI in the symbol table, it checks to make
sure it's a definition, and not simply an unresolved reference.

Fixes #29852.

Change-Id: Ice45cc41c1907919ce5750f74588e8047eaa888c
Reviewed-on: https://go-review.googlesource.com/c/159518
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-29 02:01:16 +00:00
Austin Clements
1794ee6829 cmd/link: nice error message on ABI mismatch
Currently, if a symbol is only defined under one ABI and referenced
under another ABI, you simply get a "relocation target X not defined".
This is confusing because it seems like the symbol is defined.

This CL enhances the error message in this case to be "relocation
target X not defined for <ABI> (but is defined for <ABI>)".

For #27539.

Change-Id: If857a1882c3fe9af5346797d5295ca1fe50ae565
Reviewed-on: https://go-review.googlesource.com/c/147159
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-11-12 20:46:52 +00:00
Alessandro Arzilli
9c833831b2 cmd/link: move dwarf part of DWARF generation before type name mangling
Splits part of dwarfgeneratedebugsyms into a new function,
dwarfGenerateDebugInfo which is called between deadcode elimination
and type name mangling.
This function takes care of collecting and processing the DIEs for
all functions and package-level variables and also generates DIEs
for all types used in the program.

Fixes #23733

Change-Id: I75ef0608fbed2dffc3be7a477f1b03e7e740ec61
Reviewed-on: https://go-review.googlesource.com/111237
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-09-04 17:44:41 +00:00
Than McIntosh
32bc0976f9 cmd/link: use side table instead of sym.Symbol 'Reachparent' field
The sym.Symbol 'Reachparent' field is used only when field tracking
is enabled. So as to use less memory for the common case where
field tracking is not enabled, remove this field and use a side
table stored in the context to achieve the same functionality.

Updates #26186

Change-Id: Idc5f8b0aa323689d4d51dddb5d1b0341a37bb7d2
Reviewed-on: https://go-review.googlesource.com/121915
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-07-03 14:48:30 +00:00
Heschi Kreinick
726a2d04ea cmd/link: support DWARF compression on Darwin
We want to compress DWARF even on macOS, but the native toolchain isn't
going to understand it. Add a flag that can be used to disable
compression, then add Darwin to the whitelist used during internal
linking.

Unlike GNU ld, the Darwin linker doesn't have a handy linker flag to do
compression. But since we're already doing surgery to put the DWARF in
the output executable in the first place, compressing it at the same
time isn't unduly difficult. This does have the slightly odd effect of
compressing some Apple proprietary debug sections, which absolutely
nothing will understand. Leaving them uncompressed didn't make much
sense, though, since I doubt they're useful without (say) __debug_info.

Updates #11799

Change-Id: Ie00b0215c630a798c59d009a641e2d13f0e7ea01
Reviewed-on: https://go-review.googlesource.com/120155
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2018-06-22 19:16:14 +00:00
Iskander Sharipov
289ab30457 cmd/link: fix duplicated "unresolved inter-package jump" errors
This is extension to https://golang.org/cl/113955 that handled
duplicated "unresolved relocation" errors.

For platforms with trampoline support, additional error is generated
per each undefined symbol. This breaks TestUndefinedRelocErrors test
on these platforms.

Proposed fix:

	1. Changes error text to be identical to normal undefined reloc.
	   If relocation is undefined, jump to it will be unresolved
	   as well.

	2. Introduces a map that can be used by all sites that
	   handle this kind of errors which makes it easier
	   to report such errors exactly once.

Errors on ppc64 before this change (note first 4 lines):

	main.defined1: unresolved inter-package jump to main.undefined()
	main.defined1: unresolved inter-package jump to main.undefined()
	main.defined2: unresolved inter-package jump to main.undefined()
	main.defined2: unresolved inter-package jump to main.undefined()
	main.defined1: relocation target main.undefined not defined
	main.defined2: relocation target main.undefined not defined
	runtime.main_main·f: function main is undeclared in the main package

After this change:

	main.defined1: relocation target main.undefined not defined
	main.defined2: relocation target main.undefined not defined
	runtime.main_main·f: function main is undeclared in the main package

Because of (1), errors output is the same on all platforms now.

Fixes #25753

Change-Id: Ic3084202a6fc5d4a6d2d0a93344f012b37fe58ed
Reviewed-on: https://go-review.googlesource.com/116676
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-06-08 21:32:33 +00:00
David Crawshaw
38cfeb9cb5 cmd/link: move Headtype global to ctxt
For #22095

Change-Id: Idcfdfe8a94db8626392658bb93429454238f648a
Reviewed-on: https://go-review.googlesource.com/70835
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-21 00:51:14 +00:00
David Crawshaw
d05f82a11a cmd/link: move Iself global to ctxt
For #22095

Change-Id: Iba3dffc782cecc15ea0e90a971a2734729984945
Reviewed-on: https://go-review.googlesource.com/70834
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-20 23:41:16 +00:00
David Crawshaw
6738c494ad cmd/link: move FlagLinkshared global to ctxt
For #22095

Change-Id: Ica6b3391541fe5a0355620d7c4a5107cf53eee82
Reviewed-on: https://go-review.googlesource.com/70833
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-20 23:40:56 +00:00
Daniel Martí
966c459fa4 cmd/link: various cleanups using tools
* remove unnecessary explicit types
* remove dead assignments
* remove unused fields
* unindent code using early continues
* remove some unnecessary type conversions
* remove some unused func parameters

Change-Id: I202c67e92940beacbd80fc2dc179f9556dc5d9e5
Reviewed-on: https://go-review.googlesource.com/69118
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2017-10-09 22:08:17 +00:00
David Crawshaw
2e8545531e cmd/link: move build/link mode globals into ctxt
Replace Buildmode with BuildMode and Linkmode with LinkMode.

For #22095

Change-Id: I51a6f5719d107727bca29ec8e68e3e9d87e31e33
Reviewed-on: https://go-review.googlesource.com/68334
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-10-06 13:49:11 +00:00
David Crawshaw
a910fe2c83 cmd/link: move Library type to sym package
For #22095

Change-Id: I2cb0d3e0aaf9f97952cf8dda0e99a4379e275020
Reviewed-on: https://go-review.googlesource.com/68332
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-05 12:21:19 +00:00