147 Commits

Author SHA1 Message Date
Cherry Zhang
3e6ff18247 [dev.link] cmd/link, cmd/oldlink: detect object file format mismatch
When using the new(old) linker but an old(new) object file is
found, give a better error message.

Change-Id: I94786f1a4b527c15c4f5b00457eab60d215a72a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/225457
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-30 13:52:55 +00:00
Than McIntosh
912e64ba46 [dev.link] cmd/link/internal/loader: add Loader plt/got access methods
Add accessor methods to get at the symbol {plt,got} value for
PE symbols. Fix a bug in the loaders SetPlt/SetGot methods.

Change-Id: I975bd6b86122622b206487c8798f8290ecd25a57
Reviewed-on: https://go-review.googlesource.com/c/go/+/225199
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-28 23:59:16 +00:00
Cherry Zhang
c7e3d817ab [dev.link] cmd/link: migrate symbol's Value and Align when converting to sym.Symbols
Currently, in LoadFull we migrate a symbol's Value to sym.Symbol
only for external symbols. And symbol's Align is not migrated at
all. As we move LoadFull forward, there are already places where
we set symbol's Value and Align (e.g. in doelf). Migrate them
correctly.

Currently I think we only set them on external symbols, but as
we move forward I think we'll need to set them on Go symbols as
well.

Change-Id: I63e97e38fc08b653ba9faefe15697944faf21bed
Reviewed-on: https://go-review.googlesource.com/c/go/+/225658
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-27 00:06:39 +00:00
Cherry Zhang
b1a19f3cc7 [dev.link] cmd/link: convert doxcoff to new style
Change-Id: Ic1e4ed6c14e049b1ba2f7c00f986433ab7ebe932
Reviewed-on: https://go-review.googlesource.com/c/go/+/225202
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-25 15:20:40 +00:00
Than McIntosh
e14efb6865 [dev.link] cmd/link: move setupdynexp before loadlibfull
Rewrite setupdynexp to work with loader.Sym, and move the call to it
before the body of loadlibfull. After loadlibfull is complete,
construct the old *sym.Symbol version of dynexp, since not all all
clients that access this list are converted to the loader APIs.

Change-Id: I347d24958e2f3e2332fbe33f2eb6ec25cc126bdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/224382
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-25 12:47:14 +00:00
Than McIntosh
3fa98a0436 [dev.link] cmd/link: minor tweaks to PropagateLoaderChangesToSymbols
Update PropagateLoaderChangesToSymbols so that it no longer requires
a sym.Symbols pointer. The intent is to generalize it a little to
allow it to be used in more than just linker Dwarf generation.

Change-Id: I9bddc5d39839eacd9113c945bb59d2873c0b088c
Reviewed-on: https://go-review.googlesource.com/c/go/+/224381
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-25 12:22:56 +00:00
Cherry Zhang
ea93ddfaeb [dev.link] cmd/link: use Extname in doxcoff
On AIX, when external linking, we need to change the function
names to start with a dot and make function descriptors with the
names without the dot. Currently this is done through symbol
renaming, which is not friendly for switching to the loader.

In this CL we use symbol's external name for this. This allows us
to get rid of symbol renaming.

Change-Id: If72602d17e96f0339fdac2e2321f1edfb292b5f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/224940
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-24 16:43:33 +00:00
Cherry Zhang
ed7a491940 [dev.link] cmd/link: convert mangleTypeSym to new style
Use symbol's Extname, instead of symbol renaming, for the mangled
names.

The old symbol Rename has an interesting logic of "merging"
symbols, when a symbol is renamed to the name of an existing
symbol. It turns out that this is needed for linking against
shared libraries, where the Go object has a reference to a symbol
with the original name, and the shared libary provides a symbol
under the mangled name. Implement this logic with the loader.

Change-Id: Ib95d7a9c93a52f8e02f4a51ac67240d6ebfc1c6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/224939
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-24 16:43:18 +00:00
Than McIntosh
d44c9a046b [dev.link] cmd/link: set "local" attribute on symbols during preload
Set the "local" attribute on object file loader symbols during preload,
as opposed to the current strategy of only propagating the local attr
when we convert to sym.Symbols.

Change-Id: I18965e2fdaa9a0b9315d95466d17d9f41aa40a45
Reviewed-on: https://go-review.googlesource.com/c/go/+/224379
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-23 15:10:28 +00:00
Than McIntosh
49099d21f5 [dev.link] cmd/link: add loader.Loader apis for symbol dynamic id
Add SymDynid and SetSymDynid methods to the loader. This symbol
property is currently backed by a map.

Change-Id: Iaf86b1d8aaa775fa102fadea30394eb8a670e0e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/224378
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-23 15:10:20 +00:00
Than McIntosh
7055f01e12 [dev.link] cmd/link: minor perf tweak for PropagateLoaderChangesToSymbols
When fixing up relocations in PropagateLoaderChangesToSymbols, don't
reallocate the target sym.Symbol relocation slice if it already has
the desired size (this gets rid of some unneeded allocations).

Change-Id: I05287772c18cab861c2df805fa9497103fb00dcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/224420
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-20 20:31:35 +00:00
Than McIntosh
0dff5b0b9e [dev.link] cmd/compile, cmd/link: move away from DWARF file renumbering
This patch moves the compiler and linker away from the current scheme
used to generate file references in DWARF subprogram dies.

Up until now the scheme has been to have the compiler emit a special
relocation on a DIE file reference that points to the file symbol in
question. The linker then reads this relocation and updates the addend
to the index of the appropriate file in the line table of the
compilation unit of the DIE (the linker emits the comp unit file
table, so it knows at that point what number use). The drawback of
this scheme is that it requires a lot of relocation processing.

With this patch, we switch to having the compiler emit the file index
directly, and then have the linker use the compiler-generated file
table to emit the line table file section (no renumbering, no
relocations, etc).

Change-Id: Id4fbe67b28a64200a083e3c5ea358dbe091ec917
Reviewed-on: https://go-review.googlesource.com/c/go/+/223318
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-20 16:19:22 +00:00
Than McIntosh
673a02a235 [dev.link] cmd/compile, cmd/link: move DWARF info sym to anonymous aux data
Switch the primary subprogram die DWARF symbol emitted by the compiler
from named+dupOK to anonymous aux. This should help performance wise
by not having to add these symbols to the linker's symbol name lookup
tables.

Change-Id: Idf66662b8bf60b3dee9a55e6cd5137b24a9f5ab6
Reviewed-on: https://go-review.googlesource.com/c/go/+/223669
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-20 16:19:15 +00:00
Than McIntosh
63ffa1595c [dev.link] cmd/link: cleanup of attribute handling code
Minor cleanup of symbol attributes. Specifically:

- if a symbol originally begins life as an object file symbol,
  then is converted to external in cloneToExternal, use the
  previously recorded object file index for the sym to figure
  out if it has read-only data (in the case that there is no
  entry for it in the map in question).

- remove SetAttrShared; only the loader should be populating this
  attribute at symbol creation (it never gets updated later)

- remove unused copyAttributes() method

- comment fixes

Change-Id: Iac7bc6ac310ec89bfe733ddc783970d2a8017478
Reviewed-on: https://go-review.googlesource.com/c/go/+/223667
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-18 22:49:17 +00:00
Cherry Zhang
626c89bfa3 [dev.link] cmd/link: remove AddExtSym
They used to be different at some point, but now AddExtSym and
LookupOrCreateSym are identical. Remove one.

Change-Id: I299444d987e32a7f43915b3c1bbcc5ae906e9b6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/223977
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-18 22:26:24 +00:00
Cherry Zhang
8a074fa275 [dev.link] cmd/internal/goobj2, cmd/link: experiment new aux symbol accessors
Following the previous CLs, do the same for aux symbols. This has
some small speedup:

(linking cmd/compile)
Dostkcheck    41.0ms ± 1%    38.6ms ± 1%    -6.00%  (p=0.008 n=5+5)

Change-Id: Id62b2fc9e4ef1be92e60e4c03faec0a953eee94e
Reviewed-on: https://go-review.googlesource.com/c/go/+/222303
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-16 17:33:43 +00:00
Cherry Zhang
8e100a05a5 [dev.link] cmd/internal/goobj2, cmd/link: avoid some repeated offset calculations
When iterating relocations, do the offset calculation just once.
This gives some speedup:

(linking cmd/compile)
Deadcode      52.8ms ± 1%    47.6ms ± 1%  -10.01%  (p=0.008 n=5+5)
Dostkcheck    44.2ms ± 1%    41.0ms ± 1%   -7.29%  (p=0.008 n=5+5)

Change-Id: I09e38bc29afc379a81f99e3ee4ff467bc1b5f8a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/222302
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-16 17:33:34 +00:00
Cherry Zhang
0e06775eeb [dev.link] cmd/link: don't truncate external relocation type
In CL 222244, we store external relocations in the same format as
Go object files. In Go object file format, the relocation type is
a uint8. However, for external relocations the type may not
always fit in a uint8. Truncating it will result in a bad
relocation. Fix this by storing the external reloc type on the
side. (An alternative is to extend the Go object file format to
use a uint16, but it is not necessary for Go relocations and
will waste some binary size.)

Fix ARM build.

Change-Id: I343e240d38ee0e2cc91e0e7754d03b19b525a014
Reviewed-on: https://go-review.googlesource.com/c/go/+/223338
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-13 19:28:06 +00:00
Cherry Zhang
e5f7a8334a [dev.link] cmd/link: convert dope to new style
Also convert a manually managed array to slice.

Change-Id: I7d0dd3d5f569ab237893f589b6022f0f351bca16
Reviewed-on: https://go-review.googlesource.com/c/go/+/223337
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-13 17:26:32 +00:00
Cherry Zhang
cf5c718cba [dev.link] cmd/link: experiment new reloc accessors in deadcode pass
There is a small speedup:

(linking cmd/compile)

name       old time/op    new time/op    delta
Deadcode     57.1ms ± 1%    53.5ms ± 1%   -6.44%  (p=0.008 n=5+5)

With this, we don't need a slice to read the relocations, reduce
some allocations.

name       old alloc/op   new alloc/op   delta
Deadcode     4.16MB ± 0%    3.84MB ± 0%   -7.85%  (p=0.008 n=5+5)

Change-Id: Icd41c05682ba3f293a8cb9d2fe818e39d7276e5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/222244
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-13 01:19:15 +00:00
Cherry Zhang
c951514da9 [dev.link] cmd/internal/goobj2, cmd/link: experiment another way of accessing relocations
Use a different mechanism to access relocations from the object
files, and use it in the stack bounds check pass. This shows some
speedup.

(linking cmd/compile)
Dostkcheck     76.9ms ± 1%    55.1ms ± 1%  -28.36%  (p=0.008 n=5+5)

Change-Id: I2ac42da515dccd64719fb557ffff6cdc69e4319b
Reviewed-on: https://go-review.googlesource.com/c/go/+/222240
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-13 01:04:29 +00:00
Than McIntosh
8634642f9a [dev.link] cmd/link: demote dwarf {range,loc} sub-symbols to aux
When the compiler emits DWARF for a function F, in addition to the
text symbol for F, it emits a set of sibling or child symbols that
carry the various DWARF bits for F (for example, go.info.F,
go.ranges.F, go.loc.F, and so on).

Prior to the linker modernization work, name lookup was the way you
made your way from a function symbol to one of its child DWARF
symbols. We now have a new mechanism (aux symbols), so there is really
no need for the DWARF sub-symbols to be named or to be dupok.

This patch converts DWARF "range" and "loc" sub-symbols to be pure aux
syms: unnamed, and connected to their parent text symbol only via aux
data. This should presumably have performance benefits in that we add
fewer symbols to the linker lookup tables.

Other related DWARF sub-symbols (ex: go.line.*) will be handled in a
subsequent patch.

Change-Id: Iae3ec2d42452962d4afc1df4a1bd89ccdeadc6e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/222673
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-12 18:56:09 +00:00
Than McIntosh
b0b185c5fa [dev.link] cmd/link: remove patchDWARFName from the linker
Remove the various bits of code in the linker that patch DWARF name
attributes (rewrite "". to packagepath).

Change-Id: I29a38d680d411ccf1c1eeb760f2d15fc8ec22d90
Reviewed-on: https://go-review.googlesource.com/c/go/+/222719
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-10 18:59:18 +00:00
Than McIntosh
d86e94a4c8 [dev.link] cmd/link: remove debugging code
Remove a few lines of debugging trace output code left in
by accident when the final version of CL 220987 was submitted.

Change-Id: Ib84a4658194266597140a0e88a2f90f57f78cf53
Reviewed-on: https://go-review.googlesource.com/c/go/+/222299
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-06 15:45:15 +00:00
Than McIntosh
0c41a37c6c [dev.link] cmd/link: convert second phase of DWARF-gen to use loader
This converts the second portion of DWARF-gen (dwarfGenerateDebugSyms
function and friends) to use the loader, as opposed to sym.Symbols.

Second phase is invoked as part of dodata(), which is fairly far along
in the linker, meaning that the "converted to loader.Symbol" wavefront
has not yet reached that point.

To account for this, the patch contains some shim code that runs
initially to copy back things from sym.Symbols into the loader
(notable symbol values), and then a second hook to be code after
dwarf-gen is complete to that copies the results back into the
appropriate sym.Symbols. This code is obviously temporary, but needed
for the time being.

Change-Id: I9b830b08b16480a1a5230cb52b592db1c550af18
Reviewed-on: https://go-review.googlesource.com/c/go/+/220987
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-05 16:50:21 +00:00
Than McIntosh
4a6c1caae2 [dev.link] cmd/link: small cleanup of loader ocache
Remove references to the loader.Loader's "ocache" field, which is no
longer being used the way it was. Add in a new field for object index
to the oReader type.

Change-Id: Icae6c54dbbdbbfa63ab9de3e4dc09e10abd57ee3
Reviewed-on: https://go-review.googlesource.com/c/go/+/222063
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-05 16:21:07 +00:00
Than McIntosh
ef44bd8413 [dev.link] cmd/link: expose loader PatchDWARFName hook for asm subprogram DIEs
Add a new loader method PatchDWARFName to patch up live DWARF function
DIEs during DWARF generation. This is needed to handle subprogram DIE
symbols emitted by the assembler, which still embeds "". package
tokens into the data sections of these dies.

Note: this is expected to be a temporary hack, as we are going to
transition the assembler to do the patching itself when passed the
"-p" option (once this happens the plan is to toss all of the various
PatchDWARFName helpers).

Change-Id: Id689a751f08d7f4c096c4ac2f99991f9641959e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/220986
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-05 16:20:57 +00:00
Than McIntosh
85d62a91ee [dev.link] cmd/link: fix up ctxt.Textp2 handling in AssignTextSymbolOrder
Change the loader method AssignTextSymbolOrder to return a slice of
all reachable textp symbols, since it will be needed in second-phase
DWARF gen.

Change-Id: Iaf16ee9cf0d5266aeb0d3df596e8117263b35d8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/220985
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-05 16:20:42 +00:00
Cherry Zhang
6ae47d1354 [dev.link] cmd/link: don't always load type symbols
The type symbols were always loaded in LoadFull, as the old
sym.Symbol-based DWARF generation needs them to present. Now we
have converted the DWARF type generation pass, which uses the
loader symbols directly. No need to load the type symbols if they
are not live.

This reduces loaded symbols by ~18% for cmd/compile.

Change-Id: Ic1f6fcdff845f3c62784d05ad716b1742f249481
Reviewed-on: https://go-review.googlesource.com/c/go/+/222018
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-04 16:07:49 +00:00
Cherry Zhang
5ee515fd8c [dev.link] cmd/link: remove sym.Symbols lookup map
Let the loader provide lookup functionalities.

Change-Id: I7d90166d12e6e6fd30e75cbda5d3097e93f9af1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/221137
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-03 15:34:18 +00:00
Than McIntosh
3fd7d78dec [dev.link] cmd/link: fix up 'top frame' attribute in preload
Populate the 'top frame' attribute for function symbols during preload
(needed for DWARF debug frame generation).

Change-Id: If27836f51ebafa3c1c29b268a22d72d1b59b505a
Reviewed-on: https://go-review.googlesource.com/c/go/+/220983
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-28 14:58:15 +00:00
Cherry Zhang
23c52de5fc [dev.link] cmd/link: export loader's symbol bitmap
Export loader's bitmap. As more things converted to new style,
I think it may be not uncommon to have bitmaps for symbols used
outside of the loader.

Change-Id: Ida68dac87988957a29985143121d781296820fcf
Reviewed-on: https://go-review.googlesource.com/c/go/+/220980
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-26 17:38:11 +00:00
Cherry Zhang
e00da38b81 [dev.link] cmd/internal/goobj2, cmd/link: add accessors for field of FuncInfo
Add accessors for fields of FuncInfo, so we don't have to read
the whole FuncInfo.

TODO: explore/experiment with an alternative idea -- splitting
FuncInfo to separate Aux symbols.

Change-Id: Ie4bc2613fd76d08fc63fd86956802920da63dd2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/220979
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-25 20:34:34 +00:00
Cherry Zhang
0f99ab4e86 [dev.link] cmd/internal/goobj2: move NoSplit to symbol flag
Move NoSplit flag from FuncInfo to symbol flag, so this can be
accessed easily without reading the FuncInfo.

The CFunc flag is never used. Remove.

Change-Id: I8bf4fcb2f209434bb90ccc4987a4c3f28f003323
Reviewed-on: https://go-review.googlesource.com/c/go/+/220058
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-19 23:40:15 +00:00
Than McIntosh
25f1b093fe [dev.link] cmd/link: convert DWARF type generation to use loader
Converts the portion of DWARF generation that deals with creation of
type DIEs and constant DIEs to use the new loader interfaces. Creation
of subprogram DIE and compilation unit DIE content still operates on
sym.Symbols at the moment, and happens much later in the linker.

The new code for type DIE generation is gated/guarded by the linker
flag "-newdw", which currently defaults to true. At some point in the
near future this flag should be removed, but it is handy for triage at
the moment.

This patch also includes shim code designed to run after loadlibfull()
that walks through the DIE chains and to converts loader.Sym
references back into sym.Symbol references for the remainder of the
compilation, since the second phase of DWARF has not yet been
converted.

Change-Id: I681a00fb8a1f3c37884a79b373d86411332e07c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/208230
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-19 21:15:12 +00:00
Jeremy Faller
860f12a2fc [dev.link] cmd/link: dostrdata and fieldtrack with new syms
Move the wavefront past fieldtrack and dostrdata.

Change-Id: Ia327ece0202e24031fec7e1f70b40e15fbb4f728
Reviewed-on: https://go-review.googlesource.com/c/go/+/219226
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-18 15:29:12 +00:00
Cherry Zhang
5a8b1509dd [dev.link] cmd/link: clean up unused function and arguments
Change-Id: Idacb027c5ae11e029ee4ab10f6c9a76ff435e5b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/219228
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-13 15:47:21 +00:00
Cherry Zhang
b0275bfbf4 [dev.link] cmd/link: add defined package symbols first, before any non-package symbols
Currently, the loader adds defined package symbols and
non-package symbols to the global index space object by object.
This CL changes it to add all the defined package symbols first,
then all the non-package symbols. The advantage of doing this is
that when adding package symbols, by definition they cannot be
dup to each other, so we don't need to do a name lookup when
adding them.

We still add them to the lookup table (for now), since they may
still be referenced by name (e.g. through linkname). This CL is
also a prerequisite if we want to move to not adding package
symbols to the lookup table entirely (e.g. by using
pre-generated in-file lookup table).

Also update some comments to reflect the current state.

Change-Id: Ib757e070b48a9ef6215e47dc3421fc5c055b746c
Reviewed-on: https://go-review.googlesource.com/c/go/+/219078
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-12 22:07:52 +00:00
Cherry Zhang
ab1253b59a [dev.link] cmd/link: don't read symbol name if not needed
Most of the time of goobj2.Sym.Read is spent in reading the
symbol name. Sometimes the name is not needed, so don't read
it.

This brings the linking time pretty much back to the old linker.
On my Mac,

$ time go build cmd/compile     # (new)

real	0m1.016s
user	0m1.164s
sys	0m0.297s

$ time go-tip build cmd/compile # (old)

real	0m0.980s
user	0m1.229s
sys	0m0.309s

Again, this is a bit ugly. Maybe we want to consider moving
symbol names out of Sym, to a separate section of the object
file?

Change-Id: Ia9085c15aa4d614b7eae126e71a79d6806c75055
Reviewed-on: https://go-review.googlesource.com/c/go/+/218480
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-07 20:50:06 +00:00
Cherry Zhang
38437ce118 [dev.link] cmd/link: only read necessary fields in deadcode pass
Reading unnecessary fields from object file take time, even from
memory mapping. In deadcode, we do this a alot, so only read
necessary ones. In particular, for relocations, for non-type
symbols, we only need their target symbols and reloc types to
build the dependency graph, so don't read other fields. This
should speed up the deadcode pass.

Also cache an object's number of defined package symbols, as it
is accessed a lot in resolve.

This is a bit ugly from an API point of view. It would be nice if
the compiler could figure out some fields are not used so don't
fill them.

Change-Id: I5c4e9526b8a3b0aead9fa71901a51fb214f013a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/218479
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-07 20:39:23 +00:00
Cherry Zhang
06e2339dfe [dev.link] cmd/link: clean up some fields
With the new index mapping , we can clean up some fields.
Loader.max should always be equal to len(loader.objSyms) now. And
for external symbols we now give its "local index" as its index
in the payload array, so Relocs.extIdx is no longer useful.
Delete those fields.

Change-Id: If387ff9201ea0d347b954f651f5d4b4ae74937aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/218478
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-07 18:33:37 +00:00
Cherry Zhang
2f22143cd5 [dev.link] cmd/link: remove holes from global index space
In CL 217064, we made symbol's global index unique, but we still
reserve index space for each object file, which means we may
leave holes in the index space if the symbol is a dup or is
overwritten. In this CL, we stop reserving index spaces. Instead,
symbols are added one at a time, and only added if it does not
already exist. There is no more holes in the index space.

Change-Id: I3c4e67163c556ba1198e13065706510dac4692fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/217519
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-05 20:53:18 +00:00
Cherry Zhang
7cf907606d [dev.link] cmd/link: remove sym.Symbol based host object support
We have converted all the host object readers to the new loader
based mechanism, and no longer do sym.Symbol host object loading.
Remove that support. So we don't have to take care of that in
future changes to the loader.

TODO: there are still code outside the loader than can be
removed. This CL only focuses on the loader.

Change-Id: I67dd75f631964921a7c7a6576ed95b071241484a
Reviewed-on: https://go-review.googlesource.com/c/go/+/217518
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-05 20:51:57 +00:00
Cherry Zhang
29d95be875 [dev.link] cmd/link: simplify named symbol resolution
Now that we have local-global index mappings, just use that for
symbol reference resolution.

Change-Id: I6bc5405853fe040ff21b624ccd8da7965d66ec8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/217065
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-05 20:50:45 +00:00
Jeremy Faller
83ba044be6 [dev.link] cmd/link: use new sym format in pe loader
Change-Id: Ib784b8432ff4355b7ff4068801a0bcfcaf108950
Reviewed-on: https://go-review.googlesource.com/c/go/+/216718
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-05 20:12:48 +00:00
Cherry Zhang
8e2b5d3e71 [dev.link] cmd/link: make symbol's global index unique
Currently, when mapping symbol's local index to global index, for
duplicated and overwriting/overwritten symbols, each appearance
of the symbol gets a global index, with one being the "primary",
and others "redirect" to it through the overwrite map. Basically,
the local-global index mapping is one to one, with overwrite/
dedup happening in global index level.

This has a few drawbacks:
- All symbol accesses effectively need to query the overwrite
  map. This may hurt performance.
- For multi-level overwrites, (Y overwrites X, Z overwrites Y),
  this can get quite complicated, and we have to follow the
  redirection recursively.
- Failed to follow or to update the overwrite map leads to bugs.

In this CL, we change the index mapping mechanism so that each
symbol get a unique global index. Multiple appearances of the
same symbol get the same index. Now the local-global index
mapping is N to one. Overwrite/dedup happens directly in the
local-global mapping.

We keep both mapping directions in arrays. Each object carries
an array for its local-global mapping. The loader carries an
array mapping global index to the "primary" local index, which is
the one we should load from. This way, we can get rid of the
overwrite map, and index conversions are simply array accesses.

TODO: we still make reservation of the index space upfront, and
leave holes for dup symbols. Maybe get rid of the reservation and
holes.

Change-Id: Ia251489d5f2ff16a0b3156a71d141a70cdf03a4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/217064
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-01-31 23:08:46 +00:00
Cherry Zhang
b1f9f47982 [dev.link] cmd/link: fix payload pointer liveness
Currently, the symbol updater uses a pointer pointing to the
loader's payloads array. If the payloads slice grows (and moves),
the pointer may become stale and no longer point to the symbol's
actual payload. Specifically, consider

	sb, sym := l.MakeSymbolUpdater(...)
	// add a bunch of external symbols, which grows payload slice
	sb.SetType(t)
	l.SymType(sym) // may not return t

sb.SetType on line 3 may not have the desired effect, as
sb.extSymPayload may no longer point to the right payload. As a
result, the type we get on line 4 may be not the one we set.

Fix this by making the payload's address permanent. Once it is
allocated it will never move.

Change-Id: Iab190ea5aceb5c37f91d09ad4ffd458e881b03f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/217063
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-01-31 17:41:52 +00:00
Cherry Zhang
af98efc545 [dev.link] cmd/link: correct overwrite logic
This was mistakenly changed during the refactor in  CL 201728.
Restore the old behavior.

Change-Id: I9991859e7587f5f567bbe86ae19dede904b3a399
Reviewed-on: https://go-review.googlesource.com/c/go/+/217062
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-01-31 17:40:47 +00:00
Jeremy Faller
4ad94a5db9 [dev.link] cmd/link: use new sym builders in macho loader
Change-Id: Ia055559d1eb12736d0bdd5a30103cd4b9788d36e
Reviewed-on: https://go-review.googlesource.com/c/go/+/215917
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-28 16:26:01 +00:00
Than McIntosh
2d2590e94a [dev.link] cmd/link: add loader methods to access unit, gotype
Add loader hooks to get at the 'gotype' and 'unit' symbol properties.

Change-Id: Icdf5dd151e84a548fa2574efdb998c3b698a17d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/214137
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-10 12:37:31 +00:00