Since CL 522318, all closures are now hidden. Thus this CL removes all
codes that worries about hidden vs non-hidden closures.
Change-Id: I1ea124168c76cedbfc4053d2f150937a382aa330
Reviewed-on: https://go-review.googlesource.com/c/go/+/523275
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Runtime functions, e.g. internal/abi.NoEscape, should not be
instrumented with checkptr. But if they are inlined into a
checkptr-enabled function, they will be instrumented, and may
result in a check failure.
Let the compiler not inline runtime functions into checkptr-
enabled functions.
Also undo the change in the strings package in CL 598295, as the
compiler handles it now.
Fixes#68511.
Updates #68415.
Change-Id: I78eb380855ac9dd53c1a1a628ec0da75c3e5a1a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/599435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
Modify rangefunc #next protocol to make it more robust
Extra-terrible nests of rangefunc iterators caused the
prior implementation to misbehave non-locally (in outer loops).
Add more rangefunc exit flag tests, parallel and tricky
This tests the assertion that a rangefunc iterator running
in parallel can trigger the race detector if any of the
parallel goroutines attempts an early exit. It also
verifies that if everything else is carefully written,
that it does NOT trigger the race detector if all the
parts run time completion.
Another test tries to rerun a yield function within a loop,
so that any per-line shared checking would be fooled.
Added all the use-of-body/yield-function checking.
These checks handle pathological cases that would cause
rangefunc for loops to behave in surprising ways (compared
to "regular" for loops). For example, a rangefunc iterator
might defer-recover a panic thrown in the syntactic body
of a loop; this notices the fault and panics with an
explanation
Modified closure naming to ID rangefunc bodies
Add a "-range<N>" suffix to the name of any closure generated for
a rangefunc loop body, as provided in Alessandro Arzilli's CL
(which is merged into this one).
Fix return values for panicky range functions
This removes the delayed implementation of "return x" by
ensuring that return values (in rangefunc-return-containing
functions) always have names and translating the "return x"
into "#rv1 = x" where #rv1 is the synthesized name of the
first result.
Updates #61405.
Change-Id: I933299ecce04ceabcf1c0c2de8e610b2ecd1cfd8
Reviewed-on: https://go-review.googlesource.com/c/go/+/584596
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Tim King <taking@google.com>
This appears to be useful only on amd64, and was specifically
benchmarked on Apple Silicon and did not produce any benefit there.
This CL adds the assembly instruction `PCALIGNMAX align,amount`
which aligns to `align` if that can be achieved with `amount`
or fewer bytes of padding. (0 means never, but will align the
enclosing function.)
Specifically, if low-order-address-bits + amount are
greater than or equal to align; thus, `PCALIGNMAX 64,63` is
the same as `PCALIGN 64` and `PCALIGNMAX 64,0` will never
emit any alignment, but will still cause the function itself
to be aligned to (at least) 64 bytes.
Change-Id: Id51a056f1672f8095e8f755e01f72836c9686aa3
Reviewed-on: https://go-review.googlesource.com/c/go/+/577935
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Currently in a lot of packages we define functions for appending/decoding
mostly BigEndian data (see internal/chacha8rand, net/netip,
internal/boring/sha, hash/crc64, and probably more), because we don't
want to depend on encoding/binary, because of #54097.
This change introduces a new package internal/byteorder, that
will allow us to remove all of the functions and replace them with
internal/byteorder.
Updates #54097
Change-Id: I03e5ea1eb721dd98bdabdb25786f889cc5de54c5
GitHub-Last-Rev: 3f07d3dfb453a9e679395711f9b93e25f9340a3b
GitHub-Pull-Request: golang/go#67183
Reviewed-on: https://go-review.googlesource.com/c/go/+/583298
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Commit-Queue: Ian Lance Taylor <iant@golang.org>
Minor refactoring to eliminate one of the ir.Name flag values used
when building in coverage mode (no changes to functionality). This is
intended to free up a bit in the uint16 flags field to be used in a
subsequent patch.
Change-Id: I4aedb9a55fde24c808ff3f7b077ee0552aa979af
Reviewed-on: https://go-review.googlesource.com/c/go/+/572055
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This helps reduce confusion with cmd/internal/pgo, which performs
compilation-independent analysis. pgoir associates that data with the
IR from the current package compilation.
For #58102.
Change-Id: I9ef1c8bc41db466d3340f41f6d071b95c09566de
Reviewed-on: https://go-review.googlesource.com/c/go/+/569338
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The processing performed in cmd/preprofile is a simple version of the
same initial processing performed by cmd/compile/internal/pgo. Refactor
this processing into the new IR-independent cmd/internal/pgo package.
Now cmd/preprofile and cmd/compile run the same code for initial
processing of a pprof profile, guaranteeing that they always stay in
sync.
Since it is now trivial, this CL makes one change to the serialization
format: the entries are ordered by weight. This allows us to avoid
sorting ByWeight on deserialization.
Impact on PGO parsing when compiling cmd/compile with PGO:
* Without preprocessing: PGO parsing ~13.7% of CPU time
* With preprocessing (unsorted): ~2.9% of CPU time (sorting ~1.7%)
* With preprocessing (sorted): ~1.3% of CPU time
The remaining 1.3% of CPU time approximately breaks down as:
* ~0.5% parsing the preprocessed profile
* ~0.7% building weighted IR call graph
* ~0.5% walking function IR to find direct calls
* ~0.2% performing lookups for indirect calls targets
For #58102.
Change-Id: Iaba425ea30b063ca195fb2f7b29342961c8a64c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/569337
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This CL separates the pass that computes inlinability from the pass
that performs inlinability. In particular, the latter can now happen
in any flat order, rather than bottom-up order. This also allows
inlining of calls exposed by devirtualization.
Change-Id: I389c0665fdc8288a6e25129a6744bfb1ace1eff7
Reviewed-on: https://go-review.googlesource.com/c/go/+/562319
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Change-Id: Id9103aa4bda221f5eb34a0ede8676364c574b696
Reviewed-on: https://go-review.googlesource.com/c/go/+/560616
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This patch revises the compiler's "-m=2" status messages related to
inlining. The "can inline" remarks will continue to use the same
format, but the remarks when a specific call site is inlined will be
changed to refer to the score used; before we had
runtime/traceback.go:1131:28: inlining call to gotraceback
runtime/traceback.go:1183:25: inlining call to readgstatus
and with GOEXPERIMENT=newinliner the new messages will be:
runtime/traceback.go:1131:28: inlining call to gotraceback with score 62
runtime/traceback.go:1183:25: inlining call to readgstatus with score 9
Change-Id: Ia86cf5351d29eda64a5426ca0a2a2ec0c2900d81
Reviewed-on: https://go-review.googlesource.com/c/go/+/540775
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL interleaves devirtualization and inlining, so that
devirtualized calls can be inlined.
Fixes#52193.
Change-Id: I681e7c55bdb90ebf6df315d334e7a58f05110d9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/528321
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Bypass: Matthew Dempsky <mdempsky@google.com>
There doesn't appear to be any need for this code. EditChildren won't
recurse into the closure body anyway.
Split out into a separate commit in case I'm overlooking something.
Change-Id: I004d1aa04865896de972bf3323b1622cc08a0d18
Reviewed-on: https://go-review.googlesource.com/c/go/+/543659
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
The early return here is meant to suppress inlining of the function
call itself. However, it also suppresses recursing to visit the call
arguments, which are safe to inline.
Change-Id: I75887574c00931cb622277d04a822bc84c29bfa2
Reviewed-on: https://go-review.googlesource.com/c/go/+/543658
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The devirtualizer and inliner both want to recognize call expressions
that are part of a go or defer statement. This CL refactors them to
use a single CallExpr.GoDefer flag, which gets set during
normalization of go/defer statements during typecheck.
While here, drop some OCALLMETH assertions. Typecheck has been
responsible for desugaring them into OCALLFUNC for a while now, and
ssagen will check this again for us later anyway.
Change-Id: I3fc370f4417431aae97239313da6fe523f512a2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/543657
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
This patch reworks how inlheur.AnalyzeFunc is called by the top level
inliner. Up until this point the strategy was to analyze a function at
the point where CanInline is invoked on it, however it simplifies
things to instead make the call outside of CanInline (for example, so
that directly recursive functions can be analyzed).
Also as part of this patch, change things so that we no longer run
some of the more compile-time intensive analysis on functions that
haven't been marked inlinable (so as to safe compile time), and add a
teardown/cleanup hook in the inlheur package to be invoked by the
inliner when we're done inlining.
Change-Id: Id0772a285d891b0bed66dd86adaffa69d973c26a
Reviewed-on: https://go-review.googlesource.com/c/go/+/539318
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Add a debugging flag "-d=inlscoreadj" intended to support running
experiments in which the inliner uses different score adjustment
values for specific heuristics. The flag argument is a series of
clauses separated by the "/" char where each clause takes the form
"adjK:valK". For example, in this build
go build -gcflags=-d=inlscoreadj=inLoopAdj:10/returnFeedsConstToIfAdj:-99
the "in loop" score adjustments would be reset to a value of 15 (effectively
penalizing calls in loops) adn the "return feeds constant to foldable if/switch"
score adjustment would be boosted from -15 to -99.
Change-Id: Ibd1ee334684af5992466556a69baa6dfefb246b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/532116
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
treat the panic, like a panic. It helps with inlining,
and thus reduced closure allocation and performance, for
many examples of function range iterators.
Change-Id: Ib1a656cdfa56eb2dee400089c4c94ac14f1d2104
Reviewed-on: https://go-review.googlesource.com/c/go/+/541235
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
After performing an inline of function A into function B, collect up
any call sites in the inlined-body-of-A and add them to B's callsite
table, and apply scoring to those new sites.
Change-Id: I4bf563db04e33ba31fb4210f1e484a3cc83f0ee7
Reviewed-on: https://go-review.googlesource.com/c/go/+/530579
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This patch fixes some problems with call site scoring, adds some new
tests, and moves more of the scoring-related code (for example, the
function "ScoreCalls") into "scoring.go". This also fixes some
problems with scoring of calls in non-inlinable functions (when new
inliner is turned on, scoring has to happen for all functions run
through the inliner, not just for inlinable functions). For such
functions, we build a table of inlinable call sites immediately prior
to scoring; the storage for this table is preserved between functions
so as to reduce allocations.
Change-Id: Ie6f691a3ad04fb7a03ab39f882a60aadaf957f6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/542217
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The current GOEXPERIMENT=newinliner strategy us to run "CanInline" for
a given function F with an expanded/relaxed budget of 160 (instead of
the default 80), and then only inline a call to F if the adjustments
we made to F's original score are below 80.
This way of doing things winds up writing out many more functions to
export data that have size between 80 and 160, on the theory that they
might be inlinable somewhere given the right context, which is
expensive from a compile time perspective.
This patch changes things to add a pass that revises the inlinability
of a function after its properties are computed by looking at its
properties and estimating the largest possible negative score
adjustment that could happen given the various return and param props.
If the computed score for the function minus the max adjust is not
less than 80, then we demote it from inlinable to non-inlinable to
save compile time.
Change-Id: Iedaac520d47f632be4fff3bd15d30112b46ec573
Reviewed-on: https://go-review.googlesource.com/c/go/+/529118
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Remove the global package-level call site table; no need to have this
around since we can just iterate over the function-level tables where
needed, saving a bit of memory. No change in inliner or heuristics
functionality.
Change-Id: I319a56cb766178e98b7eebc7c577a0336828ce0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/530576
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
The code that analyzes function return values checks for cases where a
function F always returns the same inlinable function, e.g.
func returnsFunc() func(*int, int) { return setit }
func setit(p *int, v int) { *p = v }
The check for inlinability was being done by looking at "fn.Inl !=
nil", which is probably not what we want, since it includes functions
whose cost value is between 80 and 160 and may only be inlined if lots
of other heuristics kick in.
This patch changes the "always returns same inlinable func" heuristic
to ensure that the func in question has a size of 80 or less, so as to
restrict this case to functions that have a high likelihood of being
inlined.
Change-Id: I06003bca1c56c401df8fd51c922a59c61aa86bea
Reviewed-on: https://go-review.googlesource.com/c/go/+/529116
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Construction of Profile is getting more complex. Currently, we construct
a partial Profile and then use methods to slowly complete the structure.
This can hide dependencies and make refactoring fragile as the
requirements and outputs of the methods is not clearly specified.
Refactor construction to build the Profile only once all of the parts
are complete. The intermediate states explicitly pass input and outputs
as arguments.
Additionally, rename Profile.NodeMap to NamedEdgeMap to make its
contents more clear (edges, specified by caller/callee name rather than
IR). Remove the node flat/cumulative weight from this map; they are
unused.
Change-Id: I2079cd991daac6398d74375b04dfe120b473d908
Reviewed-on: https://go-review.googlesource.com/c/go/+/529558
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Today, the PGO IR graph only contains entries for ir.Func loaded into
the package. This can include functions from transitive dependencies,
but only if they happen to be referenced by something in the current
package. If they are not referenced, noder never bothers to load them.
This leads to a deficiency in PGO devirtualization: some callee methods
are available in transitive dependencies but do not devirtualize because
they happen to not get loaded from export data.
Resolve this by adding an explicit lookup from export data of callees
mentioned in the profile.
I have chosen to do this during loading of the profile for simplicity:
the PGO IR graph always contains all of the functions we might need.
That said, it isn't strictly necessary. PGO devirtualization could do
the lookup lazily if it decides it actually needs a method. This saves
work at the expense of a bit more complexity, but I've chosen the
simpler approach for now as I measured the cost of this as significantly
less than the rest of PGO loading.
For #61577.
Change-Id: Ieafb2a549510587027270ee6b4c3aefd149a901f
Reviewed-on: https://go-review.googlesource.com/c/go/+/497175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
When computing function cost, hairyVisitor.doNode has two primary cases
for determining the cost of a call inside the function:
* Normal calls are simply cost 57.
* Calls that can be inlined have the cost of the inlined function body,
since that body will end up in this function.
Determining which "calls can be inlined" is where this breaks down.
doNode simply assumes that any function with `fn.Inl != nil` will get
inlined. However, this are more complex in mkinlcall, which has a
variety of cases where it may not inline.
For standard builds, most of these reasons are fairly rare (recursive
calls, calls to runtime functions in instrumented builds, etc), so this
simplification isn't too build.
However, for PGO builds, any function involved in at least one inlinable
hot callsite will have `fn.Inl != nil`, even though mkinlcall will only
inline at the hot callsites. As a result, cold functions calling hot
functions will use the (potentially very large) hot function inline body
cost in their call budget. This could make these functions too expensive
to inline even though they won't actually inline the hot function.
Handle this case plus the other inlinability cases (recursive calls,
etc) by consolidating mkinlcall's inlinability logic into
canInlineCallExpr, which is shared by doNode.
mkinlcall and doNode now have identical logic, except for one case: we
check for recursive cycles via inlined functions by looking at the
inline tree. Since we haven't actually done any inlining yet when in
doNode, we will miss those cases.
This CL doesn't change any inlining decisions in a standard build of the
compiler.
In the new inliner, the inlining decision is also based on the call
site, so this synchronization is also helpful.
Fixes#59484
Change-Id: I6ace66e37d50526535972215497ef75cd71f8b9d
Reviewed-on: https://go-review.googlesource.com/c/go/+/483196
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
CallExpr.X -> CallExpr.Fun
This consistent with go/ast and cmd/compile/internal/syntax.
OPRINTN -> OPRINTLN
This op represents the "println" builtin; might as well spell it the
same way.
Change-Id: Iead1b007776658c717879cf0997b3c48028428f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/532795
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
When a PGO build fails or produces incorrect program, it is often
unclear what the problem is. Add pgo hash so we can bisect to
individual optimization decisions, which often helps debugging.
Related to #58153.
Change-Id: I651ffd9c53bad60f2f28c8ec2a90a3f532982712
Reviewed-on: https://go-review.googlesource.com/c/go/+/528400
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Add a new debug flag "-d=dumpinlcallsitescores" that dumps out a
summary of all callsites in the package being compiled with info on
inlining heuristics, for human consumption. Sample output lines:
Score Adjustment Status Callee CallerPos ScoreFlags
...
115 40 DEMOTED cmd/compile/internal/abi.(*ABIParamAssignment).Offset expand_calls.go:1679:14|6 panicPathAdj
...
76 -5 PROMOTED runtime.persistentalloc mcheckmark.go:48:45|3 inLoopAdj
...
201 0 --- PGO unicode.DecodeRuneInString utf8.go:312:30|1
...
7 -5 --- PGO internal/abi.Name.DataChecked type.go:625:22|0 inLoopAdj
Here "Score" is the final score calculated for the callsite,
"Adjustment" is the amount added to or subtracted from the original
hairyness estimate to form the score. "Status" shows whether anything
changed with the site -- did the adjustment bump it down just below
the threshold ("PROMOTED") or instead bump it above the threshold
("DEMOTED") or did nothing happen as a result of the heuristics
("---"); "Status" also shows whether PGO was involved. "Callee" is the
name of the function called, "CallerPos" is the position of the
callsite, and "ScoreFlags" is a digest of the specific properties we
used to make adjustments to callsite score via heuristics.
Change-Id: Iea4b1cbfee038bc68df6ab81e9973f145636300b
Reviewed-on: https://go-review.googlesource.com/c/go/+/513455
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This patch changes the inliner to use callsite scores when deciding to
inline as opposed to looking only at callee cost/hairyness.
For this to work, we have to relax the inline budget cutoff as part of
CanInline to allow for the possibility that a given function might
start off with a cost of N where N > 80, but then be called from a
callsites whose score is less than 80. Once a given function F in
package P has been approved by CanInline (based on the relaxed budget)
it will then be emitted as part of the export data, meaning that other
packages importing P will need to also need to compute callsite scores
appropriately.
For a function F that calls function G, if G is marked as potentially
inlinable then the hairyness computation for F will use G's cost for
the call to G as opposed to the default call cost; for this to work
with the new scheme (given relaxed cost change described above) we
use G's cost only if it falls below inlineExtraCallCost, otherwise
just use inlineExtraCallCost.
Included in this patch are a bunch of skips and workarounds to
selected 'errorcheck' tests in the <GOROOT>/test directory to deal
with the additional "can inline" messages emitted when the new inliner
is turned on.
Change-Id: I9be5f8cd0cd8676beb4296faf80d2f6be7246335
Reviewed-on: https://go-review.googlesource.com/c/go/+/519197
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Field.Sym now always contains the original symbol as it appeared in Go
source, so we don't need OrigSym anymore.
Instead, when the mangled name is desired, Field.Nname.Sym() can be
used instead, which is always non-nil if Nname is non-nil.
Change-Id: I96cd61db6458d4a2e07ec5810239236e3dfba747
Reviewed-on: https://go-review.googlesource.com/c/go/+/527516
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Assign scores to callsites based on previously computed function
properties and callsite properties. This currently works by taking the
size score for the function (as computed by CanInline) and then making
a series of adjustments, positive or negative based on various
function and callsite properties.
NB: much work also remaining on deciding what are the best score
adjustment values for specific heuristics. I've picked a bunch of
arbitrary constants, but they will almost certainly need tuning and
tweaking to arrive at something that has good performance.
Updates #61502.
Change-Id: I887403f95e76d7aa2708494b8686c6026861a6ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/511566
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Augment the ir.Inline container to include an entry for function
properties (currently serialized as a string), and if
GOEXPERIMENT=newinliner is set, compute and store function
properties for all inline candidates processed by the inliner.
The idea here is that if the function properties are going to drive
inlining decisions, we'd like to have the same info from non-local /
imported functions as for local / in-package functions, hence we need
to include the properties in the export data.
Hand testing on the compiler itself and with k8s kubelet shows that
this increases the size of export data overall by about 2-3 percent,
so a pretty modest increase.
Updates #61502.
Change-Id: I9d1c311aa8418d02ffea3629c3dd9d8076886d15
Reviewed-on: https://go-review.googlesource.com/c/go/+/511562
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Add code to analyze properties of function result values, specifically
heuristics for cases where we always return allocated memory, always
return the same constant, or always return the same function.
Updates #61502.
Change-Id: I8b0a3295b5be7f7ad4c2d5b9803925aea0639376
Reviewed-on: https://go-review.googlesource.com/c/go/+/511559
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
We don't actually depend on Inl.Body anywhere, except it implicitly
serves to indicate whether Inl.Dcl has been populated. So replace it
with a boolean so we don't need to keep a useless copy of every
inlinable function body in memory.
While here, also add some Fatalfs to make sure there are no unused
local variables. The unified frontend now omits unreachable code
during export data writing, so there shouldn't be unused local
variables.
Also, since unified IR uses the same code/data to construct the
original function as inlined and/or imported functions, the Dcl list
should always be the same, which addresses the real root issue (i.e.,
that export/import could skew the Dcl lists).
Change-Id: I6e3435f3a0352f6efbae787344006efac1891e84
Reviewed-on: https://go-review.googlesource.com/c/go/+/523315
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, the types package has IsRuntimePkg and IsReflectPkg
predicates for testing if a Pkg is the runtime or reflect packages.
IsRuntimePkg returns "true" for any "CompilingRuntime" package, which
includes all of the packages imported by the runtime. This isn't
inherently wrong, except that all but one use of it is of the form "is
this Sym a specific runtime.X symbol?" for which we clearly only want
the package "runtime" itself. IsRuntimePkg was introduced (as
isRuntime) in CL 37538 as part of separating the real runtime package
from the compiler built-in fake runtime package. As of that CL, the
"runtime" package couldn't import any other packages, so this was
adequate at the time.
We could fix this by just changing the implementation of IsRuntimePkg,
but the meaning of this API is clearly somewhat ambiguous. Instead, we
replace it with a new RuntimeSymName function that returns the name of
a symbol if it's in package "runtime", or "" if not. This is what
every call site (except one) actually wants, which lets us simplify
the callers, and also more clearly addresses the ambiguity between
package "runtime" and the general concept of a runtime package.
IsReflectPkg doesn't have the same issue of ambiguity, but it
parallels IsRuntimePkg and is used in the same way, so we replace it
with a new ReflectSymName for consistency.
Change-Id: If3a81d7d11732a9ab2cac9488d17508415cfb597
Reviewed-on: https://go-review.googlesource.com/c/go/+/521696
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This CL removes a lot of the redundant methods for accessing struct
fields and signature parameters. In particular, users never have to
write ".Slice()" or ".FieldSlice()" anymore; the exported APIs just do
what you want.
Further internal refactorings to follow.
Change-Id: I45212f6772fe16aad39d0e68b82d71b0796e5639
Reviewed-on: https://go-review.googlesource.com/c/go/+/521295
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
This CL updates several frontend passes to stop relying on
ir.CurFunc (at least directly).
Change-Id: I3c3529e81e27fb05d54a828f081f7c7efc31af67
Reviewed-on: https://go-review.googlesource.com/c/go/+/520606
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
This CL moves more common Func-setup logic into ir.NewFunc. In
particular, it now handles constructing the Name and wiring them
together, setting the Typecheck bit, and setting Sym.Func.
Relatedly, this CL also extends typecheck.DeclFunc to append the
function to typecheck.Target.Funcs, so that callers no longer need to
do this.
Change-Id: Ifa0aded8df0517188eb295d0dccc107af85f1e8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/520338
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Start making progress towards constructing IR with proper types.
Change-Id: Iad32c1cf60f30ceb8e07c31c8871b115570ac3bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/520263
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
These aren't constructed by the unified frontend.
Change-Id: Ied87baa9656920bd11055464bc605933ff448e21
Reviewed-on: https://go-review.googlesource.com/c/go/+/520264
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This CL extends ir.StaticValue to also work on closure variables.
Also, it extracts the code from escape analysis that's responsible for
determining the static callee of a function. This will be useful when
go/defer statement normalization is moved to typecheck.
Change-Id: I69e1f7fb185658dc9fbfdc69d0f511c84df1d3ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/518959
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This never belonged in escape analysis, but the non-unified generics
frontend didn't use typecheck. That frontend is gone, so now we can
desugar it earlier.
Change-Id: I70f34a851f27fce1133777c5eeca0f549fc60ede
Reviewed-on: https://go-review.googlesource.com/c/go/+/518958
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Instead of having the inliner specially recognize that eq/hash
functions can't be inlined, change the geneq and genhash to mark them
as //go:noinline.
This is a prereq for a subsequent CL that will move more logic for
handling rtypes from package types to package reflectdata.
Change-Id: I091a9ededcc083fe8305cf5443a9af7d3a9053b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/518955
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Decls used to contain initializer statement for package-level
variables, but now it only contains ir.Funcs. So we might as well
rename it to Funcs and tighten its type to []*ir.Func.
Similarly, Externs always contains *ir.Names, so its type can be
constrained too.
Change-Id: I85b833e2f83d9d3559ab0ef8ab5d8324f4bc37b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/517855
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Split out the code that computes the initial inline "hairyness" budget
for a function so that it can be reused (in a later patch). This is a
pure refactoring; no change in compiler functionality.
Change-Id: I9b1b7b10a7c480559b837492b10eb08771b7a145
Reviewed-on: https://go-review.googlesource.com/c/go/+/514795
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Add some machinery to support computing function "properties" for use
in driving inlining heuristics, and a unit testing framework to check
to see if the property computations are correct for a given set of
canned Go source files. This CL is mainly the analysis skeleton and a
testing framework; the code to compute the actual props will arrive in
a later patch.
Updates #61502.
Change-Id: I7970b64f713d17d7fdd7e8e9ccc7d9b0490571bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/511557
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The new PGO-driven indirect call specialization from CL 492436
in theory should allow for devirtualization on methods
in another package when those methods are directly referenced
in the current package.
However, inline.InlineImpossible was checking for a zero-length
fn.Body and would cause devirtualization to fail
with a debug log message like:
"should not PGO devirtualize (*Speaker1).Speak: no function body"
Previously, the logic in inline.InlineImpossible was only
called on local functions, but with PGO-based devirtualization,
it can now be called on imported functions, where inlinable
imported functions will have a zero-length fn.Body but a
non-nil fn.Inl.
We update inline.InlineImpossible to handle imported functions
by adding a call to typecheck.HaveInlineBody in the check
that was previously failing.
For the test, we need to have a hopefully temporary workaround
of adding explicit references to the callees in another package
for devirtualization to work. CL 497175 or similar should
enable removing this workaround.
Fixes#60561
Updates #59959
Change-Id: I48449b7d8b329d84151bd3b506b8093c262eb2a3
GitHub-Last-Rev: 2d53c55fd895ad8fefd25510a6e6969e89d54a6d
GitHub-Pull-Request: golang/go#60565
Reviewed-on: https://go-review.googlesource.com/c/go/+/500155
Run-TryBot: thepudds <thepudds1460@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
