Escape analysis is currently very naive about identifying calls to
known functions: it only recognizes direct calls to a declared
function, or direct calls to a closure.
This CL adds a new "staticValue" helper function that can trace back
through local variables that were initialized and never reassigned
based on a similar optimization already used by inlining. (And to be
used by inlining in a followup CL.)
Updates #41474.
Change-Id: I8204fd3b1e150ab77a27f583985cf099a8572b2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/256458
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Normally, when variables are declared and initialized using ":=", we
set the variable's n.Name.Defn to point to the initialization
assignment node (i.e., OAS or OAS2). Further, some frontend
optimizations look for variables that are initialized but never
reassigned.
However, when inl.go inlines calls, it was declaring the inlined
variables, and then separately assigning to them. This CL changes
inl.go tweaks the AST to fit the combined declaration+initialization
pattern.
This isn't terribly useful by itself, but it allows further followup
optimizations.
Updates #41474.
Change-Id: I62a9752c60414305679e0ed15a6563baa0224efa
Reviewed-on: https://go-review.googlesource.com/c/go/+/256457
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
I've been experimenting with this, success is the wrong thing to report
even though it seems to log much less.
Change-Id: I7c25a45d2f41e82b6c8dd8b0a56ba848c63fb21a
Reviewed-on: https://go-review.googlesource.com/c/go/+/223298
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
We still disallow inlining for an immediately-recursive function, but allow
inlining if a function is in a recursion chain.
If all functions in the recursion chain are simple, then we could inline
forever down the recursion chain (eventually running out of stack on the
compiler), so we add a map to keep track of the functions we have
already inlined at a call site. We stop inlining when we reach a
function that we have already inlined in the recursive chain. Of course,
normally the inlining will have stopped earlier, because of the cost
function.
We could also limit the depth of inlining by a simple count (say, limit
max inlining of 10 at any given site). Would that limit other
opportunities too much?
Added a test in test/inline.go. runtime.BenchmarkStackCopyNoCache() is
also already a good test that triggers the check to stop inlining
when we reach the start of the recursive chain again.
For the bent benchmark suite, the performance improvement was mostly not
statistically significant, but the geomean averaged out to: -0.68%. The text size
increase was less than .1% for all bent benchmarks. The cmd/go text size increase
was 0.02% and the cmd/compile text size increase was .1%.
Fixes#29737
Change-Id: I892fa84bb07a947b3125ec8f25ed0e508bf2bdf5
Reviewed-on: https://go-review.googlesource.com/c/go/+/226818
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
We were assigning a cost of 2 for intrinsics:
One when we recognized an intrinsic,
and one for the OCALLFUNC node.
I believe that the intent was that intrinsics should
cost 1, since they are typically an arithmetic op,
and because they tend to occur in performance-sensitive code.
(Not that any of this is particularly principled right now.)
Stop charging when we recognize an intrinsic;
let the OCALLFUNC node cover the cost.
This has a negligible impact on std+cmd.
Change-Id: Ie6ae5c18da3ae7e40aec425aed95c6999c2831a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/221357
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The existing implementation outputs inline cost iff function cannot be inlined with Debug['m'] > 1, the cost info is also useful if the function is inlineable.
Fixes#36780
Change-Id: Ic96f6baf96aee25fb4b33d31d4d644dc2310e536
Reviewed-on: https://go-review.googlesource.com/c/go/+/216778
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This CL adds -d=checkptr as a compile-time option for adding
instrumentation to check that Go code is following unsafe.Pointer
safety rules dynamically. In particular, it currently checks two
things:
1. When converting unsafe.Pointer to *T, make sure the resulting
pointer is aligned appropriately for T.
2. When performing pointer arithmetic, if the result points to a Go
heap object, make sure we can find an unsafe.Pointer-typed operand
that pointed into the same object.
These checks are currently disabled for the runtime, and can also be
disabled through a new //go:nocheckptr annotation. The latter is
necessary for functions like strings.noescape, which intentionally
violate safety rules to workaround escape analysis limitations.
Fixes#22218.
Change-Id: If5a51273881d93048f74bcff10a3275c9c91da6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/162237
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The IsClosureVar, IsOutputParamHeapAddr, Assigned, Addrtaken,
InlFormal, and InlLocal flags are only interesting for ONAME nodes, so
it's better to set these flags on Name.flags instead of Node.flags.
Two caveats though:
1. Previously, we would set Assigned and Addrtaken on the entire
expression tree involved in an assignment or addressing operation.
However, the rest of the compiler only actually cares about knowing
whether the underlying ONAME (if any) was assigned/addressed.
2. This actually requires bumping Name.flags from bitset8 to bitset16,
whereas it doesn't allow shrinking Node.flags any. However, Name has
some trailing padding bytes, so expanding Name.flags doesn't cost any
memory.
Passes toolstash-check.
Change-Id: I7775d713566a38d5b9723360b1659b79391744c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/200898
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL changes cmd/compile to use Node.Right instead of
Node.Rlist for OAS2FUNC/OAS2RECV/OAS2MAPR/OAS2DOTTYPE nodes.
Fixes#32293
Change-Id: I4c9d9100be2d98d15e016797f934f64d385f5faa
Reviewed-on: https://go-review.googlesource.com/c/go/+/197817
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Mid-stack inlining is enable now, see #19348, but we still can not
remove the special case for runtime.heapBits.nextArena, because
runtime.heapBits.next is too complex to be inlined
(cost 96 exceeds budget 80).
Change-Id: I04ea86509074afdc83a3f70d68b8a1a8829763d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/174839
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In 31618, we end up comparing the is-stmt-ness of positions
to repurpose real instructions as inline marks. If the is-stmt-ness
doesn't match, we end up not being able to remove the inline mark.
Always use statement-full positions to do the matching, so we
always find a match if there is one.
Also always use positions that are statements for inline marks.
Fixes#31618
Change-Id: Idaf39bdb32fa45238d5cd52973cadf4504f947d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/173324
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
A check in inl.go to prevent inlining of functions calling
either getcallerpc or getcallersp does not work when these
functions are intrinsics. Swap checks to fix.
Includes test.
No bug, this was discovered in the course of a ridiculous
experiment with inlining.
Change-Id: Ie1392523bb89882d586678f2674e1a4eadc5e431
Reviewed-on: https://go-review.googlesource.com/c/go/+/172217
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
If no other instruction mentions an inline mark, we can get rid of it.
This normally happens when the inlined function is empty, or when all
of its code is folded into other instructions.
Also use consistent statement-ness for inline mark positions, so that
more of them can be removed in favor of existing instructions.
Update #29571Fixes#31172
Change-Id: I71f84d355101f37a27960d9e8528f42f92767496
Reviewed-on: https://go-review.googlesource.com/c/go/+/170445
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This is a re-attempt at CL 153841, which caused two regressions:
1. crypto/ecdsa failed to build with -gcflags=-l=4. This was because
when "t1, t2, ... := g(); f(t1, t2, ...)" was exported, we were losing
the first assignment from the call's Ninit field.
2. net/http/pprof failed to run with -gcflags=-N. This is due to a
conflict with CL 159717: as of that CL, package-scope initialization
statements are executed within the "init.ializer" function, rather
than the "init" function, and the generated temp variables need to be
moved accordingly too.
[Rest of description is as before.]
This CL moves order.go's copyRet logic for rewriting f(g()) into t1,
t2, ... := g(); f(t1, t2, ...) earlier into typecheck. This allows the
rest of the compiler to stop worrying about multi-value functions
appearing outside of OAS2FUNC nodes.
This changes compiler behavior in a few observable ways:
1. Typechecking error messages for builtin functions now use general
case error messages rather than unnecessarily differing ones.
2. Because f(g()) is rewritten before inlining, saved inline bodies
now see the rewritten form too. This could be addressed, but doesn't
seem worthwhile.
3. Most notably, this simplifies escape analysis and fixes a memory
corruption issue in esc.go. See #29197 for details.
Fixes#15992.
Fixes#29197.
Change-Id: I930b10f7e27af68a0944d6c9bfc8707c3fab27a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/166983
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This CL moves order.go's copyRet logic for rewriting f(g()) into t1,
t2, ... = g(); f(t1, t2, ...) earlier into typecheck. This allows the
rest of the compiler to stop worrying about multi-value functions
appearing outside of OAS2FUNC nodes.
This changes compiler behavior in a few observable ways:
1. Typechecking error messages for builtin functions now use general
case error messages rather than unnecessarily differing ones.
2. Because f(g()) is rewritten before inlining, saved inline bodies
now see the rewritten form too. This could be addressed, but doesn't
seem worthwhile.
3. Most notably, this simplifies escape analysis and fixes a memory
corruption issue in esc.go. See #29197 for details.
Fixes#15992.
Fixes#29197.
Change-Id: I86a70668301efeec8fbd11fe2d242e359a3ad0af
Reviewed-on: https://go-review.googlesource.com/c/153841
Reviewed-by: Robert Griesemer <gri@golang.org>
Work involved in getting a stack trace is divided between
runtime.Callers and runtime.CallersFrames.
Before this CL, runtime.Callers returns a pc per runtime frame.
runtime.CallersFrames is responsible for expanding a runtime frame
into potentially multiple user frames.
After this CL, runtime.Callers returns a pc per user frame.
runtime.CallersFrames just maps those to user frame info.
Entries in the result of runtime.Callers are now pcs
of the calls (or of the inline marks), not of the instruction
just after the call.
Fixes#29007Fixes#28640
Update #26320
Change-Id: I1c9567596ff73dc73271311005097a9188c3406f
Reviewed-on: https://go-review.googlesource.com/c/152537
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
A Go user made a well-documented request for a slightly
lower threshold. I tested against a selection of other
people's benchmarks, and saw a tiny benefit (possibly noise)
at equally tiny cost, and no unpleasant surprises observed
in benchmarking.
I.e., might help, doesn't hurt, low risk, request was
delivered on a silver platter.
It did, however, change the behavior of one test because
now bytes.Buffer.Grow is eligible for inlining.
Updates #19348.
Change-Id: I85e3088a4911290872b8c6bda9601b5354c48695
Reviewed-on: https://go-review.googlesource.com/c/151977
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This change does a bulk rename of several identifiers in the compiler.
See #27167 and https://docs.google.com/document/d/19_ExiylD9MRfeAjKIfEsMU1_RGhuxB9sA0b5Zv7byVI/
for context and for discussion of these particular renames.
Commands run to generate this change:
gorename -from '"cmd/compile/internal/gc".OPROC' -to OGO
gorename -from '"cmd/compile/internal/gc".OCOM' -to OBITNOT
gorename -from '"cmd/compile/internal/gc".OMINUS' -to ONEG
gorename -from '"cmd/compile/internal/gc".OIND' -to ODEREF
gorename -from '"cmd/compile/internal/gc".OARRAYBYTESTR' -to OBYTES2STR
gorename -from '"cmd/compile/internal/gc".OARRAYBYTESTRTMP' -to OBYTES2STRTMP
gorename -from '"cmd/compile/internal/gc".OARRAYRUNESTR' -to ORUNES2STR
gorename -from '"cmd/compile/internal/gc".OSTRARRAYBYTE' -to OSTR2BYTES
gorename -from '"cmd/compile/internal/gc".OSTRARRAYBYTETMP' -to OSTR2BYTESTMP
gorename -from '"cmd/compile/internal/gc".OSTRARRAYRUNE' -to OSTR2RUNES
gorename -from '"cmd/compile/internal/gc".Etop' -to ctxStmt
gorename -from '"cmd/compile/internal/gc".Erv' -to ctxExpr
gorename -from '"cmd/compile/internal/gc".Ecall' -to ctxCallee
gorename -from '"cmd/compile/internal/gc".Efnstruct' -to ctxMultiOK
gorename -from '"cmd/compile/internal/gc".Easgn' -to ctxAssign
gorename -from '"cmd/compile/internal/gc".Ecomplit' -to ctxCompLit
Not altered: parameters and local variables (mostly in typecheck.go) named top,
which should probably now be called ctx (and which should probably have a named type).
Also not altered: Field called Top in gc.Func.
gorename -from '"cmd/compile/internal/gc".Node.Isddd' -to IsDDD
gorename -from '"cmd/compile/internal/gc".Node.SetIsddd' -to SetIsDDD
gorename -from '"cmd/compile/internal/gc".nodeIsddd' -to nodeIsDDD
gorename -from '"cmd/compile/internal/types".Field.Isddd' -to IsDDD
gorename -from '"cmd/compile/internal/types".Field.SetIsddd' -to SetIsDDD
gorename -from '"cmd/compile/internal/types".fieldIsddd' -to fieldIsDDD
Not altered: function gc.hasddd, params and local variables called isddd
Also not altered: fmt.go prints nodes using "isddd(%v)".
cd cmd/compile/internal/gc; go generate
I then manually found impacted comments using exact string match
and fixed them up by hand. The comment changes were trivial.
Passes toolstash-check.
Fixes#27167. If this experiment is deemed a success,
we will open a new tracking issue for renames to do
at the end of the 1.13 cycles.
Change-Id: I2dc541533d2ab0d06cb3d31d65df205ecfb151e8
Reviewed-on: https://go-review.googlesource.com/c/150140
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This is a simple tweak to allow a bit more mid-stack inlining.
In cases like this:
func f() {
g()
}
We'd really like to inline f into its callers. It can't hurt.
We implement this optimization by making calls a bit cheaper, enough
to afford a single call in the function body, but not 2.
The remaining budget allows for some argument modification, or perhaps
a wrapping conditional:
func f(x int) {
g(x, 0)
}
func f(x int) {
if x > 0 {
g()
}
}
Update #19348
Change-Id: Ifb1ea0dd1db216c3fd5c453c31c3355561fe406f
Reviewed-on: https://go-review.googlesource.com/c/147361
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
Avoids allocating an ONAME for OLABEL, OGOTO, and named OBREAK and
OCONTINUE nodes.
Passes toolstash-check.
Change-Id: I359142cd48e8987b5bf29ac100752f8c497261c1
Reviewed-on: https://go-review.googlesource.com/c/145200
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Nowadays there are better ways to safely run untrusted Go programs, like
NaCl and gVisor.
Change-Id: I20c45f13a50dbcf35c343438b720eb93e7b4e13a
Reviewed-on: https://go-review.googlesource.com/c/142717
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Prior to stack tracing, inlining could cause
dead pointers to be kept alive in some loops.
See #18336 and CL 31674.
The adjustment removed by this change preserved the inlining status quo
in the face of Node structure changes, to avoid creating new problems.
Now that stack tracing provides precision, these hacks can be removed.
Of course, our inlining code model is already hacky (#17566),
but at least now there will be fewer epicyclical hacks.
Newly inline-able functions in std cmd as a result of this change:
hash/adler32/adler32.go:65:6: can inline (*digest).UnmarshalBinary
hash/fnv/fnv.go:281:6: can inline (*sum32).UnmarshalBinary
hash/fnv/fnv.go:292:6: can inline (*sum32a).UnmarshalBinary
reflect/value.go:1298:6: can inline Value.OverflowComplex
compress/bzip2/bit_reader.go:25:6: can inline newBitReader
encoding/xml/xml.go:365:6: can inline (*Decoder).switchToReader
vendor/golang_org/x/crypto/cryptobyte/builder.go:77:6: can inline (*Builder).AddUint16
crypto/x509/x509.go:1851:58: can inline buildExtensions.func2.1.1
crypto/x509/x509.go:1871:58: can inline buildExtensions.func2.3.1
crypto/x509/x509.go:1883:58: can inline buildExtensions.func2.4.1
cmd/vet/internal/cfg/builder.go:463:6: can inline (*builder).labeledBlock
crypto/tls/handshake_messages.go:1450:6: can inline (*newSessionTicketMsg).marshal
crypto/tls/handshake_server.go:769:6: can inline (*serverHandshakeState).clientHelloInfo
crypto/tls/handshake_messages.go:1171:6: can inline (*nextProtoMsg).unmarshal
cmd/link/internal/amd64/obj.go:40:6: can inline Init
cmd/link/internal/ppc64/obj.go:40:6: can inline Init
net/http/httputil/persist.go:54:6: can inline NewServerConn
net/http/fcgi/child.go:83:6: can inline newResponse
cmd/compile/internal/ssa/poset.go:245:6: can inline (*poset).newnode
Change-Id: I19e8e383a6273849673d35189a9358870665f82f
Reviewed-on: https://go-review.googlesource.com/c/141117
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The new indexed package export format appears stable, and no reports
of needing to revert back to binary package export.
This CL disables the binary package export format by mechanically
replacing 'flagiexport' with 'true', and then superficial code
cleanups to keep the resulting code idiomatic. The resulting dead code
is removed in a followup CL.
Change-Id: Ic30d85f78778a31d279a56b9ab14e80836d50135
Reviewed-on: https://go-review.googlesource.com/c/139337
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This CL adds the source code of all inlined functions
into the function specified in $GOSSAFUNC.
The code is appended to the sources column of ssa.html.
ssaDumpInlined is populated with references to inlined functions.
Then it is used for dumping the sources in buildssa.
The source columns contains code in following order:
target function, inlined functions sorted by filename, lineno.
Fixes#25904
Change-Id: I4f6d4834376f1efdfda1f968a5335c0543ed36bc
Reviewed-on: https://go-review.googlesource.com/126606
Run-TryBot: Yury Smolsky <yury@smolsky.by>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
If we're compiling a large function, be more picky about how big
the function we're inlining is. If the function is >5000 nodes,
we lower the inlining threshold from a cost of 80 to 20.
Turns out reflect.Value's cost is exactly 80. That's the function
at issue in #26546.
20 was chosen as a proxy for "inlined body is smaller than the call would be".
Simple functions still get inlined, like this one at cost 7:
func ifaceIndir(t *rtype) bool {
return t.kind&kindDirectIface == 0
}
5000 nodes was chosen as the big function size. Here are all the
5000+ node (~~1000+ lines) functions in the stdlib:
5187 cmd/internal/obj/arm (*ctxt5).asmout
6879 cmd/internal/obj/s390x (*ctxtz).asmout
6567 cmd/internal/obj/ppc64 (*ctxt9).asmout
9643 cmd/internal/obj/arm64 (*ctxt7).asmout
5042 cmd/internal/obj/x86 (*AsmBuf).doasm
8768 cmd/compile/internal/ssa rewriteBlockAMD64
8878 cmd/compile/internal/ssa rewriteBlockARM
8344 cmd/compile/internal/ssa rewriteValueARM64_OpARM64OR_20
7916 cmd/compile/internal/ssa rewriteValueARM64_OpARM64OR_30
5427 cmd/compile/internal/ssa rewriteBlockARM64
5126 cmd/compile/internal/ssa rewriteValuePPC64_OpPPC64OR_50
6152 cmd/compile/internal/ssa rewriteValuePPC64_OpPPC64OR_60
6412 cmd/compile/internal/ssa rewriteValuePPC64_OpPPC64OR_70
6486 cmd/compile/internal/ssa rewriteValuePPC64_OpPPC64OR_80
6534 cmd/compile/internal/ssa rewriteValuePPC64_OpPPC64OR_90
6534 cmd/compile/internal/ssa rewriteValuePPC64_OpPPC64OR_100
6534 cmd/compile/internal/ssa rewriteValuePPC64_OpPPC64OR_110
6675 cmd/compile/internal/gc typecheck1
5433 cmd/compile/internal/gc walkexpr
14070 cmd/vendor/golang.org/x/arch/arm64/arm64asm decodeArg
There are a lot more smaller (~1000 node) functions in the stdlib.
The function in #26546 has 12477 nodes.
At some point it might be nice to have a better heuristic for "inlined
body is smaller than the call", a non-cliff way to scale down the cost
as the function gets bigger, doing cheaper inlined calls first, etc.
All that can wait for another release. I'd like to do this CL for
1.11.
Fixes#26546
Update #17566
Change-Id: Idda13020e46ec2b28d79a17217f44b189f8139ac
Reviewed-on: https://go-review.googlesource.com/125516
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Adds the appropriate check to inl.go.
Includes tests of both -race+go:norace and plain go:norace.
Fixes#24651.
Change-Id: Id806342430c20baf4679a985d12eea3b677092e0
Reviewed-on: https://go-review.googlesource.com/119195
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This line of the inlining tuning experiment
https://go-review.googlesource.com/c/go/+/109918/1/src/cmd/compile/internal/gc/inl.go#347
was incorrectly rewritten in a later patch to use the call
cost, not the panic cost, and thus the inlining of panic
didn't occur when it should. I discovered this when I
realized that tests should have failed, but didn't.
Fix is to make the correct change, and also to modify the
tests that this causes to fail. One test now asserts the
new normal, the other calls "ppanic" instead which is
designed to behave like panic but not be inlined.
Change-Id: I423bb7f08bd66a70d999826dd9b87027abf34cdf
Reviewed-on: https://go-review.googlesource.com/116656
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Fix a problem in DWARF inline debug generation relating to handling of
statically unreachable local variables. For a function such as:
const always = true
func HasDeadLocal() int {
if always {
return 9
}
x := new(Something)
...
return x.y
}
the variable "x" is placed onto the Dcl list for the function during
parsing, but the actual declaration node is deleted later on when
gc.Main invokes "deadcode". Later in the compile the DWARF code emits
an abstract function with "x" (since "x" was on the Dcl list at the
point of the inline), but the export data emitted does not contain
"x". This then creates clashing/inconsistant DWARF abstract function
DIEs later on if HasDeadLocal is inlined in somewhere else.
As a fix, the inliner now pruned away variables such as "x" when
creating a copy of the Dcl list as part of the inlining; this means
that both the export data generator and the DWARF emitter wind up
seeing a consistent picture.
Fixes#25459
Change-Id: I753dc4e9f9ec694340adba5f43c907ba8cc9badc
Reviewed-on: https://go-review.googlesource.com/114090
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Inlining was refactored to perform tuning experiments,
with the "knobs" now set to also inline functions/methods
that include panic(), and -l=4 (inline calls) now expressed
as a change to costs, rather than scattered if-thens.
The -l=4 inline-calls penalty is chosen to be the best
found during experiments; it makes some programs much
larger and slower (notably, the compiler itself) and is
believed to be risky for machine-generated code in general,
which is why it is not the default. It is also not
well-tested with the debugger and DWARF output.
This change includes an explicit go:noinline applied to the
method that is the largest cause of compiler binary growth
and slowdown for midstack inlining; there are others,
ideally whatever heuristic eventually appears will make
this unnecessary.
Change-Id: Idf7056ed2f961472cf49d2fd154ee98bef9421e2
Reviewed-on: https://go-review.googlesource.com/109918
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
For struct fields and methods, Field.Nname was only used to store
position information, which means we're allocating an entire ONAME
Node+Name+Param structure just for one field. We can optimize away
these ONAME allocations by instead adding a Field.Pos field.
Unfortunately, we can't get rid of Field.Nname, because it's needed
for function parameters, so Field grows a little bit and now has more
redundant information in those cases. However, that was already the
case (e.g., Field.Sym and Field.Nname.Sym), and it's still a net win
for allocations as demonstrated by the benchmarks below.
Additionally, by moving the ONAME allocation for function parameters
to funcargs, we can avoid allocating them for function parameters that
aren't used in corresponding function bodies (e.g., interface methods,
function-typed variables, and imported functions/methods without
inline bodies).
name old time/op new time/op delta
Template 254ms ± 6% 251ms ± 6% -1.04% (p=0.000 n=487+488)
Unicode 128ms ± 7% 128ms ± 7% ~ (p=0.294 n=482+467)
GoTypes 862ms ± 5% 860ms ± 4% ~ (p=0.075 n=488+471)
Compiler 3.91s ± 4% 3.90s ± 4% -0.39% (p=0.000 n=468+473)
name old user-time/op new user-time/op delta
Template 339ms ±14% 336ms ±14% -1.02% (p=0.001 n=498+494)
Unicode 176ms ±18% 176ms ±25% ~ (p=0.940 n=491+499)
GoTypes 1.13s ± 8% 1.13s ± 9% ~ (p=0.157 n=496+493)
Compiler 5.24s ± 6% 5.21s ± 6% -0.57% (p=0.000 n=485+489)
name old alloc/op new alloc/op delta
Template 38.3MB ± 0% 37.3MB ± 0% -2.58% (p=0.000 n=499+497)
Unicode 29.1MB ± 0% 29.1MB ± 0% -0.03% (p=0.000 n=500+493)
GoTypes 116MB ± 0% 115MB ± 0% -0.65% (p=0.000 n=498+499)
Compiler 492MB ± 0% 487MB ± 0% -1.00% (p=0.000 n=497+498)
name old allocs/op new allocs/op delta
Template 364k ± 0% 360k ± 0% -1.15% (p=0.000 n=499+499)
Unicode 336k ± 0% 336k ± 0% -0.01% (p=0.000 n=500+493)
GoTypes 1.16M ± 0% 1.16M ± 0% -0.30% (p=0.000 n=499+499)
Compiler 4.54M ± 0% 4.51M ± 0% -0.58% (p=0.000 n=494+495)
Passes toolstash-check -gcflags=-dwarf=false. Changes DWARF output
because position information is now tracked more precisely for
function parameters.
Change-Id: Ib8077d70d564cc448c5e4290baceab3a4396d712
Reviewed-on: https://go-review.googlesource.com/108217
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Focus on "isfoo" funcs that take a *Node, and conver them to isFoo
methods instead. This makes for more idiomatic Go code, and also more
readable func names.
Found candidates with grep, and applied most changes with sed. The funcs
chosen were isgoconst, isnil, and isblank. All had the same signature,
func(*Node) bool.
While at it, camelCase the isliteral and iszero function names. Don't
move these to methods, as they are only used in the backend part of gc,
which might one day be split into a separate package.
Passes toolstash -cmp on std cmd.
Change-Id: I4df081b12d36c46c253167c8841c5a841f1c5a16
Reviewed-on: https://go-review.googlesource.com/105555
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Inl, Inldcl, and InlCost are only applicable to functions with bodies
that can be inlined, so pull them out into a separate Inline type to
make understanding them easier.
A side benefit is that we can check if a function can be inlined by
just checking if n.Func.Inl is non-nil, which simplifies handling of
empty function bodies.
While here, remove some unnecessary Curfn twiddling, and make imported
functions use Inl.Dcl instead of Func.Dcl for consistency for local
functions.
Passes toolstash-check.
Change-Id: Ifd4a80349d85d9e8e4484952b38ec4a63182e81f
Reviewed-on: https://go-review.googlesource.com/104756
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
When making a shallow copy of a node, various methods were used,
including calling nod(OXXX, nil, nil) and then overwriting it, or
"n1 := *n" and then using &n1.
Add a copy method instead, simplifying all of those and making them
consistent.
Passes toolstash -cmp on std cmd.
Change-Id: I3f3fc88bad708edc712bf6d87214cda4ddc43b01
Reviewed-on: https://go-review.googlesource.com/72710
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
We used to have three Sym flags for dealing with export/reexport:
Export, Package, and Exported.
Export and Package were used to distinguish whether a symbol is
exported or package-scope (i.e., mutually exclusive), except that for
local declarations Export served double-duty as tracking whether the
symbol had been added to exportlist.
Meanwhile, imported declarations that needed reexporting could be
added to exportlist multiple times, necessitating a flag to track
whether they'd already been written out by exporter.
Simplify all of these into a single OnExportList flag so that we can
ensure symbols on exportlist are present exactly once. Merge
reexportsym into exportsym so there's a single place where we append
to exportlist.
Code that used to set Exported to prevent a symbol from being exported
can now just set OnExportList before calling declare to prevent it
from even appearing on exportlist.
Lastly, drop the IsAlias check in exportsym: we call exportsym too
early for local symbols to detect if they're an alias, and we never
reexport aliases.
Passes toolstash-check.
Change-Id: Icdea3719105dc169fcd7651606589cd08b0a80ff
Reviewed-on: https://go-review.googlesource.com/103865
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently, we reexport any package-scope constant, function, type, or
variable declarations needed by an inlineable function body. However,
now that we have an early pass to walk inlineable function bodies
(golang.org/cl/74110), we can simplify the logic for finding these
declarations.
The binary export format supports writing out type declarations
in-place at their first use. Also, it always writes out constants by
value, so their declarations never need to be reexported.
Notably, we attempted this before (golang.org/cl/36170) and had to
revert it (golang.org/cl/45911). However, this was because while
writing out inline bodies, we could discover variable/function
dependencies. By collecting variable/function dependencies during
inlineable function discovery, we avoid this problem.
While here, get rid of isInlineable. We already typecheck inlineable
function bodies during inlFlood, so it's become a no-op. Just move the
comment explaining parameter numbering to its caller.
Change-Id: Ibbfaafce793733675d3a2ad98791758583055666
Reviewed-on: https://go-review.googlesource.com/103864
Reviewed-by: Robert Griesemer <gri@golang.org>
Rather than checking for each function whether the package supports
instrumentation, check once up front.
Relatedly, tweak the logic for preventing inlining calls to runtime
functions from instrumented packages. Previously, we simply disallowed
inlining runtime functions altogether when instrumenting. With this
CL, it's only disallowed from packages that are actually being
instrumented. That is, now intra-runtime calls can be inlined.
Updates #19054.
Change-Id: I88c97b48bf70193a8a3ee18d952dcb26b0369d55
Reviewed-on: https://go-review.googlesource.com/102815
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Eliminates an inconsistency between user functions and generated
functions.
Passes toolstash-check.
Change-Id: I946b511ca81d88a0024b5932cb50f3d8b9e808f4
Reviewed-on: https://go-review.googlesource.com/103863
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
The main thing is we now eagerly create the ODCLFUNC node for
closures, immediately cross-link them, and assign fields (e.g., Nbody,
Dcl, Parents, Marks) directly on the ODCLFUNC (previously they were
assigned on the OCLOSURE and later moved to the ODCLFUNC).
This allows us to set Curfn to the ODCLFUNC instead of the OCLOSURE,
which makes things more consistent with normal function declarations.
(Notably, this means Cvars now hang off the ODCLFUNC instead of the
OCLOSURE.)
Assignment of xfunc symbol names also now happens before typechecking
their body, which means debugging output now provides a more helpful
name than "<S>".
In golang.org/cl/66810, we changed "x := y" statements to avoid
creating false closure variables for x, but we still create them for
struct literals like "s{f: x}". Update comment in capturevars
accordingly.
More opportunity for cleanups still, but this makes some substantial
progress, IMO.
Passes toolstash-check.
Change-Id: I65a4efc91886e3dcd1000561348af88297775cd7
Reviewed-on: https://go-review.googlesource.com/100197
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
As a side effect of working on mid-stack inlining, we've fixed support
for inlining variadic functions. Might as well enable it.
Change-Id: I7f555f8b941969791db7eb598c0b49f6dc0820aa
Reviewed-on: https://go-review.googlesource.com/100456
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
These are always set to n.Isddd(), which is readily available within
mkinlcall.
Change-Id: I3d7fbc9dc19a40d6b905691c666eee9bcd031a00
Reviewed-on: https://go-review.googlesource.com/100455
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Add helper methods that validate n.Op and convert to/from the
appropriate type.
Notably, there was a lot of code in walk.go that thought setting
Etype=1 on an OADDR node affected escape analysis.
Passes toolstash-check.
TBR=marvin
Change-Id: Ieae7c67225c1459c9719f9e6a748a25b975cf758
Reviewed-on: https://go-review.googlesource.com/99535
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We accidentally overlooked needing to still visit Ninit for OIF
statements with constant conditions in golang.org/cl/96778.
Fixes#24120.
Change-Id: I5b341913065ff90e1163fb872b9e8d47e2a789d2
Reviewed-on: https://go-review.googlesource.com/97475
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, the heap arena map is a single, large array that covers
every possible arena frame in the entire address space. This is
practical up to about 48 bits of address space with 64 MB arenas.
However, there are two problems with this:
1. mips64, ppc64, and s390x support full 64-bit address spaces (though
on Linux only s390x has kernel support for 64-bit address spaces).
On these platforms, it would be good to support these larger
address spaces.
2. On Windows, processes are charged for untouched memory, so for
processes with small heaps, the mostly-untouched 32 MB arena map
plus a 64 MB arena are significant overhead. Hence, it would be
good to reduce both the arena map size and the arena size, but with
a single-level arena, these are inversely proportional.
This CL adds support for a two-level arena map. Arena frame numbers
are now divided into arenaL1Bits of L1 index and arenaL2Bits of L2
index.
At the moment, arenaL1Bits is always 0, so we effectively have a
single level map. We do a few things so that this has no cost beyond
the current single-level map:
1. We embed the L2 array directly in mheap, so if there's a single
entry in the L2 array, the representation is identical to the
current representation and there's no extra level of indirection.
2. Hot code that accesses the arena map is structured so that it
optimizes to nearly the same machine code as it does currently.
3. We make some small tweaks to hot code paths and to the inliner
itself to keep some important functions inlined despite their
now-larger ASTs. In particular, this is necessary for
heapBitsForAddr and heapBits.next.
Possibly as a result of some of the tweaks, this actually slightly
improves the performance of the x/benchmarks garbage benchmark:
name old time/op new time/op delta
Garbage/benchmem-MB=64-12 2.28ms ± 1% 2.26ms ± 1% -1.07% (p=0.000 n=17+19)
(https://perf.golang.org/search?q=upload:20180223.2)
For #23900.
Change-Id: If5164e0961754f97eb9eca58f837f36d759505ff
Reviewed-on: https://go-review.googlesource.com/96779
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
The front-end dead code elimination is very simple. Currently, it just
looks for if statements with constant boolean conditions. Its main
purpose is to reduce load on the compiler and shrink code before
inlining computes hairiness.
This CL teaches front-end dead code elimination about short-circuiting
boolean expressions && and ||, since they're essentially the same as
if statements.
This also teaches the inliner that the constant 'if' form left behind
by deadcode is free.
These changes will help with runtime modifications in the next CL that
would otherwise inhibit inlining in some hot code paths. Currently,
however, they have no significant impact on benchmarks.
Change-Id: I886203b3c4acdbfef08148fddd7f3a7af5afc7c1
Reviewed-on: https://go-review.googlesource.com/96778
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
