It is possible to have an unexported name with a nil package,
for an embedded field whose type is a pointer to an unexported type.
We must encode that fact in the type..namedata symbol name,
to avoid incorrectly merging an unexported name with an exported name.
Fixes#21120
Change-Id: I2e3879d77fa15c05ad92e0bf8e55f74082db5111
Reviewed-on: https://go-review.googlesource.com/50710
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Also change type from map[*types.Type]bool to map[*types.Type]struct{}.
This is basically a clean-up.
Change-Id: I167583eff0fa1070a7522647219476033b52b840
Reviewed-on: https://go-review.googlesource.com/41859
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 39915 introduced sorting of signats by ShortString
for reproducible builds. But ShortString treats types
byte and uint8 identically; same for rune and uint32.
CL 39915 attempted to compensate for this by only
adding the underlying type (uint8) to signats in addsignat.
This only works for byte and uint8. For e.g. *byte and *uint,
both get added, and their sort order is random,
leading to non-reproducible builds.
One fix would be to add yet another type printing mode
that doesn't eliminate byte and rune, and use it
for sorting signats. But the formatting routines
are complicated enough as it is.
Instead, just sort first by ShortString and then by String.
We can't just use String, because ShortString makes distinctions
that String doesn't. ShortString is really preferred here;
String is serving only as a backstop for handling of bytes and runes.
The long series of types in the test helps increase the odds of
failure, allowing a smaller number of iterations in the test.
On my machine, a full test takes 700ms.
Passes toolstash-check.
Updates #19961Fixes#20272
name old alloc/op new alloc/op delta
Template 37.9MB ± 0% 37.9MB ± 0% +0.12% (p=0.032 n=5+5)
Unicode 28.9MB ± 0% 28.9MB ± 0% ~ (p=0.841 n=5+5)
GoTypes 110MB ± 0% 110MB ± 0% ~ (p=0.841 n=5+5)
Compiler 463MB ± 0% 463MB ± 0% ~ (p=0.056 n=5+5)
SSA 1.11GB ± 0% 1.11GB ± 0% +0.02% (p=0.016 n=5+5)
Flate 24.7MB ± 0% 24.8MB ± 0% +0.14% (p=0.032 n=5+5)
GoParser 31.1MB ± 0% 31.1MB ± 0% ~ (p=0.421 n=5+5)
Reflect 73.9MB ± 0% 73.9MB ± 0% ~ (p=1.000 n=5+5)
Tar 25.8MB ± 0% 25.8MB ± 0% +0.15% (p=0.016 n=5+5)
XML 41.2MB ± 0% 41.2MB ± 0% ~ (p=0.310 n=5+5)
[Geo mean] 72.0MB 72.0MB +0.07%
name old allocs/op new allocs/op delta
Template 384k ± 0% 385k ± 1% ~ (p=0.056 n=5+5)
Unicode 343k ± 0% 344k ± 0% ~ (p=0.548 n=5+5)
GoTypes 1.16M ± 0% 1.16M ± 0% ~ (p=0.421 n=5+5)
Compiler 4.43M ± 0% 4.44M ± 0% +0.26% (p=0.032 n=5+5)
SSA 9.86M ± 0% 9.87M ± 0% +0.10% (p=0.032 n=5+5)
Flate 237k ± 1% 238k ± 0% +0.49% (p=0.032 n=5+5)
GoParser 319k ± 1% 320k ± 1% ~ (p=0.151 n=5+5)
Reflect 957k ± 0% 957k ± 0% ~ (p=1.000 n=5+5)
Tar 251k ± 0% 252k ± 1% +0.49% (p=0.016 n=5+5)
XML 399k ± 0% 401k ± 1% ~ (p=0.310 n=5+5)
[Geo mean] 739k 741k +0.26%
Change-Id: Ic27995a8d374d012b8aca14546b1df9d28d30df7
Reviewed-on: https://go-review.googlesource.com/42955
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This CL adds initial support for concurrent backend compilation.
BACKGROUND
The compiler currently consists (very roughly) of the following phases:
1. Initialization.
2. Lexing and parsing into the cmd/compile/internal/syntax AST.
3. Translation into the cmd/compile/internal/gc AST.
4. Some gc AST passes: typechecking, escape analysis, inlining,
closure handling, expression evaluation ordering (order.go),
and some lowering and optimization (walk.go).
5. Translation into the cmd/compile/internal/ssa SSA form.
6. Optimization and lowering of SSA form.
7. Translation from SSA form to assembler instructions.
8. Translation from assembler instructions to machine code.
9. Writing lots of output: machine code, DWARF symbols,
type and reflection info, export data.
Phase 2 was already concurrent as of Go 1.8.
Phase 3 is planned for eventual removal;
we hope to go straight from syntax AST to SSA.
Phases 5–8 are per-function; this CL adds support for
processing multiple functions concurrently.
The slowest phases in the compiler are 5 and 6,
so this offers the opportunity for some good speed-ups.
Unfortunately, it's not quite that straightforward.
In the current compiler, the latter parts of phase 4
(order, walk) are done function-at-a-time as needed.
Making order and walk concurrency-safe proved hard,
and they're not particularly slow, so there wasn't much reward.
To enable phases 5–8 to be done concurrently,
when concurrent backend compilation is requested,
we complete phase 4 for all functions
before starting later phases for any functions.
Also, in reality, we automatically generate new
functions in phase 9, such as method wrappers
and equality and has routines.
Those new functions then go through phases 4–8.
This CL disables concurrent backend compilation
after the first, big, user-provided batch of
functions has been compiled.
This is done to keep things simple,
and because the autogenerated functions
tend to be small, few, simple, and fast to compile.
USAGE
Concurrent backend compilation still defaults to off.
To set the number of functions that may be backend-compiled
concurrently, use the compiler flag -c.
In future work, cmd/go will automatically set -c.
Furthermore, this CL has been intentionally written
so that the c=1 path has no backend concurrency whatsoever,
not even spawning any goroutines.
This helps ensure that, should problems arise
late in the development cycle,
we can simply have cmd/go set c=1 always,
and revert to the original compiler behavior.
MUTEXES
Most of the work required to make concurrent backend
compilation safe has occurred over the past month.
This CL adds a handful of mutexes to get the rest of the way there;
they are the mutexes that I didn't see a clean way to avoid.
Some of them may still be eliminable in future work.
In no particular order:
* gc.funcsymsmu. The global funcsyms slice is populated
lazily when we need function symbols for closures.
This occurs during gc AST to SSA translation.
The function funcsym also does a package lookup,
which is a source of races on types.Pkg.Syms;
funcsymsmu also covers that package lookup.
This mutex is low priority: it adds a single global,
it is in an infrequently used code path, and it is low contention.
Since funcsyms may now be added in any order,
we must sort them to preserve reproducible builds.
* gc.largeStackFramesMu. We don't discover until after SSA compilation
that a function's stack frame is gigantic.
Recording that error happens basically never,
but it does happen concurrently.
Fix with a low priority mutex and sorting.
* obj.Link.hashmu. ctxt.hash stores the mapping from
types.Syms (compiler symbols) to obj.LSyms (linker symbols).
It is accessed fairly heavily through all the phases.
This is the only heavily contended mutex.
* gc.signatlistmu. The global signatlist map is
populated with types through several of the concurrent phases,
including notably via ngotype during DWARF generation.
It is low priority for removal.
* gc.typepkgmu. Looking up symbols in the types package
happens a fair amount during backend compilation
and DWARF generation, particularly via ngotype.
This mutex helps us to avoid a broader mutex on types.Pkg.Syms.
It has low-to-moderate contention.
* types.internedStringsmu. gc AST to SSA conversion and
some SSA work introduce new autotmps.
Those autotmps have their names interned to reduce allocations.
That interning requires protecting types.internedStrings.
The autotmp names are heavily re-used, and the mutex
overhead and contention here are low, so it is probably
a worthwhile performance optimization to keep this mutex.
TESTING
I have been testing this code locally by running
'go install -race cmd/compile'
and then doing
'go build -a -gcflags=-c=128 std cmd'
for all architectures and a variety of compiler flags.
This obviously needs to be made part of the builders,
but it is too expensive to make part of all.bash.
I have filed #19962 for this.
REPRODUCIBLE BUILDS
This version of the compiler generates reproducible builds.
Testing reproducible builds also needs automation, however,
and is also too expensive for all.bash.
This is #19961.
Also of note is that some of the compiler flags used by 'toolstash -cmp'
are currently incompatible with concurrent backend compilation.
They still work fine with c=1.
Time will tell whether this is a problem.
NEXT STEPS
* Continue to find and fix races and bugs,
using a combination of code inspection, fuzzing,
and hopefully some community experimentation.
I do not know of any outstanding races,
but there probably are some.
* Improve testing.
* Improve performance, for many values of c.
* Integrate with cmd/go and fine tune.
* Support concurrent compilation with the -race flag.
It is a sad irony that it does not yet work.
* Minor code cleanup that has been deferred during
the last month due to uncertainty about the
ultimate shape of this CL.
PERFORMANCE
Here's the buried lede, at last. :)
All benchmarks are from my 8 core 2.9 GHz Intel Core i7 darwin/amd64 laptop.
First, going from tip to this CL with c=1 has almost no impact.
name old time/op new time/op delta
Template 195ms ± 3% 194ms ± 5% ~ (p=0.370 n=30+29)
Unicode 86.6ms ± 3% 87.0ms ± 7% ~ (p=0.958 n=29+30)
GoTypes 548ms ± 3% 555ms ± 4% +1.35% (p=0.001 n=30+28)
Compiler 2.51s ± 2% 2.54s ± 2% +1.17% (p=0.000 n=28+30)
SSA 5.16s ± 3% 5.16s ± 2% ~ (p=0.910 n=30+29)
Flate 124ms ± 5% 124ms ± 4% ~ (p=0.947 n=30+30)
GoParser 146ms ± 3% 146ms ± 3% ~ (p=0.150 n=29+28)
Reflect 354ms ± 3% 352ms ± 4% ~ (p=0.096 n=29+29)
Tar 107ms ± 5% 106ms ± 3% ~ (p=0.370 n=30+29)
XML 200ms ± 4% 201ms ± 4% ~ (p=0.313 n=29+28)
[Geo mean] 332ms 333ms +0.10%
name old user-time/op new user-time/op delta
Template 227ms ± 5% 225ms ± 5% ~ (p=0.457 n=28+27)
Unicode 109ms ± 4% 109ms ± 5% ~ (p=0.758 n=29+29)
GoTypes 713ms ± 4% 721ms ± 5% ~ (p=0.051 n=30+29)
Compiler 3.36s ± 2% 3.38s ± 3% ~ (p=0.146 n=30+30)
SSA 7.46s ± 3% 7.47s ± 3% ~ (p=0.804 n=30+29)
Flate 146ms ± 7% 147ms ± 3% ~ (p=0.833 n=29+27)
GoParser 179ms ± 5% 179ms ± 5% ~ (p=0.866 n=30+30)
Reflect 431ms ± 4% 429ms ± 4% ~ (p=0.593 n=29+30)
Tar 124ms ± 5% 123ms ± 5% ~ (p=0.140 n=29+29)
XML 243ms ± 4% 242ms ± 7% ~ (p=0.404 n=29+29)
[Geo mean] 415ms 415ms +0.02%
name old obj-bytes new obj-bytes delta
Template 382k ± 0% 382k ± 0% ~ (all equal)
Unicode 203k ± 0% 203k ± 0% ~ (all equal)
GoTypes 1.18M ± 0% 1.18M ± 0% ~ (all equal)
Compiler 3.98M ± 0% 3.98M ± 0% ~ (all equal)
SSA 8.28M ± 0% 8.28M ± 0% ~ (all equal)
Flate 230k ± 0% 230k ± 0% ~ (all equal)
GoParser 287k ± 0% 287k ± 0% ~ (all equal)
Reflect 1.00M ± 0% 1.00M ± 0% ~ (all equal)
Tar 190k ± 0% 190k ± 0% ~ (all equal)
XML 416k ± 0% 416k ± 0% ~ (all equal)
[Geo mean] 660k 660k +0.00%
Comparing this CL to itself, from c=1 to c=2
improves real times 20-30%, costs 5-10% more CPU time,
and adds about 2% alloc.
The allocation increase comes from allocating more ssa.Caches.
name old time/op new time/op delta
Template 202ms ± 3% 149ms ± 3% -26.15% (p=0.000 n=49+49)
Unicode 87.4ms ± 4% 84.2ms ± 3% -3.68% (p=0.000 n=48+48)
GoTypes 560ms ± 2% 398ms ± 2% -28.96% (p=0.000 n=49+49)
Compiler 2.46s ± 3% 1.76s ± 2% -28.61% (p=0.000 n=48+46)
SSA 6.17s ± 2% 4.04s ± 1% -34.52% (p=0.000 n=49+49)
Flate 126ms ± 3% 92ms ± 2% -26.81% (p=0.000 n=49+48)
GoParser 148ms ± 4% 107ms ± 2% -27.78% (p=0.000 n=49+48)
Reflect 361ms ± 3% 281ms ± 3% -22.10% (p=0.000 n=49+49)
Tar 109ms ± 4% 86ms ± 3% -20.81% (p=0.000 n=49+47)
XML 204ms ± 3% 144ms ± 2% -29.53% (p=0.000 n=48+45)
name old user-time/op new user-time/op delta
Template 246ms ± 9% 246ms ± 4% ~ (p=0.401 n=50+48)
Unicode 109ms ± 4% 111ms ± 4% +1.47% (p=0.000 n=44+50)
GoTypes 728ms ± 3% 765ms ± 3% +5.04% (p=0.000 n=46+50)
Compiler 3.33s ± 3% 3.41s ± 2% +2.31% (p=0.000 n=49+48)
SSA 8.52s ± 2% 9.11s ± 2% +6.93% (p=0.000 n=49+47)
Flate 149ms ± 4% 161ms ± 3% +8.13% (p=0.000 n=50+47)
GoParser 181ms ± 5% 192ms ± 2% +6.40% (p=0.000 n=49+46)
Reflect 452ms ± 9% 474ms ± 2% +4.99% (p=0.000 n=50+48)
Tar 126ms ± 6% 136ms ± 4% +7.95% (p=0.000 n=50+49)
XML 247ms ± 5% 264ms ± 3% +6.94% (p=0.000 n=48+50)
name old alloc/op new alloc/op delta
Template 38.8MB ± 0% 39.3MB ± 0% +1.48% (p=0.008 n=5+5)
Unicode 29.8MB ± 0% 30.2MB ± 0% +1.19% (p=0.008 n=5+5)
GoTypes 113MB ± 0% 114MB ± 0% +0.69% (p=0.008 n=5+5)
Compiler 443MB ± 0% 447MB ± 0% +0.95% (p=0.008 n=5+5)
SSA 1.25GB ± 0% 1.26GB ± 0% +0.89% (p=0.008 n=5+5)
Flate 25.3MB ± 0% 25.9MB ± 1% +2.35% (p=0.008 n=5+5)
GoParser 31.7MB ± 0% 32.2MB ± 0% +1.59% (p=0.008 n=5+5)
Reflect 78.2MB ± 0% 78.9MB ± 0% +0.91% (p=0.008 n=5+5)
Tar 26.6MB ± 0% 27.0MB ± 0% +1.80% (p=0.008 n=5+5)
XML 42.4MB ± 0% 43.4MB ± 0% +2.35% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Template 379k ± 0% 378k ± 0% ~ (p=0.421 n=5+5)
Unicode 322k ± 0% 321k ± 0% ~ (p=0.222 n=5+5)
GoTypes 1.14M ± 0% 1.14M ± 0% ~ (p=0.548 n=5+5)
Compiler 4.12M ± 0% 4.11M ± 0% -0.14% (p=0.032 n=5+5)
SSA 9.72M ± 0% 9.72M ± 0% ~ (p=0.421 n=5+5)
Flate 234k ± 1% 234k ± 0% ~ (p=0.421 n=5+5)
GoParser 316k ± 1% 315k ± 0% ~ (p=0.222 n=5+5)
Reflect 980k ± 0% 979k ± 0% ~ (p=0.095 n=5+5)
Tar 249k ± 1% 249k ± 1% ~ (p=0.841 n=5+5)
XML 392k ± 0% 391k ± 0% ~ (p=0.095 n=5+5)
From c=1 to c=4, real time is down ~40%, CPU usage up 10-20%, alloc up ~5%:
name old time/op new time/op delta
Template 203ms ± 3% 131ms ± 5% -35.45% (p=0.000 n=50+50)
Unicode 87.2ms ± 4% 84.1ms ± 2% -3.61% (p=0.000 n=48+47)
GoTypes 560ms ± 4% 310ms ± 2% -44.65% (p=0.000 n=50+49)
Compiler 2.47s ± 3% 1.41s ± 2% -43.10% (p=0.000 n=50+46)
SSA 6.17s ± 2% 3.20s ± 2% -48.06% (p=0.000 n=49+49)
Flate 126ms ± 4% 74ms ± 2% -41.06% (p=0.000 n=49+48)
GoParser 148ms ± 4% 89ms ± 3% -39.97% (p=0.000 n=49+50)
Reflect 360ms ± 3% 242ms ± 3% -32.81% (p=0.000 n=49+49)
Tar 108ms ± 4% 73ms ± 4% -32.48% (p=0.000 n=50+49)
XML 203ms ± 3% 119ms ± 3% -41.56% (p=0.000 n=49+48)
name old user-time/op new user-time/op delta
Template 246ms ± 9% 287ms ± 9% +16.98% (p=0.000 n=50+50)
Unicode 109ms ± 4% 118ms ± 5% +7.56% (p=0.000 n=46+50)
GoTypes 735ms ± 4% 806ms ± 2% +9.62% (p=0.000 n=50+50)
Compiler 3.34s ± 4% 3.56s ± 2% +6.78% (p=0.000 n=49+49)
SSA 8.54s ± 3% 10.04s ± 3% +17.55% (p=0.000 n=50+50)
Flate 149ms ± 6% 176ms ± 3% +17.82% (p=0.000 n=50+48)
GoParser 181ms ± 5% 213ms ± 3% +17.47% (p=0.000 n=50+50)
Reflect 453ms ± 6% 499ms ± 2% +10.11% (p=0.000 n=50+48)
Tar 126ms ± 5% 149ms ±11% +18.76% (p=0.000 n=50+50)
XML 246ms ± 5% 287ms ± 4% +16.53% (p=0.000 n=49+50)
name old alloc/op new alloc/op delta
Template 38.8MB ± 0% 40.4MB ± 0% +4.21% (p=0.008 n=5+5)
Unicode 29.8MB ± 0% 30.9MB ± 0% +3.68% (p=0.008 n=5+5)
GoTypes 113MB ± 0% 116MB ± 0% +2.71% (p=0.008 n=5+5)
Compiler 443MB ± 0% 455MB ± 0% +2.75% (p=0.008 n=5+5)
SSA 1.25GB ± 0% 1.27GB ± 0% +1.84% (p=0.008 n=5+5)
Flate 25.3MB ± 0% 26.9MB ± 1% +6.31% (p=0.008 n=5+5)
GoParser 31.7MB ± 0% 33.2MB ± 0% +4.61% (p=0.008 n=5+5)
Reflect 78.2MB ± 0% 80.2MB ± 0% +2.53% (p=0.008 n=5+5)
Tar 26.6MB ± 0% 27.9MB ± 0% +5.19% (p=0.008 n=5+5)
XML 42.4MB ± 0% 44.6MB ± 0% +5.20% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Template 380k ± 0% 379k ± 0% -0.39% (p=0.032 n=5+5)
Unicode 321k ± 0% 321k ± 0% ~ (p=0.841 n=5+5)
GoTypes 1.14M ± 0% 1.14M ± 0% ~ (p=0.421 n=5+5)
Compiler 4.12M ± 0% 4.14M ± 0% +0.52% (p=0.008 n=5+5)
SSA 9.72M ± 0% 9.76M ± 0% +0.37% (p=0.008 n=5+5)
Flate 234k ± 1% 234k ± 1% ~ (p=0.690 n=5+5)
GoParser 316k ± 0% 317k ± 1% ~ (p=0.841 n=5+5)
Reflect 981k ± 0% 981k ± 0% ~ (p=1.000 n=5+5)
Tar 250k ± 0% 249k ± 1% ~ (p=0.151 n=5+5)
XML 393k ± 0% 392k ± 0% ~ (p=0.056 n=5+5)
Going beyond c=4 on my machine tends to increase CPU time and allocs
without impacting real time.
The CPU time numbers matter, because when there are many concurrent
compilation processes, that will impact the overall throughput.
The numbers above are in many ways the best case scenario;
we can take full advantage of all cores.
Fortunately, the most common compilation scenario is incremental
re-compilation of a single package during a build/test cycle.
Updates #15756
Change-Id: I6725558ca2069edec0ac5b0d1683105a9fff6bea
Reviewed-on: https://go-review.googlesource.com/40693
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
dumptypestructs did several different jobs.
Split them into separate functions
and call them in turn.
Hand dumptypestructs a list of dcls,
rather than reading the global.
Rename dumpptabs for (marginal) clarity.
This is groundwork for compiling autogenerated
functions concurrently.
Passes toolstash-check.
Change-Id: I627a1dffc70a7e4b7b4436ab19af1406267f01dc
Reviewed-on: https://go-review.googlesource.com/41501
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change a few functions so that instead of
accepting a *types.Sym and calling Linksym
themselves, they accept an *obj.LSym.
Adapt the callsites.
Passes toolstash-check.
Change-Id: Ic5d3f306f2fdd3913281215a1f54d893a966bb1f
Reviewed-on: https://go-review.googlesource.com/41404
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This batch from reflect.go.
Changes made manually, since they are simple,
few, and typechecked by the compiler.
Passes toolstash-check.
Change-Id: I0030daab2dac8e7c95158678c0f7141fd90441f9
Reviewed-on: https://go-review.googlesource.com/41399
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The only remaining uses of duintxx
are in the implementation of duintNN.
I hope to inline those once I figure out why
CL 40864 is broken.
Note that some uses of duintxx with width Widthint
were converted into duintptr.
I did that, since #19954 is officially going to move forward.
Passes toolstash-check.
Change-Id: Id25253b711ea589d0199b51be9a3c18ca1af59ce
Reviewed-on: https://go-review.googlesource.com/41398
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This is an automated refactoring to eliminate
all dxxx calls in gc/obj.go that accept types.Sym
instead of obj.LSym parameters.
The refactoring was of the form:
gorename -from '"cmd/compile/internal/gc".duintxx' -to Duintxx
gorename -from '"cmd/compile/internal/gc".duintxxLSym' -to DuintxxLSym
eg -t t.go -w cmd/compile/internal/gc
gofmt -r 'DuintxxLSym -> duintxxLSym' -w cmd/compile/internal/gc
where t.go looked like:
func before(s *types.Sym, off int, v uint64, wid int) int {
return gc.Duintxx(s, off, v, wid)
}
func after(s *types.Sym, off int, v uint64, wid int) int {
return gc.DuintxxLSym(s.Linksym(), off, v, wid)
}
The rename/gofmt shenanigans were to work around
limitations and bugs in eg and gorename.
The resulting code in reflect.go looks temporarily ugly,
but it makes refactoring and cleanup opportunities
much clearer.
Next step is to rename all the dxxx methods to rename the -LSym suffix
and clean up reflect.go.
The renaming is left for a separate CL to make the changes in
this CL more obvious, and thus hopefully easier to review.
Passes toolstash-check.
Change-Id: Ib31a2b6fd146ed03a855d20ecb0433f0f74e2f10
Reviewed-on: https://go-review.googlesource.com/41396
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Response to code review feedback on CL 40693.
It is now only accessible by types.TypePkgLookup.
Passes toolstash-check.
Change-Id: I0c422c1a271f97467ae38de53af9dc33f4b31bdb
Reviewed-on: https://go-review.googlesource.com/41304
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Response to code review feedback on CL 40693.
Remove the final reference to it from package gc,
and manually unexport.
Passes toolstash-check.
Change-Id: I7fc48edd43263d8f7c56b47aeb7573408463dc22
Reviewed-on: https://go-review.googlesource.com/41303
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
There were only two versions, 0 and 1,
and the only user of version 1 was the assembler,
to indicate that a symbol was static.
Rename LSym.Version to Static,
and add it to LSym.Attributes.
Simplify call-sites.
Passes toolstash-check.
Change-Id: Iabd39918f5019cce78f381d13f0481ae09f3871f
Reviewed-on: https://go-review.googlesource.com/41201
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
- PkgMap was only needed to test import/export in a "cleanroom"
environment, with debugFormat set. Provided helper function
instead.
- PkgList was only used to identify directly imported packages.
Instead, compute that list explicitly from the package map.
It happens only once, the list is small, and it's more robust
than keeping two data structures in sync.
Change-Id: I82dce3c0b5cb816faae58708e877799359c20fcb
Reviewed-on: https://go-review.googlesource.com/41078
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
There's already special code to access it.
Change-Id: I28ca4f44a04262407ee9f1c826ada4e7eba44775
Reviewed-on: https://go-review.googlesource.com/41073
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
That's where it belongs. Also, moved pkgMap and pkgs globals.
Change-Id: I531727fe5ce162c403efefec82f4cc90afa326d7
Reviewed-on: https://go-review.googlesource.com/41071
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Now only cmd/asm and cmd/compile depend on cmd/internal/obj. Changing
the assembler backends no longer requires reinstalling cmd/link or
cmd/addr2line.
There's also now one canonical definition of the object file format in
cmd/internal/objabi/doc.go, with a warning to update all three
implementations.
objabi is still something of a grab bag of unrelated code (e.g., flag
and environment variable handling probably belong in a separate "tool"
package), but this is still progress.
Fixes#15165.
Fixes#20026.
Change-Id: Ic4b92fac7d0d35438e0d20c9579aad4085c5534c
Reviewed-on: https://go-review.googlesource.com/40972
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
sort.Slice was added in Go 1.8.
It's nice to use, and faster than sort.Sort,
so it'd be nice to be able to use it in the toolchain.
This CL adds obj.SortSlice, which is sort.Slice,
but with a slower fallback version for bootstrapping.
This CL also includes a single demo+test use.
Change-Id: I2accc60b61f8e48c8ab4f1a63473e3b87af9b691
Reviewed-on: https://go-review.googlesource.com/40114
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
pkgByPath was added in d78c84c4 to eliminate the differences between the
export formats around the time of Go 1.7.
The last remnants of the textual export format was removed by Josh in
39850 making the pkgByPath sorting type unused.
Change-Id: I168816d6401f45119475a4fe5ada00d9ce571a9e
Reviewed-on: https://go-review.googlesource.com/40050
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This is a re-roll of CL 39710,
which broke deterministic builds.
typenamesym is called from three places:
typename, ngotype, and Type.Symbol.
Only in typename do we actually need a Node.
ngotype and Type.Symbol require only a Sym.
And writing the newly created Node to
Sym.Def is unsafe in a concurrent backend.
Rather than use a mutex protect to Sym.Def,
make typenamesym not touch Sym.Def.
The assignment to Sym.Def was serving a second purpose,
namely to prevent duplicate entries on signatlist.
Preserve that functionality by switching signatlist to a map.
This in turn requires that we sort signatlist
when exporting it, to preserve reproducibility.
We sort using exactly the same mechanism
that the export code (dtypesym) uses.
Failure to do that led to non-deterministic builds (#19872).
Since we've already calculated the Type's export name,
we could pass it to dtypesym, sparing it a bit of work.
That can be done as a future optimization.
Updates #15756
name old alloc/op new alloc/op delta
Template 39.2MB ± 0% 39.3MB ± 0% ~ (p=0.075 n=10+10)
Unicode 29.8MB ± 0% 29.8MB ± 0% ~ (p=0.393 n=10+10)
GoTypes 113MB ± 0% 113MB ± 0% +0.06% (p=0.027 n=10+8)
SSA 1.25GB ± 0% 1.25GB ± 0% +0.05% (p=0.000 n=8+10)
Flate 25.3MB ± 0% 25.3MB ± 0% ~ (p=0.105 n=10+10)
GoParser 31.7MB ± 0% 31.8MB ± 0% ~ (p=0.165 n=10+10)
Reflect 78.2MB ± 0% 78.2MB ± 0% ~ (p=0.190 n=10+10)
Tar 26.6MB ± 0% 26.6MB ± 0% ~ (p=0.481 n=10+10)
XML 42.2MB ± 0% 42.2MB ± 0% ~ (p=0.968 n=10+9)
name old allocs/op new allocs/op delta
Template 384k ± 1% 386k ± 1% +0.43% (p=0.019 n=10+10)
Unicode 320k ± 0% 321k ± 0% +0.36% (p=0.015 n=10+10)
GoTypes 1.14M ± 0% 1.14M ± 0% +0.33% (p=0.000 n=10+8)
SSA 9.69M ± 0% 9.71M ± 0% +0.18% (p=0.000 n=10+9)
Flate 233k ± 1% 233k ± 1% ~ (p=0.481 n=10+10)
GoParser 315k ± 1% 316k ± 1% ~ (p=0.113 n=9+10)
Reflect 979k ± 0% 979k ± 0% ~ (p=0.971 n=10+10)
Tar 250k ± 1% 250k ± 1% ~ (p=0.481 n=10+10)
XML 391k ± 1% 392k ± 0% ~ (p=1.000 n=10+9)
Change-Id: Ia9f21cc29c047021fa8a18c2a3d861a5146aefac
Reviewed-on: https://go-review.googlesource.com/39915
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
- created new package cmd/compile/internal/types
- moved Pkg, Sym, Type to new package
- to break cycles, for now we need the (ugly) types/utils.go
file which contains a handful of functions that must be installed
early by the gc frontend
- to break cycles, for now we need two functions to convert between
*gc.Node and *types.Node (the latter is a dummy type)
- adjusted the gc's code to use the new package and the conversion
functions as needed
- made several Pkg, Sym, and Type methods functions as needed
- renamed constructors typ, typPtr, typArray, etc. to types.New,
types.NewPtr, types.NewArray, etc.
Passes toolstash-check -all.
Change-Id: I8adfa5e85c731645d0a7fd2030375ed6ebf54b72
Reviewed-on: https://go-review.googlesource.com/39855
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The textual import/export format is ancient history.
Change-Id: Iebe90bfd9bd3074eb191186d86e5f4286ce3b1f3
Reviewed-on: https://go-review.googlesource.com/39850
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
typenamesym is called from three places:
typename, ngotype, and Type.Symbol.
Only in typename do we actually need a Node.
ngotype and Type.Symbol require only a Sym.
And writing the newly created Node to
Sym.Def is unsafe in a concurrent backend.
Rather than use a mutex protect to Sym.Def,
make typenamesym not touch Sym.Def.
The assignment to Sym.Def was serving a second purpose,
namely to prevent duplicate entries on signatlist.
Preserve that functionality by switching signatlist to a map.
This in turn requires that we sort signatlist
when exporting it, to preserve reproducibility.
We'd like to use Type.cmp for sorting,
but that causes infinite recursion at the moment;
see #19869.
For now, use Type.LongString as the sort key,
which is a complete description of the type.
Type.LongString is relatively expensive,
but we calculate it only once per type,
and signatlist is generally fairly small,
so the performance impact is minimal.
Updates #15756
name old alloc/op new alloc/op delta
Template 39.4MB ± 0% 39.4MB ± 0% ~ (p=0.222 n=5+5)
Unicode 29.8MB ± 0% 29.8MB ± 0% ~ (p=0.151 n=5+5)
GoTypes 113MB ± 0% 113MB ± 0% ~ (p=0.095 n=5+5)
SSA 1.25GB ± 0% 1.25GB ± 0% +0.04% (p=0.008 n=5+5)
Flate 25.3MB ± 0% 25.4MB ± 0% ~ (p=0.056 n=5+5)
GoParser 31.8MB ± 0% 31.8MB ± 0% ~ (p=0.310 n=5+5)
Reflect 78.3MB ± 0% 78.3MB ± 0% ~ (p=0.690 n=5+5)
Tar 26.7MB ± 0% 26.7MB ± 0% ~ (p=0.548 n=5+5)
XML 42.2MB ± 0% 42.2MB ± 0% ~ (p=0.222 n=5+5)
name old allocs/op new allocs/op delta
Template 387k ± 0% 388k ± 0% ~ (p=0.056 n=5+5)
Unicode 320k ± 0% 321k ± 0% +0.32% (p=0.032 n=5+5)
GoTypes 1.14M ± 0% 1.15M ± 0% ~ (p=0.095 n=5+5)
SSA 9.70M ± 0% 9.72M ± 0% +0.18% (p=0.008 n=5+5)
Flate 234k ± 0% 235k ± 0% +0.60% (p=0.008 n=5+5)
GoParser 317k ± 0% 317k ± 0% ~ (p=1.000 n=5+5)
Reflect 982k ± 0% 983k ± 0% ~ (p=0.841 n=5+5)
Tar 252k ± 1% 252k ± 0% ~ (p=0.310 n=5+5)
XML 393k ± 0% 392k ± 0% ~ (p=0.548 n=5+5)
Change-Id: I53a3b95d19cf1a7b7511a94fba896706addf84fb
Reviewed-on: https://go-review.googlesource.com/39710
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
It was simply a wrapper around Link.Lookup.
Unwrap everything.
CL prepared using eg with template:
package p
import "cmd/internal/obj"
func before(ctxt *obj.Link, name string, version int) *obj.LSym {
return obj.Linklookup(ctxt, name, version)
}
func after(ctxt *obj.Link, name string, version int) *obj.LSym {
return ctxt.Lookup(name, version)
}
Then one comment in cmd/asm/internal/asm/parse.go
was manually updated (and gofmt'ed!),
and func Linklookup deleted.
Passes toolstash-check (as a sanity measure).
Change-Id: Icc4d56b0b2b5c8888d3184c1898c48359ea1e638
Reviewed-on: https://go-review.googlesource.com/39715
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The node in typenamesym requires neither
a position nor a curfn.
Passes toolstash-check.
Updates #15756
Change-Id: I6d39a8961e5578fe5924aaceb29045b6de2699df
Reviewed-on: https://go-review.googlesource.com/39194
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Convert yyerrors into Fatals.
Remove the goto.
Move variable declaration closer to use.
Unify printing strings a bit.
Convert an int param into a bool.
Passes toolstash-check. No compiler performance impact.
Change-Id: I9017681417b785cf8693d18b124ac4f1ff37f2b5
Reviewed-on: https://go-review.googlesource.com/39170
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Remove one of the many lookup variants.
Change-Id: I4095aa030da4227540badd6724bbf50b728fbe93
Reviewed-on: https://go-review.googlesource.com/38990
Reviewed-by: Dave Cheney <dave@cheney.net>
