cybercyst/go
1
0
Fork 0
mirror of https://github.com/golang/go.git synced 2025-05-05 15:43:04 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Include suggestions from #31118 and add documentation about new features

Michael Knyszek 2022-11-18 16:18:00 -05:00
parent 602768cab9
commit 71cf0c199c
1 changed files with 144 additions and 84 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

228
Gomote.md

@ -1,96 +1,157 @@
# Using gomote
# Gomote
## Installing
The gomote command is a client for the Go builder infrastructure. It's a remote control for remote Go builder machines.
## Installation
```
$ go install golang.org/x/build/cmd/gomote@latest # Go 1.16 and latest
```
## Using
## Usage
TODO: examples. For now:
The most basic usage of the gomote tool involves just a few steps:
1. Create an instance.
1. Push code to the instance.
1. Run commands on the instance.
Running the `create` command will list available instance types.
```
The gomote command is a client for the Go builder infrastructure. It's a
remote control for remote Go builder machines.
See https://go.dev/wiki/Gomote
Usage:
gomote [global-flags] cmd [cmd-flags]
For example,
$ gomote create openbsd-amd64-60
user-username-openbsd-amd64-60-0
$ gomote push user-username-openbsd-amd64-60-0
$ gomote run user-username-openbsd-amd64-60-0 go/src/make.bash
$ gomote run user-username-openbsd-amd64-60-0 go/bin/go test -v -short os
To list the subcommands, run "gomote" without arguments:
Commands:
create create a buildlet; with no args, list types of buildlets
destroy destroy a buildlet
gettar extract a tar.gz from a buildlet
list list active buildlets
ls list the contents of a directory on a buildlet
ping test whether a buildlet is alive and reachable
push sync your GOROOT directory to the buildlet
put put files on a buildlet
put14 put Go 1.4 in place
puttar extract a tar.gz to a buildlet
rm delete files or directories
run run a command on a buildlet
ssh ssh to a buildlet
To list all the builder types available, run "create" with no arguments:
$ gomote create
(list tons of buildlet types)
The "gomote run" command has many of its own flags:
$ gomote run -h
run usage: gomote run [run-opts] <instance> <cmd> [args...]
-builderenv string
Optional alternate builder to act like. Must share the same
underlying buildlet host type, or it's an error. For
instance, linux-amd64-race or linux-386-387 are compatible
with linux-amd64, but openbsd-amd64 and openbsd-386 are
different hosts.
-debug
write debug info about the command's execution before it begins
-dir string
Directory to run from. Defaults to the directory of the
command, or the work directory if -system is true.
-e value
Environment variable KEY=value. The -e flag may be repeated
multiple times to add multiple things to the environment.
-path string
Comma-separated list of ExecOpts.Path elements. The special
string 'EMPTY' means to run without any $PATH. The empty
string (default) does not modify the $PATH. Otherwise, the
following expansions apply: the string '$PATH' expands to
the current PATH element(s), the substring '$WORKDIR'
expands to the buildlet's temp workdir.
-system
run inside the system, and not inside the workdir; this is implicit if cmd starts with '/'
Debugging buildlets directly
Using "gomote create" contacts the build coordinator (farmer.golang.org) and
requests that it create the buildlet on your behalf. All subsequent commands
(such as "gomote run" or "gomote ls") then proxy your request via the
coordinator. To access a buildlet directly (for example, when working on the
buildlet code), you can skip the "gomote create" step and use the special
builder name "<build-config-name>@ip[:port>", such as
"windows-amd64-2008@10.1.5.3".
$ gomote create
(list tons of buildlet types)
```
## Tricks
Then, an instance can be created by specifying an instance type. The instance's name will be printed to stdout. There may be other logging messages, but they will be prefixed with a '#' character.
```
$ gomote create linux-amd64
# Creating instance of type linux-amd64... (0s)
user-linux-amd64-0
```
With that instance's name you can now push (more specifically sync the contents of) a local Go repository to the instance and install a bootstrap toolchain. The repository you sync will appear at the `go` subdirectory of `$WORKDIR` (the default directory of all gomote operations). The bootstrap toolchain will always go into the `go1.4` subdirectory (even if the bootstrap toolchain isn't from version 1.4).
```
$ GOROOT=/path/to/local/go/repo gomote push user-linux-amd64-0
$ gomote ls user-linux-amd64-0
go1.4
go
```
Note that `push` really is a "sync" operation, so next time you push the gomote tool will only push what has changed (files added, modified, or removed).
With a toolchain installed, you can now build it by running commands on the instance. The `run` command allows you to specify an executable to run. The executable must be specified relative to `$WORKDIR` (e.g. `go/bin/go`) or via an absolute path (e.g. `/bin/bash`). That executable will then run with its current working directory set to the directory containing the executable.
```
$ gomote run user-linux-amd64-0 go/src/make.bash
```
To then run the built Go toolchain, use `go/bin/go`.
```
$ gomote run user-linux-amd64-0 go/bin/go test -run="TestSomething" -v runtime
```
You can additionally specify a working directory and environment variables to `run` that will be applied before the command is executed.
Note that gomote instances will automatically disappear after 30 minutes of inactivity. Use the `list` command to check how long they have left.
```
$ gomote list
(list of instances and how long they have left to live)
```
The `ping` command can be used to keep an instance alive if no other commands are being run against it.
For more information on each command run `gomote help <command>`. For more commands, run `gomote help`.
### Debugging buildlets directly
The `create` command contacts the build coordinator (farmer.golang.org) and requests that it create the buildlet on your behalf. All subsequent commands (such as `gomote run` or `gomote ls`) then proxy your request via the coordinator. To access a buildlet directly (for example, when working on the buildlet code), you can skip the `gomote create` step and use the special builder name `<build-config-name>@ip[:port>`, such as `windows-amd64-2008@10.1.5.3`.
### Groups
Instances may be managed in named groups, and commands are broadcast to all instances in the group.
A group is specified either by the -group global flag or through the `GOMOTE_GROUP` environment variable. The -group flag must always specify a valid group, whereas `GOMOTE_GROUP` may contain an invalid group. Instances may be part of more than one group.
Groups may be explicitly managed with the "group" subcommand, but there are several short-cuts that make this unnecessary in most cases:
- The `create` command can create a new group for instances with the `-new-group` flag.
- The `create` command will automatically create the group in `GOMOTE_GROUP` if it does not exist and no other group is explicitly specified.
- The `destroy` command can destroy a group in addition to its instances with the `-destroy-group` flag.
As a result, the easiest way to use groups is to just set the `GOMOTE_GROUP` environment variable:
```
$ export GOMOTE_GROUP=debug
$ gomote create linux-amd64
$ GOROOT=/path/to/goroot gomote create linux-amd64
$ gomote run go/src/make.bash
```
As this example demonstrates, groups are useful even if the group contains only a single instance: it can dramatically shorten most gomote commands.
## Tips and tricks
### General
The `create` command accepts the `-setup` flag which also pushes a `GOROOT` and runs the appropriate equivalent of `make.bash` for the instance.
Example:
```
$ GOROOT=/path/to/my/goroot gomote create -setup linux-amd64
# Creating user-linux-amd64-0...
# Pushing /path/to/my/goroot to user-linux-amd64-0
# Running make.bash on user-linux-amd64-0...
```
The `create` command accepts the `-count` flag for creating several instances at once.
Example:
```
$ gomote create -count=3 linux-amd64
# Creating user-linux-amd64-0...
# Creating user-linux-amd64-1...
# Creating user-linux-amd64-2...
```
The `run` command accepts the `-collect` flag for automatically writing the output from the command to a file in the current working directory, as well as a copy of the full file tree from the instance. This command is useful for capturing the output of long-running commands in a set-and-forget manner.
Example:
```
$ gomote run -collect user-linux-amd64-0 /bin/bash -c 'echo hi'
# Writing output to user-linux-amd64-0.stdout...
$ cat user-linux-amd64-0.stdout
hi
$ ls user-linux-amd64-0.tar.gz
user-linux-amd64-0.tar.gz
```
The `run` command accepts the `-until` flag for continuously executing a command until the output of the command matches some pattern. This is useful for reproducing rare issues, and especially useful when used together with `-collect`.
Example:
```
$ gomote run -collect -until 'FAIL' user-linux-amd64-0 go/bin/go test -run 'TestFlaky' -count=1000 runtime
# Writing output to user-linux-amd64-0.stdout...
$ cat user-linux-amd64-0.stdout
--- FAIL: TestFlaky ---
$ ls user-linux-amd64-0.tar.gz
user-linux-amd64-0.tar.gz
```
Note that the `run` command always streams output to a temporary file regardless of any additional flags to avoid losing output due to terminal scrollback. It always prints the location of the file.
#### Reproducing a rare failure
Putting together some of the tricks above and making use of groups, it's much easier to hammer on some test to try to reproduce a rare failure. For example:
```
$ export GOMOTE_GROUP=debug
$ GOROOT=/path/to/goroot gomote create -setup -count=10 linux-amd64
$ gomote run -until='unexpected return pc' -collect go/bin/go run -run="TestFlaky" -count=100 runtime
```
### Windows
@ -140,12 +201,11 @@ PATH must contain the exec wrapper, go_android_*_exec, built by make.bash.
gomote run -path '$PATH,$WORKDIR/go/bin' $MOTE go/bin/go test math/big
```
## About Buildlets
## About buildlets
https://farmer.golang.org/builders lists information about how each buildlet is deployed and configured.
The information is from golang.org/x/build/dashboard and golang.org/x/build/env.
## Access
**On August 2022, a new infrastructure was deployed which required the removal of all gomote access from previously approved users. Please re-request access if you still require access.**