When prepared on a DB, prepared statement code in database/sql handles everything to keep the prepared statement alive as it moves across the connection pool. Understanding this is an important part of using this API correctly, but it was only documented indirectly via `(*Tx) Prepare*`.
Change-Id: Ic8757e0150d59e675d9f0252f6c15aef2cc2e831
GitHub-Last-Rev: 55dba87458542cb631baac80aeea0c3607d8f421
GitHub-Pull-Request: golang/go#29890
Reviewed-on: https://go-review.googlesource.com/c/159077
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
The logic for pulling a database connection from the DB pool should
proceed as follows: attempt to pull either a cached connection or
new connection N times in a loop. If each connection results
in a bad connection, then create a new connection (no cache).
Previously pulling a Conn from the pool, the last step also
looked at the cache, rather then always creating a new connection.
Fixes#29684
Change-Id: I8f436fd9b96eb35502a620ebe8da4ab89fb06a2e
Reviewed-on: https://go-review.googlesource.com/c/157637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL add support for converting a returned cursor (presented
to this package as a driver.Rows) and scanning it into a *Rows.
Fixes#28515
Change-Id: Id8191c568dc135af9e5e8555efcd01987708edcb
Reviewed-on: https://go-review.googlesource.com/c/145738
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Go documentation style for boolean funcs is to say:
// Foo reports whether ...
func Foo() bool
(rather than "returns true if")
This CL also replaces 4 uses of "iff" with the same "reports whether"
wording, which doesn't lose any meaning, and will prevent people from
sending typo fixes when they don't realize it's "if and only if". In
the past I think we've had the typo CLs updated to just say "reports
whether". So do them all at once.
(Inspired by the addition of another "returns true if" in CL 146938
in fd_plan9.go)
Created with:
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor)
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor)
Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f
Reviewed-on: https://go-review.googlesource.com/c/147037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Previously the MaxIdleClosed counter was incremented when added
to the free connection list, rather then when it wasn't added
to the free connection list. Flip this logic to correct.
Fixes#27792
Change-Id: I405302c14fb985369dab48fbe845e5651afc4ccf
Reviewed-on: https://go-review.googlesource.com/c/138578
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It was unclear that users must copy values out of the src value
for value types like []byte.
Fixes#24492
Change-Id: I99ad61e0ad0075b9efc5ee4e0d067f752f91b8fa
Reviewed-on: https://go-review.googlesource.com/108535
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Provide better statistics for the database pool. Add counters
for waiting on the pool and closes. Too much waiting or too many
connection closes could indicate a problem.
Fixes#24683Fixes#22138
Change-Id: I9e1e32a0487edf41c566b8d9c07cb55e04078fec
Reviewed-on: https://go-review.googlesource.com/108536
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The connRequest may return a nil conn value. However in a rare
case that is difficult to test for it was being passed to
DB.putConn without a nil check. This was an error as this
made no sense if the driverConn is nil. This also caused
a panic in putConn.
A test for this would be nice, but didn't find a sane
way to test for this condition.
Fixes#24445
Change-Id: I827316e856788a5a3ced913f129bb5869b7bcf68
Reviewed-on: https://go-review.googlesource.com/102477
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Ensure all error prefixes in the "database/sql" package start with
"sql: ". Do not prefix errors for type conversions because they
are always embedded in another error message with a specific
context.
Fixes#25251
Change-Id: I349d9804f3bfda4eeb755b32b508ec5992c28e07
Reviewed-on: https://go-review.googlesource.com/111637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit includes efficiency improvements in two places in the
database/sql package where an "if err != nil" was redundant and
the error can be returned as-is (most of the code in the standard
library and even in the file I changed does it my suggested way).
Change-Id: Ib9dac69ed01ee846e570a776164cb87c2caee6ca
Reviewed-on: https://go-review.googlesource.com/106555
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The previous implementation would return "sql: Rows are closed" for any
context errors, which can be confusing for context timeouts or
cancelations.
Fixes#24431
Change-Id: I884904ec43204c43f4e94e2335b2802aab77a888
Reviewed-on: https://go-review.googlesource.com/104276
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It wasn't clear for existing docs if DB.Close forcefully closed
connections or waited for them to finish.
Fixes#23753
Change-Id: Id7df31224c93181c8d01bab7b0b23da25b42a288
Reviewed-on: https://go-review.googlesource.com/103397
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Prevent queries from starting a goroutine if the context is
not able to be canceled.
Fixes#23879
Change-Id: I392047bd53d7f796219dd12ee11b07303658fdaf
Reviewed-on: https://go-review.googlesource.com/102478
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Yasuhiro MATSUMOTO <mattn.jp@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This aims to expand the coverage of examples showing how the sql
package works, as well as to address a number of issues I've observed
while explaining how the database package works:
- The best way to issue UPDATE or INSERT queries, that don't need
to scan anything in return. (Previously, we had no examples for any
Execute statement).
- How to use prepared statements and transactions.
- How to aggregate arguments from a Query/QueryContext query into
a slice.
Furthermore just having examples in more places should help, as users
click on e.g. the "Rows" return parameter and are treated with the
lack of any example about how Rows is used.
Switch package examples to use QueryContext/QueryRowContext; I think
it is a good practice to prepare users to issue queries with a timeout
attached, even if they are not using it immediately.
Change-Id: I4e63af91c7e4fff88b25f820906104ecefde4cc3
Reviewed-on: https://go-review.googlesource.com/91015
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Since https://golang.org/cl/38533, this validation is performed in
driverArgs.
Change-Id: I13a3ca46a1aa3197370de1095fb46ab83ea4628c
Reviewed-on: https://go-review.googlesource.com/91115
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When 'convertAssign' gives an error, instead of giving just the index of
the failing column -- which is not always helpful, especially when there
are lots of columns in the query -- utilize 'rs.rowsi.Columns()' to
extract the underlying column name and include that in the error string:
sql: Scan error on column index 0, name "some_column": ...
Fixes#23362
Change-Id: I0fe71ff3c25f4c0dd9fc6aa2c2da2360dd93e3e0
Reviewed-on: https://go-review.googlesource.com/86537
Reviewed-by: Harald Nordgren <haraldnordgren@gmail.com>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
During the refactor in 1126d1483f0397648905fcd4590ae45352cabd69 I
introduced a logical error within one withLock function that used
the result of the call before checking for the error. Change
the order so that the error is checked before the result is used.
None of the other withLock uses have similar issues.
Fixes#23208
Change-Id: I6c5dcf262e36bad4369c850f1e0131066360a82e
Reviewed-on: https://go-review.googlesource.com/85175
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Caleb Spare <cespare@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When the user context which passed in (*DB)BeginTx is canceled or
timeout, the current implementation could cause db transaction leak
in some extreme scenario.
Goroutine 1:
Call (*DB) BeginTx begins a transaction with a userContext.
In (*DB)BeginTx, a new goroutine (*Tx)awaitDone
which monitor context and rollback tx if needed will be created
Goroutine 2(awaitDone):
block on tx.ctx.Done()
Goroutine 1:
Execute some insert or update sqls on the database
Goroutine 1:
Commit the transaction, (*Tx)Commit set
the atomic variable tx.done to 1
Goroutine 3(maybe global timer):
Cancel userContext which be passed in Tx
Goroutine 1:
(*Tx)Commit checks tx.ctx.Done().
Due to the context has been canceled, it will return
context.Canceled or context.DeadlineExceeded error immediately
and abort the real COMMIT operation of transaction
Goroutine 2:
Release with tx.ctx.Done() signal, execute (*Tx)rollback.
However the atomic variable tx.done is 1 currently,
it will return ErrTxDone error immediately and
abort the real ROLLBACK operation of transaction
Fixes#22976
Change-Id: I3bc23adf25db823861d91e33d3cca6189fb1171d
Reviewed-on: https://go-review.googlesource.com/81736
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Check for the expected number of arguments in a SQL statement
after arguments are eliminated in the argument converter.
This situation was already tested for in TestNamedValueChecker.
However the test used Exec which didn't have any check for
NumInput on it at all, thus this issue was never caught.
In addition to moving the NumInput check on the Query
methods after the converter, add the NumInput check
to the Exec methods as well.
Fixes#22630
Change-Id: If45920c6e1cf70dca63822a0cedec2cdc5cc611c
Reviewed-on: https://go-review.googlesource.com/76732
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
While driver.Connector was previously added to allow non-string
connection arguments and access to the context, most users of
the sql package will continue to rely on a string DSN.
Allow drivers to implement a string DSN to Connector interface
that both allows a single parsing of the string DSN and uses
the Connector interface which passes available context to
the driver dialer.
Fixes#22713
Change-Id: Ia0b862262f4c4670effe2538d0d6d43733fea18d
Reviewed-on: https://go-review.googlesource.com/77550
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Originally we tried the strict -er suffix as the rule in this case
but eventually we decided it was too awkward: io.WriteByter
became io.ByteWriter. By analogy, here the interface should be
named SessionResetter instead of the awkward ResetSessioner.
This change should not affect any drivers that have already
implemented the interface, because the method name is not changing.
(This was added during the Go 1.10 cycle and has not been
released yet, so we can change it.)
Change-Id: Ie50e4e090d3811f85965da9da37d966e9f45e79d
Reviewed-on: https://go-review.googlesource.com/78127
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Before terminating the connectionResetter goroutine the connection
pool processes all of the connections on the channel to unlock the
driverConn instances so everthing can shutdown cleanly. However
the channel was never closed so the goroutine hangs on the range.
Close the channel prior to ranging over it. Also prevent additional
connections from being sent to the resetter after the connection
pool has been closed.
Fixes#22699
Change-Id: I440d2b13cbedec2e04621557f5bd0b1526933dd7
Reviewed-on: https://go-review.googlesource.com/77390
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Russ pointed out in a previous CL golang.org/cl/65731 that not only
was the locking incomplete, previous changes did not correctly
lock driver calls in other sections. After inspecting
driverConn, driverStmt, driverResult, Tx, and Rows structs
where driver interfaces are stored, I discovered a few more places
that failed to lock driver calls. The largest of these
was the parameter type converter "driverArgs".
driverArgs was typically called right before another call to the
driver in a locked region, so I made the entire driverArgs expect
a locked driver mutex and combined the region. This should not
be a problem because the connection is pulled out of the connection
pool either way so there shouldn't be contention.
Fixes#21117
Change-Id: I88d46f74dca25fb11a30f0bf8e79785a73133d23
Reviewed-on: https://go-review.googlesource.com/71433
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
A single database connection ususally maps to a single session.
A connection pool is logically also a session pool. Most
sessions have a way to reset the session state which is desirable
to prevent one bad query from poisoning another later query with
temp table name conflicts or other persistent session resources.
It also lets drivers provide users with better error messages from
queryies when the underlying transport or query method fails.
Internally the driver connection should now be marked as bad, but
return the actual connection. When ResetSession is called on the
connection it should return driver.ErrBadConn to remove it from
the connection pool. Previously drivers had to choose between
meaningful error messages or poisoning the connection pool.
Lastly update TestPoolExhaustOnCancel from relying on a
WAIT query fixing a flaky timeout issue exposed by this
change.
Fixes#22049Fixes#20807
Change-Id: I2b5df6d954a38d0ad93bf1922ec16e74c827274c
Reviewed-on: https://go-review.googlesource.com/73033
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts commit 2620ac3aeafe75a62fa81bd5094a8e1e4ef1ca8b.
Reason for revert: broke all the builds.
Change-Id: I26fc09a13f5f80fa708de66c843442ff9d934694
Reviewed-on: https://go-review.googlesource.com/73050
Reviewed-by: Russ Cox <rsc@golang.org>
A single database connection ususally maps to a single session.
A connection pool is logically also a session pool. Most
sessions have a way to reset the session state which is desirable
to prevent one bad query from poisoning another later query with
temp table name conflicts or other persistent session resources.
It also lets drivers provide users with better error messages from
queryies when the underlying transport or query method fails.
Internally the driver connection should now be marked as bad, but
return the actual connection. When ResetSession is called on the
connection it should return driver.ErrBadConn to remove it from
the connection pool. Previously drivers had to choose between
meaningful error messages or poisoning the connection pool.
Lastly update TestPoolExhaustOnCancel from relying on a
WAIT query fixing a flaky timeout issue exposed by this
change.
Fixes#22049Fixes#20807
Change-Id: Idffa1a7ca9ccfe633257c4a3ae299b864f46c5b6
Reviewed-on: https://go-review.googlesource.com/67630
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Drivers shouldn't need to implement both Queryer and QueryerContext,
they should just implement QueryerContext. Same with Execer and
ExecerContext. This CL tests for QueryContext and ExecerContext
first so drivers do not need to implement Queryer and Execer
with an empty definition.
Fixes#21663
Change-Id: Ifbaa71da669f4bc60f8da8c41a04a4afed699a9f
Reviewed-on: https://go-review.googlesource.com/65733
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts commit 897080d5cbb1793f8ad3ef5fb7c6fafba2e97d42.
Reason for revert: Fails to fix all the locking issues.
Updates #21117
Change-Id: I6fc9cb7897244d6e1af78c089a2bf383258ec049
Reviewed-on: https://go-review.googlesource.com/71450
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Database drivers should be called from a single goroutine to ease
driver's design. If a driver chooses to handle context
cancels internally it may do so.
The sql package violated this agreement when calling Next or
NextResultSet. It was possible for a concurrent rollback
triggered from a context cancel to call a Tx.Rollback (which
takes a driver connection lock) while a Rows.Next is in progress
(which does not tack the driver connection lock).
The current internal design of the sql package is each call takes
roughly two locks: a closemu lock which prevents an disposing of
internal resources (assigning nil or removing from lists)
and a driver connection lock that prevents calling driver code from
multiple goroutines.
Fixes#21117
Change-Id: Ie340dc752a503089c27f57ffd43e191534829360
Reviewed-on: https://go-review.googlesource.com/65731
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Replace the work "session" with "connection" in docs. Fix
The ErrConnDone documentation. Clarify what the context is used
for in StmtContext.
Change-Id: I2f07e58d0cd6321b386a73b038cf6070cb8e2572
Reviewed-on: https://go-review.googlesource.com/65732
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The current Open method limits the ability for driver maintainers
to expose options for their drivers by forcing all the configuration
to pass through the DSN in order to create a *DB.
This CL allows driver maintainers to write their own initialization
functions that return a *DB making configuration of the underlying
drivers easier.
Fixes#20268
Change-Id: Ib10b794f36a201bbb92c23999c8351815d38eedb
Reviewed-on: https://go-review.googlesource.com/53430
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ensure a Stmt prepared on a Conn executes on the same driver.Conn.
This also removes another instance of duplicated prepare logic
as a side effect.
Fixes#20647
Change-Id: Ia00a19e4dd15e19e4d754105babdff5dc127728f
Reviewed-on: https://go-review.googlesource.com/45391
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In addition to adding a guard to the Rows close, add a var
in the fakeConn that gets read and written to on each
operation, simulating writing or reading from the server.
TestConcurrency/TxStmt* tests have been commented out
as they now fail after checking for races on the fakeConn.
See issue #20646 for more information.
Fixes#20622
Change-Id: I80b36ea33d776e5b4968be1683ff8c61728ee1ea
Reviewed-on: https://go-review.googlesource.com/45275
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
