When drivers return driver.ErrBadConn, no meaningful
information about what the cause of the problem is
returned. Ideally the driver.ErrBadConn would be
always caught with the retry loop, but this is not
always the case. Drivers today must choose between
returning a useful error and use the rety logic.
This allows supporting both.
Fixes#47142
Change-Id: I454573028f041dfdf874eed6c254fb194ccf6d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/333949
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fix idle connection reuse so that ConnMaxIdleTime clears down excessive
idle connections.
This now ensures that db.freeConn is ordered by returnedAt and that
connections that have been idle for the shortest period are reused
first.
In addition connectionCleanerRunLocked updates the next check deadline
based on idle and maximum life time information so that we avoid waiting
up to double MaxIdleTime to close connections.
Corrected the calling timer of connectionCleaner.
Fixes#39471
Change-Id: I6d26b3542179ef35aa13e5265a89bc0f08ba7fa1
Reviewed-on: https://go-review.googlesource.com/c/go/+/237337
Reviewed-by: Tamás Gulácsi <tgulacsi78@gmail.com>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Trust: Ian Lance Taylor <iant@golang.org>
This follows the spelling choices that the Go project has made for English words.
https://github.com/golang/go/wiki/Spelling
Change-Id: Ie7c586d2cf23020cb492cfff58c0831d2d8d3a78
GitHub-Last-Rev: e16a32cd225a275f73d236bcb33703986d110ded
GitHub-Pull-Request: golang/go#45442
Reviewed-on: https://go-review.googlesource.com/c/go/+/308291
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
The Go standard library retrofitted context support onto existing APIs
using context.Background and later offered variants that directly
supported user-defined context value specification. This commit makes
that behavior clear in documentation and suggests context-aware
alternatives if the user is looking for one.
An example motivation is supporting code for use in systems that expect
APIs to be cancelable for lifecycle correctness or load
shedding/management reasons, as alluded to in
https://blog.golang.org/context-and-structs.
Updates #44143
Change-Id: I2d7f954ddf9b48264d5ebc8d0007058ff9bddf14
Reviewed-on: https://go-review.googlesource.com/c/go/+/296152
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Jean de Klerk <deklerk@google.com>
Trust: Jean de Klerk <deklerk@google.com>
Run-TryBot: Jean de Klerk <deklerk@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
This change allows driver implementations to manage resources in
driver.Connector, e.g. to share the same underlying database handle
between multiple connections. That is, it allows embedded databases
with in-memory backends like SQLite and Genji to safely release the
resources once the sql.DB is closed.
This makes it possible to address oddities with in-memory stores in
SQLite and Genji drivers without introducing too much complexity in
the driver implementations.
See also:
- https://github.com/mattn/go-sqlite3/issues/204
- https://github.com/mattn/go-sqlite3/issues/511
- https://github.com/genjidb/genji/issues/210Fixes#41790
Change-Id: Idbd19763134438ed38288b9d44f16608e4e97fd7
GitHub-Last-Rev: 962c785dfb3bb6ad98b2216bcedd84ba383fe872
GitHub-Pull-Request: golang/go#41710
Reviewed-on: https://go-review.googlesource.com/c/go/+/258360
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Tx acquires tx.closemu W-lock and then acquires stmt.closemu.W-lock
to fully close the transaction and associated prepared statement.
Stmt query and execution run in reverse ways - acquires
stmt.closemu.R-lock and then acquires tx.closemu.R-lock to grab tx
connection, which may cause deadlock.
Prevent the lock is held around tx.closePrepared to ensure no
deadlock happens.
Fixes#40985
Change-Id: If53909822b87bce11861a6e3035ecb9476d2cd17
Reviewed-on: https://go-review.googlesource.com/c/go/+/250178
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Previously, 2 goroutines were created in OpenDB and a comment in the DB.close() field indicated that they were canceled. Later, session Resetter () was removed, but the comment remained the same. This commit just fixes this message
Change-Id: Ie81026f51d7770e9cf8004818154021f626fb2e8
GitHub-Last-Rev: 38b338a0d1cd713d71fa547aa842d395e6d75484
GitHub-Pull-Request: golang/go#42191
Reviewed-on: https://go-review.googlesource.com/c/go/+/264838
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
When zero or less, maxIdleTime and maxLifetime means unlimited.
Helper function shortestIdleTimeLocked must not return the
minimum of the two until both are verified to be greater
then zero.
Fixes#40841
Change-Id: I1130332baf4ad259cd90c10f4221f5def8510655
Reviewed-on: https://go-review.googlesource.com/c/go/+/248817
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
The prior implementation used the format verb %v which unfortunately
improperly wrapped any underlying scanner errors, and we couldn't use
errors.Is nor errors.As. This change fixes that by using the %w verb.
Added a unit to ensure that both error sub string matching works, but
also that errors.Is works as expected.
Fixes#38099
Change-Id: Iea667041dd8081d961246f77f2542330417292dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/248337
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Do not cancel rows during test. Only cancel the Tx.
Correct the referenced issue number on the test.
Fixes#38597
Change-Id: I0e8ba1bf2a8ba638d121c9c6938501fec1d5e961
Reviewed-on: https://go-review.googlesource.com/c/go/+/229478
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Previously the connection pool would only count connections
expired in the background connectionCleaner goroutine towards the
MaxLifetimeClosed stat.
This change increments the stat correctly when checking for
expiry in when acquiring and releasing a connection.
Fixes#38058
Change-Id: Id707ddd40a42a4c38658d5f2931da131647d6c29
GitHub-Last-Rev: 0f205ede439efa355a628b6112450f2ca9ced005
GitHub-Pull-Request: golang/go#38263
Reviewed-on: https://go-review.googlesource.com/c/go/+/227278
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Previously the Tx would drop the connection after rolling back from
a context cancel. Now if the driver can reset the session,
keep the connection.
Change-Id: Ie6a3124275632787629844d91a06bb2e70cc060b
Reviewed-on: https://go-review.googlesource.com/c/go/+/216241
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It was possible for a Tx that was aborted for rollback
asynchronously to execute a query after the rollback had completed
on the database, which often would auto commit the query outside
of the transaction.
By W-locking the tx.closemu prior to issuing the rollback
connection it ensures any Tx query either fails or finishes
on the Tx, and never after the Tx has rolled back.
Fixes#34775Fixes#32942
Change-Id: I017b7932082f2f4ead70bae08b61ed9068ac1d01
Reviewed-on: https://go-review.googlesource.com/c/go/+/216240
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
With the original connection reuse strategy, it was possible that
when a new connection was requested, the pool would wait for an
an existing connection to return for re-use in a full connection
pool, and then it would check if the returned connection was expired.
If the returned connection expired while awaiting re-use, it would
return an error to the location requestiong the new connection.
The existing call sites requesting a new connection was often the last
attempt at returning a connection for a query. This would then
result in a failed query.
This change ensures that we perform the expiry check right
before a connection is inserted back in to the connection pool
for while requesting a new connection. If requesting a new connection
it will no longer fail due to the connection expiring.
Fixes#32530
Change-Id: If16379befe0e14d90160219c0c9396243fe062f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/216197
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This addresses comments made by Russ after
https://golang.org/cl/174122 was merged. It addes a test
for the connection validator and renames the interface to just
"Validator".
Change-Id: Iea53e9b250c9be2e86e9b75906e7353e26437c5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/223963
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
The Row.Err method is intended to assist wrapping sql.DB.
Because sql.Row is a struct with private fields,
a wrapper in an existing code base cannot easily provide users
with a different implementation without large rewrites.
Adding this method allows query level errors to be handled
centrally.
Fixes#35804
Change-Id: I94e6329de89a7ee1284ce9ef76af4363d2d081f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/214317
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Adds a new interface, driver.ConnectionValidator, to allow
drivers to signal they should not be used again,
separatly from the session resetter interface.
This is done now that the session reset is done
after the connection is put into the connection pool.
Previous behavior attempted to run Session Resets
in a background worker. This implementation had two
problems: untested performance gains for additional
complexity, and failures when the pool size
exceeded the connection reset channel buffer size.
Fixes#31480
Change-Id: I7d483b883c24a362c292471e87a88db5b204d1d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/174122
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Allow removing a connection from the connection pool after
it has been idle for a period of time, without regard to the
total lifespan of the connection.
Fixes#25232
Change-Id: Icff157b906769a2d2d45c67525e04a72feb8d832
Reviewed-on: https://go-review.googlesource.com/c/go/+/145758
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The required slice capacity is already known. Thus, preallocate a slice with the correct capacity before appending to it.
Change-Id: I45ac2c5f1701caeb3dda20451d371713ae7e7365
GitHub-Last-Rev: 2bf575be65e9a449322540270988eaf87cec4245
GitHub-Pull-Request: golang/go#34602
Reviewed-on: https://go-review.googlesource.com/c/go/+/197917
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Use the following (suboptimal) script to obtain a list of possible
typos:
#!/usr/bin/env sh
set -x
git ls-files |\
grep -e '\.\(c\|cc\|go\)$' |\
xargs -n 1\
awk\
'/\/\// { gsub(/.*\/\//, ""); print; } /\/\*/, /\*\// { gsub(/.*\/\*/, ""); gsub(/\*\/.*/, ""); }' |\
hunspell -d en_US -l |\
grep '^[[:upper:]]\{0,1\}[[:lower:]]\{1,\}$' |\
grep -v -e '^.\{1,4\}$' -e '^.\{16,\}$' |\
sort -f |\
uniq -c |\
awk '$1 == 1 { print $2; }'
Then, go through the results manually and fix the most obvious typos in
the non-vendored code.
Change-Id: I3cb5830a176850e1a0584b8a40b47bde7b260eae
Reviewed-on: https://go-review.googlesource.com/c/go/+/193848
Reviewed-by: Robert Griesemer <gri@golang.org>
Exposing the underlying driver conn will allow the use of the
standard connection pool while still able to run special function
directly on the driver.
Fixes#29835
Change-Id: Ib6d3b9535e730f008916805ae3bf76e4494c88f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174182
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It is common for database integers to be represented as int32
internally. Although NullInt64 is already defined,
this should remove some type casts and make working with those eaiser.
For #31231
Change-Id: Ia0c37ecef035fee0734c1d1fb6f58aef6905cf5e
Reviewed-on: https://go-review.googlesource.com/c/go/+/174178
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Though there is variation in the spelling of canceled,
cancellation is always spelled with a double l.
Reference: https://www.grammarly.com/blog/canceled-vs-cancelled/
Change-Id: I240f1a297776c8e27e74f3eca566d2bc4c856f2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/170060
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When prepared on a DB, prepared statement code in database/sql handles everything to keep the prepared statement alive as it moves across the connection pool. Understanding this is an important part of using this API correctly, but it was only documented indirectly via `(*Tx) Prepare*`.
Change-Id: Ic8757e0150d59e675d9f0252f6c15aef2cc2e831
GitHub-Last-Rev: 55dba87458542cb631baac80aeea0c3607d8f421
GitHub-Pull-Request: golang/go#29890
Reviewed-on: https://go-review.googlesource.com/c/159077
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
The logic for pulling a database connection from the DB pool should
proceed as follows: attempt to pull either a cached connection or
new connection N times in a loop. If each connection results
in a bad connection, then create a new connection (no cache).
Previously pulling a Conn from the pool, the last step also
looked at the cache, rather then always creating a new connection.
Fixes#29684
Change-Id: I8f436fd9b96eb35502a620ebe8da4ab89fb06a2e
Reviewed-on: https://go-review.googlesource.com/c/157637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL add support for converting a returned cursor (presented
to this package as a driver.Rows) and scanning it into a *Rows.
Fixes#28515
Change-Id: Id8191c568dc135af9e5e8555efcd01987708edcb
Reviewed-on: https://go-review.googlesource.com/c/145738
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Go documentation style for boolean funcs is to say:
// Foo reports whether ...
func Foo() bool
(rather than "returns true if")
This CL also replaces 4 uses of "iff" with the same "reports whether"
wording, which doesn't lose any meaning, and will prevent people from
sending typo fixes when they don't realize it's "if and only if". In
the past I think we've had the typo CLs updated to just say "reports
whether". So do them all at once.
(Inspired by the addition of another "returns true if" in CL 146938
in fd_plan9.go)
Created with:
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor)
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor)
Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f
Reviewed-on: https://go-review.googlesource.com/c/147037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Previously the MaxIdleClosed counter was incremented when added
to the free connection list, rather then when it wasn't added
to the free connection list. Flip this logic to correct.
Fixes#27792
Change-Id: I405302c14fb985369dab48fbe845e5651afc4ccf
Reviewed-on: https://go-review.googlesource.com/c/138578
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It was unclear that users must copy values out of the src value
for value types like []byte.
Fixes#24492
Change-Id: I99ad61e0ad0075b9efc5ee4e0d067f752f91b8fa
Reviewed-on: https://go-review.googlesource.com/108535
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Provide better statistics for the database pool. Add counters
for waiting on the pool and closes. Too much waiting or too many
connection closes could indicate a problem.
Fixes#24683Fixes#22138
Change-Id: I9e1e32a0487edf41c566b8d9c07cb55e04078fec
Reviewed-on: https://go-review.googlesource.com/108536
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The connRequest may return a nil conn value. However in a rare
case that is difficult to test for it was being passed to
DB.putConn without a nil check. This was an error as this
made no sense if the driverConn is nil. This also caused
a panic in putConn.
A test for this would be nice, but didn't find a sane
way to test for this condition.
Fixes#24445
Change-Id: I827316e856788a5a3ced913f129bb5869b7bcf68
Reviewed-on: https://go-review.googlesource.com/102477
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Ensure all error prefixes in the "database/sql" package start with
"sql: ". Do not prefix errors for type conversions because they
are always embedded in another error message with a specific
context.
Fixes#25251
Change-Id: I349d9804f3bfda4eeb755b32b508ec5992c28e07
Reviewed-on: https://go-review.googlesource.com/111637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit includes efficiency improvements in two places in the
database/sql package where an "if err != nil" was redundant and
the error can be returned as-is (most of the code in the standard
library and even in the file I changed does it my suggested way).
Change-Id: Ib9dac69ed01ee846e570a776164cb87c2caee6ca
Reviewed-on: https://go-review.googlesource.com/106555
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The previous implementation would return "sql: Rows are closed" for any
context errors, which can be confusing for context timeouts or
cancelations.
Fixes#24431
Change-Id: I884904ec43204c43f4e94e2335b2802aab77a888
Reviewed-on: https://go-review.googlesource.com/104276
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It wasn't clear for existing docs if DB.Close forcefully closed
connections or waited for them to finish.
Fixes#23753
Change-Id: Id7df31224c93181c8d01bab7b0b23da25b42a288
Reviewed-on: https://go-review.googlesource.com/103397
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Prevent queries from starting a goroutine if the context is
not able to be canceled.
Fixes#23879
Change-Id: I392047bd53d7f796219dd12ee11b07303658fdaf
Reviewed-on: https://go-review.googlesource.com/102478
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Yasuhiro MATSUMOTO <mattn.jp@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
