| e5360fbf3efe85427f7e7f59afe7bbeddb4949ac |
01-Nov-2011 |
Jeff Brown <jeffbrown@google.com> |
Rewrite SQLite database wrappers.
The main theme of this change is encapsulation. This change
preserves all existing functionality but the implementation
is now much cleaner.
Instead of a "database lock", access to the database is treated
as a resource acquisition problem. If a thread's owns a database
connection, then it can access the database; otherwise, it must
acquire a database connection first, and potentially wait for other
threads to give up theirs. The SQLiteConnectionPool encapsulates
the details of how connections are created, configured, acquired,
released and disposed.
One new feature is that SQLiteConnectionPool can make scheduling
decisions about which thread should next acquire a database
connection when there is contention among threads. The factors
considered include wait queue ordering (fairness among peers),
whether the connection is needed for an interactive operation
(unfairness on behalf of the UI), and whether the primary connection
is needed or if any old connection will do. Thus one goal of the
new SQLiteConnectionPool is to improve the utilization of
database connections.
To emulate some quirks of the old "database lock," we introduce
the concept of the primary database connection. The primary
database connection is the one that is typically used to perform
write operations to the database. When a thread holds the primary
database connection, it effectively prevents other threads from
modifying the database (although they can still read). What's
more, those threads will block when they try to acquire the primary
connection, which provides the same kind of mutual exclusion
features that the old "database lock" had. (In truth, we
probably don't need to be requiring use of the primary database
connection in as many places as we do now, but we can seek to refine
that behavior in future patches.)
Another significant change is that native sqlite3_stmt objects
(prepared statements) are fully encapsulated by the SQLiteConnection
object that owns them. This ensures that the connection can
finalize (destroy) all extant statements that belong to a database
connection when the connection is closed. (In the original code,
this was very complicated because the sqlite3_stmt objects were
managed by SQLiteCompiledSql objects which had different lifetime
from the original SQLiteDatabase that created them. Worse, the
SQLiteCompiledSql finalizer method couldn't actually destroy the
sqlite3_stmt objects because it ran on the finalizer thread and
therefore could not guarantee that it could acquire the database
lock in order to do the work. This resulted in some rather
tortured logic involving a list of pending finalizable statements
and a high change of deadlocks or leaks.)
Because sqlite3_stmt objects never escape the confines of the
SQLiteConnection that owns them, we can also greatly simplify
the design of the SQLiteProgram, SQLiteQuery and SQLiteStatement
objects. They no longer have to wrangle a native sqlite3_stmt
object pointer and manage its lifecycle. So now all they do
is hold bind arguments and provide a fancy API.
All of the JNI glue related to managing database connections
and performing transactions is now bound to SQLiteConnection
(rather than being scattered everywhere). This makes sense because
SQLiteConnection owns the native sqlite3 object, so it is the
only class in the system that can interact with the native
SQLite database directly. Encapsulation for the win.
One particularly tricky part of this change is managing the
ownership of SQLiteConnection objects. At any given time,
a SQLiteConnection is either owned by a SQLiteConnectionPool
or by a SQLiteSession. SQLiteConnections should never be leaked,
but we handle that case too (and yell about it with CloseGuard).
A SQLiteSession object is responsible for acquiring and releasing
a SQLiteConnection object on behalf of a single thread as needed.
For example, the session acquires a connection when a transaction
begins and releases it when finished. If the session cannot
acquire a connection immediately, then the requested operation
blocks until a connection becomes available.
SQLiteSessions are thread-local. A SQLiteDatabase assigns a
distinct session to each thread that performs database operations.
This is very very important. First, it prevents two threads
from trying to use the same SQLiteConnection at the same time
(because two threads can't share the same session).
Second, it prevents a single thread from trying to acquire two
SQLiteConnections simultaneously from the same database (because
a single thread can't have two sessions for the same database which,
in addition to being greedy, could result in a deadlock).
There is strict layering between the various database objects,
objects at lower layers are not aware of objects at higher layers.
Moreover, objects at higher layers generally own objects at lower
layers and are responsible for ensuring they are properly disposed
when no longer needed (good for the environment).
API layer: SQLiteDatabase, SQLiteProgram, SQLiteQuery, SQLiteStatement.
Session layer: SQLiteSession.
Connection layer: SQLiteConnectionPool, SQLiteConnection.
Native layer: JNI glue.
By avoiding cyclic dependencies between layers, we make the
architecture much more intelligible, maintainable and robust.
Finally, this change adds a great deal of new debugging information.
It is now possible to view a list of the most recent database
operations including how long they took to run using
"adb shell dumpsys dbinfo". (Because most of the interesting
work happens in SQLiteConnection, it is easy to add debugging
instrumentation to track all database operations in one place.)
Change-Id: Iffb4ce72d8bcf20b4e087d911da6aa84d2f15297
atabaseConnectionPoolTest.java
QLiteCursorTest.java
QLiteDatabaseTest.java
QLiteStatementTest.java
QLiteUnfinalizedExceptionTest.java
|
| cc1eaf6f7b81eaae21c01cbf4cf6f27dc3970774 |
15-Mar-2011 |
Vasu Nori <vnori@google.com> |
bug:4090903 allow bindargs on attach database statement
code incorrectly assumed that certain statements (like ATTACH DATABASE)
will never have bindargs.
Change-Id: I62eddbdf8e15947e8debf5052c7248113a3b9b57
QLiteDatabaseTest.java
|
| 12669b6c8073defcfae369e9f7b6f879b1ac218e |
18-Feb-2011 |
Vasu Nori <vnori@google.com> |
bring back sqlite WAL tests now that the feature is in.
Change-Id: I92103915845b7cf3f41f95e909051b6669c63ec4
QLiteCursorTest.java
QLiteDatabaseTest.java
|
| e7dea4ec2479002f5ee15bc6fc98d8c8acd123a6 |
10-Feb-2011 |
Vasu Nori <vnori@google.com> |
bug:3443154 fix database tests after disabling sqlite wal
sqlite wal is disabled in Change-Id: I283ad26ba7e1793772a372aa8e24df0cb96ce2ef
Change-Id: I2b370c65390a9293e4a82f650c9cee8023b861d7
QLiteCursorTest.java
QLiteDatabaseTest.java
|
| c2ce721fcf684189b7251a5ecbf386426490d68e |
06-Oct-2010 |
Vasu Nori <vnori@google.com> |
fix broken build
Change-Id: Ic00e8b98780d48928e05ee27f855e8211fb1ec47
QLiteCursorTest.java
|
| 790762ca8f4fa458b87f75e7e8cde965c9efe3e5 |
06-Oct-2010 |
Vasu Nori <vnori@google.com> |
annotate some tests large - just in case they take longer. bug:3066136
Change-Id: I89ab0454f592748f971b678a3ae23724eb6fc2e5
QLiteCursorTest.java
QLiteStatementTest.java
|
| ffe06127f6ea4e9ea8e797f8ba0365d1f47fe297 |
27-Sep-2010 |
Vasu Nori <vnori@google.com> |
remove public API setConnectionPoolSize()
because connection pool size is now settable by modifying config.xml
of the device.
Change-Id: I973cc11d7dc10c8d95ab83e2f691068f6019fa33
QLiteDatabaseTest.java
|
| b729dcc8a94bc2c2a1ecda47d791be0d6f1d160a |
14-Sep-2010 |
Vasu Nori <vnori@google.com> |
Revert "caching code retooled to reduce locking + handle SMP"
This reverts commit 992f7d52fad590d90edc166cd74380e96d627605.
Change-Id: Ia5b789d1b2195d6ce43baffe24296c857f9b30f6
QLiteCacheTest.java
QLiteCompiledSqlTest.java
QLiteDatabaseTest.java
QLiteStatementTest.java
|
| 992f7d52fad590d90edc166cd74380e96d627605 |
03-Sep-2010 |
Vasu Nori <vnori@google.com> |
caching code retooled to reduce locking + handle SMP
1. Moved all code related to compiled-sql statement cache to SQLiteCache.java
Removed all caching related code from everywhere else.
2. Moved all code related to compiling a sql statement and caching it to
SQLiteCompiledSql.java. There was some code in SQLiteProgram.java
releated to this. moved it out.
3. Added state to SQLiteCompiledSql. This is to help in debugging.
Change-Id: I63ab0c9c4419e964eb9796d284dd389985763d83
QLiteCacheTest.java
QLiteCompiledSqlTest.java
QLiteDatabaseTest.java
QLiteStatementTest.java
|
| b18f27dbf43ee9028a11cafbca23d3fa318e278b |
13-Aug-2010 |
Vasu Nori <vnori@google.com> |
Return count of rows in a resultset only once (when startPos = 0)
If a query returns 100 rows and say only 10 rows fit in 1MB, then client
receiving the cursor from the ContentProvider needs to paginate.
ContentProvider returns count of total data everytime it returns a page
(= 1MB) of data to the client.
Returning total count causes reading (and skipping unwanted) data
from sqlite.
Instead, it should be sufficient to get total count once
and re-use the count value during the life of the cursor
until a requery is performed on the cursor.
(Count won't change unless data is changed - in which case
the cursor is asked to perform requery anyway. So doing count
once and reusing it should work)
Change-Id: I3520d94524dda07be9bcff56b6fbae5276af1d3b
QLiteCursorTest.java
|
| 422dad0f5069a96c002faf31540bf471a7052585 |
04-Sep-2010 |
Vasu Nori <vnori@google.com> |
android change to handle Change-Id: Idbeed81b5b7349059e467b33a8641abf0b4aaeff
Change-Id: Icf221a8e8d4c281f7719875816835ad7dfe7f3d1
QLiteUnfinalizedExceptionTest.java
|
| cc6f54910d2431e31af176163f61b34d50a33647 |
24-Aug-2010 |
Vasu Nori <vnori@google.com> |
SQLiteOpenHelper should discard closed singleton database objects
bug:2943028
Change-Id: I4b6263cc25413995363158c976d3c723231cc050
QLiteDatabaseTest.java
|
| ce38b98feb1e7c9c1799eb270c40798d833aa9ae |
22-Jul-2010 |
Vasu Nori <vnori@google.com> |
do begin-end transaction before standalone insert/update/delete sql
also fix bug# 2871037
Change-Id: I13325f8eabff4f218d3206905010803b61d8e2cd
QLiteDatabaseTest.java
|
| 65a8883f0e605bb8a73a692987b47ce5da632e72 |
17-Jul-2010 |
Vasu Nori <vnori@google.com> |
don't store mDatabase in SQLiteCursor as it is already in SQLiteQuery
SQLiteCursor has two members: mQuery, mDatabase
but mQuery already has mDatabase.
there is no need for SQLiteCursor.mDatabase.
and everytime SQLiteQuery.mDatabase is to be used, try to use a pooled database
connection handle, if possible.
Change-Id: I42b2376d714a1a4091c843e245a45b882bb7fee6
atabaseConnectionPoolTest.java
QLiteCursorTest.java
QLiteDatabaseTest.java
|
| e25539fdfdf884eee55107efbcc893f44e82e00e |
09-Jul-2010 |
Vasu Nori <vnori@google.com> |
reduce locking when using SQLiteStatement
Do compiling of sql, binding of args and execution of the SQL
statement within one single database lock period.
This reduces the number of times the database lock
needs to be acquired during the course of compilation, binding
and execution of a SQLiteStatement.
Change-Id: I22b090ec9e10fc0aa2532a93bafe610af2546b58
QLiteDatabaseTest.java
QLiteStatementTest.java
|
| 57feb5d9a7fd5329c2ae5daeca0ce8f10a7372a7 |
22-Jun-2010 |
Vasu Nori <vnori@google.com> |
STOPSHIP CL in master - make sqlite WAL default for all apps
for testing the sqlite WAL feature in the next few weeks.
Change-Id: I07234647b5e90d9e3bb0a5310b6cc449d58db428
QLiteDatabaseTest.java
|
| 2827d6d974beabb12344040a002dcb52dd7106b5 |
04-Jul-2010 |
Vasu Nori <vnori@google.com> |
for WAL to work, can't keep prepared SQL stmt_id in SQLiteStatement
Some (including the Contacts app) do the following:
1. Open database
2. As part of database_connection.onCreate(),
Create some SQLiteStatement objects to cache them in the process
3. attach databases
WAL doesn't work with attached databases. so, apps doing the above
should enable WAL only if there are no attached databases.
But we would like to enable WAL automatically for all apps after step #1 above
and disable WAL if the app subsequently does 'attach database' SQL.
this works only if there are no SQLiteStatement objects created in step # 2,
because SQLiteStatements cwmaintain a hard-reference to the database connection
for life and also to the prepared SQL statement id.
It is quite difficult to disable WAL in step # 3
if it is enabled in step # 1
and then a connection pool gets used by step # 2
would make WAL disabling easier if SQLiteStatement refers to prepared SQL
statement id only when it is needed (during binding and execute calls)
and thus NOT tied to a spacific database conenction.
also, from the standpoint of not blocking readers, it helps NOT to have
SQLiteStatement be married to a database connection and prepared SQL statement
id for life.
Change-Id: I464d57042965a28d2bde88e0f44b66ec119b40dc
QLiteDatabaseTest.java
QLiteStatementTest.java
|
| 6ac21d30be06c1c6d5ef04a88cf5171a446633f1 |
07-Jul-2010 |
Vasu Nori <vnori@google.com> |
unittests for DatabaseConnectionPool (and fix bugs)
Change-Id: I6f251b4bdd472bd840ea1be6497660f8c31cd3e3
atabaseConnectionPoolTest.java
|
| 59d60420ba9246eee152852b6a597a0aba7f704d |
04-Jul-2010 |
Vasu Nori <vnori@google.com> |
deprecate method returning prepared sql statement id
this method causes sql statement in a SQLiteProgram object to be never
re-compiled. thats not desirable, is it?
there should be no need for this method.
Change-Id: I207fad6415c1e2ef4097ee65a3ff347b5435b994
QLiteDatabaseTest.java
QLiteStatementTest.java
|
| 2776d57403f08e4ffcc05d476d11d58861823f08 |
04-Jul-2010 |
Vasu Nori <vnori@google.com> |
Merge "fix broken build: renaming a testfile"
|
| 7af57126b557bfe08a21952134a8712f29009aaa |
04-Jul-2010 |
Vasu Nori <vnori@google.com> |
fix broken build: renaming a testfile
Change-Id: I973f604f4f87792b3f8734a6aea4a73d29e0b34b
QLiteStatementTest.java
|
| d960ef8469b385124fe1a83c28c60ddaef5099ec |
04-Jul-2010 |
Vasu Nori <vnori@google.com> |
Merge "change name of a test to be more accurate"
|
| 709493707c22d8794a0f82b07caae96addeff460 |
04-Jul-2010 |
Vasu Nori <vnori@google.com> |
change name of a test to be more accurate
Change-Id: Ie8c7d6ab4b80ae0446f9d67c8a04cef436948692
QLiteGeneralTest.java
QLiteStatementTest.java
|
| 150daa1ae1cf81b676349a7dedef61767c5a68a9 |
02-Jul-2010 |
Vasu Nori <vnori@google.com> |
Merge "remove broken, unused and unwanted tests"
|
| ed57af9e89f1d27ae544ccebb88e2728c2ab379d |
02-Jul-2010 |
Vasu Nori <vnori@google.com> |
remove broken, unused and unwanted tests
Change-Id: I804b777b7c434c8a190d70d6a2556e1f157abe70
bstractJDBCDriverTest.java
QLiteJDBCDriverTest.java
|
| 7501010b71d57264a06f82937f5fb29cb9f4b509 |
02-Jul-2010 |
Vasu Nori <vnori@google.com> |
some refactoring and multi-threading fixes
Change-Id: I7a0497dc2ed7b1e21471d71532558ef243eb9f73
QLiteDatabaseTest.java
|
| e8bf6dbc4025f5e3973f2902af7dfdb501e5d6ff |
18-Jun-2010 |
Vasu Nori <vnori@google.com> |
fix broken test similar to I32c30f7d05e5bec7c6b2776ce9a80c8be61d856f
Change-Id: Idf075ea888dfab35afba0a28a838e3a15b7de545
QLiteDatabaseTest.java
|
| 0bcd289b99786567931b56658e37c8fcb965b513 |
17-Jun-2010 |
Vasu Nori <vnori@google.com> |
move a sqlite test from framework-tests to coretests
Change-Id: Ic8d42a3c477e4ea0ad8eaa4e1869ae221b95ba46
QLiteDatabaseTest.java
|
| e8de28415b4362824a52c180adf10dd882d12eaf |
18-Feb-2010 |
Vasu Nori <vnori@google.com> |
bug fix for 2419869. also included 2 unittests.
bug fix for 2419869 is the following
1. only one object can use the prepared statement object
(SQLiteCompiledSql in SQLIteProgram)
2. if two objects are requesting to use it, then create a new prepared
statement object for exclusive use by the newcomer and let it be
be finalized by the newcomer.
3. add mInUse flag to SQLiteCompiledSql - to be set when SQLiteProgram
requests it and to be released when that SQLiteProgram is done with it
a couple more changes included are
1. unitests to simulate bug # 2419869 (and the fix's repair to it)
2. better logging in SQLiteCloseable when it prints log messages
QLiteGeneralTest.java
|
| 3ef94e25b4c896ecaa85aa2c12b8863ecdf98df0 |
05-Feb-2010 |
Vasu Nori <vnori@google.com> |
use sqlite 3.6.22 to print and profile the sql statetements
instead of rolling our own trace/profile code in java, lets use
sqlite 3.6.22 features. turns out sqlite does a good job of
printing the sql statements - including the ones from the triggers.
QLiteDebugTest.java
|
| 4dd4ab4cc322e82401f380aeb877daa4a20d7069 |
30-Jan-2010 |
Vasu Nori <vnori@google.com> |
add instrumentation to log the sql statement + bindargs + databasename
capture the sql statement along with the bindargs passed in. this will help
one to see the sql statements being executed and hopefully will help
debug incorrect-sql bugs.
QLiteDebugTest.java
|
| 1d3165f10b12165f02b7015ac1a817c5f60e6399 |
12-Jan-2010 |
Neal Nguyen <tommyn@google.com> |
Phase 2 of test file cleanup: relocating test files from FrameworkTest closer to their sources in core.
In addition to the file moves, the package names of the tests have been updated and adjusted to match their new locations.
bstractJDBCDriverTest.java
QLiteJDBCDriverTest.java
|
| 22e31e5b609136d5bf7d77b1dccd6b042b83ebdf |
07-Jan-2010 |
Neal Nguyen <tommyn@google.com> |
Moving framework core tests closer to their source files.
Phase 2 of test case cleanup; distributing CoreTests files closer to their respective sources under frameworks.
bstractJDBCDriverTest.java
QLiteJDBCDriverTest.java
|