| 1 |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" |
|---|
| 2 |
"http://www.w3.org/TR/xhtml1/DTD/strict.dtd"> |
|---|
| 3 |
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
|---|
| 4 |
|
|---|
| 5 |
<head> |
|---|
| 6 |
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> |
|---|
| 7 |
<title>Dejavu: Configuring Storage</title> |
|---|
| 8 |
<link href='dejavu.css' rel='stylesheet' type='text/css' /> |
|---|
| 9 |
|
|---|
| 10 |
<style type='text/css'> |
|---|
| 11 |
|
|---|
| 12 |
td.notsup { |
|---|
| 13 |
background-color: #FFCCCC; |
|---|
| 14 |
} |
|---|
| 15 |
|
|---|
| 16 |
td.python { |
|---|
| 17 |
background-color: #FFFFCC; |
|---|
| 18 |
} |
|---|
| 19 |
|
|---|
| 20 |
</style> |
|---|
| 21 |
</head> |
|---|
| 22 |
|
|---|
| 23 |
<body> |
|---|
| 24 |
|
|---|
| 25 |
<h2>Deployers: Configuring Storage</h2> |
|---|
| 26 |
|
|---|
| 27 |
<p>Storage Managers insulate an application developer from the specifics of |
|---|
| 28 |
databases, query languages, and cache mechanisms. As the <i>deployer</i> of |
|---|
| 29 |
a Dejavu application, you get to be in control of these specifics. But |
|---|
| 30 |
don't worry; in the vast majority of cases, you will set up a single |
|---|
| 31 |
database with just two lines in a configuration file. Often, the |
|---|
| 32 |
application developer will have already prepared default config files |
|---|
| 33 |
which you can simply "plug and play". But if you <i>need</i> more control |
|---|
| 34 |
over your data storage, you have it, without becoming a programmer.</p> |
|---|
| 35 |
|
|---|
| 36 |
|
|---|
| 37 |
<a name='configuration'><h3>Configuration Files</h3></a> |
|---|
| 38 |
|
|---|
| 39 |
<p>When you deploy an app built with Dejavu, you must specify Storage |
|---|
| 40 |
Managers to use for persisting application objects. This is usually |
|---|
| 41 |
done through an ini-style configuration file. Here's a short example: |
|---|
| 42 |
<pre>[Junct] |
|---|
| 43 |
Class: access |
|---|
| 44 |
Connect: "PROVIDER=MICROSOFT.JET.OLEDB.4.0;DATA SOURCE=D:\data\junct.mdb;" |
|---|
| 45 |
</pre> |
|---|
| 46 |
The first line of our example ("[Junct]") names the Storage Manager; |
|---|
| 47 |
each [section] in your conf file defines a different SM. You can use whatever |
|---|
| 48 |
name you like here; in this example, we used the name of the application. |
|---|
| 49 |
The second line tells Dejavu the <i>class</i> of SM we'd like to use. |
|---|
| 50 |
For most applications, you'll decide which class to use based on the |
|---|
| 51 |
database you want to use. Our example declares that we want to persist our |
|---|
| 52 |
application data in an "MS Access" (i.e., Jet) database. The third line in |
|---|
| 53 |
our example is a standard ADO Connect string. The MS Access class requires |
|---|
| 54 |
this entry; other SM's may not.</p> |
|---|
| 55 |
|
|---|
| 56 |
<h4>Common Configuration Entries</h4> |
|---|
| 57 |
<p>There are a few configuration entries which (probably) apply to all |
|---|
| 58 |
Storage Managers:</p> |
|---|
| 59 |
|
|---|
| 60 |
<table> |
|---|
| 61 |
<tr><th>Key</th><th>Example Value</th><th>Description</th></tr> |
|---|
| 62 |
<tr> |
|---|
| 63 |
<td>Class</td> |
|---|
| 64 |
<td><tt>cache</tt>, or <tt>dejavu.storage.CachingProxy</tt></td> |
|---|
| 65 |
<td>Which backend to use when instantiating this <tt>StorageManager</tt>. |
|---|
| 66 |
You may supply a known short name or the full dotted-package name. |
|---|
| 67 |
</td> |
|---|
| 68 |
</tr> |
|---|
| 69 |
<tr> |
|---|
| 70 |
<td>Load Order</td> |
|---|
| 71 |
<td><tt>5</tt></td> |
|---|
| 72 |
<td>Optional. The order in which to load this SM. Lower numbers are |
|---|
| 73 |
loaded first. SM's without a Load Order default to 0.</td> |
|---|
| 74 |
</tr> |
|---|
| 75 |
<tr> |
|---|
| 76 |
<td>Shutdown Order</td> |
|---|
| 77 |
<td><tt>10</tt></td> |
|---|
| 78 |
<td>Optional. The order in which to shut down this SM. Lower numbers are |
|---|
| 79 |
shut down first. SM's without a Shutdown Order default to 0.</td> |
|---|
| 80 |
</tr> |
|---|
| 81 |
<tr> |
|---|
| 82 |
<td>Units</td> |
|---|
| 83 |
<td><tt>[UnitCollection, UnitEngine, UnitEngineRule, FieldDashboardSumSet]</tt></td> |
|---|
| 84 |
<td>Optional. Declares which Unit classes to manage with this SM |
|---|
| 85 |
(see below).</td> |
|---|
| 86 |
</tr> |
|---|
| 87 |
</table> |
|---|
| 88 |
|
|---|
| 89 |
<p>The "Units" entry is what you will use to separate application objects |
|---|
| 90 |
into separate stores (if you need to). The objects in an application which |
|---|
| 91 |
need to be stored are called "Units", and each Unit is of a certain Unit |
|---|
| 92 |
class. If you specify a "Units" entry, then only Units of those classes |
|---|
| 93 |
will be managed by that Storage Manager. If you do <i>not</i> specify such |
|---|
| 94 |
an entry, then <b>all</b> Units will be handled by that Storage Manager. |
|---|
| 95 |
This means that only <i>one</i> SM should be missing this entry.</p> |
|---|
| 96 |
|
|---|
| 97 |
|
|---|
| 98 |
<a name='databases'><h3>Database Storage Managers</h3></a> |
|---|
| 99 |
|
|---|
| 100 |
<h4>Microsoft SQL Server / Microsoft Access (Jet)</h4> |
|---|
| 101 |
<p>This module was developed against ADO 2.7 and 2.8, |
|---|
| 102 |
using MSDE, SQL Server 2000, and Access 2000. |
|---|
| 103 |
Configuration entries:</p> |
|---|
| 104 |
<ul> |
|---|
| 105 |
<li><b>Class:</b> "sqlserver" (<tt>dejavu.storage.storeado.StorageManagerADO_SQLServer</tt>) |
|---|
| 106 |
or "access" (<tt>dejavu.storage.storeado.StorageManagerADO_MSAccess</tt>)</li> |
|---|
| 107 |
<li><b>Connect:</b> A valid ADO connect string. There are plenty of |
|---|
| 108 |
online references for how to form these; for example, at |
|---|
| 109 |
<a href='http://support.microsoft.com/?kbid=193332'>Microsoft</a>.</li> |
|---|
| 110 |
</ul> |
|---|
| 111 |
|
|---|
| 112 |
<h4>PostgreSQL (pyPgSQL)</h4> |
|---|
| 113 |
<p>This class was developed against |
|---|
| 114 |
PostgreSQL 8.0.0 rc-1 on Win2k, |
|---|
| 115 |
and also tested on |
|---|
| 116 |
PostgreSQL 7.6.6-6 on Debian "sarge". |
|---|
| 117 |
Configuration entries:</p> |
|---|
| 118 |
<ul> |
|---|
| 119 |
<li><b>Class:</b> "postgres" (<tt>dejavu.storage.storepypgsql.StorageManagerPgSQL</tt>)</li> |
|---|
| 120 |
<li><b>Connect:</b> A connect string of the form "k=v k=v". For example, |
|---|
| 121 |
<tt>"host=localhost dbname=myapp user=postgres password=hilar1ous"</tt>. |
|---|
| 122 |
See the <a href='http://www.postgresql.org/docs/current/static/libpq.html'>libpq</a> |
|---|
| 123 |
docs for complete information.</li> |
|---|
| 124 |
</ul> |
|---|
| 125 |
|
|---|
| 126 |
<h4>PostgreSQL (psycopg2)</h4> |
|---|
| 127 |
<p>This class was developed against |
|---|
| 128 |
PostgreSQL 8.0.0 rc-1 on Win2k, using psycopg2 version '2.0.5.1 (dec dt ext pq3)'. |
|---|
| 129 |
Configuration entries:</p> |
|---|
| 130 |
<ul> |
|---|
| 131 |
<li><b>Class:</b> "psycopg" (<tt>dejavu.storage.storepsycopg.StorageManagerPsycoPg</tt>)</li> |
|---|
| 132 |
<li><b>Connect:</b> A connect string of the form "k=v k=v". For example, |
|---|
| 133 |
<tt>"host=localhost dbname=myapp user=postgres password=hilar1ous"</tt>. |
|---|
| 134 |
See the <a href='http://www.postgresql.org/docs/current/static/libpq.html'>libpq</a> |
|---|
| 135 |
docs for complete information.</li> |
|---|
| 136 |
</ul> |
|---|
| 137 |
|
|---|
| 138 |
<h4>MySQL (MySQLdb)</h4> |
|---|
| 139 |
<p>This class was developed against |
|---|
| 140 |
mysql Ver 14.7 Distrib 4.1.8, for Win95/Win98 (i32), |
|---|
| 141 |
and also tested on |
|---|
| 142 |
mysql Ver 12.22 Distrib 4.0.23, for pc-linux-gnu (i386). |
|---|
| 143 |
Configuration entries:</p> |
|---|
| 144 |
<ul> |
|---|
| 145 |
<li><b>Class:</b> "mysql" (<tt>dejavu.storage.storemysql.StorageManagerMySQL</tt>)</li> |
|---|
| 146 |
<li>Connection arguments: any of "host", "user", "passwd", "db", "port", |
|---|
| 147 |
"unix_socket", "client_flag".<br />See the |
|---|
| 148 |
<a href='http://dev.mysql.com/doc/mysql/en/mysql_real_connect.html'>docs</a> |
|---|
| 149 |
for complete info.</li> |
|---|
| 150 |
</ul> |
|---|
| 151 |
|
|---|
| 152 |
<h4>SQLite (pysqlite/sqlite3)</h4> |
|---|
| 153 |
<p>This class was developed against |
|---|
| 154 |
sqlite 3.0.8 (pysqlite-1.1.6.win32-py2.3), |
|---|
| 155 |
sqlite 3.3.3 (pysqlite-1.1.7.win32-py2.4), |
|---|
| 156 |
sqlite 2.8.15-3 on Debian "sarge", |
|---|
| 157 |
and sqlite 3.3.4 (python 2.5 on win2k). |
|---|
| 158 |
If you have Python 2.5 or later, the builtin _sqlite3 library |
|---|
| 159 |
will be used; otherwise, you need to install pysqlite 1.x. |
|---|
| 160 |
Configuration entries:</p> |
|---|
| 161 |
<ul> |
|---|
| 162 |
<li><b>Class:</b> "sqlite" (<tt>dejavu.storage.storesqlite.StorageManagerSQLite</tt>)</li> |
|---|
| 163 |
<li><b>Database:</b> Filename of the database. May be a relative path. |
|---|
| 164 |
If the DB does not already exist, it will be created.</li> |
|---|
| 165 |
<li><b>Mode:</b> Optional. DB file mode. Defaults to 0755.</li> |
|---|
| 166 |
</ul> |
|---|
| 167 |
|
|---|
| 168 |
|
|---|
| 169 |
<h4>Firebird (kinterbasdb)</h4> |
|---|
| 170 |
<p>This class was developed against |
|---|
| 171 |
KInterbasDB Version: (3, 2, 0, 'alpha', 1) and |
|---|
| 172 |
Server Version: 'WI-V1.5.2.4731 Firebird 1.5' on Win2k. |
|---|
| 173 |
Configuration entries:</p> |
|---|
| 174 |
<ul> |
|---|
| 175 |
<li><b>Class:</b> "firebird" (<tt>dejavu.storage.storefirebird.StorageManagerFirebird</tt>)</li> |
|---|
| 176 |
<li><b>Name:</b> Filename of the database. Must be an absolute path.</li> |
|---|
| 177 |
<li><b>Host:</b> The TCP host name, usually "localhost".</li> |
|---|
| 178 |
<li><b>User:</b> The user name (e.g. "sysdba").</li> |
|---|
| 179 |
<li><b>Password:</b> The password for the given user name.</li> |
|---|
| 180 |
<li><b>Encoding:</b> The charset to be used in each connect() call.</li> |
|---|
| 181 |
</ul> |
|---|
| 182 |
|
|---|
| 183 |
<p><b>The Firebird Storage Manager is new and not yet fully thread-safe. |
|---|
| 184 |
Patches welcome.</b></p> |
|---|
| 185 |
|
|---|
| 186 |
<h4>Common Database Configuration Entries</h4> |
|---|
| 187 |
<p>In addition to the above, Storage Managers for databases (probably) |
|---|
| 188 |
accept these additional options:</p> |
|---|
| 189 |
|
|---|
| 190 |
<table> |
|---|
| 191 |
<tr><th>Key</th><th>Example Value</th><th>Description</th></tr> |
|---|
| 192 |
<tr> |
|---|
| 193 |
<td>poolsize</td> |
|---|
| 194 |
<td><tt>10</tt></td> |
|---|
| 195 |
<td>Optional. Defaults to 10. If nonzero, connections will be pooled |
|---|
| 196 |
(up to a total equal to <i>Pool Size</i>). If zero, no pool |
|---|
| 197 |
will be used; each statement (!) will use a new connection.</td> |
|---|
| 198 |
</tr> |
|---|
| 199 |
<tr> |
|---|
| 200 |
<td>Prefix</td> |
|---|
| 201 |
<td><tt>myapp_</tt></td> |
|---|
| 202 |
<td>Optional. If specified, all tables in the database will have names |
|---|
| 203 |
starting with this prefix. If not provided, it defaults to "" (empty). |
|---|
| 204 |
This helps if you need to mix Dejavu tables with tables from another |
|---|
| 205 |
application. Leave blank if you want no prefix.</td> |
|---|
| 206 |
</tr> |
|---|
| 207 |
<tr> |
|---|
| 208 |
<td>Type Adapter</td> |
|---|
| 209 |
<td><tt>myapp.storage.FieldTypeAdapterForMyDB</tt></td> |
|---|
| 210 |
<td>Optional. The "Type Adapter" is used to map Python types to database |
|---|
| 211 |
column types for use in <tt>CREATE TABLE</tt> statements; for |
|---|
| 212 |
example, the Python <tt>float</tt> type might be mapped to a |
|---|
| 213 |
<tt>REAL</tt> column type. If you don't like the default column |
|---|
| 214 |
types which your Storage Manager provides, you can write your own |
|---|
| 215 |
adapter and declare its use here. The value should be the full |
|---|
| 216 |
dotted package name of the class you wish to use.</td> |
|---|
| 217 |
</tr> |
|---|
| 218 |
<tr> |
|---|
| 219 |
<td>To Adapter</td> |
|---|
| 220 |
<td><tt>myapp.storage.AdapterToMyDBSQL</tt></td> |
|---|
| 221 |
<td>Optional. The "To Adapter" is used to map Python values to database |
|---|
| 222 |
values for use in SQL statements; for example, the Python <tt>str</tt> |
|---|
| 223 |
type usually needs to be wrapped in quote marks. If you don't like |
|---|
| 224 |
the SQL which your Storage Manager generates, you can write your |
|---|
| 225 |
own adapter and declare its use here. The value should be the full |
|---|
| 226 |
dotted package name of the class you wish to use.</td> |
|---|
| 227 |
</tr> |
|---|
| 228 |
<tr> |
|---|
| 229 |
<td>From Adapter</td> |
|---|
| 230 |
<td><tt>myapp.storage.AdapterFromMyDB</tt></td> |
|---|
| 231 |
<td>Optional. The "From Adapter" is used to map incoming database values |
|---|
| 232 |
(i.e., the results of a <tt>SELECT</tt> query) to Python values; for |
|---|
| 233 |
example, your database may return a date value as a string, which |
|---|
| 234 |
must then be converted to the Python <tt>datetime.date</tt> type. |
|---|
| 235 |
If you don't like the default coercions which your Storage Manager |
|---|
| 236 |
provides, you can write your own adapter and declare its use here. |
|---|
| 237 |
The value should be the full dotted package name of the class you |
|---|
| 238 |
wish to use.</td> |
|---|
| 239 |
</tr> |
|---|
| 240 |
<tr> |
|---|
| 241 |
<td>default_isolation</td> |
|---|
| 242 |
<td><tt>"READ COMMITTED"</tt></td> |
|---|
| 243 |
<td>Optional. All database SM's already have a value for this, but you |
|---|
| 244 |
can select another if you wish. This value should be a "native value" |
|---|
| 245 |
for your database's particular transaction mechanisms. For example, |
|---|
| 246 |
PostgreSQL uses ANSI/SQL names like "READ COMMITTED", but Firebird |
|---|
| 247 |
uses library constants like <tt>kinterbasdb.isc_tpb_read_committed</tt>.</td> |
|---|
| 248 |
</tr> |
|---|
| 249 |
</table> |
|---|
| 250 |
|
|---|
| 251 |
|
|---|
| 252 |
<a name='other'><h3>Other Storage Managers</h3></a> |
|---|
| 253 |
|
|---|
| 254 |
<h4>RAM</h4> |
|---|
| 255 |
<p>Persists Units in RAM; all Units are lost when the process exits.</p> |
|---|
| 256 |
|
|---|
| 257 |
<h4>Shelve</h4> |
|---|
| 258 |
<p>Persists Units to shelve-type files. Extremely simple implementation; |
|---|
| 259 |
everything is pickled. Querying will be slow--every Unit is sucked in |
|---|
| 260 |
one-by-one and tested in pure Python using <tt>Expression(unit)</tt>. |
|---|
| 261 |
But for many applications, you don't need heavyweight query tools; |
|---|
| 262 |
for example, an online forum may only need topic content looked up by ID. |
|---|
| 263 |
Or small system tables that only get read at startup might benefit.</p> |
|---|
| 264 |
|
|---|
| 265 |
<p class='warning'><b>Developers note:</b> The shelve implementation in |
|---|
| 266 |
Dejavu does not use "writeback"; that is, changes you make to data are |
|---|
| 267 |
stored only in memory until each shelf has its <tt>close</tt> method |
|---|
| 268 |
called. If <tt>close</tt> is never called, your changes are lost! |
|---|
| 269 |
The easiest way to ensure that your changes are saved is to call |
|---|
| 270 |
store.shutdown() when your app is closing. Since one of the design |
|---|
| 271 |
goals of Dejavu is to allow deployers to choose which backend to use, |
|---|
| 272 |
your applications should <i>always</i> guarantee that store.shutdown() |
|---|
| 273 |
is called on program exit.</p> |
|---|
| 274 |
|
|---|
| 275 |
Configuration entries:</p> |
|---|
| 276 |
<ul> |
|---|
| 277 |
<li><b>Class:</b> "shelve" |
|---|
| 278 |
(<tt>dejavu.storage.storeshelve.StorageManagerShelve</tt>)</li> |
|---|
| 279 |
<li><b>Path:</b> The file path (directory) in which to place db files. |
|---|
| 280 |
Each Unit subclass will get its own file, of the same name as the |
|---|
| 281 |
subclass.</li> |
|---|
| 282 |
</ul> |
|---|
| 283 |
|
|---|
| 284 |
|
|---|
| 285 |
<h4>Folders</h4> |
|---|
| 286 |
<p>Persists Units to a filesystem, one folder per class. Each folder |
|---|
| 287 |
contains subfolders, one per Unit, with the Unit identity as the folder |
|---|
| 288 |
name. Each of those unit folders contains one file for each Unit |
|---|
| 289 |
Property. For example:</p> |
|---|
| 290 |
|
|---|
| 291 |
<pre> |
|---|
| 292 |
root/ |
|---|
| 293 |
Album/ |
|---|
| 294 |
| 78952/ |
|---|
| 295 |
| Name.txt |
|---|
| 296 |
| Artist.txt |
|---|
| 297 |
Song/ |
|---|
| 298 |
1372/ |
|---|
| 299 |
| AlbumID.txt |
|---|
| 300 |
| Data.mp3 |
|---|
| 301 |
88/ |
|---|
| 302 |
AlbumID.txt |
|---|
| 303 |
Data.mp3 |
|---|
| 304 |
</pre> |
|---|
| 305 |
|
|---|
| 306 |
<p>This is an extremely simple implementation; every value that is not |
|---|
| 307 |
of type <tt>str</tt> is pickled. Querying will be slow--every Unit is |
|---|
| 308 |
sucked in one-by-one and tested in pure Python. |
|---|
| 309 |
But for many applications, you don't need heavyweight query tools; |
|---|
| 310 |
for example, an upload site may only need files looked up by ID.</p> |
|---|
| 311 |
|
|---|
| 312 |
Configuration entries:</p> |
|---|
| 313 |
<ul> |
|---|
| 314 |
<li><b>Class:</b> "folders" |
|---|
| 315 |
(<tt>dejavu.storage.storeshelve.StorageManagerShelve</tt>)</li> |
|---|
| 316 |
<li><b>root:</b> Required. The file path (directory) in which to |
|---|
| 317 |
place db files. Each Unit class will get its own subfolder, |
|---|
| 318 |
of the same name as the class.</li> |
|---|
| 319 |
<li><b>mode:</b> Optional. The mode arg to pass to <tt>os.mkdir</tt> |
|---|
| 320 |
when creating folders. Defaults to '0777'.</li> |
|---|
| 321 |
<li><b>idsepchar:</b> Optional. The character to use for separating |
|---|
| 322 |
unit identities which are multivalent. Defaults to '_' (underscore). |
|---|
| 323 |
For example, a Unit with <tt>identifiers = ('Name', 'DOB')</tt> |
|---|
| 324 |
would get a folder name like 'Fred_20040321'.</li> |
|---|
| 325 |
<li><b>extdefault:</b> Optional. The default file extension to use |
|---|
| 326 |
for Unit Property files. Defaults to '.txt'.</li> |
|---|
| 327 |
<li><b><unit>.<propname>:</b> Optional. The value should |
|---|
| 328 |
be the file extension for properties of the given propname |
|---|
| 329 |
for the given unit class. For example, <tt>Song.Data = .mp3</tt> |
|---|
| 330 |
(be sure to include the leading 'dot' if you want one).</li> |
|---|
| 331 |
</ul> |
|---|
| 332 |
|
|---|
| 333 |
|
|---|
| 334 |
<a name='middleware'><h3>Middleware</h3></a> |
|---|
| 335 |
|
|---|
| 336 |
<p>Some Storage Managers act as "middleware", and can be chained together |
|---|
| 337 |
to provide layered functionality. Consider, for example, the |
|---|
| 338 |
<tt>CachingProxy</tt> class; it has another Storage Manager |
|---|
| 339 |
"behind it", which it proxies. It can be used to cache objects between |
|---|
| 340 |
client connections independently from the underlying, database-specific |
|---|
| 341 |
Storage Manager. The beauty of this design is that the decision to |
|---|
| 342 |
use a CachingProxy is completely up to the deployer, <i>not</i> the |
|---|
| 343 |
application developer. The deployer can separate stores, test response |
|---|
| 344 |
times, and address other integration concerns on their own systems.</p> |
|---|
| 345 |
|
|---|
| 346 |
<h4>Caching Proxy</h4> |
|---|
| 347 |
<p>Use this class to persist Units in memory between client connections. |
|---|
| 348 |
It must proxy another Storage Manager. Configuration entries:</p> |
|---|
| 349 |
<ul> |
|---|
| 350 |
<li><b>Class:</b> <tt>dejavu.storage.CachingProxy</tt></li> |
|---|
| 351 |
<li><b>Next Store:</b> Required. The name of the next Storage Manager |
|---|
| 352 |
in the chain.</li> |
|---|
| 353 |
<li><b>Lifetime:</b> Optional. The recurrence string which declares |
|---|
| 354 |
how often to sweep Units out of the in-memory cache. The string you |
|---|
| 355 |
supply should be one of the following types: |
|---|
| 356 |
<ul> |
|---|
| 357 |
<li><b>By units (intervals):</b> "3 hours" will run every 3 |
|---|
| 358 |
hours. "7 days" or "1 week" will run once each week.</li> |
|---|
| 359 |
<li><b>Daily:</b> "14:00 each day" will run at 2:00 P.M. |
|---|
| 360 |
every day.</li> |
|---|
| 361 |
<li><b>Weekly:</b> "Mon", "Monday", or "Mondays" will run once |
|---|
| 362 |
each Monday.</li> |
|---|
| 363 |
<li><b>Monthly:</b> "20 each month" will run on the 20th of |
|---|
| 364 |
each month. "0 every month" will run on the <i>last</i> |
|---|
| 365 |
day of each month.</li> |
|---|
| 366 |
</ul> |
|---|
| 367 |
See the <tt>recur</tt> module for complete options. |
|---|
| 368 |
</li> |
|---|
| 369 |
</ul> |
|---|
| 370 |
|
|---|
| 371 |
|
|---|
| 372 |
<h4>Burned Proxy</h4> |
|---|
| 373 |
<p>Use this class to persist Units in memory between client connections. |
|---|
| 374 |
It needs another Storage Manager to proxy. Unlike the Caching Proxy above, |
|---|
| 375 |
this Storage Manager recalls all Units at once upon the first request, |
|---|
| 376 |
and won't recall them again from storage. They are "burned" into memory |
|---|
| 377 |
for the lifetime of the application. Configuration entries:</p> |
|---|
| 378 |
<ul> |
|---|
| 379 |
<li><b>Class:</b> <tt>dejavu.storage.BurnedProxy</tt></li> |
|---|
| 380 |
<li><b>Next Store:</b> Required. The name of the next Storage Manager |
|---|
| 381 |
in the chain.</li> |
|---|
| 382 |
<li><b>Lifetime:</b> Optional. The recurrence string which declares |
|---|
| 383 |
how often to sweep Units out of the in-memory cache. See the |
|---|
| 384 |
Caching Proxy, above, for recurrence string formats. In general, |
|---|
| 385 |
you should <b>not</b> set this value for BurnedProxy stores.</li> |
|---|
| 386 |
</ul> |
|---|
| 387 |
|
|---|
| 388 |
|
|---|
| 389 |
<a name='comparison'><h3>SM Comparison Chart</h3></a> |
|---|
| 390 |
|
|---|
| 391 |
<p>When selecting a storage implementation, you should be aware of the |
|---|
| 392 |
strengths and limitations of each option. The following chart should help |
|---|
| 393 |
you decide.</p> |
|---|
| 394 |
|
|---|
| 395 |
<p>First, it shows you which stores do and do not support certain |
|---|
| 396 |
optional features of Dejavu. Your application developer should provide you |
|---|
| 397 |
with a list of any features which they <i>require</i>.</p> |
|---|
| 398 |
|
|---|
| 399 |
<p>Second, it shows you which stores have performance or boundary issues |
|---|
| 400 |
and where. When developing applications, you should avoid these issues |
|---|
| 401 |
either by coding alternative solutions, or by recommending to your |
|---|
| 402 |
deployers that they avoid the problematic stores. Note that some |
|---|
| 403 |
limitations are inherent in the storage mechanism itself, while some |
|---|
| 404 |
are limitations of the current Storage Manager for that mechanism.</p> |
|---|
| 405 |
|
|---|
| 406 |
<ul> |
|---|
| 407 |
<li><b>Y</b>: The store supports the feature natively.</li> |
|---|
| 408 |
<li><b>P</b>: The store does not provide the feature natively, but |
|---|
| 409 |
Dejavu provides a fallback in pure Python (which may be slower). |
|---|
| 410 |
Boundaries and limitations are therefore Python limits.</li> |
|---|
| 411 |
<li><b>N</b>: The store does not allow the feature at all.</li> |
|---|
| 412 |
<li><blank>: Unknown/not yet documented.</li> |
|---|
| 413 |
</ul> |
|---|
| 414 |
|
|---|
| 415 |
<table> |
|---|
| 416 |
<tr> |
|---|
| 417 |
<th></th> |
|---|
| 418 |
<th>access</th> |
|---|
| 419 |
<th>firebird</th> |
|---|
| 420 |
<th>mysql</th> |
|---|
| 421 |
<th>postgres</th> |
|---|
| 422 |
<th>ram</th> |
|---|
| 423 |
<th>shelve</th> |
|---|
| 424 |
<th>sqlite</th> |
|---|
| 425 |
<th>sqlserver</th> |
|---|
| 426 |
<th>folders</th> |
|---|
| 427 |
</tr> |
|---|
| 428 |
|
|---|
| 429 |
<tr> |
|---|
| 430 |
<td>Connection Pool <a href='#connpool'>[5]</a></td> |
|---|
| 431 |
<td class='notsup'>N (single only)</td> |
|---|
| 432 |
<td class='python'>P</td> |
|---|
| 433 |
<td class='python'>P</td> |
|---|
| 434 |
<td class='python'>P</td> |
|---|
| 435 |
<td class='notsup'>N</td> |
|---|
| 436 |
<td class='notsup'>N</td> |
|---|
| 437 |
<td class='python'>P</td> |
|---|
| 438 |
<td class='python'>P</td> |
|---|
| 439 |
<td class='notsup'>N</td> |
|---|
| 440 |
</tr> |
|---|
| 441 |
|
|---|
| 442 |
<tr> |
|---|
| 443 |
<td>Transactions</td> |
|---|
| 444 |
<td>Y</td> |
|---|
| 445 |
<td>Y</td> |
|---|
| 446 |
<td>Y</td> |
|---|
| 447 |
<td>Y</td> |
|---|
| 448 |
<td class='notsup'>N</td> |
|---|
| 449 |
<td class='notsup'>N</td> |
|---|
| 450 |
<td>Y</td> |
|---|
| 451 |
<td>Y</td> |
|---|
| 452 |
<td class='notsup'>N</td> |
|---|
| 453 |
</tr> |
|---|
| 454 |
|
|---|
| 455 |
<tr> |
|---|
| 456 |
<td>Indexes</td> |
|---|
| 457 |
<td>Y</td> |
|---|
| 458 |
<td>Y</td> |
|---|
| 459 |
<td>Y</td> |
|---|
| 460 |
<td>Y</td> |
|---|
| 461 |
<td class='notsup'>N</td> |
|---|
| 462 |
<td class='notsup'>N</td> |
|---|
| 463 |
<td>Y</td> |
|---|
| 464 |
<td>Y</td> |
|---|
| 465 |
<td class='notsup'>N</td> |
|---|
| 466 |
</tr> |
|---|
| 467 |
|
|---|
| 468 |
<tr> |
|---|
| 469 |
<td>Max identifier length</td> |
|---|
| 470 |
<td>64</td> |
|---|
| 471 |
<td>31</td> |
|---|
| 472 |
<td>64</td> |
|---|
| 473 |
<td>63</td> |
|---|
| 474 |
<td class='python'>P</td> |
|---|
| 475 |
<td class='python'>P</td> |
|---|
| 476 |
<td>no limit?</td> |
|---|
| 477 |
<td>128</td> |
|---|
| 478 |
<td>OS-dependent</td> |
|---|
| 479 |
</tr> |
|---|
| 480 |
|
|---|
| 481 |
<tr> |
|---|
| 482 |
<td>Case-sensitive identifiers</td> |
|---|
| 483 |
<td>Y</td> |
|---|
| 484 |
<td>Y</td> |
|---|
| 485 |
<td>Unix only</td> |
|---|
| 486 |
<td>Y</td> |
|---|
| 487 |
<td>Y</td> |
|---|
| 488 |
<td>Y</td> |
|---|
| 489 |
<td>Y</td> |
|---|
| 490 |
<td>Y</td> |
|---|
| 491 |
<td>Y <a href='#filenames'>[3]</a></td> |
|---|
| 492 |
</tr> |
|---|
| 493 |
|
|---|
| 494 |
<tr> |
|---|
| 495 |
<td>Case-sensitive LIKE ("a in b")</td> |
|---|
| 496 |
<td class='python'>P</td> |
|---|
| 497 |
<td>Y</td> |
|---|
| 498 |
<td>Y</td> |
|---|
| 499 |
<td>Y</td> |
|---|
| 500 |
<td class='python'>P</td> |
|---|
| 501 |
<td class='python'>P</td> |
|---|
| 502 |
<td>Y</td> |
|---|
| 503 |
<td class='python'>P</td> |
|---|
| 504 |
<td class='python'>P</td> |
|---|
| 505 |
</tr> |
|---|
| 506 |
|
|---|
| 507 |
<tr> |
|---|
| 508 |
<td>Case-sensitive string comparison ("a" > "A")</td> |
|---|
| 509 |
<td><tt><, <=, ==, !=, >, >=</tt></td> |
|---|
| 510 |
<td>Y</td> |
|---|
| 511 |
<td>Y</td> |
|---|
| 512 |
<td>Y</td> |
|---|
| 513 |
<td class='python'>P</td> |
|---|
| 514 |
<td class='python'>P</td> |
|---|
| 515 |
<td>Y</td> |
|---|
| 516 |
<td><tt><, <=, ==, !=, >, >=</tt></td> |
|---|
| 517 |
<td class='python'>P</td> |
|---|
| 518 |
</tr> |
|---|
| 519 |
|
|---|
| 520 |
<tr> |
|---|
| 521 |
<td>Wildcard literals in LIKE ("a in b")</td> |
|---|
| 522 |
<td>Y</td> |
|---|
| 523 |
<td>Y</td> |
|---|
| 524 |
<td>Y</td> |
|---|
| 525 |
<td>Y</td> |
|---|
| 526 |
<td class='python'>P</td> |
|---|
| 527 |
<td class='python'>P</td> |
|---|
| 528 |
<td>3.0.8+</td> |
|---|
| 529 |
<td>Y</td> |
|---|
| 530 |
<td class='python'>P</td> |
|---|
| 531 |
</tr> |
|---|
| 532 |
|
|---|
| 533 |
<tr> |
|---|
| 534 |
<td>Autoincrement</td> |
|---|
| 535 |
<td>Y</td> |
|---|
| 536 |
<td>Y</td> |
|---|
| 537 |
<td>Y</td> |
|---|
| 538 |
<td>Y</td> |
|---|
| 539 |
<td class='python'>P</td> |
|---|
| 540 |
<td class='python'>P</td> |
|---|
| 541 |
<td>3.1.0+</td> |
|---|
| 542 |
<td>Y</td> |
|---|
| 543 |
<td class='python'>P</td> |
|---|
| 544 |
</tr> |
|---|
| 545 |
|
|---|
| 546 |
<tr> |
|---|
| 547 |
<td>add/drop/rename property</td> |
|---|
| 548 |
<td>Y</td> |
|---|
| 549 |
<td>Y</td> |
|---|
| 550 |
<td>Y</td> |
|---|
| 551 |
<td>Y</td> |
|---|
| 552 |
<td>Y</td> |
|---|
| 553 |
<td>Y</td> |
|---|
| 554 |
<td class='python'>P <a href='#sqlite-alter-table'>[2]</a><br />(add: 3.2.0+)</td> |
|---|
| 555 |
<td>Y</td> |
|---|
| 556 |
<td>Y</td> |
|---|
| 557 |
</tr> |
|---|
| 558 |
|
|---|
| 559 |
<tr> |
|---|
| 560 |
<th></th> |
|---|
| 561 |
<th>access</th> |
|---|
| 562 |
<th>firebird</th> |
|---|
| 563 |
<th>mysql</th> |
|---|
| 564 |
<th>postgres</th> |
|---|
| 565 |
<th>ram</th> |
|---|
| 566 |
<th>shelve</th> |
|---|
| 567 |
<th>sqlite</th> |
|---|
| 568 |
<th>sqlserver</th> |
|---|
| 569 |
<th>folders</th> |
|---|
| 570 |
</tr> |
|---|
| 571 |
|
|---|
| 572 |
<tr> |
|---|
| 573 |
<td>fixed point/decimal precision (in decimal digits)</td> |
|---|
| 574 |
<td>12</td> |
|---|
| 575 |
<td>18</td> |
|---|
| 576 |
<td>16</td> |
|---|
| 577 |
<td>1000</td> |
|---|
| 578 |
<td class='python'>P (pickle)</td> |
|---|
| 579 |
<td class='python'>P (pickle)</td> |
|---|
| 580 |
<td>0 (always uses TEXT instead)</td> |
|---|
| 581 |
<td>12</td> |
|---|
| 582 |
<td class='python'>P (pickle)</td> |
|---|
| 583 |
</tr> |
|---|
| 584 |
|
|---|
| 585 |
<tr> |
|---|
| 586 |
<td>Max str/unicode bytes</td> |
|---|
| 587 |
<td>1 GB <a href='#memofields'>[6]</a></td> |
|---|
| 588 |
<td>32765 (255 for an index)</td> |
|---|
| 589 |
<td>8000 (row limit)</td> |
|---|
| 590 |
<td>1 GB?</td> |
|---|
| 591 |
<td class='python'>P (pickle)</td> |
|---|
| 592 |
<td class='python'>P (pickle)</td> |
|---|
| 593 |
<td>1 MB (row limit)</td> |
|---|
| 594 |
<td>8000 <a href='#ntext-bytes'>[4]</a></td> |
|---|
| 595 |
<td class='python'>P (pickle)</td> |
|---|
| 596 |
</tr> |
|---|
| 597 |
|
|---|
| 598 |
<tr> |
|---|
| 599 |
<td>datetime ranges</td> |
|---|
| 600 |
<td>0100-01-01 to 9999-12-31</td> |
|---|
| 601 |
<td>1753-01-01 to 9999-12-31</td> |
|---|
| 602 |
<td>1000-01-01 00:00:00 to 9999-12-31 23:59:59</td> |
|---|
| 603 |
<td>4713 BC to 5874897 AD</td> |
|---|
| 604 |
<td class='python'>P</td> |
|---|
| 605 |
<td class='python'>P</td> |
|---|
| 606 |
<td>4714-11-24 BC to ???</td> |
|---|
| 607 |
<td>1753-01-01 00:00:00.0 to 9999-12-31 23:59:59.997</td> |
|---|
| 608 |
<td class='python'>P</td> |
|---|
| 609 |
</tr> |
|---|
| 610 |
|
|---|
| 611 |
<tr> |
|---|
| 612 |
<td>datetime precision</td> |
|---|
| 613 |
<td>1 second</td> |
|---|
| 614 |
<td>1 second</td> |
|---|
| 615 |
<td>1 second</td> |
|---|
| 616 |
<td>1 microsecond</td> |
|---|
| 617 |
<td class='python'>P</td> |
|---|
| 618 |
<td class='python'>P</td> |
|---|
| 619 |
<td>1 second</td> |
|---|
| 620 |
<td>1 second</td> |
|---|
| 621 |
<td class='python'>P</td> |
|---|
| 622 |
</tr> |
|---|
| 623 |
|
|---|
| 624 |
<tr> |
|---|
| 625 |
<td>dejavu.year, month, day functions</td> |
|---|
| 626 |
<td>Y</td> |
|---|
| 627 |
<td class='python'>P</td> |
|---|
| 628 |
<td>Y</td> |
|---|
| 629 |
<td>Y</td> |
|---|
| 630 |
<td class='python'>P</td> |
|---|
| 631 |
<td class='python'>P</td> |
|---|
| 632 |
<td>3.2.3+ <a href='#perfect-dates'>[1]</a></td> |
|---|
| 633 |
<td>Y</td> |
|---|
| 634 |
<td class='python'>P</td> |
|---|
| 635 |
</tr> |
|---|
| 636 |
|
|---|
| 637 |
<tr> |
|---|
| 638 |
<td>dejavu.now, today functions</td> |
|---|
| 639 |
<td>Y</td> |
|---|
| 640 |
<td>now</td> |
|---|
| 641 |
<td>Y</td> |
|---|
| 642 |
<td>Y</td> |
|---|
| 643 |
<td class='python'>P</td> |
|---|
| 644 |
<td class='python'>P</td> |
|---|
| 645 |
<td>3.2.3+ <a href='#perfect-dates'>[1]</a></td> |
|---|
| 646 |
<td>Y</td> |
|---|
| 647 |
<td class='python'>P</td> |
|---|
| 648 |
</tr> |
|---|
| 649 |
|
|---|
| 650 |
<tr> |
|---|
| 651 |
<td>startswith, endswith, containedby, |
|---|
| 652 |
dejavu.icontainedby, dejavu.icontains, |
|---|
| 653 |
dejavu.istartswith, dejavu.iendswith</td> |
|---|
| 654 |
<td>Y</td> |
|---|
| 655 |
<td>Y</td> |
|---|
| 656 |
<td>Y</td> |
|---|
| 657 |
<td>Y</td> |
|---|
| 658 |
<td class='python'>P</td> |
|---|
| 659 |
<td class='python'>P</td> |
|---|
| 660 |
<td>Y</td> |
|---|
| 661 |
<td>Y</td> |
|---|
| 662 |
<td class='python'>P</td> |
|---|
| 663 |
</tr> |
|---|
| 664 |
|
|---|
| 665 |
<tr> |
|---|
| 666 |
<td>builtin function: len</td> |
|---|
| 667 |
<td>Y</td> |
|---|
| 668 |
<td class='python'>P</td> |
|---|
| 669 |
<td>Y</td> |
|---|
| 670 |
<td>Y</td> |
|---|
| 671 |
<td class='python'>P</td> |
|---|
| 672 |
<td class='python'>P</td> |
|---|
| 673 |
<td>Y</td> |
|---|
| 674 |
<td>Y</td> |
|---|
| 675 |
<td class='python'>P</td> |
|---|
| 676 |
</tr> |
|---|
| 677 |
|
|---|
| 678 |
<tr> |
|---|
| 679 |
<th></th> |
|---|
| 680 |
<th>access</th> |
|---|
| 681 |
<th>firebird</th> |
|---|
| 682 |
<th>mysql</th> |
|---|
| 683 |
<th>postgres</th> |
|---|
| 684 |
<th>ram</th> |
|---|
| 685 |
<th>shelve</th> |
|---|
| 686 |
<th>sqlite</th> |
|---|
| 687 |
<th>sqlserver</th> |
|---|
| 688 |
<th>folders</th> |
|---|
| 689 |
</tr> |
|---|
| 690 |
|
|---|
| 691 |
<tr> |
|---|
| 692 |
<td>READ UNCOMMITTED</td> |
|---|
| 693 |
<td>Y</td> |
|---|
| 694 |
<td class='notsup'>N</td> |
|---|
| 695 |
<td>Y</td> |
|---|
| 696 |
<td class='notsup'>N <a href='#too-isolated'>[7]</a></td> |
|---|
| 697 |
<td></td> |
|---|
| 698 |
<td></td> |
|---|
| 699 |
<td class='notsup'>N</td> |
|---|
| 700 |
<td>Y</td> |
|---|
| 701 |
<td></td> |
|---|
| 702 |
</tr> |
|---|
| 703 |
|
|---|
| 704 |
<tr> |
|---|
| 705 |
<td>READ COMMITTED</td> |
|---|
| 706 |
<td class='notsup'>N</td> |
|---|
| 707 |
<td>Y</td> |
|---|
| 708 |
<td>Y</td> |
|---|
| 709 |
<td>Y</td> |
|---|
| 710 |
<td></td> |
|---|
| 711 |
<td></td> |
|---|
| 712 |
<td class='notsup'>N</td> |
|---|
| 713 |
<td>Y (timeout)</td> |
|---|
| 714 |
<td></td> |
|---|
| 715 |
</tr> |
|---|
| 716 |
<tr> |
|---|
| 717 |
<td>REPEATABLE READ</td> |
|---|
| 718 |
<td class='notsup'>N</td> |
|---|
| 719 |
<td class='notsup'>N <a href='#too-isolated'>[7]</a></td> |
|---|
| 720 |
<td class='notsup'>N <a href='#too-isolated'>[7]</a></td> |
|---|
| 721 |
<td class='notsup'>N <a href='#too-isolated'>[7]</a></td> |
|---|
| 722 |
<td></td> |
|---|
| 723 |
<td></td> |
|---|
| 724 |
<td class='notsup'>N</td> |
|---|
| 725 |
<td>Y (timeout)</td> |
|---|
| 726 |
<td></td> |
|---|
| 727 |
</tr> |
|---|
| 728 |
<tr> |
|---|
| 729 |
<td>SERIALIZABLE</td> |
|---|
| 730 |
<td class='notsup'>N</td> |
|---|
| 731 |
<td>Y</td> |
|---|
| 732 |
<td>Y (timeout)</td> |
|---|
| 733 |
<td>Y</td> |
|---|
| 734 |
<td></td> |
|---|
| 735 |
<td></td> |
|---|
| 736 |
<td>Y <a href='#memory-trans'>[8]</a></td> |
|---|
| 737 |
<td>Y (timeout)</td> |
|---|
| 738 |
<td></td> |
|---|
| 739 |
</tr> |
|---|
| 740 |
<tr> |
|---|
| 741 |
<td>Change isolation inside transaction</td> |
|---|
| 742 |
<td class='notsup'>N</td> |
|---|
| 743 |
<td class='notsup'>N</td> |
|---|
| 744 |
<td>Y</td> |
|---|
| 745 |
<td>Y</td> |
|---|
| 746 |
<td></td> |
|---|
| 747 |
<td></td> |
|---|
| 748 |
<td class='notsup'>N</td> |
|---|
| 749 |
<td>Y</td> |
|---|
| 750 |
<td></td> |
|---|
| 751 |
</tr> |
|---|
| 752 |
</table> |
|---|
| 753 |
|
|---|
| 754 |
<p><a name='perfect-dates'>[1]</a> In order to use native date functions in |
|---|
| 755 |
SQLite, you must be storing your date and time values in one of the |
|---|
| 756 |
acceptable formats. See the |
|---|
| 757 |
<a href='http://www.sqlite.org/cvstrac/wiki?p=DateAndTimeFunctions'>SQLite wiki</a> |
|---|
| 758 |
for more information. Once you have verified that you are using such a format, |
|---|
| 759 |
you must then set AdapterToSQLite.using_perfect_dates to True. This can be |
|---|
| 760 |
done with the configuration entry: <tt>Perfect Dates: True</tt>.</p> |
|---|
| 761 |
|
|---|
| 762 |
<p><a name='sqlite-alter-table'>[2]</a> SQLite must copy the entire table |
|---|
| 763 |
to an intermediate table and then to a new, final table in order to alter |
|---|
| 764 |
tables. Beginning in 3.2.0, adding columns may now be performed natively |
|---|
| 765 |
(but not renaming or dropping them).</p> |
|---|
| 766 |
|
|---|
| 767 |
<p><a name='filenames'>[3]</a> The Folders store keeps identifer values and |
|---|
| 768 |
property names in folder and file names. Not all filesystems support |
|---|
| 769 |
case-sensitive file/folder names.</p> |
|---|
| 770 |
|
|---|
| 771 |
<p><a name='ntext-bytes'>[4]</a> Microsoft SQL Server does not allow |
|---|
| 772 |
comparisons on string fields larger than 8000 characters.</p> |
|---|
| 773 |
|
|---|
| 774 |
<p><a name='connpool'>[5]</a> Dejavu provides connection pool factories |
|---|
| 775 |
in pure Python, and does not yet make any attempt to use native pooling |
|---|
| 776 |
features.</p> |
|---|
| 777 |
|
|---|
| 778 |
<p><a name='memofields'>[6]</a> Microsoft Access "MEMO" fields have a 1 GB |
|---|
| 779 |
limit, but so does the entire database. Memo fields also cannot be used as |
|---|
| 780 |
join keys; set <tt>hints['bytes'] = 255</tt> or less to use VARCHAR instead.</p> |
|---|
| 781 |
|
|---|
| 782 |
<p><a name='memofields'>[7]</a> Some databases over-protect at various |
|---|
| 783 |
isolation levels. For example, "REPEATABLE READ" should prevent fuzzy |
|---|
| 784 |
reads but allow phantoms, but MySQL's and Firebird's REPEATABLE READ |
|---|
| 785 |
prevent both. |
|---|
| 786 |
PostgreSQL only uses two isolation levels internally, so that selecting |
|---|
| 787 |
"READ UNCOMMITTED" behaves like "READ COMMITTED" and "REPEATABLE READ" |
|---|
| 788 |
behaves like "SERIALIZABLE".</p> |
|---|
| 789 |
|
|---|
| 790 |
<p><a name='memory-trans'>[8]</a> SQLite <tt>:memory:</tt> databases |
|---|
| 791 |
cannot use multiple connections, so a single connection is used for |
|---|
| 792 |
all threads. However, this means that transactions are generally not |
|---|
| 793 |
allowed for <tt>:memory:</tt> databases when using multiple threads |
|---|
| 794 |
(because multiple transactions would overlap on the same connection |
|---|
| 795 |
and not be isolated at all!).</p> |
|---|
| 796 |
|
|---|
| 797 |
</body> |
|---|
| 798 |
</html> |
|---|