| 1 |
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" |
|---|
| 2 |
"http://www.w3.org/TR/xhtml1/DTD/strict.dtd"> |
|---|
| 3 |
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
|---|
| 4 |
|
|---|
| 5 |
<head> |
|---|
| 6 |
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> |
|---|
| 7 |
<title>Dejavu: Introduction</title> |
|---|
| 8 |
<link href='dejavu.css' rel='stylesheet' type='text/css' /> |
|---|
| 9 |
</head> |
|---|
| 10 |
|
|---|
| 11 |
<body> |
|---|
| 12 |
|
|---|
| 13 |
<h2>Introduction</h2> |
|---|
| 14 |
|
|---|
| 15 |
<p>Dejavu is a thread-safe Object-Relational Mapper for Python applications. |
|---|
| 16 |
It is designed to provide the "Model" third of an MVC application. When you |
|---|
| 17 |
build an application using Dejavu, you must supply the Controller(s) and |
|---|
| 18 |
View(s) yourself. Dejavu does not provide these, and does its best to not |
|---|
| 19 |
limit your choices regarding them.</p> |
|---|
| 20 |
|
|---|
| 21 |
<p>If you're familiar with Martin Fowler's work <a href='#fowler'>[1]</a>, |
|---|
| 22 |
you can think of Dejavu as providing a Data Source layer, plus the tools |
|---|
| 23 |
to write your own Domain layer. For the Presentation layer, you're on your |
|---|
| 24 |
own. ;) It primarily uses a generic Data Mapper architecture (as opposed |
|---|
| 25 |
to the more tightly-coupled Active Record architecture).</p> |
|---|
| 26 |
|
|---|
| 27 |
<h3>Basic Structure</h3> |
|---|
| 28 |
<p>Developers build their Model by creating classes which subclass |
|---|
| 29 |
<tt>dejavu.Unit</tt>; in RDBMS terminology, |
|---|
| 30 |
each Unit subclass corresponds to a table; instances of the class |
|---|
| 31 |
correspond to the rows. Each subclass possesses a set of attributes known |
|---|
| 32 |
as "properties", which you can think of as columns in your database |
|---|
| 33 |
table. These attributes are generally formed from a <tt>UnitProperty</tt> |
|---|
| 34 |
descriptor. Any Unit data which needs to be persisted ought to be contained |
|---|
| 35 |
in a Unit Property. However, Unit classes can also possess arbitrary |
|---|
| 36 |
methods and attributes which aid their use within the application.</p> |
|---|
| 37 |
|
|---|
| 38 |
<p>Unit classes can be <i>associated</i> to other Unit classes. This means |
|---|
| 39 |
that one of the properties of UnitA maps to one of the properties of UnitB. |
|---|
| 40 |
Related objects may then be looked up more easily.</p> |
|---|
| 41 |
|
|---|
| 42 |
<p>Units are managed in memory by <tt>Sandbox</tt> objects, which function |
|---|
| 43 |
as "Identity Maps" <a href='#fowler'>[1]</a>: in-memory caches of Units |
|---|
| 44 |
which keep commit conflicts to a minimum. Unit objects can be "memorized" |
|---|
| 45 |
and "recalled" from a <tt>Sandbox</tt>, using pure Python lambda expressions |
|---|
| 46 |
<a href='#cpython'>[2]</a> as a query language. The lambda is wrapped |
|---|
| 47 |
in an <tt>Expression</tt> object to make it portable.</p> |
|---|
| 48 |
|
|---|
| 49 |
<p>Sandboxes persist Unit data by <tt>StorageManager</tt> objects. Each |
|---|
| 50 |
persistence mechanism has its own subclass of the <tt>StorageManager</tt> |
|---|
| 51 |
class; for example, persisting Unit data to a Microsoft SQL Server database |
|---|
| 52 |
requires a <tt>StorageManagerADO_SQLServer</tt> object. When recalling data, |
|---|
| 53 |
Storage Managers receive Expression objects; database SM's, for example, |
|---|
| 54 |
will typically examine these Expressions and produce SQL statements from |
|---|
| 55 |
them, which they then use to retrieve data. Storage Managers also handle |
|---|
| 56 |
the creation of new Units, and their destruction.</p> |
|---|
| 57 |
|
|---|
| 58 |
<p>Finally, Dejavu provides a core <tt>Arena</tt> class which you should be |
|---|
| 59 |
able to leverage for any sort of application you are building. The Arena |
|---|
| 60 |
object functions as a top-level "Application" object, collecting the global |
|---|
| 61 |
settings for an application into one place. It doles out Sandboxes, |
|---|
| 62 |
maintains a registry of Units and their associations, and manages startup |
|---|
| 63 |
and shutdown operations.</p> |
|---|
| 64 |
|
|---|
| 65 |
|
|---|
| 66 |
<h3>Simple Example</h3> |
|---|
| 67 |
|
|---|
| 68 |
<p>Since a block of code is often worth a thousand words, here's a minimal |
|---|
| 69 |
example of a Dejavu application:</p> |
|---|
| 70 |
|
|---|
| 71 |
zookeeper.py |
|---|
| 72 |
<pre>import dejavu |
|---|
| 73 |
|
|---|
| 74 |
class Zoo(dejavu.Unit): |
|---|
| 75 |
Name = dejavu.UnitProperty() |
|---|
| 76 |
Size = dejavu.UnitProperty(int) |
|---|
| 77 |
|
|---|
| 78 |
def total_legs(self): |
|---|
| 79 |
return sum([x.Legs for x in self.Animal()]) |
|---|
| 80 |
|
|---|
| 81 |
class Animal(dejavu.Unit): |
|---|
| 82 |
Legs = dejavu.UnitProperty(int, default=4) |
|---|
| 83 |
|
|---|
| 84 |
Animal.set_properties({"Name": unicode, |
|---|
| 85 |
"ZooID": int, |
|---|
| 86 |
}) |
|---|
| 87 |
Animal.many_to_one('ZooID', Zoo, 'ID') |
|---|
| 88 |
|
|---|
| 89 |
# Set up a global Arena object. |
|---|
| 90 |
arena = dejavu.Arena() |
|---|
| 91 |
conf = {u'Connect': r"PROVIDER=MICROSOFT.JET.OLEDB.4.0;DATA SOURCE=C:\zookeeper.mdb;"} |
|---|
| 92 |
arena.add_store("main", "access", conf) |
|---|
| 93 |
arena.register_all(globals())</pre> |
|---|
| 94 |
|
|---|
| 95 |
<p>The above creates the model for the zookeeper application. |
|---|
| 96 |
There are three basic things happening: |
|---|
| 97 |
<ol> |
|---|
| 98 |
<li>The <tt>Zoo</tt> and <tt>Animal</tt> classes, which subclass |
|---|
| 99 |
<tt>dejavu.Unit</tt>. These will correspond to the Zoo and |
|---|
| 100 |
Animal tables within the database. Notice the two different |
|---|
| 101 |
methods of declaring Unit properties. Each class also inherits |
|---|
| 102 |
an 'ID' property (an int) from <tt>dejavu.Unit</tt>.</li> |
|---|
| 103 |
<li>The association between the Animal class and the Zoo class |
|---|
| 104 |
(many-to-one).</li> |
|---|
| 105 |
<li>The setup of a dejavu <tt>Arena</tt> object, including a Storage |
|---|
| 106 |
Manager which uses a Microsoft Access (Jet) database.</li> |
|---|
| 107 |
</ol> |
|---|
| 108 |
</p> |
|---|
| 109 |
|
|---|
| 110 |
<p>Here's a simple interactive session which uses the above (assume that |
|---|
| 111 |
tables have been created and populated elsewhere):</p> |
|---|
| 112 |
|
|---|
| 113 |
<pre>>>> import zookeeper |
|---|
| 114 |
>>> box = zookeeper.arena.new_sandbox() |
|---|
| 115 |
>>> box.recall(zookeeper.Animal) |
|---|
| 116 |
[<zookeeper.Animal object at 0x013281F0>, <zookeeper.Animal object at 0x01328150>, |
|---|
| 117 |
<zookeeper.Animal object at 0x01328130>, <zookeeper.Animal object at 0x01328230>] |
|---|
| 118 |
>>> box.recall(zookeeper.Zoo) |
|---|
| 119 |
[] |
|---|
| 120 |
>>> zoo = zookeeper.Zoo(Name='San Diego Zoo', Size='38') |
|---|
| 121 |
>>> box.memorize(zoo) |
|---|
| 122 |
>>> zoo.ID |
|---|
| 123 |
1 |
|---|
| 124 |
>>> box.unit(zookeeper.Zoo, ID=1) is zoo |
|---|
| 125 |
True |
|---|
| 126 |
>>> for creature in box.recall(zookeeper.Animal): |
|---|
| 127 |
zoo.add(creature) |
|---|
| 128 |
>>> len(zoo.Animal()) |
|---|
| 129 |
4</pre> |
|---|
| 130 |
|
|---|
| 131 |
|
|---|
| 132 |
<h3>Design Goals</h3> |
|---|
| 133 |
|
|---|
| 134 |
<p>Dejavu is designed to function in environments with complex integration |
|---|
| 135 |
needs, and tends to separate concerns as much as possible. In particular, |
|---|
| 136 |
Dejavu tries to avoid making decisions in the framework which are better |
|---|
| 137 |
left to developers. Some of those decisions are: |
|---|
| 138 |
<ul> |
|---|
| 139 |
<li>Uer interface. Dejavu works well in all sorts of applications, |
|---|
| 140 |
whether desktop, thin-client or web.</li> |
|---|
| 141 |
<li>Application package architecture. You can place your application |
|---|
| 142 |
within a single Python module, develop complete packages, |
|---|
| 143 |
or use Dejavu inside a larger framework.</li> |
|---|
| 144 |
<li>Which types to use for <tt>Unit</tt> properties. Builtin types |
|---|
| 145 |
are fully supported out of the box, including datetime and |
|---|
| 146 |
decimal. Tim Peters' excellent <tt>fixedpoint</tt> module |
|---|
| 147 |
is also available. New types are easily added.</li> |
|---|
| 148 |
<li>Which keys to use when associating <tt>Unit</tt> classes.</li> |
|---|
| 149 |
<li>What to name identifiers.</li> |
|---|
| 150 |
</ul> |
|---|
| 151 |
|
|---|
| 152 |
In the same way, Dejavu tries to avoid having developers make decisions |
|---|
| 153 |
which are better left to deployers. Some of those decisions are: |
|---|
| 154 |
<ul> |
|---|
| 155 |
<li>Where (and how) to log error messages.</li> |
|---|
| 156 |
<li>Which storage mechanism (database or ...?) to use. In particular, |
|---|
| 157 |
deployers are allowed to mix and match stores, including how |
|---|
| 158 |
and when to cache objects in memory. Dejavu tries to make |
|---|
| 159 |
it easy for <i>deployers</i> to tune applications to their |
|---|
| 160 |
particular environment.</li> |
|---|
| 161 |
</ul> |
|---|
| 162 |
</p> |
|---|
| 163 |
|
|---|
| 164 |
<p>Unlike most generic storage wrappers, Dejavu does not <i>require</i> you |
|---|
| 165 |
to have complete control of your back end. For example, consider Mission |
|---|
| 166 |
Control, the first application built on Dejavu. Mission Control required |
|---|
| 167 |
an ORM which transparently supported two very different backends. Half of |
|---|
| 168 |
the data was to be stored in an MS Access database, |
|---|
| 169 |
over which the application developers had full control. |
|---|
| 170 |
But half of the data was stored in a third-party application, "The Raiser's |
|---|
| 171 |
Edge" (RE) from Blackbaud. RE provides read-only database access; all writes |
|---|
| 172 |
must go through their object-oriented API. Further, reading via that API was |
|---|
| 173 |
found to be too slow. Therefore, a custom Storage Manager (about 2500 lines |
|---|
| 174 |
of code) was developed, which searches for and loads objects via SQL, but |
|---|
| 175 |
writes Unit data via the REAPI. Dejavu allows the application logic to be |
|---|
| 176 |
completely ignorant of this complex mass of storage details. If Blackbaud |
|---|
| 177 |
closed its doors tomorrow, the solution could be quickly migrated to another |
|---|
| 178 |
data store; business downtime is reduced in the face of inevitable change.<p> |
|---|
| 179 |
|
|---|
| 180 |
<h3>Obtaining and Installing</h3> |
|---|
| 181 |
|
|---|
| 182 |
<p>You can obtain Dejavu from its Subversion repository at |
|---|
| 183 |
<tt>http://projects.amor.org/dejavu/svn/trunk</tt>. Dejavu is designed to be |
|---|
| 184 |
installed in <tt>site-packages/dejavu</tt> or some other root python |
|---|
| 185 |
path.</p> |
|---|
| 186 |
|
|---|
| 187 |
<p>Dejavu was built using Python 2.3.2. You should probably use |
|---|
| 188 |
at least 2.3; Dejavu depends upon the <tt>datetime</tt> module. |
|---|
| 189 |
Although Dejavu <i>supports</i> additional modules like |
|---|
| 190 |
<tt>fixedpoint</tt> and <tt>decimal</tt>, it does not <i>require</i> |
|---|
| 191 |
them.</p> |
|---|
| 192 |
|
|---|
| 193 |
<p>Dejavu uses bytecode hacks, and therefore requires CPython |
|---|
| 194 |
<a href='#cpython'>[2]</a>.</p> |
|---|
| 195 |
|
|---|
| 196 |
<h3>Compared To Other Database Wrappers</h3> |
|---|
| 197 |
<h4>SQLObject</h4> |
|---|
| 198 |
<p>No matter what project I start on, odds are I'll discover that Ian |
|---|
| 199 |
Bicking has already done the same thing, usually better. |
|---|
| 200 |
<br />See http://blog.ianbicking.org/another-less-sleepy-alternative-to-hibernate.html |
|---|
| 201 |
<br />Which was a reply to Ruby's ActiveRecord: |
|---|
| 202 |
http://www.loudthinking.com/arc/000297.html |
|---|
| 203 |
<br />Which was a reply to Java's Hibernate: |
|---|
| 204 |
http://informit.com/guides/content.asp?g=java&seqNum=127&f1=rss</p> |
|---|
| 205 |
|
|---|
| 206 |
<p>Using dejavu, the application developer supplies the following code |
|---|
| 207 |
to define the Units and their relationships:</p> |
|---|
| 208 |
|
|---|
| 209 |
<pre>from dejavu import * |
|---|
| 210 |
import fixedpoint # or decimal, for Python 2.4+ |
|---|
| 211 |
import datetime |
|---|
| 212 |
|
|---|
| 213 |
class Book(Unit): |
|---|
| 214 |
# The ID field is already set to 'int' for all Unit subclasses. |
|---|
| 215 |
title = UnitProperty(str) |
|---|
| 216 |
price = UnitProperty(fixedpoint.Fixedpoint) |
|---|
| 217 |
publishDate = UnitProperty(datetime.datetime) |
|---|
| 218 |
publisher = UnitProperty(int) |
|---|
| 219 |
|
|---|
| 220 |
def addAuthor(self, author): |
|---|
| 221 |
a = Authorship(authorID=author.ID, bookID=self.ID) |
|---|
| 222 |
self.sandbox.memorize(a) |
|---|
| 223 |
|
|---|
| 224 |
def author_names(self): |
|---|
| 225 |
names = [] |
|---|
| 226 |
for authorship in self.Authorship(): |
|---|
| 227 |
author = authorship.Author() |
|---|
| 228 |
if author: |
|---|
| 229 |
names.append(author.name) |
|---|
| 230 |
return u', '.join(names) |
|---|
| 231 |
|
|---|
| 232 |
class Publisher(Unit): |
|---|
| 233 |
name = UnitProperty(str) |
|---|
| 234 |
|
|---|
| 235 |
class Author(Unit): |
|---|
| 236 |
name = UnitProperty(str) |
|---|
| 237 |
|
|---|
| 238 |
class Authorship(Unit): |
|---|
| 239 |
authorID = UnitProperty(int) |
|---|
| 240 |
bookID = UnitProperty(int) |
|---|
| 241 |
|
|---|
| 242 |
Book.many_to_one('publisher', Publisher, 'ID') |
|---|
| 243 |
Authorship.many_to_one('bookID', Book, 'ID') |
|---|
| 244 |
Authorship.many_to_one('authorID', Author, 'ID') |
|---|
| 245 |
|
|---|
| 246 |
arena = Arena() |
|---|
| 247 |
arena.register_all(globals()) |
|---|
| 248 |
</pre> |
|---|
| 249 |
|
|---|
| 250 |
|
|---|
| 251 |
<p>The deployer would write in a .conf file:</p> |
|---|
| 252 |
<pre>[Books] |
|---|
| 253 |
Class: dejavu.storage.storepypgsql.StorageManagerPgSQL |
|---|
| 254 |
Connect: host=localhost dbname=bookstore user=postgres password=****</pre> |
|---|
| 255 |
|
|---|
| 256 |
<p>To create the tables:</p> |
|---|
| 257 |
<pre>for cls in (Author, Publisher, Book): |
|---|
| 258 |
arena.create_storage(cls)</pre> |
|---|
| 259 |
|
|---|
| 260 |
<p>The app developer's runtime code reads as follows:</p> |
|---|
| 261 |
<pre> |
|---|
| 262 |
box = arena.new_sandbox() |
|---|
| 263 |
ppython = Book(title='Programming Python', price=20, |
|---|
| 264 |
publishDate=datetime.datetime(2001, 3, 1)) |
|---|
| 265 |
# This next line is redundant; all properties default to None. |
|---|
| 266 |
# But explicitness is rarely a bad thing. |
|---|
| 267 |
ppython.publisher = None |
|---|
| 268 |
box.memorize(ppython) |
|---|
| 269 |
|
|---|
| 270 |
print ppython.title # output: 'Programming Python' |
|---|
| 271 |
|
|---|
| 272 |
mlutz = Author(name = 'Mark Lutz') |
|---|
| 273 |
box.memorize(mlutz) # give mlutz an ID |
|---|
| 274 |
ppython.addAuthor(mlutz) |
|---|
| 275 |
|
|---|
| 276 |
print len(ppython.Authorship()) # output: 1 |
|---|
| 277 |
print ppython.author_names() # output: 'Mark Lutz' |
|---|
| 278 |
|
|---|
| 279 |
oreilly = Publisher(name="O'Reilly") |
|---|
| 280 |
box.memorize(oreilly) # give oreilly an ID |
|---|
| 281 |
|
|---|
| 282 |
ppython.publisher = oreilly.ID |
|---|
| 283 |
print ppython.Publisher().name # output: "O'Reilly" |
|---|
| 284 |
|
|---|
| 285 |
print len(oreilly.Book()) # output: 1 |
|---|
| 286 |
|
|---|
| 287 |
print 'Hi,', oreilly.Book().author_names() # output: "Hi, Mark Lutz" |
|---|
| 288 |
</pre> |
|---|
| 289 |
</p> |
|---|
| 290 |
|
|---|
| 291 |
<hr /> |
|---|
| 292 |
|
|---|
| 293 |
<p><a name='fowler'>[1]</a> Fowler, |
|---|
| 294 |
<a href='http://www.martinfowler.com/eaaCatalog/identityMap.html'>Patterns |
|---|
| 295 |
of Enterprise Application Architecture</a>.<br /> |
|---|
| 296 |
<a name='cpython'>[2]</a> Dejavu relies upon bytecode hacking to achieve |
|---|
| 297 |
its clean lambda syntax for data queries. Therefore, it is CPython-specific. |
|---|
| 298 |
In addition, the bytecode of Python may change from one version of Python |
|---|
| 299 |
to another; if you find your version of Python does not work with Dejavu's |
|---|
| 300 |
<tt>codewalk</tt> and <tt>logic</tt> modules, please let me know.<br /> |
|---|
| 301 |
</p> |
|---|
| 302 |
|
|---|
| 303 |
</body> |
|---|
| 304 |
</html> |
|---|
