Contact: fumanchu@aminus.org

Log in as guest/dejavu to create tickets

I think I've seen this ORM somewhere before...

Changeset 388

Show
Ignore:
Timestamp:
01/10/07 08:42:36
Author:
fumanchu
Message:

Doc updates, mostly.

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved
  • trunk/doc/index.html

    r387 r388  
    6262    <li><a href='modeling.html#schemas'>Managing Schemas</a> 
    6363        <ul> 
    64         <li>Schema Objects: Managing Changes</li> 
    65         <li>The DeployedVersion Unit</li> 
     64        <li>Installation</li> 
     65        <li>Modifying Storage Structures</li> 
     66        <li>Upgrading: Schema Objects</li> 
     67        <li>Versions: The DeployedVersion Unit</li> 
    6668        </ul> 
    6769    </li> 
     
    100102<li><a href='storage.html'>Deployers: Configuring Storage</a> 
    101103    <ul> 
    102     <li>Storage Managers 
     104    <li><a href='storage.html#configuration'>Common Configuration Entries</a></li> 
     105    <li><a href='storage.html#databases'>Database Storage Managers</a> 
    103106        <ul> 
    104         <li>Common Configuration Entries</li> 
    105         <li>Database Storage Managers 
    106             <ul> 
    107             <li>Microsoft SQL Server / Microsoft Access (Jet)</li> 
    108             <li>PostgreSQL</li> 
    109             <li>MySQL</li> 
    110             <li>SQLite</li> 
    111             <li>Shelve</li> 
    112             <li>Common Database Configuration Entries</li> 
    113             </ul> 
    114         </li> 
    115         <li>Other Storage Managers 
    116             <ul> 
    117             <li>RAM</li> 
    118             </ul> 
    119         </li> 
    120         <li>Middleware 
    121             <ul> 
    122             <li>Caching Proxy</li> 
    123             <li>Burned Proxy</li> 
    124             </ul> 
    125         </li> 
     107        <li>ADO: Microsoft SQL Server / Microsoft Access (Jet)</li> 
     108        <li>PostgreSQL (pyPgSQL)</li> 
     109        <li>PostgreSQL (psycopg2)</li> 
     110        <li>MySQL</li> 
     111        <li>SQLite</li> 
     112        <li>Common Database Configuration Entries</li> 
     113        </ul> 
     114    </li> 
     115    <li><a href='storage.html#other'>Other Storage Managers</a> 
     116        <ul> 
     117        <li>RAM</li> 
     118        <li>Shelve</li> 
     119        <li>Folders</li> 
     120        </ul> 
     121    </li> 
     122    <li><a href='storage.html#middleware'>Middleware</a> 
     123        <ul> 
     124        <li>Caching Proxy</li> 
     125        <li>Burned Proxy</li> 
    126126        </ul> 
    127127    </li> 
  • trunk/doc/modeling.html

    r387 r388  
    826826<a name='schemas'><h3>Managing Schemas</h3></a> 
    827827 
    828 <h4>Schema Objects: Managing Changes</h4> 
    829  
    830 <p>The <tt>Schema</tt> class helps you manage changes to your Dejavu model 
    831 throughout its lifetime. For example, let's say that we deploy our 
     828 
     829<h4>Installation</h4> 
     830 
     831<p>Since this procedure typically happens once per deployed application, 
     832Dejavu doesn't try to over-engineer it. But the deployer will still have 
     833to go through an installation step at some point. Dejavu offers minimal 
     834library calls which you can then build installation (and upgrade, and 
     835uninstall) tools on top of.</p> 
     836 
     837<p>For example, a simple install process could look like this:</p> 
     838 
     839<pre> 
     840elif cmd == "install": 
     841    arena.log = getlogger(os.path.join(os.getcwd(), localDir, "install.log") 
     842    arena.logflags = logflags.ERROR + logflags.SQL + logflags.SANDBOX 
     843     
     844    print "Creating databases..." 
     845    for store in arena.stores.itervalues(): 
     846        store.create_database() 
     847     
     848    print "Creating tables..." 
     849    for cls in arena._registered_classes: 
     850        arena.create_storage(cls) 
     851     
     852    print "done" 
     853    sys.exit(0) 
     854</pre> 
     855 
     856<p>In addition to <tt class='def'>create_database()</tt>, all Storage 
     857Managers also have a <tt class='def'>drop_database()</tt> method. 
     858 
     859 
     860<h4>Modifying Storage Structures</h4> 
     861 
     862<p>The <tt>Arena</tt> class has some methods to help you make changes 
     863to keep storage structures in sync with changes to your Unit classes. 
     864For example, let's say that we deploy our 
    832865Archaeology-Biography application at various libraries around the world. 
    833866After a year, one of the developers wishes to implement a new reporting 
     
    836869Biography class isn't very informative. It would be better if we could 
    837870rename that to "ArchaeologistID":</p> 
     871 
     872<pre> 
     873arena.rename_property(Biography, "ArchID", "ArchaeologistID") 
     874</pre> 
     875 
     876<p>Assuming we've already made the change to our model, the above example 
     877renames the property in the persistence layer (the database) using the 
     878<tt class='def'>rename_property(cls, oldname, newname)</tt> method. 
     879Additional <tt>arena</tt> methods:</p> 
     880 
     881<p>Unit classes (tables):</p> 
     882<ul> 
     883<li><tt class='def'>create_storage(cls)</tt></li> 
     884<li><tt class='def'>has_storage(cls)</tt></li> 
     885<li><tt class='def'>drop_storage(cls)</tt></li> 
     886</ul> 
     887 
     888<p>Unit properties (columns):</p> 
     889<ul> 
     890<li><tt class='def'>add_property(cls, name)</tt></li> 
     891<li><tt class='def'>has_property(cls, name)</tt></li> 
     892<li><tt class='def'>drop_property(cls, name)</tt></li> 
     893</ul> 
     894 
     895<p>Unit property (column) indices:</p> 
     896<ul> 
     897<li><tt class='def'>add_index(cls, name)</tt></li> 
     898<li><tt class='def'>has_index(cls, name)</tt></li> 
     899<li><tt class='def'>drop_index(cls, name)</tt></li> 
     900</ul> 
     901 
     902 
     903<h4>Upgrading: Schema Objects</h4> 
     904 
     905<p>The <tt>Schema</tt> class helps you manage changes to your Dejavu model 
     906throughout its lifetime. Taking our <tt>rename_property</tt> example from 
     907above, we can rewrite it in a Schema obejcts like this:</p> 
    838908 
    839909<pre>class ArchBioSchema(dejavu.Schema): 
     
    849919</pre> 
    850920 
    851 <p>Assuming we've already made the change to our model, the above example 
    852 renames the property in the persistence layer (the database). There are 
    853 also <tt class='def'>add_property(cls, name)</tt> and 
    854 <tt class='def'>drop_property(cls, name)</tt> methods. In addition, 
    855 there are <tt class='def'>create_storage(cls)</tt>, 
    856 <tt class='def'>has_storage(cls)</tt>, and 
    857 <tt class='def'>drop_storage(cls)</tt>.</p> 
    858  
    859 <p>The example also declares this change to be "version 2" of our schema. 
     921<p>The example declares this change to be "version 2" of our schema. 
    860922If you examine the base Schema class, you will see that it already has an 
    861923<tt>upgrade_to_0</tt> method. The "zeroth" upgrade makes no schema changes; 
     
    8639251 in the example, just in case I need some setup code in the future ;).</p> 
    864926 
    865 <p>If you call <tt class='def'>schema.upgrade(version)</tt> with a version argument, 
    866 then your deployment will be upgraded to that version. If no argument is 
    867 given, the installation will be upgraded to <tt>schema.latest</tt>. You 
    868 can even skip steps (i.e. remove methods for broken steps) if it comes to 
    869 that.</p> 
    870  
    871 <p>The Schema superclass also has a <tt class='def'>stage</tt> attribute. 
     927<p>If you call <tt class='def'>schema.upgrade(version)</tt> with a 
     928version argument, then your deployment will be upgraded to that version. 
     929If no argument is given, the installation will be upgraded to 
     930<tt>schema.latest</tt>. You can even skip steps (i.e. remove methods 
     931for broken steps) if it comes to that.</p> 
     932 
     933<p>Each Schema also has a <tt class='def'>stage</tt> attribute. 
    872934While an upgrade is in process, this value will be an int, the same number 
    873935as that of the upgrade method. That is, while upgrade_to_2 is running, 
     
    876938 
    877939<p>After you run <tt>upgrade</tt>, you can call the 
    878 <tt class='def'>assert_storage</tt> method to tell Dejavu to create storage 
    879 (tables in your database) for all the Unit classes registered in your arena. 
     940<tt class='def'>assert_storage</tt> method of the Schema object 
     941to tell Dejavu to create storage (tables in your database) 
     942for all the Unit classes registered in your arena. 
    880943If storage already exists for a given class, it is skipped.</p> 
    881944 
     
    889952and deployed schema).</p> 
    890953 
    891 <h4>The DeployedVersion Unit</h4> 
     954 
     955<h4>Versions: The DeployedVersion Unit</h4> 
    892956 
    893957<p>The <tt>Schema</tt> class uses a magic table in the database to keep 
  • trunk/doc/storage.html

    r387 r388  
    1212 
    1313<h2>Deployers: Configuring Storage</h2> 
    14  
    15 <h3>Storage Managers</h3> 
    1614 
    1715<p>Storage Managers insulate an application developer from the specifics of 
     
    2321which you can simply "plug and play". But if you <i>need</i> more control 
    2422over your data storage, you have it, without becoming a programmer.</p> 
     23 
     24 
     25<a name='configuration'><h3>Configuration Files</h3></a> 
    2526 
    2627<p>When you deploy an app built with Dejavu, you must specify Storage 
     
    8384 
    8485 
    85 <h4>Database Storage Managers</h4
    86  
    87 <h5>Microsoft SQL Server / Microsoft Access (Jet)</h5
     86<a name='databases'><h3>Database Storage Managers</h3></a
     87 
     88<h4>Microsoft SQL Server / Microsoft Access (Jet)</h4
    8889<p>This module was developed against ADO 2.7 and 2.8, 
    8990    using MSDE, SQL Server 2000, and Access 2000. 
     
    9798</ul> 
    9899 
    99 <h5>PostgreSQL (pyPgSQL)</h5
     100<h4>PostgreSQL (pyPgSQL)</h4
    100101<p>This class was developed against 
    101102    PostgreSQL 8.0.0 rc-1 on Win2k, 
     
    111112</ul> 
    112113 
    113 <h5>MySQL (MySQLdb)</h5> 
     114<h4>PostgreSQL (psycopg2)</h4> 
     115<p>This class was developed against 
     116    PostgreSQL 8.0.0 rc-1 on Win2k, using psycopg2 version '2.0.5.1 (dec dt ext pq3)'. 
     117    Configuration entries:</p> 
     118<ul> 
     119    <li><b>Class:</b> "psycopg" (<tt>dejavu.storage.storepsycopg.StorageManagerPsycoPg</tt>)</li> 
     120    <li><b>Connect:</b> A connect string of the form "k=v k=v". For example, 
     121        <tt>"host=localhost dbname=myapp user=postgres password=hilar1ous"</tt>. 
     122        See the <a href='http://www.postgresql.org/docs/current/static/libpq.html'>libpq</a> 
     123        docs for complete information.</li> 
     124</ul> 
     125 
     126<h4>MySQL (MySQLdb)</h4> 
    114127<p>This class was developed against 
    115128    mysql  Ver 14.7 Distrib 4.1.8, for Win95/Win98 (i32), 
     
    125138</ul> 
    126139 
    127 <h5>SQLite (pysqlite/sqlite3)</h5
     140<h4>SQLite (pysqlite/sqlite3)</h4
    128141<p>This class was developed against 
    129142    sqlite 3.0.8 (pysqlite-1.1.6.win32-py2.3), 
     
    142155 
    143156 
    144 <h5>Firebird (kinterbasdb)</h5
     157<h4>Firebird (kinterbasdb)</h4
    145158<p>This class was developed against 
    146159    KInterbasDB Version: (3, 2, 0, 'alpha', 1) and  
     
    159172Patches welcome.</b></p> 
    160173 
    161 <h5>Common Database Configuration Entries</h5
     174<h4>Common Database Configuration Entries</h4
    162175<p>In addition to the above, Storage Managers for databases (probably) 
    163176accept these additional options:</p> 
     
    216229 
    217230 
    218 <h4>Other Storage Managers</h4
    219  
    220 <h5>RAM</h5
     231<a name='other'><h3>Other Storage Managers</h3></a
     232 
     233<h4>RAM</h4
    221234<p>Persists Units in RAM; all Units are lost when the process exits.</p> 
    222235 
    223 <h5>Shelve</h5
     236<h4>Shelve</h4
    224237<p>Persists Units to shelve-type files. Extremely simple implementation; 
    225238everything is pickled. Querying will be slow--every Unit is sucked in 
     
    241254Configuration entries:</p> 
    242255<ul> 
    243     <li><b>Class:</b> <tt>dejavu.storage.storeshelve.StorageManagerShelve</tt></li> 
     256    <li><b>Class:</b> "shelve" 
     257        (<tt>dejavu.storage.storeshelve.StorageManagerShelve</tt>)</li> 
    244258    <li><b>Path:</b> The file path (directory) in which to place db files. 
    245259        Each Unit subclass will get its own file, of the same name as the 
     
    248262 
    249263 
    250 <h4>Middleware</h4> 
     264<h4>Folders</h4> 
     265<p>Persists Units to a filesystem, one folder per class. Each folder 
     266contains subfolders, one per Unit, with the Unit identity as the folder 
     267name. Each of those unit folders contains one file for each Unit 
     268Property. For example:</p> 
     269 
     270<pre> 
     271root/ 
     272    Album/ 
     273    |   78952/ 
     274    |       Name.txt 
     275    |       Artist.txt 
     276    Song/ 
     277        1372/ 
     278        |   AlbumID.txt 
     279        |   Data.mp3 
     280        88/ 
     281            AlbumID.txt 
     282            Data.mp3 
     283</pre> 
     284 
     285<p>This is an extremely simple implementation; every value that is not 
     286of type <tt>str</tt> is pickled. Querying will be slow--every Unit is 
     287sucked in one-by-one and tested in pure Python. 
     288But for many applications, you don't need heavyweight query tools; 
     289for example, an upload site may only need files looked up by ID.</p> 
     290 
     291Configuration entries:</p> 
     292<ul> 
     293    <li><b>Class:</b> "folders" 
     294        (<tt>dejavu.storage.storeshelve.StorageManagerShelve</tt>)</li> 
     295    <li><b>root:</b> Required. The file path (directory) in which to 
     296        place db files. Each Unit class will get its own subfolder, 
     297        of the same name as the class.</li> 
     298    <li><b>mode:</b> Optional. The mode arg to pass to <tt>os.mkdir</tt> 
     299        when creating folders. Defaults to '0777'.</li> 
     300    <li><b>idsepchar:</b> Optional. The character to use for separating 
     301        unit identities which are multivalent. Defaults to '_' (underscore). 
     302        For example, a Unit with <tt>identifiers = ('Name', 'DOB')</tt> 
     303        would get a folder name like 'Fred_20040321'.</li> 
     304    <li><b>extdefault:</b> Optional. The default file extension to use 
     305        for Unit Property files. Defaults to '.txt'.</li> 
     306    <li><b>&lt;unit&gt;.&lt;propname&gt;:</b> Optional. The value should 
     307        be the file extension for properties of the given propname 
     308        for the given unit class. For example, <tt>Song.Data = .mp3</tt> 
     309        (be sure to include the leading 'dot' if you want one).</li> 
     310</ul> 
     311 
     312 
     313<a name='middleware'><h3>Middleware</h3></a> 
    251314 
    252315<p>Some Storage Managers act as "middleware", and can be chained together 
     
    260323times, and address other integration concerns on their own systems.</p> 
    261324 
    262 <h5>Caching Proxy</h5
     325<h4>Caching Proxy</h4
    263326<p>Use this class to persist Units in memory between client connections. 
    264327It must proxy another Storage Manager. Configuration entries:</p> 
     
    286349 
    287350 
    288 <h5>Burned Proxy</h5
     351<h4>Burned Proxy</h4
    289352<p>Use this class to persist Units in memory between client connections. 
    290353It needs another Storage Manager to proxy. Unlike the Caching Proxy above, 
  • trunk/storage/__init__.py

    r387 r388  
    8888                                  % self.__class__) 
    8989     
     90    def has_property(self, cls, name): 
     91        raise NotImplementedError("%s has no has_property method." 
     92                                  % self.__class__) 
     93     
    9094    def drop_property(self, cls, name): 
    9195        raise NotImplementedError("%s has no drop_property method." 
     
    9599        raise NotImplementedError("%s has no rename_property method." 
    96100                                  % self.__class__) 
     101     
     102    def add_index(self, cls, name): 
     103        raise NotImplementedError("%s has no add_index method." 
     104                                  % self.__class__) 
     105     
     106    def has_index(self, cls, name): 
     107        raise NotImplementedError("%s has no has_index method." 
     108                                  % self.__class__) 
     109     
     110    def drop_index(self, cls, name): 
     111        raise NotImplementedError("%s has no drop_index method." 
     112                                  % self.__class__) 
     113     
    97114     
    98115    #                            Transactions                             # 
  • trunk/storage/db.py

    r387 r388  
    606606         
    607607        t.rename(oldname, newname) 
     608     
     609    def add_index(self, cls, name): 
     610        i = self.db.make_index(cls.__name__, name) 
     611        self.db[cls.__name__].indices[name] = i 
    608612     
    609613    def has_index(self, cls, name): 
  • trunk/storage/storefs.py

    r387 r388  
    4343        # Character to use for joining multiple identifiers 
    4444        # into a single folder name. 
    45         self.idsepchar = allOptions.get('sepchar', '_') 
     45        self.idsepchar = allOptions.get('idsepchar', '_') 
    4646         
    4747        # Map of file extensions. Keys should be "clsname.propname" 
  • trunk/units.py

    r387 r388  
    259259     
    260260    def assign(self, unit, sequence): 
    261         newvalue= self.initial 
     261        newvalue = self.initial 
    262262        if sequence: 
    263263            m = max(sequence)