Changeset 388
- Timestamp:
- 01/10/07 08:42:36
- Files:
-
- trunk/doc/index.html (modified) (2 diffs)
- trunk/doc/modeling.html (modified) (6 diffs)
- trunk/doc/storage.html (modified) (13 diffs)
- trunk/storage/__init__.py (modified) (2 diffs)
- trunk/storage/db.py (modified) (1 diff)
- trunk/storage/storefs.py (modified) (1 diff)
- trunk/units.py (modified) (1 diff)
Legend:
- Unmodified
- Added
- Removed
- Modified
- Copied
- Moved
trunk/doc/index.html
r387 r388 62 62 <li><a href='modeling.html#schemas'>Managing Schemas</a> 63 63 <ul> 64 <li>Schema Objects: Managing Changes</li> 65 <li>The DeployedVersion Unit</li> 64 <li>Installation</li> 65 <li>Modifying Storage Structures</li> 66 <li>Upgrading: Schema Objects</li> 67 <li>Versions: The DeployedVersion Unit</li> 66 68 </ul> 67 69 </li> … … 100 102 <li><a href='storage.html'>Deployers: Configuring Storage</a> 101 103 <ul> 102 <li>Storage Managers 104 <li><a href='storage.html#configuration'>Common Configuration Entries</a></li> 105 <li><a href='storage.html#databases'>Database Storage Managers</a> 103 106 <ul> 104 <li>Common Configuration Entries</li> 105 <li>Database Storage Managers 106 <ul> 107 <li>Microsoft SQL Server / Microsoft Access (Jet)</li> 108 <li>PostgreSQL</li> 109 <li>MySQL</li> 110 <li>SQLite</li> 111 <li>Shelve</li> 112 <li>Common Database Configuration Entries</li> 113 </ul> 114 </li> 115 <li>Other Storage Managers 116 <ul> 117 <li>RAM</li> 118 </ul> 119 </li> 120 <li>Middleware 121 <ul> 122 <li>Caching Proxy</li> 123 <li>Burned Proxy</li> 124 </ul> 125 </li> 107 <li>ADO: Microsoft SQL Server / Microsoft Access (Jet)</li> 108 <li>PostgreSQL (pyPgSQL)</li> 109 <li>PostgreSQL (psycopg2)</li> 110 <li>MySQL</li> 111 <li>SQLite</li> 112 <li>Common Database Configuration Entries</li> 113 </ul> 114 </li> 115 <li><a href='storage.html#other'>Other Storage Managers</a> 116 <ul> 117 <li>RAM</li> 118 <li>Shelve</li> 119 <li>Folders</li> 120 </ul> 121 </li> 122 <li><a href='storage.html#middleware'>Middleware</a> 123 <ul> 124 <li>Caching Proxy</li> 125 <li>Burned Proxy</li> 126 126 </ul> 127 127 </li> trunk/doc/modeling.html
r387 r388 826 826 <a name='schemas'><h3>Managing Schemas</h3></a> 827 827 828 <h4>Schema Objects: Managing Changes</h4> 829 830 <p>The <tt>Schema</tt> class helps you manage changes to your Dejavu model 831 throughout its lifetime. For example, let's say that we deploy our 828 829 <h4>Installation</h4> 830 831 <p>Since this procedure typically happens once per deployed application, 832 Dejavu doesn't try to over-engineer it. But the deployer will still have 833 to go through an installation step at some point. Dejavu offers minimal 834 library calls which you can then build installation (and upgrade, and 835 uninstall) tools on top of.</p> 836 837 <p>For example, a simple install process could look like this:</p> 838 839 <pre> 840 elif cmd == "install": 841 arena.log = getlogger(os.path.join(os.getcwd(), localDir, "install.log") 842 arena.logflags = logflags.ERROR + logflags.SQL + logflags.SANDBOX 843 844 print "Creating databases..." 845 for store in arena.stores.itervalues(): 846 store.create_database() 847 848 print "Creating tables..." 849 for cls in arena._registered_classes: 850 arena.create_storage(cls) 851 852 print "done" 853 sys.exit(0) 854 </pre> 855 856 <p>In addition to <tt class='def'>create_database()</tt>, all Storage 857 Managers also have a <tt class='def'>drop_database()</tt> method. 858 859 860 <h4>Modifying Storage Structures</h4> 861 862 <p>The <tt>Arena</tt> class has some methods to help you make changes 863 to keep storage structures in sync with changes to your Unit classes. 864 For example, let's say that we deploy our 832 865 Archaeology-Biography application at various libraries around the world. 833 866 After a year, one of the developers wishes to implement a new reporting … … 836 869 Biography class isn't very informative. It would be better if we could 837 870 rename that to "ArchaeologistID":</p> 871 872 <pre> 873 arena.rename_property(Biography, "ArchID", "ArchaeologistID") 874 </pre> 875 876 <p>Assuming we've already made the change to our model, the above example 877 renames the property in the persistence layer (the database) using the 878 <tt class='def'>rename_property(cls, oldname, newname)</tt> method. 879 Additional <tt>arena</tt> methods:</p> 880 881 <p>Unit classes (tables):</p> 882 <ul> 883 <li><tt class='def'>create_storage(cls)</tt></li> 884 <li><tt class='def'>has_storage(cls)</tt></li> 885 <li><tt class='def'>drop_storage(cls)</tt></li> 886 </ul> 887 888 <p>Unit properties (columns):</p> 889 <ul> 890 <li><tt class='def'>add_property(cls, name)</tt></li> 891 <li><tt class='def'>has_property(cls, name)</tt></li> 892 <li><tt class='def'>drop_property(cls, name)</tt></li> 893 </ul> 894 895 <p>Unit property (column) indices:</p> 896 <ul> 897 <li><tt class='def'>add_index(cls, name)</tt></li> 898 <li><tt class='def'>has_index(cls, name)</tt></li> 899 <li><tt class='def'>drop_index(cls, name)</tt></li> 900 </ul> 901 902 903 <h4>Upgrading: Schema Objects</h4> 904 905 <p>The <tt>Schema</tt> class helps you manage changes to your Dejavu model 906 throughout its lifetime. Taking our <tt>rename_property</tt> example from 907 above, we can rewrite it in a Schema obejcts like this:</p> 838 908 839 909 <pre>class ArchBioSchema(dejavu.Schema): … … 849 919 </pre> 850 920 851 <p>Assuming we've already made the change to our model, the above example 852 renames the property in the persistence layer (the database). There are 853 also <tt class='def'>add_property(cls, name)</tt> and 854 <tt class='def'>drop_property(cls, name)</tt> methods. In addition, 855 there are <tt class='def'>create_storage(cls)</tt>, 856 <tt class='def'>has_storage(cls)</tt>, and 857 <tt class='def'>drop_storage(cls)</tt>.</p> 858 859 <p>The example also declares this change to be "version 2" of our schema. 921 <p>The example declares this change to be "version 2" of our schema. 860 922 If you examine the base Schema class, you will see that it already has an 861 923 <tt>upgrade_to_0</tt> method. The "zeroth" upgrade makes no schema changes; … … 863 925 1 in the example, just in case I need some setup code in the future ;).</p> 864 926 865 <p>If you call <tt class='def'>schema.upgrade(version)</tt> with a version argument,866 then your deployment will be upgraded to that version. If no argument is 867 given, the installation will be upgraded to <tt>schema.latest</tt>. You 868 can even skip steps (i.e. remove methods for broken steps) if it comes to 869 that.</p>870 871 <p> The Schema superclassalso has a <tt class='def'>stage</tt> attribute.927 <p>If you call <tt class='def'>schema.upgrade(version)</tt> with a 928 version argument, then your deployment will be upgraded to that version. 929 If no argument is given, the installation will be upgraded to 930 <tt>schema.latest</tt>. You can even skip steps (i.e. remove methods 931 for broken steps) if it comes to that.</p> 932 933 <p>Each Schema also has a <tt class='def'>stage</tt> attribute. 872 934 While an upgrade is in process, this value will be an int, the same number 873 935 as that of the upgrade method. That is, while upgrade_to_2 is running, … … 876 938 877 939 <p>After you run <tt>upgrade</tt>, you can call the 878 <tt class='def'>assert_storage</tt> method to tell Dejavu to create storage 879 (tables in your database) for all the Unit classes registered in your arena. 940 <tt class='def'>assert_storage</tt> method of the Schema object 941 to tell Dejavu to create storage (tables in your database) 942 for all the Unit classes registered in your arena. 880 943 If storage already exists for a given class, it is skipped.</p> 881 944 … … 889 952 and deployed schema).</p> 890 953 891 <h4>The DeployedVersion Unit</h4> 954 955 <h4>Versions: The DeployedVersion Unit</h4> 892 956 893 957 <p>The <tt>Schema</tt> class uses a magic table in the database to keep trunk/doc/storage.html
r387 r388 12 12 13 13 <h2>Deployers: Configuring Storage</h2> 14 15 <h3>Storage Managers</h3>16 14 17 15 <p>Storage Managers insulate an application developer from the specifics of … … 23 21 which you can simply "plug and play". But if you <i>need</i> more control 24 22 over your data storage, you have it, without becoming a programmer.</p> 23 24 25 <a name='configuration'><h3>Configuration Files</h3></a> 25 26 26 27 <p>When you deploy an app built with Dejavu, you must specify Storage … … 83 84 84 85 85 < h4>Database Storage Managers</h4>86 87 <h 5>Microsoft SQL Server / Microsoft Access (Jet)</h5>86 <a name='databases'><h3>Database Storage Managers</h3></a> 87 88 <h4>Microsoft SQL Server / Microsoft Access (Jet)</h4> 88 89 <p>This module was developed against ADO 2.7 and 2.8, 89 90 using MSDE, SQL Server 2000, and Access 2000. … … 97 98 </ul> 98 99 99 <h 5>PostgreSQL (pyPgSQL)</h5>100 <h4>PostgreSQL (pyPgSQL)</h4> 100 101 <p>This class was developed against 101 102 PostgreSQL 8.0.0 rc-1 on Win2k, … … 111 112 </ul> 112 113 113 <h5>MySQL (MySQLdb)</h5> 114 <h4>PostgreSQL (psycopg2)</h4> 115 <p>This class was developed against 116 PostgreSQL 8.0.0 rc-1 on Win2k, using psycopg2 version '2.0.5.1 (dec dt ext pq3)'. 117 Configuration entries:</p> 118 <ul> 119 <li><b>Class:</b> "psycopg" (<tt>dejavu.storage.storepsycopg.StorageManagerPsycoPg</tt>)</li> 120 <li><b>Connect:</b> A connect string of the form "k=v k=v". For example, 121 <tt>"host=localhost dbname=myapp user=postgres password=hilar1ous"</tt>. 122 See the <a href='http://www.postgresql.org/docs/current/static/libpq.html'>libpq</a> 123 docs for complete information.</li> 124 </ul> 125 126 <h4>MySQL (MySQLdb)</h4> 114 127 <p>This class was developed against 115 128 mysql Ver 14.7 Distrib 4.1.8, for Win95/Win98 (i32), … … 125 138 </ul> 126 139 127 <h 5>SQLite (pysqlite/sqlite3)</h5>140 <h4>SQLite (pysqlite/sqlite3)</h4> 128 141 <p>This class was developed against 129 142 sqlite 3.0.8 (pysqlite-1.1.6.win32-py2.3), … … 142 155 143 156 144 <h 5>Firebird (kinterbasdb)</h5>157 <h4>Firebird (kinterbasdb)</h4> 145 158 <p>This class was developed against 146 159 KInterbasDB Version: (3, 2, 0, 'alpha', 1) and … … 159 172 Patches welcome.</b></p> 160 173 161 <h 5>Common Database Configuration Entries</h5>174 <h4>Common Database Configuration Entries</h4> 162 175 <p>In addition to the above, Storage Managers for databases (probably) 163 176 accept these additional options:</p> … … 216 229 217 230 218 < h4>Other Storage Managers</h4>219 220 <h 5>RAM</h5>231 <a name='other'><h3>Other Storage Managers</h3></a> 232 233 <h4>RAM</h4> 221 234 <p>Persists Units in RAM; all Units are lost when the process exits.</p> 222 235 223 <h 5>Shelve</h5>236 <h4>Shelve</h4> 224 237 <p>Persists Units to shelve-type files. Extremely simple implementation; 225 238 everything is pickled. Querying will be slow--every Unit is sucked in … … 241 254 Configuration entries:</p> 242 255 <ul> 243 <li><b>Class:</b> <tt>dejavu.storage.storeshelve.StorageManagerShelve</tt></li> 256 <li><b>Class:</b> "shelve" 257 (<tt>dejavu.storage.storeshelve.StorageManagerShelve</tt>)</li> 244 258 <li><b>Path:</b> The file path (directory) in which to place db files. 245 259 Each Unit subclass will get its own file, of the same name as the … … 248 262 249 263 250 <h4>Middleware</h4> 264 <h4>Folders</h4> 265 <p>Persists Units to a filesystem, one folder per class. Each folder 266 contains subfolders, one per Unit, with the Unit identity as the folder 267 name. Each of those unit folders contains one file for each Unit 268 Property. For example:</p> 269 270 <pre> 271 root/ 272 Album/ 273 | 78952/ 274 | Name.txt 275 | Artist.txt 276 Song/ 277 1372/ 278 | AlbumID.txt 279 | Data.mp3 280 88/ 281 AlbumID.txt 282 Data.mp3 283 </pre> 284 285 <p>This is an extremely simple implementation; every value that is not 286 of type <tt>str</tt> is pickled. Querying will be slow--every Unit is 287 sucked in one-by-one and tested in pure Python. 288 But for many applications, you don't need heavyweight query tools; 289 for example, an upload site may only need files looked up by ID.</p> 290 291 Configuration entries:</p> 292 <ul> 293 <li><b>Class:</b> "folders" 294 (<tt>dejavu.storage.storeshelve.StorageManagerShelve</tt>)</li> 295 <li><b>root:</b> Required. The file path (directory) in which to 296 place db files. Each Unit class will get its own subfolder, 297 of the same name as the class.</li> 298 <li><b>mode:</b> Optional. The mode arg to pass to <tt>os.mkdir</tt> 299 when creating folders. Defaults to '0777'.</li> 300 <li><b>idsepchar:</b> Optional. The character to use for separating 301 unit identities which are multivalent. Defaults to '_' (underscore). 302 For example, a Unit with <tt>identifiers = ('Name', 'DOB')</tt> 303 would get a folder name like 'Fred_20040321'.</li> 304 <li><b>extdefault:</b> Optional. The default file extension to use 305 for Unit Property files. Defaults to '.txt'.</li> 306 <li><b><unit>.<propname>:</b> Optional. The value should 307 be the file extension for properties of the given propname 308 for the given unit class. For example, <tt>Song.Data = .mp3</tt> 309 (be sure to include the leading 'dot' if you want one).</li> 310 </ul> 311 312 313 <a name='middleware'><h3>Middleware</h3></a> 251 314 252 315 <p>Some Storage Managers act as "middleware", and can be chained together … … 260 323 times, and address other integration concerns on their own systems.</p> 261 324 262 <h 5>Caching Proxy</h5>325 <h4>Caching Proxy</h4> 263 326 <p>Use this class to persist Units in memory between client connections. 264 327 It must proxy another Storage Manager. Configuration entries:</p> … … 286 349 287 350 288 <h 5>Burned Proxy</h5>351 <h4>Burned Proxy</h4> 289 352 <p>Use this class to persist Units in memory between client connections. 290 353 It needs another Storage Manager to proxy. Unlike the Caching Proxy above, trunk/storage/__init__.py
r387 r388 88 88 % self.__class__) 89 89 90 def has_property(self, cls, name): 91 raise NotImplementedError("%s has no has_property method." 92 % self.__class__) 93 90 94 def drop_property(self, cls, name): 91 95 raise NotImplementedError("%s has no drop_property method." … … 95 99 raise NotImplementedError("%s has no rename_property method." 96 100 % self.__class__) 101 102 def add_index(self, cls, name): 103 raise NotImplementedError("%s has no add_index method." 104 % self.__class__) 105 106 def has_index(self, cls, name): 107 raise NotImplementedError("%s has no has_index method." 108 % self.__class__) 109 110 def drop_index(self, cls, name): 111 raise NotImplementedError("%s has no drop_index method." 112 % self.__class__) 113 97 114 98 115 # Transactions # trunk/storage/db.py
r387 r388 606 606 607 607 t.rename(oldname, newname) 608 609 def add_index(self, cls, name): 610 i = self.db.make_index(cls.__name__, name) 611 self.db[cls.__name__].indices[name] = i 608 612 609 613 def has_index(self, cls, name): trunk/storage/storefs.py
r387 r388 43 43 # Character to use for joining multiple identifiers 44 44 # into a single folder name. 45 self.idsepchar = allOptions.get(' sepchar', '_')45 self.idsepchar = allOptions.get('idsepchar', '_') 46 46 47 47 # Map of file extensions. Keys should be "clsname.propname" trunk/units.py
r387 r388 259 259 260 260 def assign(self, unit, sequence): 261 newvalue = self.initial261 newvalue = self.initial 262 262 if sequence: 263 263 m = max(sequence)
