Index: trunk/__init__.py =================================================================== --- trunk/__init__.py (revision 13) +++ trunk/__init__.py (revision 15) @@ -646,12 +646,12 @@ return None - def distinct(self, cls, fields, expr=None): + def distinct(self, cls, attrs, expr=None): """Recall distinct Unit property values. - If only one field is specified, a list of values will be returned. - If more than one field is specified, a zipped list will be returned. + If only one attribute is specified, a list of values will be returned. + If more than one attribute is specified, a zipped list will be returned. Notice that you can also use this function as a count() function - (in fact it's the only way to do it) by using fields = ['ID']. + (in fact it's the only way to do it) by using attrs = ['ID']. """ seen = {} @@ -659,5 +659,5 @@ for unit in cache.itervalues(): if expr is None or expr.evaluate(unit): - row = tuple([getattr(unit, field) for field in fields]) + row = tuple([getattr(unit, attr) for attr in attrs]) if row not in seen: seen[row] = None @@ -665,5 +665,5 @@ store = self.arena.storage(cls) if store: - for row in store.distinct(cls, fields, expr): + for row in store.distinct(cls, attrs, expr): if row not in seen: seen[row] = None @@ -671,5 +671,5 @@ seen = seen.keys() seen.sort() - if len(fields) == 1: + if len(attrs) == 1: seen = [x[0] for x in seen] return seen Index: trunk/doc/index.html =================================================================== --- trunk/doc/index.html (revision 14) +++ trunk/doc/index.html (revision 15) @@ -25,14 +25,42 @@
  • Application Developers: Using Dejavu to construct a model +
  • Deployers: Configuring Storage
  • Index: trunk/doc/modeling.html =================================================================== --- trunk/doc/modeling.html (revision 14) +++ trunk/doc/modeling.html (revision 15) @@ -193,6 +193,26 @@ not fire.

    +

    You may define special methods on your Units to provide start-of-life +behaviors. If a Unit possesses an on_memorize method, it will +be called after the Unit has been 'reserved' in storage, and after the +Unit has ben placed in the Sandbox cache.

    +

    Sequencing

    -

    +

    Every Unit has an ID property. The default ID property +is of type int; however, you can override that to whatever type +you like. As long as you provide your own IDs for Units, nothing will +break--you can memorize and recall Units without problems. However, if +you memorize a Unit with an ID of None, the Sandbox may attempt +to provide an ID for it.

    + +

    The Unit base class possesses a sequencer attribute +to help Sandboxes generate new IDs. The default value is an instance of +UnitSequencerInteger, which examines all existing Units, finds +the maximum integer ID, adds 1, and uses that value for the new ID.

    + +

    The other useful Sequencer is UnitSequencerNull, which simply +raises an error when asked to generate an ID. If your ID's are strings, +you'll probably want to make that class' .sequencer one of +these, and form ID values in your own code.

    Recalling

    @@ -239,4 +259,44 @@ (although the rest are probably loaded into memory).

    +

    Forgetting and Repressing

    +

    To forget a Unit is to destroy it forever. You have two options +for forgetting Units: you can call Sandbox().forget(unit) or +the simpler version, Unit().forget(). Either of these will clear +the Unit from the Sandbox' cache, and the Sandbox will tell the appropriate +Storage Manager to destroy the stored Unit data. If a Unit has not yet +been memorized, you do not need to forget it.

    + +

    In some circumstances, you may wish to only clear the Unit from the +Sandbox without destroying it. You can do this by calling either +Sandbox().repress(unit) or the simpler version, +Unit().repress().

    + +

    You may define special methods on your Units to provide end-of-life +behaviors. If a Unit possesses an on_forget method, it will +be called after the Unit has been destroyed. If a Unit possesses an +on_repress method, it will be called before the Unit +has been repressed. I'm sure there was a good reason for this +disparity, but I've forgotten (or perhaps repressed) it.

    + +

    Flushing Sandboxes

    +

    When the client connection has closed, you should flush the +Sandbox caches. In general, a single call to flush_all() will do +the trick. Notice that flushing calls repress() for each Unit in +the Sandbox, and any on_repress() triggers will be executed.

    + +

    Aggregate Functions

    +

    Sandboxes also provide a distinct(cls, attrs, expr=None) +function. This returns values, rather than Units. Put simply, it returns +all distinct values for the given attribute(s) of the Unit class provided. +If only one attribute is specified, a list of values will be returned. +If more than one attribute is specified, a zipped list will be returned +of all distinct existing combinations. Providing an expr argument (an +Expression object, see below) will filter the set of Units before +obtaining distinct values.

    + +

    The distinct function can also be used as a count +function by passing attrs = ['ID']. Sandboxes provide a +count(cls, expr) method which does just this.

    +

    Querying

    When you retrieve Units, you often don't want to load the entire set for @@ -245,5 +305,5 @@ the filter you intend. Dejavu actually provides three ways.

    -

    Expressions

    +

    The Expression class

    Regardless of which technique you use to express your filter, you're going to end up with a logic.Expression object. You can build @@ -255,8 +315,19 @@ >>> e logic.Expression(lambda x: x.Date >= datetime.date(2004, 3, 1)) -Neat, eh? I worked hard on that __repr__. ;) What is not obvious from the -above code snippet is perhaps the most important aspect of -Expressions: any globals or cell references (from closures) in the -supplied lambda get bound early. Compare the following disassemblies: +Neat, eh? I worked hard on that __repr__. ;)

    + +

    It may be obvious, but we'll be explicit, here. The lambda which you pass +into an Expression must possess a single positional argument, which will +always be bound to a Unit instance. In the example above, it's named 'x', +but you can use any name you like. For complete Unit objects residing in +memory, this arrangement means that we can simply call Expression.func(Unit), +and receive a boolean value indicating whether our Unit "passes the test". +Attribute lookups on our 'x' object will apply to Unit Properties for that +Unit object. That is, x.Date becomes Unit.Date.

    + +

    What is not obvious from the above code snippet is perhaps the most +important aspect of Expressions: any globals or cell references (from +closures) in the supplied lambda get bound early. Compare the +following disassemblies:

    >>> import dis
     >>> dis.dis(f)
    @@ -295,10 +366,152 @@
     globals into the logic module. Note that the logic module
     already tries to import datetime, fixedpoint and
    -decimal.

    Syntactic Sugar -

    The logic module also provides some convenience functions -for creating Expression objects via the filter and +decimal.

    + +

    External functions within Expressions

    +

    Dejavu provides additional functions which can be used in Expressions. +For example, you can construct an Expression like: +

    logic.Expression(lambda x: x.Size < 3 and x.Date > dejavu.today())
    +In this example, the today() function breaks convention and is +actually bound late. That is, if you construct this Expression now +and use it six months later, the value of today() will change. +Storage Managers "know about" these dejavu functions, and can use them +to build more appropriate queries. Here are the functions supplied by +the dejavu module:

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    FunctionLate bound?Description
    icontains(a, b)Case-insensitive test b in a. Note the operand order.
    icontainedby(a, b)Case-insensitive test a in b. Note the operand order.
    istartswith(a, b)True if a starts with b (case-insensitive), False otherwise.
    iendswith(a, b)True if a ends with b (case-insensitive), False otherwise.
    ieq(a, b)True if a == b (case-insensitive), False otherwise.
    year(value)The year attribute of a date. If value is None, return None.
    now()Ydatetime.datetime.now()
    today()Ydatetime.date.today()
    iscurrentweek(value)YIf value is in the current week, return True, else False.
    + +

    It is possible for you, the application developer, to define your +own external functions. However, because Storage Managers are unaware +of your new functions, they will not be able to optimize their use; +instead, they will simply retrieve a larger set of objects from storage, +evaluate each one against the function you provide, and return those +Units which match your function.

    + +

    Combining Expressions

    +

    Expressions are combinable; by using the "&" operator, the two condition +lists are combined with an adjoining logical "and". For example: +

    >>> a = logic.Expression(lambda x: x.Size > 3)
    +>>> b = logic.Expression(lambda x: x.Size <= 15)
    +>>> c = a & b
    +>>> c
    +logic.Expression(lambda x: (x.Size > 3) and (x.Size <= 15))
    +The + operator works just like the & operator. The +| operator combines the two Expressions with a logical 'or'.

    + +

    Expressions using filter and comparison

    +

    The logic module also provides convenient methods to +create common types of Expression objects via the filter and comparison factory functions.

    + +

    The filter(**kwargs) function produces an Expression by taking +the keyword arguments you supply, and rewriting them in lambda form. The +only operator allowed is therefore the equals '==' operator. For example: +

    >>> logic.filter(Type='Cat', Mutation='Atomic')
    +logic.Expression(lambda x: (x.Type == 'Cat') and (x.Mutation == 'Atomic'))
    +

    + +

    The comparison(attr, cmp_op, criteria) function allows you to +form Expressions with dynamic operators. This can come in handy when you +are constructing Expressions on the fly from user input. For example, a +search page might prompt users for an attribute name, an operator, and an +operand (the criteria).

    + +

    Borrowing from opcode.cmp_op, the allowed values for our cmp_op +argument are as follows:

    + + + + + + + + + + + + +
    Numeric Value (cmp_op)Operator
    0<
    1<=
    2==
    3!=
    4>
    5>=
    6in
    7not in
    8is
    9is not
    + +

    Here's an example of using comparison: +

    >>> logic.comparison('Name', 3, 'Mr. Kamikaze')
    +logic.Expression(lambda x: x.Name != 'Mr. Kamikaze')
    +Although the comparison function only allows a single comparison at a time, +the resulting Expressions can be combined with the & and | +operators (described earlier) to produce more complex Expressions.

    + +

    Exporting the logic module

    +

    The logic module (and codewalk, on which it is built) +isn't limited to Dejavu. Feel free to use it in some other framework or +script! The only change you may have to make (if you relocate the module +outside of the dejavu package) would be to the single line: +from dejavu import codewalk, to point to the new location.

    + +

    In particular, logic.Expression objects can operate on any +Python object, not just dejavu Unit instances. If you wish to +provide additional logic functions (as dejavu does), simply inject them +into logic's globals.

    + +

    You may also find the underlying codewalk module useful for +other purposes on its own. The Visitor base class can be very +convenient for building bytecode hacks.

    + +

    To make a long story short, Dejavu depends on logic throughout, +but the reverse is not true.

    + + +

    Unit Engines

    +

    + + +

    Analysis Tools

    +

    +

    The Arena Object

    @@ -314,10 +527,4 @@

    -

    Unit Engines

    -

    - -

    Analysis Tools

    -

    -
    Index: trunk/logic.py =================================================================== --- trunk/logic.py (revision 12) +++ trunk/logic.py (revision 15) @@ -187,4 +187,15 @@ self.instr_index[-1:] = [obj] * (newtarget + 4) + def or_combine(self, obj): + obj = codewalk.Rewriter(obj) + bytecode = map(ord, obj.co_code) + newtarget = len(bytecode) + + self._bytecode.pop() + self._bytecode.extend([112, newtarget & 0xFF, newtarget >> 8, + 1]) + self._bytecode.extend(bytecode) + self.instr_index[-1:] = [obj] * (newtarget + 4) + def visit_LOAD_ATTR(self, lo, hi): src = self.instr_index[self.cursor] @@ -233,9 +244,18 @@ return 'logic.Expression(%s)' % self.code() - def __add__(self, other): + def __and__(self, other): """Logical-and this Expression with another.""" assert isinstance(other, Expression) ag = Aggregator(self.func) ag.and_combine(other.func) + agfunc = ag.function() + return Expression(agfunc) + __add__ = __and__ + + def __or__(self, other): + """Logical-or this Expression with another.""" + assert isinstance(other, Expression) + ag = Aggregator(self.func) + ag.or_combine(other.func) agfunc = ag.function() return Expression(agfunc)