witten
/
luminotes
Archived
1
0
Fork 0
Code Releases Activity

Merged revisions 401-446 via svnmerge from 

svn+ssh://torsion.org/home/luminotes/repos/luminotes/branches/postgres

................
  r402 | witten | 2007-10-04 00:48:49 -0700 (Thu, 04 Oct 2007) | 3 lines

  Initialized merge tracking via "svnmerge" with revisions "1-401" from
  svn+ssh://torsion.org/home/luminotes/repos/luminotes/trunk
................
  r404 | witten | 2007-10-04 01:17:07 -0700 (Thu, 04 Oct 2007) | 2 lines

  Beginning a conversion from bsddb to postgres.
................
  r405 | witten | 2007-10-04 01:18:58 -0700 (Thu, 04 Oct 2007) | 9 lines

  Merged revisions 402-404 via svnmerge from
  svn+ssh://torsion.org/home/luminotes/repos/luminotes/trunk

  ........
    r403 | witten | 2007-10-04 01:14:45 -0700 (Thu, 04 Oct 2007) | 2 lines

    Yay, no more stupid deprecation warnings from simplejson about the sre module.
  ........
................
  r406 | witten | 2007-10-04 15:34:39 -0700 (Thu, 04 Oct 2007) | 4 lines

   * Switched back to Python 2.4 because many Python modules in Debian are not packaged to work with Python 2.5
   * Began removal of all references to Scheduler, @async, yield, and so on.
   * Converted Database.py to support PostgreSQL and updated its unit tests accordingly.
................
  r407 | witten | 2007-10-04 16:34:01 -0700 (Thu, 04 Oct 2007) | 2 lines

  All unit tests for the new model classes now pass.
................
  r409 | witten | 2007-10-05 00:53:56 -0700 (Fri, 05 Oct 2007) | 2 lines

  Reordering some columns and adding some indices.
................
  r410 | witten | 2007-10-05 16:08:37 -0700 (Fri, 05 Oct 2007) | 4 lines

  Now adding trash notebooks to user_notebook table. Also switching db
  conversion/verification tools back to require Python 2.5, since they still use
  the old Scheduler, which requires 2.5 generator features.
................
  r411 | witten | 2007-10-06 16:26:56 -0700 (Sat, 06 Oct 2007) | 2 lines

  Lots more unit tests passing. Most of the recent work was on controller.Users and related stuff.
................
  r412 | witten | 2007-10-07 01:52:12 -0700 (Sun, 07 Oct 2007) | 2 lines

  controller.Users unit tests now finally pass!
................
  r413 | witten | 2007-10-07 02:14:10 -0700 (Sun, 07 Oct 2007) | 3 lines

  Got controller.Root unit tests passing.
  Moved fake sql_* function shenanigans from Test_users.py to Test_controller.py, for use by other controller unit tests.
................
  r414 | witten | 2007-10-08 23:11:11 -0700 (Mon, 08 Oct 2007) | 2 lines

  All unit tests pass! Fuck yeah!
................
  r415 | witten | 2007-10-08 23:13:07 -0700 (Mon, 08 Oct 2007) | 2 lines

  Removing all references to Scheduler from luminotes.py
................
  r416 | witten | 2007-10-08 23:54:51 -0700 (Mon, 08 Oct 2007) | 3 lines

  Converted deleted_from to deleted_from_id in a few more places.
  Fixed bug in Users.contents().
................
  r417 | witten | 2007-10-09 00:11:59 -0700 (Tue, 09 Oct 2007) | 3 lines

  Typo fix in Note sql method.
  Adding autocommit flag to Database.next_id() method.
................
  r418 | witten | 2007-10-09 00:13:19 -0700 (Tue, 09 Oct 2007) | 2 lines

  Updating unit test for new auto commit flag.
................
  r419 | witten | 2007-10-09 00:14:09 -0700 (Tue, 09 Oct 2007) | 2 lines

  Removing debugging print.
................
  r420 | witten | 2007-10-09 00:20:55 -0700 (Tue, 09 Oct 2007) | 2 lines

  More sql fixes. I really need some funtional tests that hit the database and exercise the SQL.
................
  r421 | witten | 2007-10-09 00:51:34 -0700 (Tue, 09 Oct 2007) | 3 lines

  Fixed controller.Database handling of tuple as an Object_type.
  Made SQL for user storage calculation better at handling null values and also more succinct.
................
  r422 | witten | 2007-10-09 13:32:16 -0700 (Tue, 09 Oct 2007) | 2 lines

  Converting Wiki.js to trash_id notebook member instead of trash object.
................
  r423 | witten | 2007-10-09 13:42:10 -0700 (Tue, 09 Oct 2007) | 2 lines

  No longer displaying "download as html" on the front page, as people see "download" and think they're downloading the software.
................
  r424 | witten | 2007-10-09 14:24:40 -0700 (Tue, 09 Oct 2007) | 2 lines

  Notebooks.contents() now returns notebooks with correct read-write status.
................
  r425 | witten | 2007-10-09 14:32:25 -0700 (Tue, 09 Oct 2007) | 2 lines

  Fixed reporting of validation errors to the user. Now says "The blah is missing." instead of just "is missing"
................
  r426 | witten | 2007-10-09 17:05:22 -0700 (Tue, 09 Oct 2007) | 2 lines

  No longer redirecting to trash notebook upon login.
................
  r427 | witten | 2007-10-09 17:20:33 -0700 (Tue, 09 Oct 2007) | 2 lines

  Made controller.Database use a connection pool.
................
  r429 | witten | 2007-10-09 20:13:30 -0700 (Tue, 09 Oct 2007) | 2 lines

  Converted initdb.py and updatedb.py to Postgres from bsddb.
................
  r430 | witten | 2007-10-09 20:37:14 -0700 (Tue, 09 Oct 2007) | 2 lines

  Changing error message to remove underscores from variable names.
................
  r431 | witten | 2007-10-10 13:23:30 -0700 (Wed, 10 Oct 2007) | 2 lines

  Removing unused note_title parameter from Wiki.create_editor().
................
  r432 | witten | 2007-10-10 13:25:16 -0700 (Wed, 10 Oct 2007) | 2 lines

  Revision regular expression now supports timezone notation.
................
  r433 | witten | 2007-10-10 14:43:47 -0700 (Wed, 10 Oct 2007) | 2 lines

  Finished implementing ranked ordering for startup notes. (However, there's no way to change the rank from the client yet.)
................
  r434 | witten | 2007-10-10 16:25:19 -0700 (Wed, 10 Oct 2007) | 4 lines

  More strict access checking. Fixed oversight in Postgres DB conversion where,
  in certain controller.Notebook methods, access was only checked at the
  notebook level, not at the note level as well.
................
  r435 | witten | 2007-10-10 17:45:18 -0700 (Wed, 10 Oct 2007) | 3 lines

  Now loading revisions on demand from client when the "changes" button is clicked. Also caching
  loading revisions so subsequent clicks don't have to reload.
................
  r436 | witten | 2007-10-10 21:31:20 -0700 (Wed, 10 Oct 2007) | 2 lines

  Tweaking some of the error handling in Expose and Root so that unhandled errors give a generic error message to the client.
................
  r437 | witten | 2007-10-10 21:33:49 -0700 (Wed, 10 Oct 2007) | 2 lines

  The release script no longer runs initdb.py, because the default database is no longer a single file included in the tarball.
................
  r438 | witten | 2007-10-10 21:40:11 -0700 (Wed, 10 Oct 2007) | 2 lines

  Updated install instructuctions to include use of initdb.py.
................
  r439 | witten | 2007-10-10 21:56:42 -0700 (Wed, 10 Oct 2007) | 3 lines

  Made initdb.py only nuke (drop tables/views) when given a command-line flag.
  Also made install directions more correct.
................
  r440 | witten | 2007-10-10 21:58:48 -0700 (Wed, 10 Oct 2007) | 2 lines

  IE 6 doesn't like commas.
................
  r441 | witten | 2007-10-10 22:08:50 -0700 (Wed, 10 Oct 2007) | 4 lines

  load your notebook. without clicking on "changes", edit a note that has previous revisions. click on "changes". it'll only show
  the most recent revision. fixed by not appending to changes as a result of a save unless the client-side revisions list cache has
  something in it
................
  r442 | witten | 2007-10-10 23:30:41 -0700 (Wed, 10 Oct 2007) | 2 lines

  Forgot to actually save off the new revision as editor.revision.
................
  r443 | witten | 2007-10-11 01:35:54 -0700 (Thu, 11 Oct 2007) | 13 lines

  More intelligent datetime handling:
    * convertdb.py assumes old bsddb database timestamps are Pacific, and then
      converts them to UTC before inserting them into the new PostgreSQL
      database.
    * No longer using naked timezoneless datetime objects in model/controller
      code, except in unit tests that need compatability with pysqlite. Now
      using UTC everwhere.
    * Asking PostgreSQL to give us all timestamps back in UTC.
    * New dependency on python-tz (pytz) package, noted in INSTALL doc.
    * Client now responsible for converting UTC timestamps to local time for
      display.
................
  r444 | witten | 2007-10-11 01:46:09 -0700 (Thu, 11 Oct 2007) | 2 lines

  Tweak to prevent potential race in IE.
................
  r445 | witten | 2007-10-11 01:49:58 -0700 (Thu, 11 Oct 2007) | 2 lines

  Got JavaScript "unit" tests passing again.
................
  r446 | witten | 2007-10-11 01:53:58 -0700 (Thu, 11 Oct 2007) | 2 lines

  Noting that js tests require the Luminotes server on localhost.
................
This commit is contained in:
Dan Helfman 2007-10-11 09:03:43 +00:00
parent 03bffe4676
commit 43c6f54e9f
48 changed files with 2959 additions and 2308 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
INSTALL
View File

@ -4,19 +4,17 @@ you shouldn't need if you only want to make a wiki.
First, install the prerequisites:
* Python 2.5
* Python 2.4
* CherryPy 2.2
* PostgreSQL 8.1
* psycopg 2.0
* simplejson 1.3
* pytz 2006p
In Debian GNU/Linux, you can issue the following command to install these
packages:
apt-get install python2.5 python-cherrypy postgresql-8.1 python-psycopg2 python-simplejson
If you're using Debian Etch, see the note below about "psycopg in Debian
Etch".
apt-get install python2.4 python-cherrypy postgresql-8.1 python-psycopg2 python-simplejson python-tz
development mode
@ -40,14 +38,15 @@ and set the password to "dev".
createuser -S -d -R -P -E luminotes
Initialize the database with the starting schema and basic data:
Initialize the database with the starting schema and default data:
psql -U luminotes postgres -f model/schema.sql
psql -U luminotes postgres -f model/data.sql
createdb -W -U luminotes luminotes
export PYTHONPATH=.
python2.4 tools/initdb.py
To start the server in development mode, run:
python2.5 luminotes.py -d
python2.4 luminotes.py -d
Connect to the following URL in a web browser running on the same machine:
@ -114,18 +113,20 @@ Restart postgresql so these changes take effect:
/etc/init.d/postgresql restart
As the PostgreSQL superuser (usually "postgres"), create a new database user
and set the password to "dev".
and set a new password, for instance, "mypassword".
createuser -S -d -R -P -E luminotes
Initialize the database with the starting schema and basic data:
Initialize the database with the starting schema and default data:
psql -U luminotes postgres -f model/schema.sql
psql -U luminotes postgres -f model/data.sql
createdb -W -U luminotes luminotes
export PYTHONPATH=.
export PGPASSWORD=mypassword
python2.4 tools/initdb.py
Then to actually start the production mode server, run:
python2.5 luminotes.py
python2.4 luminotes.py
You should be able to connect to the site at whatever domain you've configured
Apache to serve.
@ -137,44 +138,28 @@ Python unit tests
If you're interested in running unit tests of the server, install:
* nose 0.9.0
* pysqlite 2.3
In Debian GNU/Linux, you can issue the following command to install this
package:
apt-get install python-nose
apt-get install python-nose python-pysqlite2
Then you can run unit tests by running:
nosetests
JavaScript unit tests
---------------------
JavaScript "unit" tests
-----------------------
JsUnit is included with Luminotes, so to kick off tests of the client-side
JavaScript code, simply run:
python2.5 static/js/test/run_tests.py
python2.4 static/js/test/run_tests.py
The run_tests.py script runs the tests inside browser windows and presumes you
have both Firefox and Internet Explorer 6 installed. Edit run_tests.py if you
The run_tests.py script runs the tests inside browser windows and presumes
that you have both Firefox and Internet Explorer 6 installed, and also that
the Luminotes server is running on the local machine. Edit run_tests.py if you
need to specify different paths to the browser binaries or want to test with
additional browsers.
psycopg in Debian Etch
----------------------
As of this writing, Debian Etch does not contain a version of psycopg with
support for Python 2.5. However, the version of psycopg in Debian testing does
support Python 2.5. So you can grab the source for python-psycopg2 from Debian
testing, install the build dependencies (including python2.5-dev), and build
the package yourself on an Etch machine.
Then, edit /usr/share/python/debian_defaults and move "python2.5" from
"unsupported-versions" to "supported-versions". Finally, install the
python-psycopg2 package you've just built, and it should fully support Python
2.5.
See Debian bug #404355 for more information. Note that it was fixed in
unstable, but not in Etch.

350
controller/Database.py
View File

@ -1,219 +1,158 @@
import re
import bsddb
import os
import psycopg2 as psycopg
from psycopg2.pool import PersistentConnectionPool
import random
import cPickle
from cStringIO import StringIO
from copy import copy
from model.Persistent import Persistent
from Async import async
class Database( object ):
ID_BITS = 128 # number of bits within an id
ID_DIGITS = "0123456789abcdefghijklmnopqrstuvwxyz"
def __init__( self, scheduler, database_path = None ):
def __init__( self, connection = None ):
"""
Create a new database and return it.
@type scheduler: Scheduler
@param scheduler: scheduler to use
@type database_path: unicode
@param database_path: path to the database file
@type connection: existing connection object with cursor()/close()/commit() methods, or NoneType
@param connection: database connection to use (optional, defaults to making a connection pool)
@rtype: Database
@return: database at the given path
@return: newly constructed Database
"""
self.__scheduler = scheduler
self.__env = bsddb.db.DBEnv()
self.__env.open( None, bsddb.db.DB_CREATE | bsddb.db.DB_PRIVATE | bsddb.db.DB_INIT_MPOOL )
self.__db = bsddb.db.DB( self.__env )
self.__db.open( database_path, "database", bsddb.db.DB_HASH, bsddb.db.DB_CREATE )
self.__cache = {}
# This tells PostgreSQL to give us timestamps in UTC. I'd use "set timezone" instead, but that
# makes SQLite angry.
os.putenv( "PGTZ", "UTC" )
def __persistent_id( self, obj, skip = None ):
# save the object and return its persistent id
if obj != skip and isinstance( obj, Persistent ):
self.__save( obj )
return obj.object_id
if connection:
self.__connection = connection
self.__pool = None
else:
self.__connection = None
self.__pool = PersistentConnectionPool(
1, # minimum connections
50, # maximum connections
"dbname=luminotes user=luminotes password=%s" % os.getenv( "PGPASSWORD", "dev" ),
)
# returning None indicates that the object should be pickled normally without using a persistent id
return None
def __get_connection( self ):
if self.__connection:
return self.__connection
else:
return self.__pool.getconn()
@async
def save( self, obj, callback = None ):
def save( self, obj, commit = True ):
"""
Save the given object to the database, including any objects that it references.
Save the given object to the database.
@type obj: Persistent
@param obj: object to save
@type callback: generator or NoneType
@param callback: generator to wakeup when the save is complete (optional)
@type commit: bool
@param commit: True to automatically commit after the save
"""
self.__save( obj )
yield callback
connection = self.__get_connection()
cursor = connection.cursor()
def __save( self, obj ):
# if this object's current revision is already saved, bail
revision_id = obj.revision_id()
if revision_id in self.__cache:
return
cursor.execute( obj.sql_exists() )
if cursor.fetchone():
cursor.execute( obj.sql_update() )
else:
cursor.execute( obj.sql_create() )
object_id = unicode( obj.object_id ).encode( "utf8" )
revision_id = unicode( obj.revision_id() ).encode( "utf8" )
secondary_id = obj.secondary_id and unicode( obj.full_secondary_id() ).encode( "utf8" ) or None
if commit:
connection.commit()
# update the cache with this saved object
self.__cache[ object_id ] = obj
self.__cache[ revision_id ] = copy( obj )
if secondary_id:
self.__cache[ secondary_id ] = obj
def commit( self ):
self.__get_connection().commit()
# set the pickler up to save persistent ids for every object except for the obj passed in, which
# will be pickled normally
buffer = StringIO()
pickler = cPickle.Pickler( buffer, protocol = -1 )
pickler.persistent_id = lambda o: self.__persistent_id( o, skip = obj )
# pickle the object and write it to the database under both its id key and its revision id key
pickler.dump( obj )
pickled = buffer.getvalue()
self.__db.put( object_id, pickled )
self.__db.put( revision_id, pickled )
# write the pickled object id (only) to the database under its secondary id
if secondary_id:
buffer = StringIO()
pickler = cPickle.Pickler( buffer, protocol = -1 )
pickler.persistent_id = lambda o: self.__persistent_id( o )
pickler.dump( obj )
self.__db.put( secondary_id, buffer.getvalue() )
self.__db.sync()
@async
def load( self, object_id, callback, revision = None ):
def load( self, Object_type, object_id, revision = None ):
"""
Load the object corresponding to the given object id from the database, and yield the provided
callback generator with the loaded object as its argument, or None if the object_id is unknown.
If a revision is provided, a specific revision of the object will be loaded.
@type object_id: unicode
@param object_id: id of the object to load
@type callback: generator
@param callback: generator to send the loaded object to
@type revision: int or NoneType
@param revision: revision of the object to load (optional)
"""
obj = self.__load( object_id, revision )
yield callback, obj
def __load( self, object_id, revision = None ):
if revision is not None:
object_id = Persistent.make_revision_id( object_id, revision )
object_id = unicode( object_id ).encode( "utf8" )
# if the object corresponding to the given id has already been loaded, simply return it without
# loading it again
obj = self.__cache.get( object_id )
if obj is not None:
return obj
# grab the object for the given id from the database
buffer = StringIO()
unpickler = cPickle.Unpickler( buffer )
unpickler.persistent_load = self.__load
pickled = self.__db.get( object_id )
if pickled is None or pickled == "":
return None
buffer.write( pickled )
buffer.flush()
buffer.seek( 0 )
# unpickle the object and update the cache with this saved object
obj = unpickler.load()
if obj is None:
print "error unpickling %s: %s" % ( object_id, pickled )
return None
self.__cache[ unicode( obj.object_id ).encode( "utf8" ) ] = obj
self.__cache[ unicode( obj.revision_id() ).encode( "utf8" ) ] = copy( obj )
return obj
@async
def reload( self, object_id, callback = None ):
"""
Load and immediately save the object corresponding to the given object id or database key. This
is useful when the object has a __setstate__() method that performs some sort of schema
evolution operation.
@type object_id: unicode
@param object_id: id or key of the object to reload
@type callback: generator or NoneType
@param callback: generator to wakeup when the save is complete (optional)
"""
self.__reload( object_id )
yield callback
def __reload( self, object_id, revision = None ):
object_id = unicode( object_id ).encode( "utf8" )
# grab the object for the given id from the database
buffer = StringIO()
unpickler = cPickle.Unpickler( buffer )
unpickler.persistent_load = self.__load
pickled = self.__db.get( object_id )
if pickled is None or pickled == "":
return
buffer.write( pickled )
buffer.flush()
buffer.seek( 0 )
# unpickle the object. this should trigger __setstate__() if the object has such a method
obj = unpickler.load()
if obj is None:
print "error unpickling %s: %s" % ( object_id, pickled )
return
self.__cache[ object_id ] = obj
# set the pickler up to save persistent ids for every object except for the obj passed in, which
# will be pickled normally
buffer = StringIO()
pickler = cPickle.Pickler( buffer, protocol = -1 )
pickler.persistent_id = lambda o: self.__persistent_id( o, skip = obj )
# pickle the object and write it to the database under its id key
pickler.dump( obj )
pickled = buffer.getvalue()
self.__db.put( object_id, pickled )
self.__db.sync()
def size( self, object_id, revision = None ):
"""
Load the object corresponding to the given object id from the database, and return the size of
its pickled data in bytes. If a revision is provided, a specific revision of the object will be
Load the object corresponding to the given object id from the database and return it, or None if
the object_id is unknown. If a revision is provided, a specific revision of the object will be
loaded.
@type Object_type: type
@param Object_type: class of the object to load
@type object_id: unicode
@param object_id: id of the object whose size should be returned
@param object_id: id of the object to load
@type revision: int or NoneType
@param revision: revision of the object to load (optional)
@rtype: Object_type or NoneType
@return: loaded object, or None if no match
"""
if revision is not None:
object_id = Persistent.make_revision_id( object_id, revision )
return self.select_one( Object_type, Object_type.sql_load( object_id, revision ) )
object_id = unicode( object_id ).encode( "utf8" )
def select_one( self, Object_type, sql_command ):
"""
Execute the given sql_command and return its results in the form of an object of Object_type,
or None if there was no match.
pickled = self.__db.get( object_id )
if pickled is None or pickled == "":
@type Object_type: type
@param Object_type: class of the object to load
@type sql_command: unicode
@param sql_command: SQL command to execute
@rtype: Object_type or NoneType
@return: loaded object, or None if no match
"""
connection = self.__get_connection()
cursor = connection.cursor()
cursor.execute( sql_command )
row = cursor.fetchone()
if not row:
return None
return len( pickled )
if Object_type in ( tuple, list ):
return Object_type( row )
else:
return Object_type( *row )
def select_many( self, Object_type, sql_command ):
"""
Execute the given sql_command and return its results in the form of a list of objects of
Object_type.
@type Object_type: type
@param Object_type: class of the object to load
@type sql_command: unicode
@param sql_command: SQL command to execute
@rtype: list of Object_type
@return: loaded objects
"""
connection = self.__get_connection()
cursor = connection.cursor()
cursor.execute( sql_command )
objects = []
row = cursor.fetchone()
while row:
if Object_type in ( tuple, list ):
obj = Object_type( row )
else:
obj = Object_type( *row )
objects.append( obj )
row = cursor.fetchone()
return objects
def execute( self, sql_command, commit = True ):
"""
Execute the given sql_command.
@type sql_command: unicode
@param sql_command: SQL command to execute
@type commit: bool
@param commit: True to automatically commit after the command
"""
connection = self.__get_connection()
cursor = connection.cursor()
cursor.execute( sql_command )
if commit:
connection.commit()
@staticmethod
def generate_id():
@ -231,44 +170,45 @@ class Database( object ):
return "".join( digits )
@async
def next_id( self, callback ):
def next_id( self, Object_type, commit = True ):
"""
Generate the next available object id, and yield the provided callback generator with the
object id as its argument.
Generate the next available object id and return it.
@type callback: generator
@param callback: generator to send the next available object id to
@type Object_type: type
@param Object_type: class of the object that the id is for
@type commit: bool
@param commit: True to automatically commit after storing the next id
"""
connection = self.__get_connection()
cursor = connection.cursor()
# generate a random id, but on the off-chance that it collides with something else already in
# the database, try again
next_id = Database.generate_id()
while self.__db.get( next_id, default = None ) is not None:
cursor.execute( Object_type.sql_id_exists( next_id ) )
while cursor.fetchone() is not None:
next_id = Database.generate_id()
cursor.execute( Object_type.sql_id_exists( next_id ) )
# save the next_id as a key in the database so that it's not handed out again to another client
self.__db[ next_id ] = ""
# save a new object with the next_id to the database
obj = Object_type( next_id )
cursor.execute( obj.sql_create() )
yield callback, next_id
if commit:
connection.commit()
return next_id
@async
def close( self ):
"""
Shutdown the database.
"""
self.__db.close()
self.__env.close()
yield None
if self.__connection:
self.__connection.close()
@async
def clear_cache( self ):
"""
Clear the memory object cache.
"""
self.__cache.clear()
yield None
scheduler = property( lambda self: self.__scheduler )
if self.__pool:
self.__pool.closeall()
class Valid_id( object ):
@ -289,9 +229,9 @@ class Valid_id( object ):
class Valid_revision( object ):
"""
Validator for an object id.
Validator for an object revision timestamp.
"""
REVISION_PATTERN = re.compile( "^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d+$" )
REVISION_PATTERN = re.compile( "^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d+[+-]\d\d(:)?\d\d$" )
def __init__( self, none_okay = False ):
self.__none_okay = none_okay

22
controller/Expose.py
View File

@ -1,18 +1,10 @@
import cherrypy
from Validate import Validation_error
# module-level variable that, when set to a view, overrides the view for all exposed methods. used
# by unit tests
view_override = None
class Expose_error( Exception ):
def __init__( self, message ):
Exception.__init__( self, message )
self.__message = message
def expose( view = None, rss = None ):
"""
expose() can be used to tag a method as available for publishing to the web via CherryPy. In
@ -57,8 +49,16 @@ def expose( view = None, rss = None ):
# try executing the exposed function
try:
result = function( *args, **kwargs )
except Validation_error, error:
result = dict( name = error.name, value = error.value, error = error.message )
except cherrypy.NotFound:
raise
except Exception, error:
if hasattr( error, "to_dict" ):
result = error.to_dict()
else:
# TODO: it'd be nice to send an email to myself with the traceback
import traceback
traceback.print_exc()
result = dict( error = u"An error occurred when processing your request. Please try again or contact support." )
redirect = result.get( u"redirect", None )
@ -74,7 +74,7 @@ def expose( view = None, rss = None ):
return unicode( view_override( **result ) )
except:
if redirect is None:
raise Expose_error( result.get( u"error" ) or result )
raise
# if that doesn't work, and there's a redirect, then redirect
del( result[ u"redirect" ] )

504
controller/Notebooks.py
View File

@ -1,15 +1,13 @@
import cherrypy
from Scheduler import Scheduler
from datetime import datetime
from Expose import expose
from Validate import validate, Valid_string, Validation_error, Valid_bool
from Database import Valid_id, Valid_revision
from Users import grab_user_id
from Updater import wait_for_update, update_client
from Expire import strongly_expire
from Html_nuker import Html_nuker
from Async import async
from model.Notebook import Notebook
from model.Note import Note
from new_model.Notebook import Notebook
from new_model.Note import Note
from view.Main_page import Main_page
from view.Json import Json
from view.Html_file import Html_file
@ -18,7 +16,7 @@ from view.Html_file import Html_file
class Access_error( Exception ):
def __init__( self, message = None ):
if message is None:
message = u"You don't have access to this notebook."
message = u"Sorry, you don't have access to do that."
Exception.__init__( self, message )
self.__message = message
@ -33,12 +31,10 @@ class Notebooks( object ):
"""
Controller for dealing with notebooks and their notes, corresponding to the "/notebooks" URL.
"""
def __init__( self, scheduler, database, users ):
def __init__( self, database, users ):
"""
Create a new Notebooks object.
@type scheduler: controller.Scheduler
@param scheduler: scheduler to use for asynchronous calls
@type database: controller.Database
@param database: database that notebooks are stored in
@type users: controller.Users
@ -46,7 +42,6 @@ class Notebooks( object ):
@rtype: Notebooks
@return: newly constructed Notebooks
"""
self.__scheduler = scheduler
self.__database = database
self.__users = users
@ -83,14 +78,11 @@ class Notebooks( object ):
@expose( view = Json )
@strongly_expire
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_id = Valid_id( none_okay = True ),
revision = Valid_string( min = 0, max = 30 ),
revision = Valid_revision( none_okay = True ),
user_id = Valid_id( none_okay = True ),
)
def contents( self, notebook_id, note_id = None, revision = None, user_id = None ):
@ -108,39 +100,37 @@ class Notebooks( object ):
@param user_id: id of current logged-in user (if any), determined by @grab_user_id
@rtype: json dict
@return: { 'notebook': notebookdict, 'note': notedict or None }
@raise Access_error: the current user doesn't have access to the given notebook
@raise Access_error: the current user doesn't have access to the given notebook or note
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not self.__users.check_access( user_id, notebook_id, read_write = True ):
notebook.read_write = False
if notebook is None:
note = None
elif note_id == u"blank":
note = Note( note_id )
note = Note.create( note_id )
else:
note = notebook.lookup_note( note_id )
note = self.__database.load( Note, note_id, revision )
if note and note.notebook_id != notebook_id:
raise Access_error()
if revision:
self.__database.load( note_id, self.__scheduler.thread, revision )
note = ( yield Scheduler.SLEEP )
startup_notes = self.__database.select_many( Note, notebook.sql_load_startup_notes() )
yield dict(
return dict(
notebook = notebook,
startup_notes = notebook.startup_notes,
startup_notes = startup_notes,
note = note,
)
@expose( view = Json )
@strongly_expire
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_id = Valid_id(),
@ -161,35 +151,24 @@ class Notebooks( object ):
@param user_id: id of current logged-in user (if any), determined by @grab_user_id
@rtype: json dict
@return: { 'note': notedict or None }
@raise Access_error: the current user doesn't have access to the given notebook
@raise Access_error: the current user doesn't have access to the given notebook or note
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
note = self.__database.load( Note, note_id, revision )
if notebook is None:
note = None
else:
note = notebook.lookup_note( note_id )
if note and note.notebook_id != notebook_id:
raise Access_error()
if revision:
self.__database.load( note_id, self.__scheduler.thread, revision )
note = ( yield Scheduler.SLEEP )
yield dict(
return dict(
note = note,
)
@expose( view = Json )
@strongly_expire
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_title = Valid_string( min = 1, max = 500 ),
@ -210,28 +189,23 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if notebook is None:
note = None
else:
note = notebook.lookup_note_by_title( note_title )
note = self.__database.select_one( Notebook, notebook.sql_load_note_by_title( note_title ) )
yield dict(
return dict(
note = note,
)
@expose( view = Json )
@strongly_expire
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_title = Valid_string( min = 1, max = 500 ),
@ -252,27 +226,61 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if notebook is None:
note = None
else:
note = notebook.lookup_note_by_title( note_title )
note = self.__database.select_one( Notebook, notebook.sql_load_note_by_title( note_title ) )
yield dict(
return dict(
note_id = note and note.object_id or None,
)
@expose( view = Json )
@wait_for_update
@strongly_expire
@grab_user_id
@validate(
notebook_id = Valid_id(),
note_id = Valid_id(),
user_id = Valid_id( none_okay = True ),
)
def load_note_revisions( self, notebook_id, note_id, user_id = None ):
"""
Return the full list of revision timestamps for this note in chronological order.
@type notebook_id: unicode
@param notebook_id: id of notebook the note is in
@type note_id: unicode
@param note_id: id of note in question
@type user_id: unicode or NoneType
@param user_id: id of current logged-in user (if any), determined by @grab_user_id
@rtype: json dict
@return: { 'revisions': revisionslist or None }
@raise Access_error: the current user doesn't have access to the given notebook or note
@raise Validation_error: one of the arguments is invalid
"""
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
note = self.__database.load( Note, note_id )
if note:
if note.notebook_id != notebook_id:
raise Access_error()
revisions = self.__database.select_many( unicode, note.sql_load_revisions() )
else:
revisions = None
return dict(
revisions = revisions,
)
@expose( view = Json )
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_id = Valid_id(),
@ -310,186 +318,78 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id, read_write = True ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not notebook:
raise Access_error()
self.__database.load( note_id, self.__scheduler.thread )
note = ( yield Scheduler.SLEEP )
note = self.__database.load( Note, note_id )
# check whether the provided note contents have been changed since the previous revision
def update_note( current_notebook, old_note ):
def update_note( current_notebook, old_note, startup ):
# the note hasn't been changed, so bail without updating it
if contents == old_note.contents:
if contents == old_note.contents and startup == old_note.startup:
new_revision = None
# the note has changed, so update it
else:
notebook.update_note( note, contents )
note.contents = contents
note.startup = startup
if startup:
if note.rank is None:
note.rank = self.__database.select_one( float, notebook.sql_highest_rank() ) + 1
else:
note.rank = None
new_revision = note.revision
return new_revision
# if the note is already in the given notebook, load it and update it
if note and note in notebook.notes:
self.__database.load( note_id, self.__scheduler.thread, previous_revision )
old_note = ( yield Scheduler.SLEEP )
if note and note.notebook_id == notebook.object_id:
old_note = self.__database.load( Note, note_id, previous_revision )
previous_revision = note.revision
new_revision = update_note( notebook, old_note )
new_revision = update_note( notebook, old_note, startup )
# the note is not already in the given notebook, so look for it in the trash
elif note and notebook.trash and note in notebook.trash.notes:
self.__database.load( note_id, self.__scheduler.thread, previous_revision )
old_note = ( yield Scheduler.SLEEP )
elif note and notebook.trash_id and note.notebook_id == notebook.trash_id:
old_note = self.__database.load( Note, note_id, previous_revision )
# undelete the note, putting it back in the given notebook
previous_revision = note.revision
notebook.trash.remove_note( note )
note.deleted_from = None
notebook.add_note( note )
new_revision = update_note( notebook, old_note )
note.notebook_id = notebook.object_id
note.deleted_from_id = None
new_revision = update_note( notebook, old_note, startup )
# otherwise, create a new note
else:
if startup:
rank = self.__database.select_one( float, notebook.sql_highest_rank() ) + 1
else:
rank = None
previous_revision = None
note = Note( note_id, contents )
notebook.add_note( note )
note = Note.create( note_id, contents, notebook_id = notebook.object_id, startup = startup, rank = rank )
new_revision = note.revision
if startup:
startup_changed = notebook.add_startup_note( note )
else:
startup_changed = notebook.remove_startup_note( note )
if new_revision or startup_changed:
self.__database.save( notebook, self.__scheduler.thread )
yield Scheduler.SLEEP
self.__users.update_storage( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
self.__database.save( user )
if new_revision:
self.__database.save( note, commit = False )
user = self.__users.update_storage( user_id, commit = False )
self.__database.commit()
else:
user = None
yield dict(
return dict(
new_revision = new_revision,
previous_revision = previous_revision,
storage_bytes = user and user.storage_bytes or 0,
)
@expose( view = Json )
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_id = Valid_id(),
user_id = Valid_id( none_okay = True ),
)
def add_startup_note( self, notebook_id, note_id, user_id ):
"""
Designate a particular note to be shown upon startup, e.g. whenever its notebook is displayed.
The given note must already be within this notebook.
@type notebook_id: unicode
@param notebook_id: id of notebook the note is in
@type note_id: unicode
@param note_id: id of note to show on startup
@type user_id: unicode or NoneType
@param user_id: id of current logged-in user (if any), determined by @grab_user_id
@rtype: json dict
@return: { 'storage_bytes': current storage usage by user }
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
if not notebook:
raise Access_error()
self.__database.load( note_id, self.__scheduler.thread )
note = ( yield Scheduler.SLEEP )
if note:
notebook.add_startup_note( note )
self.__database.save( notebook, self.__scheduler.thread )
yield Scheduler.SLEEP
self.__users.update_storage( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
self.__database.save( user )
yield dict( storage_bytes = user.storage_bytes )
else:
yield dict( storage_bytes = 0 )
@expose( view = Json )
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_id = Valid_id(),
user_id = Valid_id( none_okay = True ),
)
def remove_startup_note( self, notebook_id, note_id, user_id ):
"""
Prevent a particular note from being shown on startup, e.g. whenever its notebook is displayed.
The given note must already be within this notebook.
@type notebook_id: unicode
@param notebook_id: id of notebook the note is in
@type note_id: unicode
@param note_id: id of note to no longer show on startup
@type user_id: unicode or NoneType
@param user_id: id of current logged-in user (if any), determined by @grab_user_id
@rtype: json dict
@return: { 'storage_bytes': current storage usage by user }
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
if not notebook:
raise Access_error()
self.__database.load( note_id, self.__scheduler.thread )
note = ( yield Scheduler.SLEEP )
if note:
notebook.remove_startup_note( note )
self.__database.save( notebook, self.__scheduler.thread )
yield Scheduler.SLEEP
self.__users.update_storage( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
self.__database.save( user )
yield dict( storage_bytes = user.storage_bytes )
else:
yield dict( storage_bytes = 0 )
@expose( view = Json )
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_id = Valid_id(),
@ -512,42 +412,34 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id, read_write = True ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not notebook:
raise Access_error()
self.__database.load( note_id, self.__scheduler.thread )
note = ( yield Scheduler.SLEEP )
note = self.__database.load( Note, note_id )
if note:
notebook.remove_note( note )
if note and note.notebook_id == notebook_id:
if notebook.trash_id:
note.deleted_from_id = notebook_id
note.notebook_id = notebook.trash_id
note.startup = True
else:
note.notebook_id = None
if notebook.trash:
note.deleted_from = notebook.object_id
notebook.trash.add_note( note )
notebook.trash.add_startup_note( note )
self.__database.save( note, commit = False )
user = self.__users.update_storage( user_id, commit = False )
self.__database.commit()
self.__database.save( notebook, self.__scheduler.thread )
yield Scheduler.SLEEP
self.__users.update_storage( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
self.__database.save( user )
yield dict( storage_bytes = user.storage_bytes )
return dict( storage_bytes = user.storage_bytes )
else:
yield dict( storage_bytes = 0 )
return dict( storage_bytes = 0 )
@expose( view = Json )
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
note_id = Valid_id(),
@ -569,50 +461,39 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id, read_write = True ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not notebook:
raise Access_error()
self.__database.load( note_id, self.__scheduler.thread )
note = ( yield Scheduler.SLEEP )
note = self.__database.load( Note, note_id )
if note and notebook.trash:
if note and notebook.trash_id:
# if the note isn't deleted, and it's already in this notebook, just return
if note.deleted_from is None and notebook.lookup_note( note.object_id ):
yield dict( storage_bytes = 0 )
return
if note.deleted_from_id is None and note.notebook_id == notebook_id:
return dict( storage_bytes = 0 )
# if the note was deleted from a different notebook than the notebook given, raise
if note.deleted_from != notebook_id:
if note.deleted_from_id != notebook_id:
raise Access_error()
notebook.trash.remove_note( note )
note.notebook_id = note.deleted_from_id
note.deleted_from_id = None
note.startup = True
note.deleted_from = None
notebook.add_note( note )
notebook.add_startup_note( note )
self.__database.save( note, commit = False )
user = self.__users.update_storage( user_id, commit = False )
self.__database.commit()
self.__database.save( notebook, self.__scheduler.thread )
yield Scheduler.SLEEP
self.__users.update_storage( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
self.__database.save( user )
yield dict( storage_bytes = user.storage_bytes )
return dict( storage_bytes = user.storage_bytes )
else:
yield dict( storage_bytes = 0 )
return dict( storage_bytes = 0 )
@expose( view = Json )
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
user_id = Valid_id( none_okay = True ),
@ -632,40 +513,35 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id, read_write = True ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not notebook:
raise Access_error()
for note in notebook.notes:
notebook.remove_note( note )
notes = self.__database.select_many( Note, notebook.sql_load_notes() )
if notebook.trash:
note.deleted_from = notebook.object_id
notebook.trash.add_note( note )
notebook.trash.add_startup_note( note )
for note in notes:
if notebook.trash_id:
note.deleted_from_id = notebook_id
note.notebook_id = notebook.trash_id
note.startup = True
else:
note.notebook_id = None
self.__database.save( note, commit = False )
self.__database.save( notebook, self.__scheduler.thread )
yield Scheduler.SLEEP
self.__users.update_storage( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
self.__database.save( user )
user = self.__users.update_storage( user_id, commit = False )
self.__database.commit()
yield dict(
return dict(
storage_bytes = user.storage_bytes,
)
@expose( view = Json )
@strongly_expire
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
search_text = Valid_string( min = 0, max = 100 ),
@ -688,41 +564,39 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not notebook:
raise Access_error()
search_text = search_text.lower()
if len( search_text ) == 0:
return dict( notes = [] )
title_matches = []
content_matches = []
nuker = Html_nuker()
if len( search_text ) > 0:
for note in notebook.notes:
if note is None: continue
if search_text in nuker.nuke( note.title ).lower():
title_matches.append( note )
elif search_text in nuker.nuke( note.contents ).lower():
content_matches.append( note )
notes = self.__database.select_many( Note, notebook.sql_search_notes( search_text ) )
notes = title_matches + content_matches
# further narrow the search results by making sure notes still match after all HTML tags are
# stripped out
for note in notes:
if search_text in nuker.nuke( note.title ).lower():
title_matches.append( note )
elif search_text in nuker.nuke( note.contents ).lower():
content_matches.append( note )
yield dict(
notes = notes,
return dict(
notes = title_matches + content_matches,
)
@expose( view = Json )
@strongly_expire
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
user_id = Valid_id( none_okay = True ),
@ -740,29 +614,23 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not notebook:
raise Access_error()
notes = [ note for note in notebook.notes if note is not None and note.title is not None ]
notes.sort( lambda a, b: cmp( b.revision, a.revision ) )
notes = self.__database.select_many( Note, notebook.sql_load_notes() )
yield dict(
return dict(
notes = [ ( note.object_id, note.title ) for note in notes ]
)
@expose( view = Html_file )
@strongly_expire
@wait_for_update
@grab_user_id
@async
@update_client
@validate(
notebook_id = Valid_id(),
user_id = Valid_id( none_okay = True ),
@ -780,42 +648,18 @@ class Notebooks( object ):
@raise Access_error: the current user doesn't have access to the given notebook
@raise Validation_error: one of the arguments is invalid
"""
self.check_access( notebook_id, user_id, self.__scheduler.thread )
if not ( yield Scheduler.SLEEP ):
if not self.__users.check_access( user_id, notebook_id ):
raise Access_error()
self.__database.load( notebook_id, self.__scheduler.thread )
notebook = ( yield Scheduler.SLEEP )
notebook = self.__database.load( Notebook, notebook_id )
if not notebook:
raise Access_error()
normal_notes = list( set( notebook.notes ) - set( notebook.startup_notes ) )
normal_notes.sort( lambda a, b: -cmp( a.revision, b.revision ) )
yield dict(
startup_notes = self.__database.select_many( Note, notebook.sql_load_startup_notes() )
other_notes = self.__database.select_many( Note, notebook.sql_load_non_startup_notes() )
return dict(
notebook_name = notebook.name,
notes = [ note for note in notebook.startup_notes + normal_notes if note is not None ],
notes = startup_notes + other_notes,
)
@async
def check_access( self, notebook_id, user_id, callback ):
# check if the anonymous user has access to this notebook
self.__database.load( u"User anonymous", self.__scheduler.thread )
anonymous = ( yield Scheduler.SLEEP )
access = False
if anonymous.has_access( notebook_id ):
access = True
if user_id:
# check if the currently logged in user has access to this notebook
self.__database.load( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
if user and user.has_access( notebook_id ):
access = True
yield callback, access
scheduler = property( lambda self: self.__scheduler )

303
controller/Old_database.py Normal file
View File

@ -0,0 +1,303 @@
import re
import bsddb
import random
import cPickle
from cStringIO import StringIO
from copy import copy
from model.Persistent import Persistent
from Async import async
class Old_database( object ):
ID_BITS = 128 # number of bits within an id
ID_DIGITS = "0123456789abcdefghijklmnopqrstuvwxyz"
def __init__( self, scheduler, database_path = None ):
"""
Create a new database and return it.
@type scheduler: Scheduler
@param scheduler: scheduler to use
@type database_path: unicode
@param database_path: path to the database file
@rtype: Old_database
@return: database at the given path
"""
self.__scheduler = scheduler
self.__env = bsddb.db.DBEnv()
self.__env.open( None, bsddb.db.DB_CREATE | bsddb.db.DB_PRIVATE | bsddb.db.DB_INIT_MPOOL )
self.__db = bsddb.db.DB( self.__env )
self.__db.open( database_path, "database", bsddb.db.DB_HASH, bsddb.db.DB_CREATE )
self.__cache = {}
def __persistent_id( self, obj, skip = None ):
# save the object and return its persistent id
if obj != skip and isinstance( obj, Persistent ):
self.__save( obj )
return obj.object_id
# returning None indicates that the object should be pickled normally without using a persistent id
return None
@async
def save( self, obj, callback = None ):
"""
Save the given object to the database, including any objects that it references.
@type obj: Persistent
@param obj: object to save
@type callback: generator or NoneType
@param callback: generator to wakeup when the save is complete (optional)
"""
self.__save( obj )
yield callback
def __save( self, obj ):
# if this object's current revision is already saved, bail
revision_id = obj.revision_id()
if revision_id in self.__cache:
return
object_id = unicode( obj.object_id ).encode( "utf8" )
revision_id = unicode( obj.revision_id() ).encode( "utf8" )
secondary_id = obj.secondary_id and unicode( obj.full_secondary_id() ).encode( "utf8" ) or None
# update the cache with this saved object
self.__cache[ object_id ] = obj
self.__cache[ revision_id ] = copy( obj )
if secondary_id:
self.__cache[ secondary_id ] = obj
# set the pickler up to save persistent ids for every object except for the obj passed in, which
# will be pickled normally
buffer = StringIO()
pickler = cPickle.Pickler( buffer, protocol = -1 )
pickler.persistent_id = lambda o: self.__persistent_id( o, skip = obj )
# pickle the object and write it to the database under both its id key and its revision id key
pickler.dump( obj )
pickled = buffer.getvalue()
self.__db.put( object_id, pickled )
self.__db.put( revision_id, pickled )
# write the pickled object id (only) to the database under its secondary id
if secondary_id:
buffer = StringIO()
pickler = cPickle.Pickler( buffer, protocol = -1 )
pickler.persistent_id = lambda o: self.__persistent_id( o )
pickler.dump( obj )
self.__db.put( secondary_id, buffer.getvalue() )
self.__db.sync()
@async
def load( self, object_id, callback, revision = None ):
"""
Load the object corresponding to the given object id from the database, and yield the provided
callback generator with the loaded object as its argument, or None if the object_id is unknown.
If a revision is provided, a specific revision of the object will be loaded.
@type object_id: unicode
@param object_id: id of the object to load
@type callback: generator
@param callback: generator to send the loaded object to
@type revision: int or NoneType
@param revision: revision of the object to load (optional)
"""
obj = self.__load( object_id, revision )
yield callback, obj
def __load( self, object_id, revision = None ):
if revision is not None:
object_id = Persistent.make_revision_id( object_id, revision )
object_id = unicode( object_id ).encode( "utf8" )
# if the object corresponding to the given id has already been loaded, simply return it without
# loading it again
obj = self.__cache.get( object_id )
if obj is not None:
return obj
# grab the object for the given id from the database
buffer = StringIO()
unpickler = cPickle.Unpickler( buffer )
unpickler.persistent_load = self.__load
pickled = self.__db.get( object_id )
if pickled is None or pickled == "":
return None
buffer.write( pickled )
buffer.flush()
buffer.seek( 0 )
# unpickle the object and update the cache with this saved object
obj = unpickler.load()
if obj is None:
print "error unpickling %s: %s" % ( object_id, pickled )
return None
self.__cache[ unicode( obj.object_id ).encode( "utf8" ) ] = obj
self.__cache[ unicode( obj.revision_id() ).encode( "utf8" ) ] = copy( obj )
return obj
@async
def reload( self, object_id, callback = None ):
"""
Load and immediately save the object corresponding to the given object id or database key. This
is useful when the object has a __setstate__() method that performs some sort of schema
evolution operation.
@type object_id: unicode
@param object_id: id or key of the object to reload
@type callback: generator or NoneType
@param callback: generator to wakeup when the save is complete (optional)
"""
self.__reload( object_id )
yield callback
def __reload( self, object_id, revision = None ):
object_id = unicode( object_id ).encode( "utf8" )
# grab the object for the given id from the database
buffer = StringIO()
unpickler = cPickle.Unpickler( buffer )
unpickler.persistent_load = self.__load
pickled = self.__db.get( object_id )
if pickled is None or pickled == "":
return
buffer.write( pickled )
buffer.flush()
buffer.seek( 0 )
# unpickle the object. this should trigger __setstate__() if the object has such a method
obj = unpickler.load()
if obj is None:
print "error unpickling %s: %s" % ( object_id, pickled )
return
self.__cache[ object_id ] = obj
# set the pickler up to save persistent ids for every object except for the obj passed in, which
# will be pickled normally
buffer = StringIO()
pickler = cPickle.Pickler( buffer, protocol = -1 )
pickler.persistent_id = lambda o: self.__persistent_id( o, skip = obj )
# pickle the object and write it to the database under its id key
pickler.dump( obj )
pickled = buffer.getvalue()
self.__db.put( object_id, pickled )
self.__db.sync()
def size( self, object_id, revision = None ):
"""
Load the object corresponding to the given object id from the database, and return the size of
its pickled data in bytes. If a revision is provided, a specific revision of the object will be
loaded.
@type object_id: unicode
@param object_id: id of the object whose size should be returned
@type revision: int or NoneType
@param revision: revision of the object to load (optional)
"""
if revision is not None:
object_id = Persistent.make_revision_id( object_id, revision )
object_id = unicode( object_id ).encode( "utf8" )
pickled = self.__db.get( object_id )
if pickled is None or pickled == "":
return None
return len( pickled )
@staticmethod
def generate_id():
int_id = random.getrandbits( Old_database.ID_BITS )
base = len( Old_database.ID_DIGITS )
digits = []
while True:
index = int_id % base
digits.insert( 0, Old_database.ID_DIGITS[ index ] )
int_id = int_id / base
if int_id == 0:
break
return "".join( digits )
@async
def next_id( self, callback ):
"""
Generate the next available object id, and yield the provided callback generator with the
object id as its argument.
@type callback: generator
@param callback: generator to send the next available object id to
"""
# generate a random id, but on the off-chance that it collides with something else already in
# the database, try again
next_id = Old_database.generate_id()
while self.__db.get( next_id, default = None ) is not None:
next_id = Old_database.generate_id()
# save the next_id as a key in the database so that it's not handed out again to another client
self.__db[ next_id ] = ""
yield callback, next_id
@async
def close( self ):
"""
Shutdown the database.
"""
self.__db.close()
self.__env.close()
yield None
@async
def clear_cache( self ):
"""
Clear the memory object cache.
"""
self.__cache.clear()
yield None
scheduler = property( lambda self: self.__scheduler )
class Valid_id( object ):
"""
Validator for an object id.
"""
ID_PATTERN = re.compile( "^[%s]+$" % Old_database.ID_DIGITS )
def __init__( self, none_okay = False ):
self.__none_okay = none_okay
def __call__( self, value ):
if self.__none_okay and value in ( None, "None", "" ): return None
if self.ID_PATTERN.search( value ): return str( value )
raise ValueError()
class Valid_revision( object ):
"""
Validator for an object id.
"""
REVISION_PATTERN = re.compile( "^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d\.\d+$" )
def __init__( self, none_okay = False ):
self.__none_okay = none_okay
def __call__( self, value ):
if self.__none_okay and value in ( None, "None", "" ): return None
if self.REVISION_PATTERN.search( value ): return str( value )
raise ValueError()

43
controller/Root.py
View File

@ -1,13 +1,11 @@
import cherrypy
from Scheduler import Scheduler
from Expose import expose
from Validate import validate
from Async import async
from Notebooks import Notebooks
from Users import Users
from Updater import update_client, wait_for_update
from Database import Valid_id
from new_model.Note import Note
from view.Main_page import Main_page
from view.Json import Json
from view.Error_page import Error_page
@ -18,12 +16,10 @@ class Root( object ):
"""
The root of the controller hierarchy, corresponding to the "/" URL.
"""
def __init__( self, scheduler, database, settings ):
def __init__( self, database, settings ):
"""
Create a new Root object with the given settings.
@type scheduler: controller.Scheduler
@param scheduler: scheduler to use for asynchronous calls
@type database: controller.Database
@param database: database to use for all controllers
@type settings: dict
@ -31,18 +27,16 @@ class Root( object ):
@rtype: Root
@return: newly constructed Root
"""
self.__scheduler = scheduler
self.__database = database
self.__settings = settings
self.__users = Users(
scheduler,
database,
settings[ u"global" ].get( u"luminotes.http_url", u"" ),
settings[ u"global" ].get( u"luminotes.https_url", u"" ),
settings[ u"global" ].get( u"luminotes.support_email", u"" ),
settings[ u"global" ].get( u"luminotes.rate_plans", [] ),
)
self.__notebooks = Notebooks( scheduler, database, self.__users )
self.__notebooks = Notebooks( database, self.__users )
@expose()
def default( self, password_reset_id ):
@ -72,22 +66,19 @@ class Root( object ):
return dict()
# TODO: move this method to controller.Notebooks, and maybe give it a more sensible name
@expose( view = Json )
@wait_for_update
@async
@update_client
def next_id( self ):
"""
Return the next available database object id. This id is guaranteed to be unique to the
database.
Return the next available database object id for a new note. This id is guaranteed to be unique
among all existing notes.
@rtype: json dict
@return: { 'next_id': nextid }
"""
self.__database.next_id( self.__scheduler.thread )
next_id = ( yield Scheduler.SLEEP )
next_id = self.__database.next_id( Note )
yield dict(
return dict(
next_id = next_id,
)
@ -95,28 +86,20 @@ class Root( object ):
"""
CherryPy HTTP error handler, used to display page not found and generic error pages.
"""
support_email = self.__settings[ u"global" ].get( u"luminotes.support_email" )
if status == 404:
cherrypy.response.headerMap[ u"Status" ] = u"404 Not Found"
cherrypy.response.status = status
cherrypy.response.body = [ unicode( Not_found_page( self.__settings[ u"global" ].get( u"luminotes.support_email" ) ) ) ]
cherrypy.response.body = [ unicode( Not_found_page( support_email ) ) ]
return
import sys
# TODO: it'd be nice to send an email to myself with the traceback
import traceback
traceback.print_exc()
exc_info = sys.exc_info()
if exc_info:
message = exc_info[ 1 ].message
else:
message = None
cherrypy.response.body = [ unicode( Error_page( support_email ) ) ]
cherrypy.response.body = [ unicode( Error_page(
self.__settings[ u"global" ].get( u"luminotes.support_email" ),
message,
) ) ]
scheduler = property( lambda self: self.__scheduler )
database = property( lambda self: self.__database )
notebooks = property( lambda self: self.__notebooks )
users = property( lambda self: self.__users )

72
controller/Updater.py
View File

@ -1,72 +0,0 @@
from Queue import Queue, Empty
TIMEOUT_SECONDS = 10.0
def wait_for_update( function ):
"""
A decorator that passes a "queue" keyword arugment to its decorated function, calls the function,
and then blocks until an asynchronous response comes back via the Queue. When a response is
received, wait_for_update() returns it.
For this decorator to be useful, you should use it to decorate a function that fires off some
asynchronous action and then returns immediately. A typical way to accomplish this is by using
the @async decorator after the @wait_for_update decorator.
"""
def get_message( *args, **kwargs ):
queue = Queue()
kwargs[ "queue" ] = queue
function( *args, **kwargs )
# wait until a response is available in the queue, and then return that response
try:
return queue.get( block = True, timeout = TIMEOUT_SECONDS )
except Empty:
return { "error": u"A timeout occurred when processing your request. Please try again or contact support." }
return get_message
def update_client( function ):
"""
A decorator used to wrap a generator function so that its yielded values can be issued as
updates to the client. For this to work, the generator function must be invoked with a keyword
argument "queue" containing a Queue where the result can be put().
Also supports catching Validation_error exceptions and sending appropriate errors to the client.
Note that this decorator itself is a generator function and works by passing along next()/send()
calls to its decorated generator. Only yielded values that are dictionaries are sent to the
client via the provided queue. All other types of yielded values are in turn yielded by this
decorator itself.
"""
def put_message( *args, **kwargs ):
# look in the called function's kwargs for the queue where results should be sent
queue = kwargs.pop( "queue" )
try:
generator = function( *args, **kwargs )
message = None
while True:
result = generator.send( message )
if isinstance( result, dict ):
queue.put( result )
message = ( yield None )
else:
message = ( yield result )
except StopIteration:
return
except Exception, error:
# TODO: might be better to use view.Json instead of calling to_dict() manually
if hasattr( error, "to_dict" ):
result = error.to_dict()
queue.put( result )
else:
queue.put( { "error": u"An error occurred when processing your request. Please try again or contact support." } )
raise
return put_message

331
controller/Users.py
View File

@ -1,17 +1,15 @@
import re
import cherrypy
from pytz import utc
from datetime import datetime, timedelta
from model.User import User
from model.Notebook import Notebook
from model.Note import Note
from model.Password_reset import Password_reset
from Scheduler import Scheduler
from new_model.User import User
from new_model.Notebook import Notebook
from new_model.Note import Note
from new_model.Password_reset import Password_reset
from Expose import expose
from Validate import validate, Valid_string, Valid_bool, Validation_error
from Database import Valid_id
from Updater import update_client, wait_for_update
from Expire import strongly_expire
from Async import async
from view.Json import Json
from view.Main_page import Main_page
from view.Redeem_reset_note import Redeem_reset_note
@ -123,12 +121,10 @@ class Users( object ):
"""
Controller for dealing with users, corresponding to the "/users" URL.
"""
def __init__( self, scheduler, database, http_url, https_url, support_email, rate_plans ):
def __init__( self, database, http_url, https_url, support_email, rate_plans ):
"""
Create a new Users object.
@type scheduler: controller.Scheduler
@param scheduler: scheduler to use for asynchronous calls
@type database: controller.Database
@param database: database that users are stored in
@type http_url: unicode
@ -142,7 +138,6 @@ class Users( object ):
@rtype: Users
@return: newly constructed Users
"""
self.__scheduler = scheduler
self.__database = database
self.__http_url = http_url
self.__https_url = https_url
@ -151,9 +146,6 @@ class Users( object ):
@expose( view = Json )
@update_auth
@wait_for_update
@async
@update_client
@validate(
username = ( Valid_string( min = 1, max = 30 ), valid_username ),
password = Valid_string( min = 1, max = 30 ),
@ -184,45 +176,39 @@ class Users( object ):
if password != password_repeat:
raise Signup_error( u"The passwords you entered do not match. Please try again." )
self.__database.load( "User %s" % username, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
user = self.__database.select_one( User, User.sql_load_by_username( username ) )
if user is not None:
raise Signup_error( u"Sorry, that username is not available. Please try something else." )
# create a notebook for this user, along with a trash for that notebook
self.__database.next_id( self.__scheduler.thread )
trash_id = ( yield Scheduler.SLEEP )
trash = Notebook( trash_id, u"trash" )
trash_id = self.__database.next_id( Notebook, commit = False )
trash = Notebook.create( trash_id, u"trash" )
self.__database.save( trash, commit = False )
self.__database.next_id( self.__scheduler.thread )
notebook_id = ( yield Scheduler.SLEEP )
notebook = Notebook( notebook_id, u"my notebook", trash )
notebook_id = self.__database.next_id( Notebook, commit = False )
notebook = Notebook.create( notebook_id, u"my notebook", trash_id )
self.__database.save( notebook, commit = False )
# create a startup note for this user's notebook
self.__database.next_id( self.__scheduler.thread )
note_id = ( yield Scheduler.SLEEP )
note = Note( note_id, file( u"static/html/welcome to your wiki.html" ).read() )
notebook.add_note( note )
notebook.add_startup_note( note )
note_id = self.__database.next_id( Note, commit = False )
note_contents = file( u"static/html/welcome to your wiki.html" ).read()
note = Note.create( note_id, note_contents, notebook_id, startup = True, rank = 0 )
self.__database.save( note, commit = False )
# actually create the new user
self.__database.next_id( self.__scheduler.thread )
user_id = ( yield Scheduler.SLEEP )
user_id = self.__database.next_id( User, commit = False )
user = User.create( user_id, username, password, email_address )
self.__database.save( user, commit = False )
user = User( user_id, username, password, email_address, notebooks = [ notebook ] )
self.__database.save( user )
# add the new user to the user list
self.__database.load( u"User_list all", self.scheduler.thread )
user_list = ( yield Scheduler.SLEEP )
if user_list:
user_list.add_user( user )
self.__database.save( user_list )
# record the fact that the new user has access to their new notebook
self.__database.execute( user.sql_save_notebook( notebook_id, read_write = True ), commit = False )
self.__database.execute( user.sql_save_notebook( trash_id, read_write = True ), commit = False )
self.__database.commit()
redirect = u"/notebooks/%s" % notebook.object_id
yield dict(
return dict(
redirect = redirect,
authenticated = user,
)
@ -230,9 +216,6 @@ class Users( object ):
@expose()
@grab_user_id
@update_auth
@wait_for_update
@async
@update_client
def demo( self, user_id = None ):
"""
Create a new guest User for purposes of the demo. Start that user with their own Notebook and
@ -250,54 +233,51 @@ class Users( object ):
# if the user is already logged in as a guest, then just redirect to their existing demo
# notebook
if user_id:
self.__database.load( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
if user.username is None and len( user.notebooks ) > 0:
redirect = u"/notebooks/%s" % user.notebooks[ 0 ].object_id
yield dict( redirect = redirect )
return
user = self.__database.load( User, user_id )
first_notebook = self.__database.select_one( Notebook, user.sql_load_notebooks( parents_only = True ) )
if user.username is None and first_notebook:
redirect = u"/notebooks/%s" % first_notebook.object_id
return dict( redirect = redirect )
# create a demo notebook for this user, along with a trash for that notebook
self.__database.next_id( self.__scheduler.thread )
trash_id = ( yield Scheduler.SLEEP )
trash = Notebook( trash_id, u"trash" )
trash_id = self.__database.next_id( Notebook, commit = False )
trash = Notebook.create( trash_id, u"trash" )
self.__database.save( trash, commit = False )
self.__database.next_id( self.__scheduler.thread )
notebook_id = ( yield Scheduler.SLEEP )
notebook = Notebook( notebook_id, u"my notebook", trash )
notebook_id = self.__database.next_id( Notebook, commit = False )
notebook = Notebook.create( notebook_id, u"my notebook", trash_id )
self.__database.save( notebook, commit = False )
# create startup notes for this user's notebook
self.__database.next_id( self.__scheduler.thread )
note_id = ( yield Scheduler.SLEEP )
note = Note( note_id, file( u"static/html/this is a demo.html" ).read() )
notebook.add_note( note )
notebook.add_startup_note( note )
note_id = self.__database.next_id( Note, commit = False )
note_contents = file( u"static/html/this is a demo.html" ).read()
note = Note.create( note_id, note_contents, notebook_id, startup = True, rank = 0 )
self.__database.save( note, commit = False )
self.__database.next_id( self.__scheduler.thread )
note_id = ( yield Scheduler.SLEEP )
note = Note( note_id, file( u"static/html/welcome to your wiki.html" ).read() )
notebook.add_note( note )
notebook.add_startup_note( note )
note_id = self.__database.next_id( Note, commit = False )
note_contents = file( u"static/html/welcome to your wiki.html" ).read()
note = Note.create( note_id, note_contents, notebook_id, startup = True, rank = 1 )
self.__database.save( note, commit = False )
# actually create the new user. because this is just a demo user, we're not adding it to the User_list
self.__database.next_id( self.__scheduler.thread )
user_id = ( yield Scheduler.SLEEP )
# actually create the new user
user_id = self.__database.next_id( User, commit = False )
user = User.create( user_id, username = None, password = None, email_address = None )
self.__database.save( user, commit = False )
user = User( user_id, username = None, password = None, email_address = None, notebooks = [ notebook ] )
self.__database.save( user )
# record the fact that the new user has access to their new notebook
self.__database.execute( user.sql_save_notebook( notebook_id, read_write = True ), commit = False )
self.__database.execute( user.sql_save_notebook( trash_id, read_write = True ), commit = False )
self.__database.commit()
redirect = u"/notebooks/%s" % notebook.object_id
yield dict(
return dict(
redirect = redirect,
authenticated = user,
)
@expose( view = Json )
@update_auth
@wait_for_update
@async
@update_client
@validate(
username = ( Valid_string( min = 1, max = 30 ), valid_username ),
password = Valid_string( min = 1, max = 30 ),
@ -317,28 +297,26 @@ class Users( object ):
@raise Authentication_error: invalid username or password
@raise Validation_error: one of the arguments is invalid
"""
self.__database.load( "User %s" % username, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
user = self.__database.select_one( User, User.sql_load_by_username( username ) )
if user is None or user.check_password( password ) is False:
raise Authentication_error( u"Invalid username or password." )
first_notebook = self.__database.select_one( Notebook, user.sql_load_notebooks( parents_only = True ) )
# redirect to the user's first notebook (if any)
if len( user.notebooks ) > 0:
redirect = u"/notebooks/%s" % user.notebooks[ 0 ].object_id
if first_notebook:
redirect = u"/notebooks/%s" % first_notebook.object_id
else:
redirect = u"/"
yield dict(
return dict(
redirect = redirect,
authenticated = user,
)
@expose( view = Json )
@update_auth
@wait_for_update
@async
@update_client
def logout( self ):
"""
Deauthenticate the user and log them out of their current session.
@ -346,7 +324,7 @@ class Users( object ):
@rtype: json dict
@return: { 'redirect': url, 'deauthenticated': True }
"""
yield dict(
return dict(
redirect = self.__http_url + u"/",
deauthenticated = True,
)
@ -354,9 +332,6 @@ class Users( object ):
@expose( view = Json )
@strongly_expire
@grab_user_id
@wait_for_update
@async
@update_client
@validate(
include_startup_notes = Valid_bool(),
user_id = Valid_id( none_okay = True ),
@ -382,38 +357,42 @@ class Users( object ):
@raise Validation_error: one of the arguments is invalid
"""
# if there's no logged-in user, default to the anonymous user
self.__database.load( user_id or u"User anonymous", self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
anonymous = self.__database.select_one( User, User.sql_load_by_username( u"anonymous" ) )
if user_id:
user = self.__database.load( User, user_id )
else:
user = anonymous
if not user:
yield dict(
if not user or not anonymous:
return dict(
user = None,
notebooks = None,
http_url = u"",
)
return
# in addition to this user's own notebooks, add to that list the anonymous user's notebooks
self.__database.load( u"User anonymous", self.__scheduler.thread )
anonymous = ( yield Scheduler.SLEEP )
login_url = None
notebooks = self.__database.select_many( Notebook, anonymous.sql_load_notebooks() )
if user_id:
notebooks = anonymous.notebooks
notebooks += self.__database.select_many( Notebook, user.sql_load_notebooks() )
# if the user is not logged in, return a login URL
else:
notebooks = []
if len( anonymous.notebooks ) > 0:
anon_notebook = anonymous.notebooks[ 0 ]
login_note = anon_notebook.lookup_note_by_title( u"login" )
if len( notebooks ) > 0:
main_notebook = notebooks[ 0 ]
login_note = self.__database.select_one( Note, main_notebook.sql_load_note_by_title( u"login" ) )
if login_note:
login_url = "%s/notebooks/%s?note_id=%s" % ( self.__https_url, anon_notebook.object_id, login_note.object_id )
login_url = "%s/notebooks/%s?note_id=%s" % ( self.__https_url, main_notebook.object_id, login_note.object_id )
notebooks += user.notebooks
if include_startup_notes and len( notebooks ) > 0:
startup_notes = self.__database.select_many( Note, notebooks[ 0 ].sql_load_startup_notes() )
else:
startup_notes = []
yield dict(
return dict(
user = user,
notebooks = notebooks,
startup_notes = include_startup_notes and len( notebooks ) > 0 and notebooks[ 0 ].startup_notes or [],
startup_notes = startup_notes,
http_url = self.__http_url,
login_url = login_url,
rate_plan = ( user.rate_plan < len( self.__rate_plans ) ) and self.__rate_plans[ user.rate_plan ] or {},
@ -421,54 +400,64 @@ class Users( object ):
def calculate_storage( self, user ):
"""
Calculate total storage utilization for all notebooks and all notes of the given user,
including storage for all past revisions.
Calculate total storage utilization for all notes of the given user, including storage for all
past revisions.
@type user: User
@param user: user for which to calculate storage utilization
@rtype: int
@return: total bytes used for storage
"""
total_bytes = 0
return sum( self.__database.select_one( tuple, user.sql_calculate_storage() ), 0 )
def sum_revisions( obj ):
return \
self.__database.size( obj.object_id ) + \
sum( [ self.__database.size( obj.object_id, revision ) or 0 for revision in obj.revisions_list ], 0 )
def sum_notebook( notebook ):
return \
self.__database.size( notebook.object_id ) + \
sum( [ sum_revisions( note ) for note in notebook.notes ], 0 )
for notebook in user.notebooks:
total_bytes += sum_notebook( notebook )
if notebook.trash:
total_bytes += sum_notebook( notebook.trash )
return total_bytes
@async
def update_storage( self, user_id, callback = None ):
def update_storage( self, user_id, commit = True ):
"""
Calculate and record total storage utilization for the given user.
@type user_id: unicode or NoneType
@type user_id: unicode
@param user_id: id of user for which to calculate storage utilization
@type callback: generator or NoneType
@param callback: generator to wakeup when the update is complete (optional)
@type commit: bool
@param commit: True to automatically commit after the update
@rtype: model.User
@return: object of the user corresponding to user_id
"""
self.__database.load( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
user = self.__database.load( User, user_id )
if user:
user.storage_bytes = self.calculate_storage( user )
self.__database.save( user, commit )
yield callback, user
return user
def check_access( self, user_id, notebook_id, read_write = False ):
"""
Determine whether the given user has access to the given notebook.
@type user_id: unicode
@param user_id: id of user whose access to check
@type notebook_id: unicode
@param notebook_id: id of notebook to check access for
@type read_write: bool
@param read_write: True if read-write access is being checked, False if read-only access (defaults to False)
@rtype: bool
@return: True if the user has access
"""
# check if the anonymous user has access to this notebook
anonymous = self.__database.select_one( User, User.sql_load_by_username( u"anonymous" ) )
if self.__database.select_one( bool, anonymous.sql_has_access( notebook_id, read_write ) ):
return True
if user_id:
# check if the given user has access to this notebook
user = self.__database.load( User, user_id )
if user and self.__database.select_one( bool, user.sql_has_access( notebook_id ) ):
return True
return False
@expose( view = Json )
@wait_for_update
@async
@update_client
@validate(
email_address = ( Valid_string( min = 1, max = 60 ), valid_email_address ),
send_reset_button = unicode,
@ -491,19 +480,13 @@ class Users( object ):
from email import Message
# check whether there are actually any users with the given email address
self.__database.load( u"User_list all", self.scheduler.thread )
user_list = ( yield Scheduler.SLEEP )
users = self.__database.select_many( User, User.sql_load_by_email_address( email_address ) )
if not user_list:
raise Password_reset_error( "There was an error when sending your password reset email. Please contact %s." % self.__support_email )
users = [ user for user in user_list.users if user.email_address == email_address ]
if len( users ) == 0:
raise Password_reset_error( u"There are no Luminotes users with the email address %s" % email_address )
# record the sending of this reset email
self.__database.next_id( self.__scheduler.thread )
password_reset_id = ( yield Scheduler.SLEEP )
password_reset_id = self.__database.next_id( Password_reset, commit = False )
password_reset = Password_reset( password_reset_id, email_address )
self.__database.save( password_reset )
@ -527,15 +510,12 @@ class Users( object ):
server.sendmail( message[ u"from" ], [ email_address ], message.as_string() )
server.quit()
yield dict(
return dict(
message = u"Please check your inbox. A password reset email has been sent to %s" % email_address,
)
@expose( view = Main_page )
@strongly_expire
@wait_for_update
@async
@update_client
@validate(
password_reset_id = Valid_id(),
)
@ -550,43 +530,34 @@ class Users( object ):
@raise Password_reset_error: an error occured when redeeming the password reset, such as an expired link
@raise Validation_error: one of the arguments is invalid
"""
self.__database.load( u"User anonymous", self.__scheduler.thread )
anonymous = ( yield Scheduler.SLEEP )
anonymous = self.__database.select_one( User, User.sql_load_by_username( u"anonymous" ) )
if anonymous:
main_notebook = self.__database.select_one( Notebook, anonymous.sql_load_notebooks() )
if not anonymous or len( anonymous.notebooks ) == 0:
if not anonymous or not main_notebook:
raise Password_reset_error( "There was an error when completing your password reset. Please contact %s." % self.__support_email )
self.__database.load( password_reset_id, self.__scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
password_reset = self.__database.load( Password_reset, password_reset_id )
if not password_reset or datetime.now() - password_reset.revision > timedelta( hours = 25 ):
if not password_reset or datetime.now( tz = utc ) - password_reset.revision > timedelta( hours = 25 ):
raise Password_reset_error( "Your password reset link has expired. Please request a new password reset email." )
if password_reset.redeemed:
raise Password_reset_error( "Your password has already been reset. Please request a new password reset email." )
self.__database.load( u"User_list all", self.__scheduler.thread )
user_list = ( yield Scheduler.SLEEP )
if not user_list:
raise Password_reset_error( u"There are no Luminotes users with the email address %s" % password_reset.email_address )
# find the user(s) with the email address from the password reset request
matching_users = [ user for user in user_list.users if user.email_address == password_reset.email_address ]
matching_users = self.__database.select_many( User, User.sql_load_by_email_address( password_reset.email_address ) )
if len( matching_users ) == 0:
raise Password_reset_error( u"There are no Luminotes users with the email address %s" % password_reset.email_address )
yield dict(
notebook_id = anonymous.notebooks[ 0 ].object_id,
return dict(
notebook_id = main_notebook.object_id,
note_id = u"blank",
note_contents = unicode( Redeem_reset_note( password_reset_id, matching_users ) ),
)
@expose( view = Json )
@wait_for_update
@async
@update_client
def reset_password( self, password_reset_id, reset_button, **new_passwords ):
"""
Reset all the users with the provided passwords.
@ -606,27 +577,19 @@ class Users( object ):
except ValueError:
raise Validation_error( "password_reset_id", password_reset_id, id_validator, "is not a valid id" )
self.__database.load( password_reset_id, self.__scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
password_reset = self.__database.load( Password_reset, password_reset_id )
if not password_reset or datetime.now() - password_reset.revision > timedelta( hours = 25 ):
if not password_reset or datetime.now( tz = utc ) - password_reset.revision > timedelta( hours = 25 ):
raise Password_reset_error( "Your password reset link has expired. Please request a new password reset email." )
if password_reset.redeemed:
raise Password_reset_error( "Your password has already been reset. Please request a new password reset email." )
self.__database.load( u"User_list all", self.__scheduler.thread )
user_list = ( yield Scheduler.SLEEP )
if not user_list:
raise Password_reset_error( "There was an error when resetting your password. Please contact %s." % self.__support_email )
# find the user(s) with the email address from the password reset request
matching_users = [ user for user in user_list.users if user.email_address == password_reset.email_address ]
matching_users = self.__database.select_many( User, User.sql_load_by_email_address( password_reset.email_address ) )
allowed_user_ids = [ user.object_id for user in matching_users ]
# reset any passwords that are non-blank
users_to_reset = []
at_least_one_reset = False
for ( user_id, ( new_password, new_password_repeat ) ) in new_passwords.items():
if user_id not in allowed_user_ids:
raise Password_reset_error( "There was an error when resetting your password. Please contact %s." % self.__support_email )
@ -635,8 +598,7 @@ class Users( object ):
if new_password == u"" and new_password_repeat == u"":
continue
self.__database.load( user_id, self.__scheduler.thread )
user = ( yield Scheduler.SLEEP )
user = self.__database.load( User, user_id )
if not user:
raise Password_reset_error( "There was an error when resetting your password. Please contact %s." % self.__support_email )
@ -649,19 +611,16 @@ class Users( object ):
if len( new_password ) > 30:
raise Password_reset_error( u"Your password can be no longer than 30 characters." )
users_to_reset.append( ( user, new_password ) )
for ( user, new_password ) in users_to_reset:
at_least_one_reset = True
user.password = new_password
self.__database.save( user )
self.__database.save( user, commit = False )
# if all the new passwords provided are blank, bail
if not users_to_reset:
if not at_least_one_reset:
raise Password_reset_error( u"Please enter a new password. Or, if you already know your password, just click the login link above." )
password_reset.redeemed = True
self.__database.save( password_reset )
self.__database.save( password_reset, commit = False )
self.__database.commit()
yield dict( redirect = u"/" )
scheduler = property( lambda self: self.__scheduler )
return dict( redirect = u"/" )

2
controller/Validate.py
View File

@ -32,7 +32,7 @@ class Validation_error( Exception ):
def to_dict( self ):
return dict(
error = u"The %s %s." % ( self.__name, self.__message ),
error = u"The %s %s." % ( self.__name.replace( u"_", " " ), self.__message ),
name = self.__name,
value = self.__value,
)

72
controller/test/Stub_database.py Normal file
View File

@ -0,0 +1,72 @@
from copy import copy
class Stub_database( object ):
def __init__( self, connection = None ):
# map of object id to list of saved objects (presumably in increasing order of revisions)
self.objects = {}
self.user_notebook = {} # map of user_id to ( notebook_id, read_write )
self.__next_id = 0
def save( self, obj, commit = False ):
if obj.object_id in self.objects:
self.objects[ obj.object_id ].append( copy( obj ) )
else:
self.objects[ obj.object_id ] = [ copy( obj ) ]
def load( self, Object_type, object_id, revision = None ):
obj_list = self.objects.get( object_id )
if not obj_list:
return None
# if a particular revision wasn't requested, just return the most recently saved object
# matching the given object_id
if revision is None:
if not isinstance( obj_list[ -1 ], Object_type ):
return None
return copy( obj_list[ -1 ] )
# a particular revision was requested, so pick it out of the objects matching the given id
matching_objs = [ obj for obj in obj_list if str( obj.revision ) == str( revision ) ]
if len( matching_objs ) > 0:
if not isinstance( matching_objs[ -1 ], Object_type ):
return None
return copy( matching_objs[ -1 ] )
return None
def select_one( self, Object_type, sql_command ):
if callable( sql_command ):
result = sql_command( self )
if isinstance( result, list ):
if len( result ) == 0: return None
return result[ 0 ]
return result
raise NotImplementedError( sql_command )
def select_many( self, Object_type, sql_command ):
if callable( sql_command ):
result = sql_command( self )
if isinstance( result, list ):
return result
return [ result ]
raise NotImplementedError( sql_command )
def execute( self, sql_command, commit = False ):
if callable( sql_command ):
return sql_command( self )
raise NotImplementedError( sql_command )
def next_id( self, Object_type, commit = True ):
self.__next_id += 1
return unicode( self.__next_id )
def commit( self ):
pass
def close( self ):
pass

79
controller/test/Stub_object.py Normal file
View File

@ -0,0 +1,79 @@
from datetime import datetime
from new_model.Persistent import Persistent, quote
def notz_quote( value ):
"""
Apparently, pysqlite2 chokes on timestamps that have a timezone when reading them out of the
database, so for purposes of the unit tests, strip off the timezone on all datetime objects.
"""
if isinstance( value, datetime ):
value = value.replace( tzinfo = None )
return quote( value )
class Stub_object( Persistent ):
def __init__( self, object_id, revision = None, value = None, value2 = None ):
Persistent.__init__( self, object_id, revision )
self.__value = value
self.__value2 = value2
@staticmethod
def sql_load( object_id, revision = None ):
if revision:
return "select * from stub_object where id = %s and revision = %s;" % ( quote( object_id ), notz_quote( revision ) )
return "select * from stub_object where id = %s order by revision desc limit 1;" % quote( object_id )
@staticmethod
def sql_id_exists( object_id, revision = None ):
if revision:
return "select id from stub_object where id = %s and revision = %s;" % ( quote( object_id ), notz_quote( revision ) )
return "select id from stub_object where id = %s order by revision desc limit 1;" % quote( object_id )
def sql_exists( self ):
return Stub_object.sql_id_exists( self.object_id, self.revision )
def sql_create( self ):
return \
"insert into stub_object ( id, revision, value, value2 ) " + \
"values ( %s, %s, %s, %s );" % \
( quote( self.object_id ), notz_quote( self.revision ), quote( self.__value ),
quote( self.__value2 ) )
def sql_update( self ):
return self.sql_create()
@staticmethod
def sql_load_em_all():
return "select * from stub_object;"
@staticmethod
def sql_create_table():
return \
"""
create table stub_object (
id text not null,
revision timestamp with time zone not null,
value integer,
value2 integer
);
"""
@staticmethod
def sql_tuple():
return "select 1, 2;"
def __set_value( self, value ):
self.update_revision()
self.__value = value
def __set_value2( self, value2 ):
self.update_revision()
self.__value2 = value2
value = property( lambda self: self.__value, __set_value )
value2 = property( lambda self: self.__value2, __set_value2 )

177
controller/test/Test_controller.py
View File

@ -1,18 +1,178 @@
import cherrypy
from controller.Scheduler import Scheduler
from controller.Database import Database
from controller.test.Stub_view import Stub_view
from Stub_database import Stub_database
from Stub_view import Stub_view
from config import Common
from datetime import datetime
from StringIO import StringIO
class Test_controller( object ):
def __init__( self ):
from new_model.User import User
from new_model.Notebook import Notebook
from new_model.Note import Note
# Since Stub_database isn't a real database and doesn't know SQL, replace some of the
# SQL-returning methods in User, Note, and Notebook to return functions that manipulate data in
# Stub_database directly instead. This is all a little fragile, but it's better than relying on
# the presence of a real database for unit tests.
def sql_save_notebook( self, notebook_id, read_write, database ):
if self.object_id in database.user_notebook:
database.user_notebook[ self.object_id ].append( ( notebook_id, read_write ) )
else:
database.user_notebook[ self.object_id ] = [ ( notebook_id, read_write ) ]
User.sql_save_notebook = lambda self, notebook_id, read_write = False: \
lambda database: sql_save_notebook( self, notebook_id, read_write, database )
def sql_load_notebooks( self, parents_only, database ):
notebooks = []
notebook_tuples = database.user_notebook.get( self.object_id )
if not notebook_tuples: return None
for notebook_tuple in notebook_tuples:
( notebook_id, read_write ) = notebook_tuple
notebook = database.objects.get( notebook_id )[ -1 ]
notebook._Notebook__read_write = read_write
if parents_only and notebook.trash_id is None:
continue
notebooks.append( notebook )
return notebooks
User.sql_load_notebooks = lambda self, parents_only = False: \
lambda database: sql_load_notebooks( self, parents_only, database )
def sql_load_by_username( username, database ):
users = []
for ( object_id, obj_list ) in database.objects.items():
obj = obj_list[ -1 ]
if isinstance( obj, User ) and obj.username == username:
users.append( obj )
return users
User.sql_load_by_username = staticmethod( lambda username: \
lambda database: sql_load_by_username( username, database ) )
def sql_load_by_email_address( email_address, database ):
users = []
for ( object_id, obj_list ) in database.objects.items():
obj = obj_list[ -1 ]
if isinstance( obj, User ) and obj.email_address == email_address:
users.append( obj )
return users
User.sql_load_by_email_address = staticmethod( lambda email_address: \
lambda database: sql_load_by_email_address( email_address, database ) )
def sql_calculate_storage( self, database ):
return ( 17, 3, 4, 22 ) # rather than actually calculating anything, return arbitrary numbers
User.sql_calculate_storage = lambda self: \
lambda database: sql_calculate_storage( self, database )
def sql_has_access( self, notebook_id, read_write, database ):
for ( user_id, notebook_tuples ) in database.user_notebook.items():
for notebook_tuple in notebook_tuples:
( db_notebook_id, db_read_write ) = notebook_tuple
if self.object_id == user_id and notebook_id == db_notebook_id:
if read_write is True and db_read_write is False:
return False
return True
return False
User.sql_has_access = lambda self, notebook_id, read_write = False: \
lambda database: sql_has_access( self, notebook_id, read_write, database )
def sql_load_revisions( self, database ):
note_list = database.objects.get( self.object_id )
if not note_list: return None
revisions = [ note.revision for note in note_list ]
return revisions
Note.sql_load_revisions = lambda self: \
lambda database: sql_load_revisions( self, database )
def sql_load_notes( self, database ):
notes = []
for ( object_id, obj_list ) in database.objects.items():
obj = obj_list[ -1 ]
if isinstance( obj, Note ) and obj.notebook_id == self.object_id:
notes.append( obj )
notes.sort( lambda a, b: -cmp( a.revision, b.revision ) )
return notes
Notebook.sql_load_notes = lambda self: \
lambda database: sql_load_notes( self, database )
def sql_load_startup_notes( self, database ):
notes = []
for ( object_id, obj_list ) in database.objects.items():
obj = obj_list[ -1 ]
if isinstance( obj, Note ) and obj.notebook_id == self.object_id and obj.startup:
notes.append( obj )
return notes
Notebook.sql_load_startup_notes = lambda self: \
lambda database: sql_load_startup_notes( self, database )
def sql_load_note_by_title( self, title, database ):
notes = []
for ( object_id, obj_list ) in database.objects.items():
obj = obj_list[ -1 ]
if isinstance( obj, Note ) and obj.notebook_id == self.object_id and obj.title == title:
notes.append( obj )
return notes
Notebook.sql_load_note_by_title = lambda self, title: \
lambda database: sql_load_note_by_title( self, title, database )
def sql_search_notes( self, search_text, database ):
notes = []
search_text = search_text.lower()
for ( object_id, obj_list ) in database.objects.items():
obj = obj_list[ -1 ]
if isinstance( obj, Note ) and obj.notebook_id == self.object_id and \
search_text in obj.contents.lower():
notes.append( obj )
return notes
Notebook.sql_search_notes = lambda self, search_text: \
lambda database: sql_search_notes( self, search_text, database )
def sql_highest_rank( self, database ):
max_rank = -1
for ( object_id, obj_list ) in database.objects.items():
obj = obj_list[ -1 ]
if isinstance( obj, Note ) and obj.notebook_id == self.object_id and obj.rank > max_rank:
max_rank = obj.rank
return max_rank
Notebook.sql_highest_rank = lambda self: \
lambda database: sql_highest_rank( self, database )
def setUp( self ):
from controller.Root import Root
cherrypy.lowercase_api = True
self.scheduler = Scheduler()
self.database = Database( self.scheduler, database_path = None )
self.database = Stub_database()
self.settings = {
u"global": {
u"luminotes.http_url" : u"http://luminotes.com",
@ -33,7 +193,7 @@ class Test_controller( object ):
},
}
cherrypy.root = Root( self.scheduler, self.database, self.settings )
cherrypy.root = Root( self.database, self.settings )
cherrypy.config.update( Common.settings )
cherrypy.config.update( { u"server.log_to_screen": False } )
cherrypy.server.start( init_only = True, server_class = None )
@ -45,7 +205,6 @@ class Test_controller( object ):
def tearDown( self ):
cherrypy.server.stop()
self.scheduler.shutdown()
def http_get( self, http_path, headers = None, session_id = None, pretend_https = False ):
"""
@ -64,7 +223,7 @@ class Test_controller( object ):
proxy_ip = self.settings[ "global" ].get( u"luminotes.http_proxy_ip" )
request = cherrypy.server.request( ( proxy_ip, 1234 ), u"127.0.0.5" )
response = request.run( "GET %s HTTP/1.0" % http_path, headers = headers, rfile = StringIO() )
response = request.run( "GET %s HTTP/1.0" % str( http_path ), headers = headers, rfile = StringIO() )
session_id = response.simple_cookie.get( u"session_id" )
if session_id: session_id = session_id.value
@ -103,7 +262,7 @@ class Test_controller( object ):
headers.append( ( u"Cookie", "session_id=%s" % session_id ) ) # will break if unicode is used for the value
request = cherrypy.server.request( ( u"127.0.0.1", 1234 ), u"127.0.0.5" )
response = request.run( "POST %s HTTP/1.0" % http_path, headers = headers, rfile = StringIO( post_data ) )
response = request.run( "POST %s HTTP/1.0" % str( http_path ), headers = headers, rfile = StringIO( post_data ) )
session_id = response.simple_cookie.get( u"session_id" )
if session_id: session_id = session_id.value

415
controller/test/Test_database.py
View File

@ -1,323 +1,188 @@
from pytz import utc
from pysqlite2 import dbapi2 as sqlite
from datetime import datetime
from Stub_object import Stub_object
from controller.Database import Database
from controller.Scheduler import Scheduler
from model.Persistent import Persistent
class Some_object( Persistent ):
def __init__( self, object_id, value, value2 = None, secondary_id = None ):
Persistent.__init__( self, object_id, secondary_id )
self.__value = value
self.__value2 = value2
def __set_value( self, value ):
self.update_revision()
self.__value = value
def __set_value2( self, value2 ):
self.update_revision()
self.__value2 = value2
value = property( lambda self: self.__value, __set_value )
value2 = property( lambda self: self.__value2, __set_value2 )
class Test_database( object ):
def __init__( self, clear_cache = True ):
self.clear_cache = clear_cache
def setUp( self ):
self.scheduler = Scheduler()
self.database = Database( self.scheduler )
next_id = None
# make an in-memory sqlite database to use in place of PostgreSQL during testing
self.connection = sqlite.connect( ":memory:", detect_types = sqlite.PARSE_DECLTYPES | sqlite.PARSE_COLNAMES )
cursor = self.connection.cursor()
cursor.execute( Stub_object.sql_create_table() )
self.database = Database( self.connection )
def tearDown( self ):
self.database.close()
self.scheduler.shutdown()
def test_save_and_load( self ):
def gen():
basic_obj = Some_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
self.database.load( basic_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
self.database.save( basic_obj )
obj = self.database.load( Stub_object, basic_obj.object_id )
assert obj.object_id == basic_obj.object_id
assert obj.revision == original_revision
assert obj.revisions_list == [ original_revision ]
assert obj.value == basic_obj.value
assert obj.object_id == basic_obj.object_id
assert obj.revision.replace( tzinfo = utc ) == original_revision
assert obj.value == basic_obj.value
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
def test_save_and_load_without_commit( self ):
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
def test_complex_save_and_load( self ):
def gen():
basic_obj = Some_object( object_id = "7", value = 2 )
basic_original_revision = basic_obj.revision
complex_obj = Some_object( object_id = "6", value = basic_obj )
complex_original_revision = complex_obj.revision
self.database.save( basic_obj, commit = False )
self.connection.rollback() # if commit wasn't called, this should back out the save
obj = self.database.load( Stub_object, basic_obj.object_id )
self.database.save( complex_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
self.database.load( complex_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
if self.clear_cache: self.database.clear_cache()
assert obj == None
assert obj.object_id == complex_obj.object_id
assert obj.revision == complex_original_revision
assert obj.revisions_list == [ complex_original_revision ]
assert obj.value.object_id == basic_obj.object_id
assert obj.value.value == basic_obj.value
assert obj.value.revision == basic_original_revision
assert obj.value.revisions_list == [ basic_original_revision ]
def test_save_and_load_with_explicit_commit( self ):
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
self.database.load( basic_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
self.database.save( basic_obj, commit = False )
self.database.commit()
self.connection.rollback() # should have no effect because of the call to commit
obj = self.database.load( Stub_object, basic_obj.object_id )
assert obj.object_id == basic_obj.object_id
assert obj.value == basic_obj.value
assert obj.revision == basic_original_revision
assert obj.revisions_list == [ basic_original_revision ]
assert obj.object_id == basic_obj.object_id
assert obj.revision.replace( tzinfo = utc ) == original_revision
assert obj.value == basic_obj.value
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
def test_select_one( self ):
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
def test_save_and_load_by_secondary( self ):
def gen():
basic_obj = Some_object( object_id = "5", value = 1, secondary_id = u"foo" )
original_revision = basic_obj.revision
self.database.save( basic_obj )
obj = self.database.select_one( Stub_object, Stub_object.sql_load( basic_obj.object_id ) )
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
self.database.load( u"Some_object foo", self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
assert obj.object_id == basic_obj.object_id
assert obj.revision.replace( tzinfo = utc ) == original_revision
assert obj.value == basic_obj.value
assert obj.object_id == basic_obj.object_id
assert obj.value == basic_obj.value
assert obj.revision == original_revision
assert obj.revisions_list == [ original_revision ]
def test_select_one_tuple( self ):
obj = self.database.select_one( tuple, Stub_object.sql_tuple() )
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
assert len( obj ) == 2
assert obj[ 0 ] == 1
assert obj[ 1 ] == 2
def test_duplicate_save_and_load( self ):
def gen():
basic_obj = Some_object( object_id = "9", value = 3 )
basic_original_revision = basic_obj.revision
complex_obj = Some_object( object_id = "8", value = basic_obj, value2 = basic_obj )
complex_original_revision = complex_obj.revision
def test_select_many( self ):
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
basic_obj2 = Stub_object( object_id = "6", value = 2 )
original_revision2 = basic_obj2.revision
self.database.save( complex_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
self.database.load( complex_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
if self.clear_cache: self.database.clear_cache()
self.database.save( basic_obj )
self.database.save( basic_obj2 )
objs = self.database.select_many( Stub_object, Stub_object.sql_load_em_all() )
assert obj.object_id == complex_obj.object_id
assert obj.revision == complex_original_revision
assert obj.revisions_list == [ complex_original_revision ]
assert len( objs ) == 2
assert objs[ 0 ].object_id == basic_obj.object_id
assert objs[ 0 ].revision.replace( tzinfo = utc ) == original_revision
assert objs[ 0 ].value == basic_obj.value
assert objs[ 1 ].object_id == basic_obj2.object_id
assert objs[ 1 ].revision.replace( tzinfo = utc ) == original_revision2
assert objs[ 1 ].value == basic_obj2.value
assert obj.value.object_id == basic_obj.object_id
assert obj.value.value == basic_obj.value
assert obj.value.revision == basic_original_revision
assert obj.value.revisions_list == [ basic_original_revision ]
def test_select_many_tuples( self ):
objs = self.database.select_many( tuple, Stub_object.sql_tuple() )
assert obj.value2.object_id == basic_obj.object_id
assert obj.value2.value == basic_obj.value
assert obj.value2.revision == basic_original_revision
assert obj.value2.revisions_list == [ basic_original_revision ]
assert len( objs ) == 1
assert len( objs[ 0 ] ) == 2
assert objs[ 0 ][ 0 ] == 1
assert objs[ 0 ][ 1 ] == 2
assert obj.value == obj.value2
def test_select_many_with_no_matches( self ):
objs = self.database.select_many( Stub_object, Stub_object.sql_load_em_all() )
self.database.load( basic_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
assert obj.object_id == basic_obj.object_id
assert obj.value == basic_obj.value
assert obj.revision == basic_original_revision
assert obj.revisions_list == [ basic_original_revision ]
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
assert len( objs ) == 0
def test_save_and_load_revision( self ):
def gen():
basic_obj = Some_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
self.database.save( basic_obj )
basic_obj.value = 2
basic_obj.value = 2
self.database.save( basic_obj )
obj = self.database.load( Stub_object, basic_obj.object_id )
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
self.database.load( basic_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
if self.clear_cache: self.database.clear_cache()
assert obj.object_id == basic_obj.object_id
assert obj.revision.replace( tzinfo = utc ) == basic_obj.revision
assert obj.value == basic_obj.value
assert obj.object_id == basic_obj.object_id
assert obj.revision == basic_obj.revision
assert obj.revisions_list == [ original_revision, basic_obj.revision ]
assert obj.value == basic_obj.value
revised = self.database.load( Stub_object, basic_obj.object_id, revision = original_revision )
self.database.load( basic_obj.object_id, self.scheduler.thread, revision = original_revision )
revised = ( yield Scheduler.SLEEP )
assert revised.object_id == basic_obj.object_id
assert revised.value == 1
assert revised.revision.replace( tzinfo = utc ) == original_revision
assert revised.object_id == basic_obj.object_id
assert revised.value == 1
assert revised.revision == original_revision
assert id( obj.revisions_list ) != id( revised.revisions_list )
assert revised.revisions_list == [ original_revision ]
def test_execute( self ):
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
self.database.execute( basic_obj.sql_create() )
obj = self.database.load( Stub_object, basic_obj.object_id )
assert obj.object_id == basic_obj.object_id
assert obj.revision.replace( tzinfo = utc ) == original_revision
assert obj.value == basic_obj.value
def test_execute_without_commit( self ):
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
self.database.execute( basic_obj.sql_create(), commit = False )
self.connection.rollback()
obj = self.database.load( Stub_object, basic_obj.object_id )
assert obj == None
def test_execute_with_explicit_commit( self ):
basic_obj = Stub_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
self.database.execute( basic_obj.sql_create(), commit = False )
self.database.commit()
obj = self.database.load( Stub_object, basic_obj.object_id )
assert obj.object_id == basic_obj.object_id
assert obj.revision.replace( tzinfo = utc ) == original_revision
assert obj.value == basic_obj.value
def test_load_unknown( self ):
def gen():
basic_obj = Some_object( object_id = "5", value = 1 )
self.database.load( basic_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
assert obj == None
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
def test_reload( self ):
def gen():
basic_obj = Some_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
def setstate( self, state ):
state[ "_Some_object__value" ] = 55
self.__dict__.update( state )
Some_object.__setstate__ = setstate
self.database.reload( basic_obj.object_id, self.scheduler.thread )
yield Scheduler.SLEEP
delattr( Some_object, "__setstate__" )
if self.clear_cache: self.database.clear_cache()
self.database.load( basic_obj.object_id, self.scheduler.thread )
obj = ( yield Scheduler.SLEEP )
assert obj.object_id == basic_obj.object_id
assert obj.value == 55
assert obj.revision == original_revision
assert obj.revisions_list == [ original_revision ]
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
def test_reload_revision( self ):
def gen():
basic_obj = Some_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
original_revision_id = basic_obj.revision_id()
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
basic_obj.value = 2
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
def setstate( self, state ):
state[ "_Some_object__value" ] = 55
self.__dict__.update( state )
Some_object.__setstate__ = setstate
self.database.reload( original_revision_id, self.scheduler.thread )
yield Scheduler.SLEEP
delattr( Some_object, "__setstate__" )
if self.clear_cache: self.database.clear_cache()
self.database.load( basic_obj.object_id, self.scheduler.thread, revision = original_revision )
obj = ( yield Scheduler.SLEEP )
assert obj.object_id == basic_obj.object_id
assert obj.revision == original_revision
assert obj.revisions_list == [ original_revision ]
assert obj.value == 55
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
def test_size( self ):
def gen():
basic_obj = Some_object( object_id = "5", value = 1 )
original_revision = basic_obj.revision
self.database.save( basic_obj, self.scheduler.thread )
yield Scheduler.SLEEP
if self.clear_cache: self.database.clear_cache()
size = self.database.size( basic_obj.object_id )
from cPickle import Pickler
from StringIO import StringIO
buffer = StringIO()
pickler = Pickler( buffer, protocol = -1 )
pickler.dump( basic_obj )
expected_size = len( buffer.getvalue() )
# as long as the size is close to the expected size, that's fine
assert abs( size - expected_size ) < 10
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
basic_obj = Stub_object( object_id = "5", value = 1 )
obj = self.database.load( Stub_object, basic_obj.object_id )
assert obj == None
def test_next_id( self ):
def gen():
self.database.next_id( self.scheduler.thread )
next_id = ( yield Scheduler.SLEEP )
assert next_id
prev_ids = [ next_id ]
next_id = self.database.next_id( Stub_object )
assert next_id
assert self.database.load( Stub_object, next_id )
prev_ids = [ next_id ]
self.database.next_id( self.scheduler.thread )
next_id = ( yield Scheduler.SLEEP )
assert next_id
assert next_id not in prev_ids
prev_ids.append( next_id )
next_id = self.database.next_id( Stub_object )
assert next_id
assert next_id not in prev_ids
assert self.database.load( Stub_object, next_id )
prev_ids.append( next_id )
self.database.next_id( self.scheduler.thread )
next_id = ( yield Scheduler.SLEEP )
assert next_id
assert next_id not in prev_ids
next_id = self.database.next_id( Stub_object )
assert next_id
assert next_id not in prev_ids
assert self.database.load( Stub_object, next_id )
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
def test_next_id_without_commit( self ):
next_id = self.database.next_id( Stub_object, commit = False )
self.connection.rollback()
assert self.database.load( Stub_object, next_id ) == None
class Test_database_without_clearing_cache( Test_database ):
def __init__( self ):
Test_database.__init__( self, clear_cache = False )
def test_next_id_with_explit_commit( self ):
next_id = self.database.next_id( Stub_object, commit = False )
self.database.commit()
assert next_id
assert self.database.load( Stub_object, next_id )

640
controller/test/Test_notebooks.py
View File

File diff suppressed because it is too large Load Diff

10
controller/test/Test_root.py
View File

@ -1,5 +1,5 @@
import cherrypy
from model.User import User
from new_model.User import User
from controller.Scheduler import Scheduler
from Test_controller import Test_controller
@ -14,13 +14,7 @@ class Test_root( Test_controller ):
self.user = None
self.session_id = None
thread = self.make_user()
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def make_user( self ):
self.database.next_id( self.scheduler.thread )
self.user = User( ( yield Scheduler.SLEEP ), self.username, self.password, self.email_address, [] )
self.user = User.create( self.database.next_id( User ), self.username, self.password, self.email_address )
self.database.save( self.user )
def test_index( self ):

395
controller/test/Test_users.py
View File

@ -1,15 +1,15 @@
import re
import cherrypy
import smtplib
from pytz import utc
from datetime import datetime, timedelta
from nose.tools import raises
from Test_controller import Test_controller
from Stub_smtp import Stub_smtp
from controller.Scheduler import Scheduler
from model.User import User
from model.Notebook import Notebook
from model.Note import Note
from model.User_list import User_list
from new_model.User import User
from new_model.Notebook import Notebook
from new_model.Note import Note
from new_model.Password_reset import Password_reset
class Test_users( Test_controller ):
@ -32,46 +32,42 @@ class Test_users( Test_controller ):
self.anonymous = None
self.notebooks = None
thread = self.make_users()
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
self.make_users()
def make_users( self ):
self.database.next_id( self.scheduler.thread )
notebook_id1 = ( yield Scheduler.SLEEP )
self.database.next_id( self.scheduler.thread )
notebook_id2 = ( yield Scheduler.SLEEP )
notebook_id1 = self.database.next_id( Notebook )
notebook_id2 = self.database.next_id( Notebook )
trash_id1 = self.database.next_id( Notebook )
trash_id2 = self.database.next_id( Notebook )
self.notebooks = [
Notebook( notebook_id1, u"my notebook" ),
Notebook( notebook_id2, u"my other notebook" ),
Notebook.create( notebook_id1, u"my notebook", trash_id = trash_id1 ),
Notebook.create( notebook_id2, u"my other notebook", trash_id = trash_id2 ),
]
self.database.save( self.notebooks[ 0 ] )
self.database.save( self.notebooks[ 1 ] )
self.database.next_id( self.scheduler.thread )
self.anon_notebook = Notebook( ( yield Scheduler.SLEEP ), u"anon notebook" )
self.database.next_id( self.scheduler.thread )
self.startup_note = Note( ( yield Scheduler.SLEEP ), u"<h3>login</h3>" )
self.anon_notebook.add_note( self.startup_note )
self.anon_notebook.add_startup_note( self.startup_note )
self.anon_notebook = Notebook.create( self.database.next_id( Notebook ), u"anon notebook" )
self.database.save( self.anon_notebook )
self.startup_note = Note.create(
self.database.next_id( Note ), u"<h3>login</h3>",
notebook_id = self.anon_notebook.object_id, startup = True,
)
self.database.save( self.startup_note )
self.database.next_id( self.scheduler.thread )
self.user = User( ( yield Scheduler.SLEEP ), self.username, self.password, self.email_address, self.notebooks )
self.database.next_id( self.scheduler.thread )
self.user2 = User( ( yield Scheduler.SLEEP ), self.username2, self.password2, self.email_address2 )
self.database.next_id( self.scheduler.thread )
self.anonymous = User( ( yield Scheduler.SLEEP ), u"anonymous", None, None, [ self.anon_notebook ] )
self.user = User.create( self.database.next_id( User ), self.username, self.password, self.email_address )
self.database.save( self.user, commit = False )
self.database.execute( self.user.sql_save_notebook( notebook_id1, read_write = True ), commit = False )
self.database.execute( self.user.sql_save_notebook( notebook_id2, read_write = True ), commit = False )
self.database.next_id( self.scheduler.thread )
user_list_id = ( yield Scheduler.SLEEP )
user_list = User_list( user_list_id, u"all" )
user_list.add_user( self.user )
user_list.add_user( self.user2 )
user_list.add_user( self.anonymous )
self.user2 = User.create( self.database.next_id( User ), self.username2, self.password2, self.email_address2 )
self.database.save( self.user2, commit = False )
self.database.save( self.user )
self.database.save( self.user2 )
self.database.save( self.anonymous )
self.database.save( user_list )
self.anonymous = User.create( self.database.next_id( User ), u"anonymous" )
self.database.save( self.anonymous, commit = False )
self.database.execute( self.anonymous.sql_save_notebook( self.anon_notebook.object_id ), commit = False )
self.database.commit()
def test_signup( self ):
result = self.http_post( "/users/signup", dict(
@ -103,20 +99,33 @@ class Test_users( Test_controller ):
assert result[ u"user" ].username == self.new_username
notebooks = result[ u"notebooks" ]
assert len( notebooks ) == 2
assert notebooks[ 0 ] == self.anon_notebook
assert notebooks[ 0 ].trash == None
notebook = notebooks[ 0 ]
assert notebook.object_id == self.anon_notebook.object_id
assert notebook.revision == self.anon_notebook.revision
assert notebook.name == self.anon_notebook.name
assert notebook.trash_id == None
assert notebook.read_write == False
notebook = notebooks[ 1 ]
assert notebook.object_id == new_notebook_id
assert notebook.trash
assert len( notebook.notes ) == 1
assert len( notebook.startup_notes ) == 1
assert notebook.revision
assert notebook.name == u"my notebook"
assert notebook.trash_id
assert notebook.read_write == True
notebook = notebooks[ 2 ]
assert notebook.object_id == notebooks[ 1 ].trash_id
assert notebook.revision
assert notebook.name == u"trash"
assert notebook.trash_id == None
assert notebook.read_write == True
startup_notes = result[ "startup_notes" ]
if include_startup_notes:
assert len( startup_notes ) == 1
assert startup_notes[ 0 ] == self.startup_note
assert startup_notes[ 0 ].object_id == self.startup_note.object_id
assert startup_notes[ 0 ].title == self.startup_note.title
assert startup_notes[ 0 ].contents == self.startup_note.contents
else:
assert startup_notes == []
@ -156,20 +165,34 @@ class Test_users( Test_controller ):
assert result[ u"user" ].username == None
notebooks = result[ u"notebooks" ]
assert len( notebooks ) == 2
assert notebooks[ 0 ] == self.anon_notebook
assert notebooks[ 0 ].trash == None
assert len( notebooks ) == 3
notebook = notebooks[ 0 ]
assert notebook.object_id == self.anon_notebook.object_id
assert notebook.revision == self.anon_notebook.revision
assert notebook.name == self.anon_notebook.name
assert notebook.trash_id == None
assert notebook.read_write == False
notebook = notebooks[ 1 ]
assert notebook.object_id == new_notebook_id
assert notebook.trash
assert len( notebook.notes ) == 2
assert len( notebook.startup_notes ) == 2
assert notebook.revision
assert notebook.name == u"my notebook"
assert notebook.trash_id
assert notebook.read_write == True
notebook = notebooks[ 2 ]
assert notebook.object_id == notebooks[ 1 ].trash_id
assert notebook.revision
assert notebook.name == u"trash"
assert notebook.trash_id == None
assert notebook.read_write == True
startup_notes = result[ "startup_notes" ]
if include_startup_notes:
assert len( startup_notes ) == 1
assert startup_notes[ 0 ] == self.startup_note
assert startup_notes[ 0 ].object_id == self.startup_note.object_id
assert startup_notes[ 0 ].title == self.startup_note.title
assert startup_notes[ 0 ].contents == self.startup_note.contents
else:
assert startup_notes == []
@ -260,15 +283,25 @@ class Test_users( Test_controller ):
session_id = session_id,
)
assert result[ u"user" ] == self.user
assert result[ u"notebooks" ] == [ self.anon_notebook ] + self.notebooks
assert result[ u"user" ]
assert result[ u"user" ].object_id == self.user.object_id
assert result[ u"user" ].username == self.user.username
assert len( result[ u"notebooks" ] ) == 3
assert result[ u"notebooks" ][ 0 ].object_id == self.anon_notebook.object_id
assert result[ u"notebooks" ][ 0 ].read_write == False
assert result[ u"notebooks" ][ 1 ].object_id == self.notebooks[ 0 ].object_id
assert result[ u"notebooks" ][ 1 ].read_write == True
assert result[ u"notebooks" ][ 2 ].object_id == self.notebooks[ 1 ].object_id
assert result[ u"notebooks" ][ 2 ].read_write == True
assert result[ u"http_url" ] == self.settings[ u"global" ].get( u"luminotes.http_url" )
assert result[ u"login_url" ] == None
startup_notes = result[ "startup_notes" ]
if include_startup_notes:
assert len( startup_notes ) == 1
assert startup_notes[ 0 ] == self.startup_note
assert startup_notes[ 0 ].object_id == self.startup_note.object_id
assert startup_notes[ 0 ].title == self.startup_note.title
assert startup_notes[ 0 ].contents == self.startup_note.contents
else:
assert startup_notes == []
@ -281,10 +314,13 @@ class Test_users( Test_controller ):
)
assert result[ u"user" ].username == "anonymous"
assert result[ u"notebooks" ] == [ self.anon_notebook ]
assert len( result[ u"notebooks" ] ) == 1
assert result[ u"notebooks" ][ 0 ].object_id == self.anon_notebook.object_id
assert result[ u"notebooks" ][ 0 ].name == self.anon_notebook.name
assert result[ u"notebooks" ][ 0 ].read_write == False
assert result[ u"http_url" ] == self.settings[ u"global" ].get( u"luminotes.http_url" )
login_note = self.anon_notebook.lookup_note_by_title( u"login" )
login_note = self.database.select_one( Note, self.anon_notebook.sql_load_note_by_title( u"login" ) )
assert result[ u"login_url" ] == u"%s/notebooks/%s?note_id=%s" % (
self.settings[ u"global" ][ u"luminotes.https_url" ],
self.anon_notebook.object_id,
@ -294,72 +330,37 @@ class Test_users( Test_controller ):
startup_notes = result[ "startup_notes" ]
if include_startup_notes:
assert len( startup_notes ) == 1
assert startup_notes[ 0 ] == self.startup_note
assert startup_notes[ 0 ].object_id == self.startup_note.object_id
assert startup_notes[ 0 ].title == self.startup_note.title
assert startup_notes[ 0 ].contents == self.startup_note.contents
else:
assert startup_notes == []
def test_current_with_startup_notes_without_login( self ):
self.test_current_without_login( include_startup_notes = True )
def test_calculate_user_storage( self ):
size = cherrypy.root.users.calculate_storage( self.user )
notebooks = self.user.notebooks
# expected a sum of the sizes of all of this user's notebooks, notes, and revisions
expected_size = \
self.database.size( notebooks[ 0 ].object_id ) + \
self.database.size( notebooks[ 1 ].object_id )
assert size == expected_size
def test_calculate_anon_storage( self ):
size = cherrypy.root.users.calculate_storage( self.anonymous )
expected_size = \
self.database.size( self.anon_notebook.object_id ) + \
self.database.size( self.anon_notebook.notes[ 0 ].object_id ) + \
self.database.size( self.anon_notebook.notes[ 0 ].object_id, self.anon_notebook.notes[ 0 ].revision )
assert size == expected_size
def test_update_storage( self ):
previous_revision = self.user.revision
cherrypy.root.users.update_storage( self.user.object_id )
self.scheduler.wait_until_idle()
expected_size = cherrypy.root.users.calculate_storage( self.user )
assert self.user.storage_bytes == expected_size
assert self.user.revision > previous_revision
user = self.database.load( User, self.user.object_id )
assert user.storage_bytes == expected_size
assert user.revision > previous_revision
def test_update_storage_with_unknown_user_id( self ):
original_revision = self.user.revision
cherrypy.root.users.update_storage( 77 )
self.scheduler.wait_until_idle()
expected_size = cherrypy.root.users.calculate_storage( self.user )
user = self.database.load( User, self.user.object_id )
assert self.user.storage_bytes == 0
assert self.user.revision == original_revision
def test_update_storage_with_callback( self ):
def gen():
previous_revision = self.user.revision
cherrypy.root.users.update_storage( self.user.object_id, self.scheduler.thread )
user = ( yield Scheduler.SLEEP )
expected_size = cherrypy.root.users.calculate_storage( self.user )
assert user == self.user
assert self.user.storage_bytes == expected_size
assert self.user.revision > previous_revision
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
def test_send_reset( self ):
# trick send_reset() into using a fake SMTP server
Stub_smtp.reset()
@ -408,7 +409,7 @@ class Test_users( Test_controller ):
result = self.http_get( "/users/redeem_reset/%s" % password_reset_id )
assert result[ u"notebook_id" ] == self.anonymous.notebooks[ 0 ].object_id
assert result[ u"notebook_id" ] == self.anon_notebook.object_id
assert result[ u"note_id" ]
assert u"password reset" in result[ u"note_contents" ]
assert self.user.username in result[ u"note_contents" ]
@ -434,15 +435,9 @@ class Test_users( Test_controller ):
assert password_reset_id
# to trigger expiration, pretend that the password reset was made 25 hours ago
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
password_reset._Persistent__revision = datetime.now() - timedelta( hours = 25 )
self.database.save( password_reset )
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
password_reset = self.database.load( Password_reset, password_reset_id )
password_reset._Persistent__revision = datetime.now( tz = utc ) - timedelta( hours = 25 )
self.database.save( password_reset )
result = self.http_get( "/users/redeem_reset/%s" % password_reset_id )
@ -461,15 +456,9 @@ class Test_users( Test_controller ):
password_reset_id = matches.group( 2 )
assert password_reset_id
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
password_reset.redeemed = True
self.database.save( password_reset )
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
password_reset = self.database.load( Password_reset, password_reset_id )
password_reset.redeemed = True
self.database.save( password_reset )
result = self.http_get( "/users/redeem_reset/%s" % password_reset_id )
@ -488,15 +477,9 @@ class Test_users( Test_controller ):
password_reset_id = matches.group( 2 )
assert password_reset_id
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
password_reset._Password_reset__email_address = u"unknown@example.com"
self.database.save( password_reset )
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
password_reset = self.database.load( Password_reset, password_reset_id )
password_reset._Password_reset__email_address = u"unknown@example.com"
self.database.save( password_reset )
result = self.http_get( "/users/redeem_reset/%s" % password_reset_id )
@ -525,20 +508,17 @@ class Test_users( Test_controller ):
( self.user2.object_id, u"" ),
) )
# check that the password reset is now marked as redeemed
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
assert password_reset.redeemed
assert result[ u"redirect" ]
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
# check that the password reset is now marked as redeemed
password_reset = self.database.load( Password_reset, password_reset_id )
assert password_reset.redeemed
# check that the password was actually reset for one of the users, but not the other
assert self.user.check_password( new_password )
assert self.user2.check_password( self.password2 )
assert result[ u"redirect" ]
user = self.database.load( User, self.user.object_id )
assert user.check_password( new_password )
user2 = self.database.load( User, self.user2.object_id )
assert user2.check_password( self.password2 )
def test_reset_password_unknown_reset_id( self ):
new_password = u"newpass"
@ -552,11 +532,14 @@ class Test_users( Test_controller ):
( self.user2.object_id, u"" ),
) )
# check that neither user's password has changed
assert self.user.check_password( self.password )
assert self.user2.check_password( self.password2 )
assert u"expired" in result[ "error" ]
# check that neither user's password has changed
user = self.database.load( User, self.user.object_id )
assert user.check_password( self.password )
user2 = self.database.load( User, self.user2.object_id )
assert user2.check_password( self.password2 )
def test_reset_password_invalid_reset_id( self ):
new_password = u"newpass"
password_reset_id = u"invalid reset id"
@ -569,11 +552,14 @@ class Test_users( Test_controller ):
( self.user2.object_id, u"" ),
) )
# check that neither user's password has changed
assert self.user.check_password( self.password )
assert self.user2.check_password( self.password2 )
assert u"valid" in result[ "error" ]
# check that neither user's password has changed
user = self.database.load( User, self.user.object_id )
assert user.check_password( self.password )
user2 = self.database.load( User, self.user2.object_id )
assert user2.check_password( self.password2 )
def test_reset_password_expired( self ):
Stub_smtp.reset()
smtplib.SMTP = Stub_smtp
@ -588,15 +574,9 @@ class Test_users( Test_controller ):
assert password_reset_id
# to trigger expiration, pretend that the password reset was made 25 hours ago
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
password_reset._Persistent__revision = datetime.now() - timedelta( hours = 25 )
self.database.save( password_reset )
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
password_reset = self.database.load( Password_reset, password_reset_id )
password_reset._Persistent__revision = datetime.now( tz = utc ) - timedelta( hours = 25 )
self.database.save( password_reset )
new_password = u"newpass"
result = self.http_post( "/users/reset_password", (
@ -609,86 +589,16 @@ class Test_users( Test_controller ):
) )
# check that the password reset is not marked as redeemed
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
assert password_reset.redeemed == False
password_reset = self.database.load( Password_reset, password_reset_id )
assert password_reset.redeemed == False
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
# check that neither user's password has changed
assert self.user.check_password( self.password )
assert self.user2.check_password( self.password2 )
assert u"expired" in result[ "error" ]
def test_reset_password_expired( self ):
Stub_smtp.reset()
smtplib.SMTP = Stub_smtp
self.http_post( "/users/send_reset", dict(
email_address = self.user.email_address,
send_reset_button = u"email me",
) )
matches = self.RESET_LINK_PATTERN.search( smtplib.SMTP.message )
password_reset_id = matches.group( 2 )
assert password_reset_id
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
password_reset.redeemed = True
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
new_password = u"newpass"
result = self.http_post( "/users/reset_password", (
( u"password_reset_id", password_reset_id ),
( u"reset_button", u"reset passwords" ),
( self.user.object_id, new_password ),
( self.user.object_id, new_password ),
( self.user2.object_id, u"" ),
( self.user2.object_id, u"" ),
) )
# check that neither user's password has changed
assert self.user.check_password( self.password )
assert self.user2.check_password( self.password2 )
assert u"already" in result[ "error" ]
def test_reset_password_unknown_user_id( self ):
Stub_smtp.reset()
smtplib.SMTP = Stub_smtp
self.http_post( "/users/send_reset", dict(
email_address = self.user.email_address,
send_reset_button = u"email me",
) )
matches = self.RESET_LINK_PATTERN.search( smtplib.SMTP.message )
password_reset_id = matches.group( 2 )
assert password_reset_id
new_password = u"newpass"
result = self.http_post( "/users/reset_password", (
( u"password_reset_id", password_reset_id ),
( u"reset_button", u"reset passwords" ),
( self.user.object_id, new_password ),
( self.user.object_id, new_password ),
( u"unknown", u"foo" ),
( u"unknown", u"foo" ),
( self.user2.object_id, u"" ),
( self.user2.object_id, u"" ),
) )
# check that neither user's password has changed
assert self.user.check_password( self.password )
assert self.user2.check_password( self.password2 )
assert result[ "error" ]
user = self.database.load( User, self.user.object_id )
assert user.check_password( self.password )
user2 = self.database.load( User, self.user2.object_id )
assert user2.check_password( self.password2 )
def test_reset_password_non_matching( self ):
Stub_smtp.reset()
@ -713,10 +623,13 @@ class Test_users( Test_controller ):
( self.user2.object_id, u"" ),
) )
assert u"password" in result[ "error" ]
# check that neither user's password has changed
assert self.user.check_password( self.password )
assert self.user2.check_password( self.password2 )
assert result[ "error" ]
user = self.database.load( User, self.user.object_id )
assert user.check_password( self.password )
user2 = self.database.load( User, self.user2.object_id )
assert user2.check_password( self.password2 )
def test_reset_password_blank( self ):
Stub_smtp.reset()
@ -740,10 +653,11 @@ class Test_users( Test_controller ):
( self.user2.object_id, u"" ),
) )
assert result[ "error" ]
# check that neither user's password has changed
assert self.user.check_password( self.password )
assert self.user2.check_password( self.password2 )
assert result[ "error" ]
def test_reset_password_multiple_users( self ):
Stub_smtp.reset()
@ -769,17 +683,14 @@ class Test_users( Test_controller ):
( self.user2.object_id, new_password2 ),
) )
# check that the password reset is now marked as redeemed
def gen():
self.database.load( password_reset_id, self.scheduler.thread )
password_reset = ( yield Scheduler.SLEEP )
assert password_reset.redeemed
assert result[ u"redirect" ]
g = gen()
self.scheduler.add( g )
self.scheduler.wait_for( g )
# check that the password reset is now marked as redeemed
password_reset = self.database.load( Password_reset, password_reset_id )
assert password_reset.redeemed
# check that the password was actually reset for both users
assert self.user.check_password( new_password )
assert self.user2.check_password( new_password2 )
assert result[ u"redirect" ]
user = self.database.load( User, self.user.object_id )
assert user.check_password( new_password )
user2 = self.database.load( User, self.user2.object_id )
assert user2.check_password( new_password2 )

9
luminotes.py
View File

@ -1,13 +1,11 @@
import cherrypy
from controller.Database import Database
from controller.Root import Root
from controller.Scheduler import Scheduler
from config import Common
def main( args ):
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
database = Database()
cherrypy.config.update( Common.settings )
@ -21,12 +19,9 @@ def main( args ):
cherrypy.config.update( settings )
cherrypy.lowercase_api = True
root = Root( scheduler, database, cherrypy.config.configMap )
root = Root( database, cherrypy.config.configMap )
cherrypy.root = root
if scheduler.shutdown not in cherrypy.server.on_stop_server_list:
cherrypy.server.on_stop_server_list.append( scheduler.shutdown )
cherrypy.server.start()

150
new_model/Note.py Normal file
View File

@ -0,0 +1,150 @@
import re
from Persistent import Persistent, quote
from controller.Html_nuker import Html_nuker
class Note( Persistent ):
"""
An single textual wiki note.
"""
TITLE_PATTERN = re.compile( u"<h3>(.*?)</h3>", flags = re.IGNORECASE )
def __init__( self, object_id, revision = None, title = None, contents = None, notebook_id = None,
startup = None, deleted_from_id = None, rank = None ):
"""
Create a new note with the given id and contents.
@type object_id: unicode
@param object_id: id of the note
@type revision: datetime or NoneType
@param revision: revision timestamp of the object (optional, defaults to now)
@type title: unicode or NoneType
@param title: textual title of the note (optional, defaults to derived from contents)
@type contents: unicode or NoneType
@param contents: HTML contents of the note (optional)
@type notebook_id: unicode or NoneType
@param notebook_id: id of notebook containing this note (optional)
@type startup: bool or NoneType
@param startup: whether this note should be displayed upon startup (optional, defaults to False)
@type deleted_from_id: unicode or NoneType
@param deleted_from_id: id of the notebook that this note was deleted from (optional)
@type rank: float or NoneType
@param rank: indicates numeric ordering of this note in relation to other startup notes
@rtype: Note
@return: newly constructed note
"""
Persistent.__init__( self, object_id, revision )
self.__title = title
self.__contents = contents
self.__notebook_id = notebook_id
self.__startup = startup or False
self.__deleted_from_id = deleted_from_id
self.__rank = rank
@staticmethod
def create( object_id, contents = None, notebook_id = None, startup = None, rank = None ):
"""
Convenience constructor for creating a new note.
@type object_id: unicode
@param object_id: id of the note
@type contents: unicode or NoneType
@param contents: HTML contents of the note (optional)
@type notebook_id: unicode or NoneType
@param notebook_id: id of notebook containing this note (optional)
@type startup: bool or NoneType
@param startup: whether this note should be displayed upon startup (optional, defaults to False)
@type rank: float or NoneType
@param rank: indicates numeric ordering of this note in relation to other startup notes
@rtype: Note
@return: newly constructed note
"""
note = Note( object_id, notebook_id = notebook_id, startup = startup, rank = rank )
note.contents = contents
return note
def __set_contents( self, contents ):
self.update_revision()
self.__contents = contents
if contents is None:
self.__title = None
return
# parse title out of the beginning of the contents
result = Note.TITLE_PATTERN.search( contents )
if result:
self.__title = result.groups()[ 0 ]
self.__title = Html_nuker( allow_refs = True ).nuke( self.__title )
else:
self.__title = None
def __set_notebook_id( self, notebook_id ):
self.__notebook_id = notebook_id
self.update_revision()
def __set_startup( self, startup ):
self.__startup = startup
self.update_revision()
def __set_deleted_from_id( self, deleted_from_id ):
self.__deleted_from_id = deleted_from_id
self.update_revision()
def __set_rank( self, rank ):
self.__rank = rank
self.update_revision()
@staticmethod
def sql_load( object_id, revision = None ):
if revision:
return "select * from note where id = %s and revision = %s;" % ( quote( object_id ), quote( revision ) )
return "select * from note_current where id = %s;" % quote( object_id )
@staticmethod
def sql_id_exists( object_id, revision = None ):
if revision:
return "select id from note where id = %s and revision = %s;" % ( quote( object_id ), quote( revision ) )
return "select id from note_current where id = %s;" % quote( object_id )
def sql_exists( self ):
return Note.sql_id_exists( self.object_id, self.revision )
def sql_create( self ):
rank = self.__rank
if rank is None:
rank = quote( None )
return \
"insert into note ( id, revision, title, contents, notebook_id, startup, deleted_from_id, rank ) " + \
"values ( %s, %s, %s, %s, %s, %s, %s, %s );" % \
( quote( self.object_id ), quote( self.revision ), quote( self.__title ),
quote( self.__contents ), quote( self.__notebook_id ), quote( self.__startup and 't' or 'f' ),
quote( self.__deleted_from_id ), rank )
def sql_update( self ):
return self.sql_create()
def sql_load_revisions( self ):
return "select revision from note where id = %s order by revision;" % quote( self.object_id )
def to_dict( self ):
d = Persistent.to_dict( self )
d.update( dict(
contents = self.__contents,
title = self.__title,
deleted_from_id = self.__deleted_from_id,
) )
return d
title = property( lambda self: self.__title )
contents = property( lambda self: self.__contents, __set_contents )
notebook_id = property( lambda self: self.__notebook_id, __set_notebook_id )
startup = property( lambda self: self.__startup, __set_startup )
deleted_from_id = property( lambda self: self.__deleted_from_id, __set_deleted_from_id )
rank = property( lambda self: self.__rank, __set_rank )

148
new_model/Notebook.py Normal file
View File

@ -0,0 +1,148 @@
from copy import copy
from Note import Note
from Persistent import Persistent, quote
class Notebook( Persistent ):
"""
A collection of wiki notes.
"""
def __init__( self, object_id, revision = None, name = None, trash_id = None, read_write = True ):
"""
Create a new notebook with the given id and name.
@type object_id: unicode
@param object_id: id of the notebook
@type revision: datetime or NoneType
@param revision: revision timestamp of the object (optional, defaults to now)
@type name: unicode or NoneType
@param name: name of this notebook (optional)
@type trash_id: Notebook or NoneType
@param trash_id: id of the notebook where deleted notes from this notebook go to die (optional)
@type read_write: bool or NoneType
@param read_write: whether this view of the notebook is currently read-write (optional, defaults to True)
@rtype: Notebook
@return: newly constructed notebook
"""
Persistent.__init__( self, object_id, revision )
self.__name = name
self.__trash_id = trash_id
self.__read_write = read_write
@staticmethod
def create( object_id, name = None, trash_id = None, read_write = True ):
"""
Convenience constructor for creating a new notebook.
@type object_id: unicode
@param object_id: id of the notebook
@type name: unicode or NoneType
@param name: name of this notebook (optional)
@type trash_id: Notebook or NoneType
@param trash_id: id of the notebook where deleted notes from this notebook go to die (optional)
@type read_write: bool or NoneType
@param read_write: whether this view of the notebook is currently read-write (optional, defaults to True)
@rtype: Notebook
@return: newly constructed notebook
"""
return Notebook( object_id, name = name, trash_id = trash_id, read_write = read_write )
@staticmethod
def sql_load( object_id, revision = None ):
if revision:
return "select * from notebook where id = %s and revision = %s;" % ( quote( object_id ), quote( revision ) )
return "select * from notebook_current where id = %s;" % quote( object_id )
@staticmethod
def sql_id_exists( object_id, revision = None ):
if revision:
return "select id from notebook where id = %s and revision = %s;" % ( quote( object_id ), quote( revision ) )
return "select id from notebook_current where id = %s;" % quote( object_id )
def sql_exists( self ):
return Notebook.sql_id_exists( self.object_id, self.revision )
def sql_create( self ):
return \
"insert into notebook ( id, revision, name, trash_id ) " + \
"values ( %s, %s, %s, %s );" % \
( quote( self.object_id ), quote( self.revision ), quote( self.__name ),
quote( self.__trash_id ) )
def sql_update( self ):
return self.sql_create()
def sql_load_notes( self ):
"""
Return a SQL string to load a list of all the notes within this notebook.
"""
return "select * from note_current where notebook_id = %s order by revision desc;" % quote( self.object_id )
def sql_load_non_startup_notes( self ):
"""
Return a SQL string to load a list of the non-startup notes within this notebook.
"""
return "select * from note_current where notebook_id = %s and startup = 'f' order by revision desc;" % quote( self.object_id )
def sql_load_startup_notes( self ):
"""
Return a SQL string to load a list of the startup notes within this notebook.
"""
return "select * from note_current where notebook_id = %s and startup = 't' order by rank;" % quote( self.object_id )
def sql_load_note_by_id( self, note_id ):
"""
Return a SQL string to load a particular note within this notebook by the note's id.
@type note_id: unicode
@param note_id: id of note to load
"""
return "select * from note_current where notebook_id = %s and id = %s;" % ( quote( self.object_id ), quote( note_id ) )
def sql_load_note_by_title( self, title ):
"""
Return a SQL string to load a particular note within this notebook by the note's title.
@type note_id: unicode
@param note_id: title of note to load
"""
return "select * from note_current where notebook_id = %s and title = %s;" % ( quote( self.object_id ), quote( title ) )
def sql_search_notes( self, search_text ):
"""
Return a SQL string to search for notes whose contents contain the given search_text. This
is a case-insensitive search.
@type search_text: unicode
@param search_text: text to search for within the notes
"""
return \
"select * from note_current where notebook_id = %s and contents ilike %s;" % \
( quote( self.object_id ), quote( "%" + search_text + "%" ) )
def sql_highest_rank( self ):
return "select coalesce( max( rank ), -1 ) from note_current where notebook_id = %s;" % quote( self.object_id )
def to_dict( self ):
d = Persistent.to_dict( self )
d.update( dict(
name = self.__name,
trash_id = self.__trash_id,
read_write = self.__read_write,
) )
return d
def __set_name( self, name ):
self.__name = name
self.update_revision()
def __set_read_write( self, read_write ):
self.__read_write = read_write
name = property( lambda self: self.__name, __set_name )
trash_id = property( lambda self: self.__trash_id )
read_write = property( lambda self: self.__read_write, __set_read_write )

57
new_model/Password_reset.py Normal file
View File

@ -0,0 +1,57 @@
from Persistent import Persistent, quote
class Password_reset( Persistent ):
"""
A request for a password reset.
"""
def __init__( self, object_id, email_address, redeemed = False ):
"""
Create a password reset request with the given id.
@type object_id: unicode
@param object_id: id of the password reset
@type email_address: unicode
@param email_address: where the reset confirmation was emailed
@type redeemed: bool or NoneType
@param redeemed: whether this password reset has been redeemed yet (optional, defaults to False)
@rtype: Password_reset
@return: newly constructed password reset
"""
Persistent.__init__( self, object_id )
self.__email_address = email_address
self.__redeemed = redeemed
@staticmethod
def sql_load( object_id, revision = None ):
# password resets don't track revisions
if revision:
raise NotImplementedError()
return "select * from password_reset where id = %s;" % quote( object_id )
@staticmethod
def sql_id_exists( object_id, revision = None ):
if revision:
raise NotImplementedError()
return "select id from password_reset where id = %s;" % quote( object_id )
def sql_exists( self ):
return Password_reset.sql_id_exists( self.object_id )
def sql_create( self ):
return "insert into password_reset ( id, email_address, redeemed ) values ( %s, %s, %s );" % \
( quote( self.object_id ), quote( self.__email_address ), quote( self.__redeemed and "t" or "f" ) )
def sql_update( self ):
return "update password_reset set redeemed = %s where id = %s;" % \
( quote( self.__redeemed and "t" or "f" ), quote( self.object_id ) )
def __set_redeemed( self, redeemed ):
if redeemed != self.__redeemed:
self.update_revision()
self.__redeemed = redeemed
email_address = property( lambda self: self.__email_address )
redeemed = property( lambda self: self.__redeemed, __set_redeemed )

82
new_model/Persistent.py Normal file
View File

@ -0,0 +1,82 @@
from datetime import datetime
from pytz import utc
class Persistent( object ):
"""
A persistent database object with a unique id.
"""
def __init__( self, object_id, revision = None ):
self.__object_id = object_id
self.__revision = revision
if not revision:
self.update_revision()
@staticmethod
def sql_load( object_id, revision = None ):
"""
Return a SQL string to load an object with the given information from the database. If a
revision is not provided, then the most current version of the given object will be loaded.
@type object_id: unicode
@param object_id: id of object to load
@type revision: unicode or NoneType
@param revision: revision of the object to load (optional)
"""
raise NotImplementedError()
@staticmethod
def sql_id_exists( object_id, revision = None ):
"""
Return a SQL string to determine whether the given object is present in the database. If a
revision is not provided, then the most current version of the given object will be used.
@type object_id: unicode
@param object_id: id of object to check for existence
@type revision: unicode or NoneType
@param revision: revision of the object to check (optional)
"""
raise NotImplementedError()
def sql_exists( self ):
"""
Return a SQL string to determine whether the current revision of this object is present in the
database.
"""
raise NotImplementedError()
def sql_create( self ):
"""
Return a SQL string to save this object to the database for the first time. This should be in
the form of a SQL insert.
"""
raise NotImplementedError()
def sql_update( self ):
"""
Return a SQL string to save an updated revision of this object to the database. Note that,
because of the retention of old row revisions in the database, this SQL string will usually
be in the form of an insert rather than an update to an existing row.
"""
raise NotImplementedError()
def to_dict( self ):
return dict(
object_id = self.__object_id,
revision = self.__revision,
)
def update_revision( self ):
self.__revision = datetime.now( tz = utc )
object_id = property( lambda self: self.__object_id )
revision = property( lambda self: self.__revision )
def quote( value ):
if value is None:
return "null"
value = unicode( value )
return "'%s'" % value.replace( "'", "''" ).replace( "\\", "\\\\" )

212
new_model/User.py Normal file
View File

@ -0,0 +1,212 @@
import sha
import random
from copy import copy
from Persistent import Persistent, quote
class User( Persistent ):
"""
A Luminotes user.
"""
SALT_CHARS = [ chr( c ) for c in range( ord( "!" ), ord( "~" ) + 1 ) ]
SALT_SIZE = 12
def __init__( self, object_id, revision = None, username = None, salt = None, password_hash = None,
email_address = None, storage_bytes = None, rate_plan = None ):
"""
Create a new user with the given credentials and information.
@type object_id: unicode
@param object_id: id of the user
@type revision: datetime or NoneType
@param revision: revision timestamp of the object (optional, defaults to now)
@type username: unicode or NoneType
@param username: unique user identifier for login purposes (optional)
@type salt: unicode or NoneType
@param salt: salt to use when hashing the password (optional, defaults to random)
@type password_hash: unicode or NoneType
@param password_hash: cryptographic hash of secret password for login purposes (optional)
@type email_address: unicode or NoneType
@param email_address: a hopefully valid email address (optional)
@type storage_bytes: int or NoneType
@param storage_bytes: count of bytes that the user is currently using for storage (optional)
@type rate_plan: int or NoneType
@param rate_plan: index into the rate plan array in config/Common.py (optional, defaults to 0)
@rtype: User
@return: newly created user
"""
Persistent.__init__( self, object_id, revision )
self.__username = username
self.__salt = salt
self.__password_hash = password_hash
self.__email_address = email_address
self.__storage_bytes = storage_bytes or 0
self.__rate_plan = rate_plan or 0
@staticmethod
def create( object_id, username = None, password = None, email_address = None ):
"""
Convenience constructor for creating a new user.
@type object_id: unicode
@param object_id: id of the user
@type username: unicode or NoneType
@param username: unique user identifier for login purposes (optional)
@type password: unicode or NoneType
@param password: secret password for login purposes (optional)
@type email_address: unicode or NoneType
@param email_address: a hopefully valid email address (optional)
@rtype: User
@return: newly created user
"""
salt = User.__create_salt()
password_hash = User.__hash_password( salt, password )
return User( object_id, None, username, salt, password_hash, email_address )
@staticmethod
def __create_salt():
return "".join( [ random.choice( User.SALT_CHARS ) for i in range( User.SALT_SIZE ) ] )
@staticmethod
def __hash_password( salt, password ):
if password is None or len( password ) == 0:
return None
return sha.new( salt + password ).hexdigest()
def check_password( self, password ):
"""
Check that the given password matches this user's password.
@type password: unicode
@param password: password to check
@rtype: bool
@return: True if the password matches
"""
if self.__password_hash == None:
return False
hash = User.__hash_password( self.__salt, password )
if hash == self.__password_hash:
return True
return False
@staticmethod
def sql_load( object_id, revision = None ):
if revision:
return "select * from luminotes_user where id = %s and revision = %s;" % ( quote( object_id ), quote( revision ) )
return "select * from luminotes_user_current where id = %s;" % quote( object_id )
@staticmethod
def sql_id_exists( object_id, revision = None ):
if revision:
return "select id from luminotes_user where id = %s and revision = %s;" % ( quote( object_id ), quote( revision ) )
return "select id from luminotes_user_current where id = %s;" % quote( object_id )
def sql_exists( self ):
return User.sql_id_exists( self.object_id, self.revision )
def sql_create( self ):
return \
"insert into luminotes_user ( id, revision, username, salt, password_hash, email_address, storage_bytes, rate_plan ) " + \
"values ( %s, %s, %s, %s, %s, %s, %s, %s );" % \
( quote( self.object_id ), quote( self.revision ), quote( self.__username ),
quote( self.__salt ), quote( self.__password_hash ), quote( self.__email_address ),
self.__storage_bytes, self.__rate_plan )
def sql_update( self ):
return self.sql_create()
@staticmethod
def sql_load_by_username( username ):
return "select * from luminotes_user_current where username = %s;" % quote( username )
@staticmethod
def sql_load_by_email_address( email_address ):
return "select * from luminotes_user_current where username = %s;" % quote( email_address )
def sql_load_notebooks( self, parents_only = False ):
"""
Return a SQL string to load a list of the notebooks to which this user has access.
"""
if parents_only:
parents_only_clause = " and trash_id is not null";
else:
parents_only_clause = ""
return \
"select notebook_current.*, user_notebook.read_write from user_notebook, notebook_current " + \
"where user_id = %s%s and user_notebook.notebook_id = notebook_current.id order by revision;" % \
( quote( self.object_id ), parents_only_clause )
def sql_save_notebook( self, notebook_id, read_write = True ):
"""
Return a SQL string to save the id of a notebook to which this user has access.
"""
return \
"insert into user_notebook ( user_id, notebook_id, read_write ) values " + \
"( %s, %s, %s );" % ( quote( self.object_id ), quote( notebook_id ), quote( read_write and 't' or 'f' ) )
def sql_has_access( self, notebook_id, read_write = False ):
"""
Return a SQL string to determine whether this user has access to the given notebook.
"""
if read_write is True:
return \
"select user_id from user_notebook where user_id = %s and notebook_id = %s and read_write = 't';" % \
( quote( self.object_id ), quote( notebook_id ) )
else:
return \
"select user_id from user_notebook where user_id = %s and notebook_id = %s;" % \
( quote( self.object_id ), quote( notebook_id ) )
def sql_calculate_storage( self ):
"""
Return a SQL string to calculate the total bytes of storage usage by this user. Note that this
only includes storage for all the user's notes and past revisions. It doesn't include storage
for the notebooks themselves.
"""
return \
"""
select
coalesce( sum( pg_column_size( note.* ) ), 0 )
from
luminotes_user_current, user_notebook, note
where
luminotes_user_current.id = %s and
user_notebook.user_id = luminotes_user_current.id and
note.notebook_id = user_notebook.notebook_id;
""" % quote( self.object_id )
def to_dict( self ):
d = Persistent.to_dict( self )
d.update( dict(
username = self.username,
storage_bytes = self.__storage_bytes,
rate_plan = self.__rate_plan,
) )
return d
def __set_password( self, password ):
self.update_revision()
self.__salt = User.__create_salt()
self.__password_hash = User.__hash_password( self.__salt, password )
def __set_storage_bytes( self, storage_bytes ):
self.update_revision()
self.__storage_bytes = storage_bytes
def __set_rate_plan( self, rate_plan ):
self.update_revision()
self.__rate_plan = rate_plan
username = property( lambda self: self.__username )
email_address = property( lambda self: self.__email_address )
password = property( None, __set_password )
storage_bytes = property( lambda self: self.__storage_bytes, __set_storage_bytes )
rate_plan = property( lambda self: self.__rate_plan, __set_rate_plan )

0
new_model/__init__.py Normal file
View File

8
new_model/drop.sql Normal file
View File

@ -0,0 +1,8 @@
DROP VIEW luminotes_user_current;
DROP TABLE luminotes_user;
DROP VIEW note_current;
DROP TABLE note;
DROP VIEW notebook_current;
DROP TABLE notebook;
DROP TABLE password_reset;
DROP TABLE user_notebook;

7
model/schema.sql → new_model/schema.sql
View File

@ -6,13 +6,6 @@ SET client_encoding = 'UTF8';
SET check_function_bodies = false;
SET client_min_messages = warning;
--
-- Name: SCHEMA public; Type: COMMENT; Schema: -; Owner: postgres
--
COMMENT ON SCHEMA public IS 'Standard public schema';
SET search_path = public, pg_catalog;
SET default_tablespace = '';

133
new_model/test/Test_note.py Normal file
View File

@ -0,0 +1,133 @@
from pytz import utc
from datetime import datetime, timedelta
from new_model.Note import Note
class Test_note( object ):
def setUp( self ):
self.object_id = u"17"
self.title = u"title goes here"
self.contents = u"<h3>%s</h3>blah" % self.title
self.notebook_id = u"18"
self.startup = False
self.rank = 17.5
self.delta = timedelta( seconds = 1 )
self.note = Note.create( self.object_id, self.contents, self.notebook_id, self.startup, self.rank )
def test_create( self ):
assert self.note.object_id == self.object_id
assert datetime.now( tz = utc ) - self.note.revision < self.delta
assert self.note.contents == self.contents
assert self.note.title == self.title
assert self.note.notebook_id == self.notebook_id
assert self.note.startup == self.startup
assert self.note.deleted_from_id == None
assert self.note.rank == self.rank
def test_set_contents( self ):
new_title = u"new title"
new_contents = u"<h3>%s</h3>new blah" % new_title
previous_revision = self.note.revision
self.note.contents = new_contents
assert self.note.revision > previous_revision
assert self.note.contents == new_contents
assert self.note.title == new_title
assert self.note.notebook_id == self.notebook_id
assert self.note.startup == self.startup
assert self.note.deleted_from_id == None
assert self.note.rank == self.rank
def test_set_contents_with_html_title( self ):
new_title = u"new title"
new_contents = u"<h3>new<br /> title</h3>new blah"
previous_revision = self.note.revision
self.note.contents = new_contents
# html should be stripped out of the title
assert self.note.revision > previous_revision
assert self.note.contents == new_contents
assert self.note.title == new_title
assert self.note.notebook_id == self.notebook_id
assert self.note.startup == self.startup
assert self.note.deleted_from_id == None
assert self.note.rank == self.rank
def test_set_contents_with_multiple_titles( self ):
new_title = u"new title"
new_contents = u"<h3>new<br /> title</h3>new blah<h3>other title</h3>hmm"
previous_revision = self.note.revision
self.note.contents = new_contents
# should only use the first title
assert self.note.revision > previous_revision
assert self.note.contents == new_contents
assert self.note.title == new_title
assert self.note.notebook_id == self.notebook_id
assert self.note.startup == self.startup
assert self.note.deleted_from_id == None
assert self.note.rank == self.rank
def test_set_notebook_id( self ):
previous_revision = self.note.revision
self.note.notebook_id = u"54"
assert self.note.revision > previous_revision
assert self.note.notebook_id == u"54"
def test_set_startup( self ):
previous_revision = self.note.revision
self.note.startup = True
assert self.note.revision > previous_revision
assert self.note.startup == True
def test_set_deleted_from_id( self ):
previous_revision = self.note.revision
self.note.deleted_from_id = u"55"
assert self.note.revision > previous_revision
assert self.note.deleted_from_id == u"55"
def test_set_rank( self ):
previous_revision = self.note.revision
self.note.rank = 5
assert self.note.revision > previous_revision
assert self.note.rank == 5
def test_to_dict( self ):
d = self.note.to_dict()
assert d.get( "object_id" ) == self.note.object_id
assert datetime.now( tz = utc ) - d.get( "revision" ) < self.delta
assert d.get( "contents" ) == self.contents
assert d.get( "title" ) == self.title
assert d.get( "deleted_from_id" ) == None
class Test_note_blank( Test_note ):
def setUp( self ):
self.object_id = u"17"
self.title = None
self.contents = None
self.notebook_id = None
self.startup = False
self.rank = None
self.delta = timedelta( seconds = 1 )
self.note = Note.create( self.object_id )
def test_create( self ):
assert self.note.object_id == self.object_id
assert datetime.now( tz = utc ) - self.note.revision < self.delta
assert self.note.contents == None
assert self.note.title == None
assert self.note.notebook_id == None
assert self.note.startup == False
assert self.note.deleted_from_id == None
assert self.note.rank == None

54
new_model/test/Test_notebook.py Normal file
View File

@ -0,0 +1,54 @@
from pytz import utc
from datetime import datetime, timedelta
from new_model.Notebook import Notebook
from new_model.Note import Note
class Test_notebook( object ):
def setUp( self ):
self.object_id = "17"
self.trash_id = "18"
self.name = u"my notebook"
self.trash_name = u"trash"
self.delta = timedelta( seconds = 1 )
self.trash = Notebook.create( self.trash_id, self.trash_name, read_write = False )
self.notebook = Notebook.create( self.object_id, self.name, trash_id = self.trash.object_id )
self.note = Note.create( "19", u"<h3>title</h3>blah" )
def test_create( self ):
assert self.notebook.object_id == self.object_id
assert datetime.now( tz = utc ) - self.notebook.revision < self.delta
assert self.notebook.name == self.name
assert self.notebook.read_write == True
assert self.notebook.trash_id == self.trash_id
assert self.trash.object_id == self.trash_id
assert datetime.now( tz = utc ) - self.trash.revision < self.delta
assert self.trash.name == self.trash_name
assert self.trash.read_write == False
assert self.trash.trash_id == None
def test_set_name( self ):
new_name = u"my new notebook"
previous_revision = self.notebook.revision
self.notebook.name = new_name
assert self.notebook.name == new_name
assert self.notebook.revision > previous_revision
def test_set_read_write( self ):
original_revision = self.notebook.revision
self.notebook.read_write = True
assert self.notebook.read_write == True
assert self.notebook.revision == original_revision
def test_to_dict( self ):
d = self.notebook.to_dict()
assert d.get( "name" ) == self.name
assert d.get( "trash_id" ) == self.trash.object_id
assert d.get( "read_write" ) == True
assert d.get( "object_id" ) == self.notebook.object_id
assert datetime.now( tz = utc ) - d.get( "revision" ) < self.delta

38
new_model/test/Test_password_reset.py Normal file
View File

@ -0,0 +1,38 @@
from model.User import User
from new_model.Password_reset import Password_reset
class Test_password_reset( object ):
def setUp( self ):
self.object_id = u"17"
self.email_address = u"bob@example.com"
self.password_reset = Password_reset( self.object_id, self.email_address )
def test_create( self ):
assert self.password_reset.object_id == self.object_id
assert self.password_reset.email_address == self.email_address
assert self.password_reset.redeemed == False
def test_redeem( self ):
previous_revision = self.password_reset.revision
self.password_reset.redeemed = True
assert self.password_reset.redeemed == True
assert self.password_reset.revision > previous_revision
def test_redeem_twice( self ):
self.password_reset.redeemed = True
current_revision = self.password_reset.revision
self.password_reset.redeemed = True
assert self.password_reset.redeemed == True
assert self.password_reset.revision == current_revision
def test_unredeem( self ):
self.password_reset.redeemed = True
previous_revision = self.password_reset.revision
self.password_reset.redeemed = False
assert self.password_reset.redeemed == False
assert self.password_reset.revision > previous_revision

73
new_model/test/Test_persistent.py Normal file
View File

@ -0,0 +1,73 @@
from pytz import utc
from datetime import datetime, timedelta
from new_model.Persistent import Persistent, quote
class Test_persistent( object ):
def setUp( self ):
self.object_id = "17"
self.obj = Persistent( self.object_id )
self.delta = timedelta( seconds = 1 )
def test_create( self ):
assert self.obj.object_id == self.object_id
assert datetime.now( tz = utc ) - self.obj.revision < self.delta
def test_update_revision( self ):
previous_revision = self.obj.revision
self.obj.update_revision()
assert self.obj.revision > previous_revision
assert datetime.now( tz = utc ) - self.obj.revision < self.delta
previous_revision = self.obj.revision
self.obj.update_revision()
assert self.obj.revision > previous_revision
assert datetime.now( tz = utc ) - self.obj.revision < self.delta
def test_to_dict( self ):
d = self.obj.to_dict()
assert d.get( "object_id" ) == self.object_id
assert d.get( "revision" ) == self.obj.revision
class Test_persistent_with_revision( object ):
def setUp( self ):
self.object_id = "17"
self.revision = datetime.now( tz = utc ) - timedelta( hours = 24 )
self.obj = Persistent( self.object_id, self.revision )
self.delta = timedelta( seconds = 1 )
def test_create( self ):
assert self.obj.object_id == self.object_id
assert self.revision - self.obj.revision < self.delta
def test_update_revision( self ):
previous_revision = self.obj.revision
self.obj.update_revision()
assert self.obj.revision > previous_revision
assert datetime.now( tz = utc ) - self.obj.revision < self.delta
previous_revision = self.obj.revision
self.obj.update_revision()
assert self.obj.revision > previous_revision
assert datetime.now( tz = utc ) - self.obj.revision < self.delta
def test_to_dict( self ):
d = self.obj.to_dict()
assert d.get( "object_id" ) == self.object_id
assert d.get( "revision" ) == self.obj.revision
def test_quote():
assert "'foo'" == quote( "foo" )
def test_quote_apostrophe():
assert "'it''s'" == quote( "it's" )
def test_quote_backslash():
assert r"'c:\\\\whee'" == quote( r"c:\\whee" )
def test_quote_none():
assert "null" == quote( None )

69
new_model/test/Test_user.py Normal file
View File

@ -0,0 +1,69 @@
from pytz import utc
from datetime import datetime, timedelta
from new_model.User import User
class Test_user( object ):
def setUp( self ):
self.object_id = u"17"
self.username = u"bob"
self.password = u"foobar"
self.email_address = u"bob@example.com"
self.delta = timedelta( seconds = 1 )
self.user = User.create( self.object_id, self.username, self.password, self.email_address )
def test_create( self ):
assert self.user.object_id == self.object_id
assert datetime.now( tz = utc ) - self.user.revision < self.delta
assert self.user.username == self.username
assert self.user.email_address == self.email_address
assert self.user.storage_bytes == 0
assert self.user.rate_plan == 0
def test_check_correct_password( self ):
assert self.user.check_password( self.password ) == True
def test_check_incorrect_password( self ):
assert self.user.check_password( u"wrong" ) == False
def test_set_password( self ):
previous_revision = self.user.revision
new_password = u"newpass"
self.user.password = new_password
assert self.user.check_password( self.password ) == False
assert self.user.check_password( new_password ) == True
assert self.user.revision > previous_revision
def test_set_none_password( self ):
previous_revision = self.user.revision
new_password = None
self.user.password = new_password
assert self.user.check_password( self.password ) == False
assert self.user.check_password( new_password ) == False
assert self.user.revision > previous_revision
def test_set_storage_bytes( self ):
previous_revision = self.user.revision
storage_bytes = 44
self.user.storage_bytes = storage_bytes
assert self.user.storage_bytes == storage_bytes
assert self.user.revision > previous_revision
def test_set_rate_plan( self ):
previous_revision = self.user.revision
rate_plan = 2
self.user.rate_plan = rate_plan
assert self.user.rate_plan == rate_plan
assert self.user.revision > previous_revision
def test_to_dict( self ):
d = self.user.to_dict()
assert d.get( "username" ) == self.username
assert d.get( "storage_bytes" ) == self.user.storage_bytes
assert d.get( "rate_plan" ) == self.user.rate_plan

13
static/js/Editor.js
View File

@ -1,9 +1,10 @@
function Editor( id, notebook_id, note_text, deleted_from, revisions_list, read_write, startup, highlight, focus ) {
function Editor( id, notebook_id, note_text, deleted_from_id, revision, read_write, startup, highlight, focus ) {
this.id = id;
this.notebook_id = notebook_id;
this.initial_text = note_text;
this.deleted_from = deleted_from || null;
this.revisions_list = revisions_list || new Array();
this.deleted_from_id = deleted_from_id || null;
this.revision = revision;
this.revisions_list = new Array(); // cache for this note's list of revisions, loaded from the server on-demand
this.read_write = read_write;
this.startup = startup || false; // whether this Editor is for a startup note
this.init_highlight = highlight || false;
@ -31,7 +32,7 @@ function Editor( id, notebook_id, note_text, deleted_from, revisions_list, read_
"type": "button",
"class": "note_button",
"id": "delete_" + iframe_id,
"value": "delete" + ( this.deleted_from ? " forever" : "" ),
"value": "delete" + ( this.deleted_from_id ? " forever" : "" ),
"title": "delete note [ctrl-d]"
} );
connect( this.delete_button, "onclick", function ( event ) { signal( self, "delete_clicked", event ); } );
@ -45,7 +46,7 @@ function Editor( id, notebook_id, note_text, deleted_from, revisions_list, read_
} );
connect( this.changes_button, "onclick", function ( event ) { signal( self, "changes_clicked", event ); } );
if ( this.deleted_from ) {
if ( this.deleted_from_id ) {
this.undelete_button = createDOM( "input", {
"type": "button",
"class": "note_button",
@ -66,7 +67,7 @@ function Editor( id, notebook_id, note_text, deleted_from, revisions_list, read_
}
}
if ( !this.deleted_from && ( read_write || !startup ) ) {
if ( !this.deleted_from_id && ( read_write || !startup ) ) {
this.hide_button = createDOM( "input", {
"type": "button",
"class": "note_button",

134
static/js/Wiki.js
View File

@ -72,7 +72,7 @@ Wiki.prototype.display_user = function ( result ) {
for ( var i in result.notebooks ) {
var notebook = result.notebooks[ i ];
if ( notebook.name == "Luminotes" )
if ( notebook.name == "Luminotes" || notebook.name == "trash" )
continue;
var div_class = "link_area_item";
@ -162,18 +162,20 @@ Wiki.prototype.populate = function ( result ) {
createDOM( "a", { "href": location.href, "id": "all_notes_link", "title": "View a list of all notes in this notebook." }, "all notes" )
) );
}
appendChildNodes( span, createDOM( "div", { "class": "link_area_item" },
createDOM( "a", { "href": "/notebooks/download_html/" + this.notebook.object_id, "id": "download_html_link", "title": "Download a stand-alone copy of the entire wiki notebook." }, "download as html" )
) );
if ( this.notebook.name != "Luminotes" ) {
appendChildNodes( span, createDOM( "div", { "class": "link_area_item" },
createDOM( "a", { "href": "/notebooks/download_html/" + this.notebook.object_id, "id": "download_html_link", "title": "Download a stand-alone copy of the entire wiki notebook." }, "download as html" )
) );
}
if ( this.notebook.read_write ) {
this.read_write = true;
removeElementClass( "toolbar", "undisplayed" );
if ( this.notebook.trash ) {
if ( this.notebook.trash_id ) {
appendChildNodes( span, createDOM( "div", { "class": "link_area_item" },
createDOM( "a", {
"href": "/notebooks/" + this.notebook.trash.object_id + "?parent_id=" + this.notebook.object_id,
"href": "/notebooks/" + this.notebook.trash_id + "?parent_id=" + this.notebook.object_id,
"id": "trash_link",
"title": "Look here for notes you've deleted."
}, "trash" )
@ -221,9 +223,11 @@ Wiki.prototype.populate = function ( result ) {
} );
}
connect( "download_html_link", "onclick", function ( event ) {
self.save_editor( null, true );
} );
if ( this.notebook.name != "Luminotes" ) {
connect( "download_html_link", "onclick", function ( event ) {
self.save_editor( null, true );
} );
}
// create an editor for each startup note in the received notebook, focusing the first one
var focus = true;
@ -234,7 +238,7 @@ Wiki.prototype.populate = function ( result ) {
// don't actually create an editor if a particular note was provided in the result
if ( !result.note ) {
var editor = this.create_editor( note.object_id, note.contents, note.deleted_from, note.revisions_list, undefined, this.read_write, false, focus );
var editor = this.create_editor( note.object_id, note.contents, note.deleted_from_id, note.revision, this.read_write, false, focus );
this.open_editors[ note.title ] = editor;
focus = false;
}
@ -242,14 +246,16 @@ Wiki.prototype.populate = function ( result ) {
// if one particular note was provided, then just display an editor for that note
var read_write = this.read_write;
if ( getElement( "revision" ).value ) read_write = false;
var revision_element = getElement( "revision" );
if ( revision_element && revision_element.value ) read_write = false;
if ( result.note )
this.create_editor(
result.note.object_id,
result.note.contents || getElement( "note_contents" ).value,
result.note.deleted_from,
result.note.revisions_list,
undefined, read_write, false, true
result.note.deleted_from_id,
result.note.revision,
read_write, false, true
);
if ( result.startup_notes.length == 0 && !result.note )
@ -282,7 +288,7 @@ Wiki.prototype.create_blank_editor = function ( event ) {
}
}
var editor = this.create_editor( undefined, undefined, undefined, undefined, undefined, this.read_write, true, true );
var editor = this.create_editor( undefined, undefined, undefined, undefined, this.read_write, true, true );
this.blank_editor_id = editor.id;
}
@ -448,26 +454,27 @@ Wiki.prototype.resolve_link = function ( note_title, link, callback ) {
);
}
Wiki.prototype.parse_loaded_editor = function ( result, note_title, revision, link ) {
Wiki.prototype.parse_loaded_editor = function ( result, note_title, requested_revision, link ) {
if ( result.note ) {
var id = result.note.object_id;
if ( revision ) id += " " + revision;
if ( requested_revision )
id += " " + requested_revision;
var actual_revision = result.note.revision;
var note_text = result.note.contents;
var deleted_from = result.note.deleted;
var revisions_list = result.note.revisions_list;
var deleted_from_id = result.note.deleted;
} else {
var id = null;
var note_text = "<h3>" + note_title;
var deleted_from = null;
var revisions_list = new Array();
var deleted_from_id = null;
var actual_revision = null;
}
if ( revision )
if ( requested_revision )
var read_write = false; // show previous revisions as read-only
else
var read_write = this.read_write;
var editor = this.create_editor( id, note_text, deleted_from, revisions_list, note_title, read_write, true, false );
var editor = this.create_editor( id, note_text, deleted_from_id, actual_revision, read_write, true, false );
id = editor.id;
// if a link that launched this editor was provided, update it with the created note's id
@ -475,7 +482,7 @@ Wiki.prototype.parse_loaded_editor = function ( result, note_title, revision, li
link.href = "/notebooks/" + this.notebook_id + "?note_id=" + id;
}
Wiki.prototype.create_editor = function ( id, note_text, deleted_from, revisions_list, note_title, read_write, highlight, focus ) {
Wiki.prototype.create_editor = function ( id, note_text, deleted_from_id, revision, read_write, highlight, focus ) {
var self = this;
if ( isUndefinedOrNull( id ) ) {
if ( this.read_write ) {
@ -489,13 +496,13 @@ Wiki.prototype.create_editor = function ( id, note_text, deleted_from, revisions
}
// for read-only notes within read-write notebooks, tack the revision timestamp onto the start of the note text
if ( !read_write && this.read_write && revisions_list && revisions_list.length ) {
var short_revision = this.brief_revision( revisions_list[ revisions_list.length - 1 ] );
if ( !read_write && this.read_write && revision ) {
var short_revision = this.brief_revision( revision );
note_text = "<p>Previous revision from " + short_revision + "</p>" + note_text;
}
var startup = this.startup_notes[ id ];
var editor = new Editor( id, this.notebook_id, note_text, deleted_from, revisions_list, read_write, startup, highlight, focus );
var editor = new Editor( id, this.notebook_id, note_text, deleted_from_id, revision, read_write, startup, highlight, focus );
if ( this.read_write ) {
connect( editor, "state_changed", this, "editor_state_changed" );
@ -613,7 +620,7 @@ Wiki.prototype.editor_key_pressed = function ( editor, event ) {
this.create_blank_editor( event );
// ctrl-h: hide note
} else if ( code == 72 ) {
if ( !editor.deleted_from )
if ( !editor.deleted_from_id )
this.hide_editor( event );
// ctrl-d: delete note
} else if ( code == 68 ) {
@ -727,7 +734,7 @@ Wiki.prototype.delete_editor = function ( event, editor ) {
if ( editor == this.focused_editor )
this.focused_editor = null;
if ( this.notebook.trash && !editor.empty() ) {
if ( this.notebook.trash_id && !editor.empty() ) {
var undo_button = createDOM( "input", {
"type": "button",
"class": "message_button",
@ -735,7 +742,7 @@ Wiki.prototype.delete_editor = function ( event, editor ) {
"title": "undo deletion"
} );
var trash_link = createDOM( "a", {
"href": "/notebooks/" + this.notebook.trash.object_id + "?parent_id=" + this.notebook.object_id
"href": "/notebooks/" + this.notebook.trash_id + "?parent_id=" + this.notebook.object_id
}, "trash" );
this.display_message( 'The note has been moved to the', [ trash_link, ". ", undo_button ] )
var self = this;
@ -767,7 +774,7 @@ Wiki.prototype.undelete_editor_via_trash = function ( event, editor ) {
if ( this.read_write && editor.read_write ) {
var self = this;
this.invoker.invoke( "/notebooks/undelete_note", "POST", {
"notebook_id": editor.deleted_from,
"notebook_id": editor.deleted_from_id,
"note_id": editor.id
}, function ( result ) { self.display_storage_usage( result.storage_bytes ); } );
}
@ -816,13 +823,12 @@ Wiki.prototype.save_editor = function ( editor, fire_and_forget ) {
var self = this;
if ( editor && editor.read_write && !editor.empty() ) {
var revisions = editor.revisions_list;
this.invoker.invoke( "/notebooks/save_note", "POST", {
"notebook_id": this.notebook_id,
"note_id": editor.id,
"contents": editor.contents(),
"startup": editor.startup,
"previous_revision": revisions.length ? revisions[ revisions.length - 1 ] : "None"
"previous_revision": editor.revision ? editor.revision : "None"
}, function ( result ) {
self.update_editor_revisions( result, editor );
self.display_storage_usage( result.storage_bytes );
@ -835,8 +841,8 @@ Wiki.prototype.update_editor_revisions = function ( result, editor ) {
if ( !result.new_revision )
return;
var revisions = editor.revisions_list;
var client_previous_revision = revisions.length ? revisions[ revisions.length - 1 ] : null;
var client_previous_revision = editor.revision;
editor.revision = result.new_revision;
// if the server's idea of the previous revision doesn't match the client's, then someone has
// gone behind our back and saved the editor's note from another window
@ -854,11 +860,15 @@ Wiki.prototype.update_editor_revisions = function ( result, editor ) {
self.compare_versions( event, editor, result.previous_revision );
} );
revisions.push( result.previous_revision );
if ( !editor.revisions_list || editor.revisions_list.length == 0 )
return;
editor.revisions_list.push( result.previous_revision );
}
// add the new revision to the editor's revisions list
revisions.push( result.new_revision );
if ( !editor.revisions_list || editor.revisions_list.length == 0 )
return;
editor.revisions_list.push( result.new_revision );
}
Wiki.prototype.search = function ( event ) {
@ -898,7 +908,7 @@ Wiki.prototype.display_search_results = function ( result ) {
}
// otherwise, create an editor for the one note
this.create_editor( note.object_id, note.contents, note.deleted_from, note.revisions_list, undefined, this.read_write, true, true );
this.create_editor( note.object_id, note.contents, note.deleted_from_id, note.revision, this.read_write, true, true );
return;
}
@ -936,7 +946,7 @@ Wiki.prototype.display_search_results = function ( result ) {
);
}
this.search_results_editor = this.create_editor( "search_results", "<h3>search results</h3>" + list.innerHTML, undefined, undefined, undefined, false, true, true );
this.search_results_editor = this.create_editor( "search_results", "<h3>search results</h3>" + list.innerHTML, undefined, undefined, false, true, true );
}
Wiki.prototype.display_all_notes_list = function ( result ) {
@ -954,6 +964,8 @@ Wiki.prototype.display_all_notes_list = function ( result ) {
var note_tuple = result.notes[ i ]
var note_id = note_tuple[ 0 ];
var note_title = note_tuple[ 1 ];
if ( !note_title )
note_title = "untitled note";
appendChildNodes( list,
createDOM( "li", {},
@ -962,7 +974,7 @@ Wiki.prototype.display_all_notes_list = function ( result ) {
);
}
this.all_notes_editor = this.create_editor( "all_notes", "<h3>all notes</h3>" + list.innerHTML, undefined, undefined, undefined, false, true, true );
this.all_notes_editor = this.create_editor( "all_notes", "<h3>all notes</h3>" + list.innerHTML, undefined, undefined, false, true, true );
}
Wiki.prototype.display_message = function ( text, nodes ) {
@ -1099,8 +1111,22 @@ Wiki.prototype.display_empty_message = function () {
this.display_message( "The trash is empty." )
}
DATE_PATTERN = /(\d\d\d\d)-(\d\d)-(\d\d) (\d\d):(\d\d):(\d\d).(\d+)[+-](\d\d:?\d\d)/;
Wiki.prototype.brief_revision = function ( revision ) {
return revision.split( /\.\d/ )[ 0 ]; // strip off seconds from the timestamp
var matches = DATE_PATTERN.exec( revision );
return new Date( Date.UTC(
matches[ 1 ], // year
matches[ 2 ] - 1, // month (zero-based)
matches[ 3 ], // day
matches[ 4 ], // hour
matches[ 5 ], // minute
matches[ 6 ], // second
matches[ 7 ] * 0.001 // milliseconds
) ).toLocaleString();
// return revision.split( /\.\d/ )[ 0 ]; // strip off seconds from the timestamp
}
Wiki.prototype.toggle_editor_changes = function ( event, editor ) {
@ -1112,8 +1138,26 @@ Wiki.prototype.toggle_editor_changes = function ( event, editor ) {
return;
}
new Changes_pulldown( this, this.notebook_id, this.invoker, editor );
event.stop();
// if there's already a cached revision list, display the changes pulldown with it
if ( editor.revisions_list.length > 0 ) {
new Changes_pulldown( this, this.notebook_id, this.invoker, editor );
return;
}
// otherwise, load the revision list for this note from the server
var self = this;
this.invoker.invoke(
"/notebooks/load_note_revisions", "GET", {
"notebook_id": this.notebook_id,
"note_id": editor.id
},
function ( result ) {
editor.revisions_list = result.revisions;
new Changes_pulldown( self, self.notebook_id, self.invoker, editor );
}
);
}
Wiki.prototype.toggle_editor_options = function ( event, editor ) {
@ -1236,13 +1280,13 @@ function Changes_pulldown( wiki, notebook_id, invoker, editor ) {
this.editor = editor;
this.links = new Array();
// display list of revision timestamps in reverse chronological order
if ( isUndefinedOrNull( this.editor.revisions_list ) || this.editor.revisions_list.length == 0 ) {
if ( !editor.revisions_list || editor.revisions_list.length == 0 ) {
appendChildNodes( this.div, createDOM( "span", "This note has no previous changes." ) );
return;
}
var revisions_list = clone( this.editor.revisions_list );
// display list of revision timestamps in reverse chronological order
var revisions_list = clone( editor.revisions_list );
revisions_list.reverse();
var self = this;

4
static/js/test/Editor_setup.js
View File

@ -3,14 +3,14 @@ function setUpPage() {
notebook_id = "fake_notebook_id";
title = "the title"
note_text = "<h3>" + title + "</h3>blah";
deleted_from = undefined;
deleted_from_id = undefined;
revisions_list = undefined;
read_write = true;
startup = false;
highlight = false;
editor_focus = false;
editor = new Editor( id, notebook_id, note_text, deleted_from, revisions_list, read_write, startup, highlight, editor_focus );
editor = new Editor( id, notebook_id, note_text, deleted_from_id, revisions_list, read_write, startup, highlight, editor_focus );
init_complete = false;
connect( editor, "init_complete", function () { init_complete = true; } );

4
static/js/test/Stub_editor.js
View File

@ -1,8 +1,8 @@
function Editor( id, notebook_id, note_text, deleted_from, revisions_list, read_write, startup, highlight, focus ) {
function Editor( id, notebook_id, note_text, deleted_from_id, revisions_list, read_write, startup, highlight, focus ) {
this.id = id;
this.notebook_id = notebook_id;
this.initial_text = note_text;
this.deleted_from = deleted_from || null;
this.deleted_from_id = deleted_from_id || null;
this.revisions_list = revisions_list || new Array();
this.read_write = read_write;
this.startup = startup || false; // whether this Editor is for a startup note

3
static/js/test/Test_editor.html
View File

@ -15,11 +15,10 @@ function test_Editor() {
assertNotUndefined( "editor should have changes_button member", editor.changes_button );
assertNotUndefined( "editor should have options_button member", editor.options_button );
assertFalse( "editor should not have closed flag set", editor.closed );
assertEquals( "editor should have correct deleted_from flag", editor.deleted_from, deleted_from || null );
assertEquals( "editor should have correct deleted_from_id flag", editor.deleted_from_id, deleted_from_id || null );
assertNotUndefined( "editor should have document member", editor.document );
assertEquals( "editor id should have correct id", editor.id, id );
assertNotUndefined( "editor should have iframe member", editor.iframe );
assertEquals( "editor should have empty revisions list", editor.revisions_list.length, 0 );
assertEquals( "editor should have correct startup flag", editor.startup, startup );
assertEquals( "editor should have correct title", editor.title, title );
assertEquals( "editor should have correct read_write flag", editor.read_write, read_write );

1
static/js/test/Test_wiki.html
View File

@ -101,6 +101,7 @@ function test_Wiki() {
<input type="hidden" name="note_id" id="note_id" value="" />
<input type="hidden" name="parent_id" id="parent_id" value="" />
<input type="hidden" name="revision" id="revision" value="" />
<input type="hidden" name="note_contents" id="note_contents" value="" />
</body>
</html>

28
tools/convertdb.py
View File

@ -3,21 +3,31 @@
import os
import os.path
import psycopg2 as psycopg
from controller.Database import Database
from pytz import timezone, utc
from datetime import datetime
from controller.Old_database import Old_database
from controller.Scheduler import Scheduler
pacific = timezone( "US/Pacific" )
def quote( value ):
if value is None:
return "null"
# if this is a datetime, assume it's in the Pacific timezone, and then convert it to UTC
if isinstance( value, datetime ):
value = value.replace( tzinfo = pacific ).astimezone( utc )
value = unicode( value )
return "'%s'" % value.replace( "'", "''" ).replace( "\\", "\\\\" )
class Converter( object ):
"""
Converts a Luminotes database from bsddb to PostgreSQL, using the old bsddb controller.Database.
Converts a Luminotes database from bsddb to PostgreSQL, using the old bsddb controller.Old_database.
This assumes that the PostgreSQL schema from model/schema.sql is already in the database.
"""
def __init__( self, scheduler, database ):
@ -36,8 +46,8 @@ class Converter( object ):
notes = {} # map of note object id to its notebook
startup_notes = {} # map of startup note object id to its notebook
for key in self.database._Database__db.keys():
if not self.database._Database__db.get( key ):
for key in self.database._Old_database__db.keys():
if not self.database._Old_database__db.get( key ):
continue
self.database.load( key, self.scheduler.thread )
@ -102,6 +112,14 @@ class Converter( object ):
( quote( value.object_id ), quote( notebook_id ),
quote( read_only and "f" or "t" ) )
)
if notebook.trash:
self.cursor.execute(
"insert into user_notebook " +
"( user_id, notebook_id, read_write ) " +
"values ( %s, %s, %s );" %
( quote( value.object_id ), quote( notebook.trash.object_id ),
quote( read_only and "f" or "t" ) )
)
elif class_name == "Read_only_notebook":
pass
elif class_name == "Password_reset":
@ -140,7 +158,7 @@ class Converter( object ):
def main():
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
database = Old_database( scheduler, "data.db" )
initializer = Converter( scheduler, database )
scheduler.wait_until_idle()

53
tools/deletenote.py
View File

@ -1,53 +0,0 @@
#!/usr/bin/python2.5
import os
import os.path
from config.Common import settings
from controller.Database import Database
from controller.Scheduler import Scheduler
from model.Note import Note
from tools.initdb import fix_note_contents
class Deleter( object ):
HTML_PATH = u"static/html"
def __init__( self, scheduler, database ):
self.scheduler = scheduler
self.database = database
threads = (
self.delete_note(),
)
for thread in threads:
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def delete_note( self ):
self.database.load( u"User anonymous", self.scheduler.thread )
anonymous = ( yield Scheduler.SLEEP )
read_only_main_notebook = anonymous.notebooks[ 0 ]
main_notebook = anonymous.notebooks[ 0 ]._Read_only_notebook__wrapped
startup_notes = []
for note in main_notebook.notes:
if note and note.title == "try it out": # FIXME: make the note title to delete not hard-coded
print "deleting note %s: %s" % ( note.object_id, note.title )
main_notebook.remove_note( note )
self.database.save( main_notebook )
def main( args ):
print "IMPORTANT: Stop the Luminotes server before running this program."
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Deleter( scheduler, database )
scheduler.wait_until_idle()
if __name__ == "__main__":
import sys
main( sys.argv[ 1: ] )

35
tools/dumpdb.py
View File

@ -1,35 +0,0 @@
#!/usr/bin/python2.5
import os
import os.path
from controller.Database import Database
from controller.Scheduler import Scheduler
class Dumper( object ):
def __init__( self, scheduler, database ):
self.scheduler = scheduler
self.database = database
thread = self.dump_database()
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def dump_database( self ):
for key in self.database._Database__db.keys():
self.database.load( key, self.scheduler.thread )
value = ( yield Scheduler.SLEEP )
print "%s: %s" % ( key, value )
yield None
def main():
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Dumper( scheduler, database )
scheduler.wait_until_idle()
if __name__ == "__main__":
main()

99
tools/initdb.py
View File

@ -1,14 +1,12 @@
#!/usr/bin/python2.5
#!/usr/bin/python2.4
import os
import os.path
import sys
from controller.Database import Database
from controller.Scheduler import Scheduler
from model.Notebook import Notebook
from model.Read_only_notebook import Read_only_notebook
from model.Note import Note
from model.User import User
from model.User_list import User_list
from new_model.Notebook import Notebook
from new_model.Note import Note
from new_model.User import User
class Initializer( object ):
@ -27,78 +25,68 @@ class Initializer( object ):
( u"advanced browser features.html", False ),
]
def __init__( self, scheduler, database ):
self.scheduler = scheduler
def __init__( self, database, nuke = False ):
self.database = database
self.main_notebook = None
self.read_only_main_notebook = None
self.anonymous = None
threads = (
self.create_main_notebook(),
self.create_anonymous_user(),
)
if nuke is True:
self.database.execute( file( "new_model/drop.sql" ).read(), commit = False )
for thread in threads:
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
self.database.execute( file( "new_model/schema.sql" ).read(), commit = False )
self.create_main_notebook()
self.create_anonymous_user()
self.database.commit()
def create_main_notebook( self ):
# create the main notebook and all of its notes
self.database.next_id( self.scheduler.thread )
main_notebook_id = ( yield Scheduler.SLEEP )
self.main_notebook = Notebook( main_notebook_id, u"Luminotes" )
# create the read-only view of the main notebook
self.database.next_id( self.scheduler.thread )
read_only_main_notebook_id = ( yield Scheduler.SLEEP )
self.read_only_main_notebook = Read_only_notebook( read_only_main_notebook_id, self.main_notebook )
# create the main notebook
main_notebook_id = self.database.next_id( Notebook )
self.main_notebook = Notebook.create( main_notebook_id, u"Luminotes" )
self.database.save( self.main_notebook, commit = False )
# create an id for each note
note_ids = {}
for ( filename, startup ) in self.NOTE_FILES:
self.database.next_id( self.scheduler.thread )
note_ids[ filename ] = ( yield Scheduler.SLEEP )
note_ids[ filename ] = self.database.next_id( Note )
rank = 0
for ( filename, startup ) in self.NOTE_FILES:
full_filename = os.path.join( self.HTML_PATH, filename )
contents = fix_note_contents( file( full_filename ).read(), read_only_main_notebook_id, note_ids )
note = Note( note_ids[ filename ], contents )
self.main_notebook.add_note( note )
contents = fix_note_contents( file( full_filename ).read(), main_notebook_id, note_ids )
if startup:
self.main_notebook.add_startup_note( note )
rank += 1
self.database.save( self.main_notebook )
self.database.save( self.read_only_main_notebook )
note = Note.create( note_ids[ filename ], contents, notebook_id = self.main_notebook.object_id, startup = startup, rank = startup and rank or None )
self.database.save( note, commit = False )
def create_anonymous_user( self ):
# create the anonymous user
self.database.next_id( self.scheduler.thread )
anonymous_user_id = ( yield Scheduler.SLEEP )
notebooks = [ self.read_only_main_notebook ]
self.anonymous = User( anonymous_user_id, u"anonymous", None, None, notebooks )
self.database.save( self.anonymous )
anonymous_user_id = self.database.next_id( User )
self.anonymous = User.create( anonymous_user_id, u"anonymous", None, None )
self.database.save( self.anonymous, commit = False )
# create a user list
self.database.next_id( self.scheduler.thread )
user_list_id = ( yield Scheduler.SLEEP )
user_list = User_list( user_list_id, u"all" )
user_list.add_user( self.anonymous )
self.database.save( user_list )
# give the anonymous user read-only access to the main notebook
self.database.execute( self.anonymous.sql_save_notebook( self.main_notebook.object_id, read_write = False ), commit = False )
def main():
print "IMPORTANT: Stop the Luminotes server before running this program."
def main( args = None ):
nuke = False
if os.path.exists( "data.db" ):
os.remove( "data.db" )
if args and ( "-n" in args or "--nuke" in args ):
nuke = True
print "This will nuke the contents of the database before initializing it with default data. Continue (y/n)? ",
confirmation = sys.stdin.readline().strip()
print
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Initializer( scheduler, database )
scheduler.wait_until_idle()
if confirmation.lower()[ 0 ] != 'y':
print "Exiting without touching the database."
return
print "Initializing the database with default data."
database = Database()
initializer = Initializer( database, nuke )
def fix_note_contents( contents, notebook_id, note_ids ):
@ -134,4 +122,5 @@ def fix_note_contents( contents, notebook_id, note_ids ):
if __name__ == "__main__":
main()
import sys
main( sys.argv[ 1: ] )

47
tools/listuser.py
View File

@ -1,47 +0,0 @@
#!/usr/bin/python2.5
import os
import os.path
import sys
from config.Common import settings
from controller.Database import Database
from controller.Scheduler import Scheduler
class Lister( object ):
def __init__( self, scheduler, database, username ):
self.scheduler = scheduler
self.database = database
self.username = username
threads = (
self.list_user(),
)
for thread in threads:
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def list_user( self ):
self.database.load( u"User %s" % self.username, self.scheduler.thread )
user = ( yield Scheduler.SLEEP )
if user is None:
print "user %s is unknown" % self.username
else:
print "user %s: %s" % ( self.username, user )
def main( program_name, args ):
if len( args ) == 0:
print "usage: %s username" % program_name
sys.exit( 1 )
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Lister( scheduler, database, args[ 0 ] )
scheduler.wait_until_idle()
if __name__ == "__main__":
import sys
main( sys.argv[ 0 ], sys.argv[ 1: ] )

6
tools/release.sh
View File

@ -1,11 +1,7 @@
#!/bin/sh
# Run this from the root directory of Luminotes's source. Note: This will nuke your database!
# Run this from the root directory of Luminotes's source.
ORIG_PYTHONPATH="$PYTHONPATH"
export PYTHONPATH=.
python2.5 tools/initdb.py
export PYTHONPATH="$ORIG_PYTHONPATH"
cd ..
rm -f luminotes.tar.gz
tar cvfz luminotes.tar.gz --exclude=session --exclude="*.log" --exclude="*.pyc" --exclude=".*" luminotes

36
tools/reloaddb.py
View File

@ -1,36 +0,0 @@
#!/usr/bin/python2.5
import os
import os.path
from controller.Database import Database
from controller.Scheduler import Scheduler
class Reloader( object ):
def __init__( self, scheduler, database ):
self.scheduler = scheduler
self.database = database
thread = self.reload_database()
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def reload_database( self ):
for key in self.database._Database__db.keys():
self.database.reload( key, self.scheduler.thread )
yield Scheduler.SLEEP
yield None
def main():
print "IMPORTANT: Stop the Luminotes server before running this program."
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Reloader( scheduler, database )
scheduler.wait_until_idle()
if __name__ == "__main__":
main()

61
tools/resetpw.py
View File

@ -1,61 +0,0 @@
#!/usr/bin/python2.5
import os
import os.path
import sys
from config.Common import settings
from controller.Database import Database
from controller.Scheduler import Scheduler
class Resetter( object ):
def __init__( self, scheduler, database, username ):
self.scheduler = scheduler
self.database = database
self.username = username
self.password = None
self.prompt_for_password()
threads = (
self.reset_password(),
)
for thread in threads:
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def prompt_for_password( self ):
print "enter new password for user %s: " % self.username,
sys.stdout.flush()
self.password = sys.stdin.readline().strip()
print
def reset_password( self ):
self.database.load( u"User %s" % self.username, self.scheduler.thread )
user = ( yield Scheduler.SLEEP )
if user is None:
raise Exception( "user %s is unknown" % self.username )
user.password = self.password
self.database.save( user )
print "password reset"
def main( program_name, args ):
print "IMPORTANT: Stop the Luminotes server before running this program."
if len( args ) == 0:
print "usage: %s username" % program_name
sys.exit( 1 )
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Resetter( scheduler, database, args[ 0 ] )
scheduler.wait_until_idle()
if __name__ == "__main__":
import sys
main( sys.argv[ 0 ], sys.argv[ 1: ] )

53
tools/setemail.py
View File

@ -1,53 +0,0 @@
#!/usr/bin/python2.5
import os
import os.path
import sys
from config.Common import settings
from controller.Database import Database
from controller.Scheduler import Scheduler
class Setter( object ):
def __init__( self, scheduler, database, username, email_address ):
self.scheduler = scheduler
self.database = database
self.username = username
self.email_address = email_address
self.password = None
threads = (
self.set_email_address(),
)
for thread in threads:
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def set_email_address( self ):
self.database.load( u"User %s" % self.username, self.scheduler.thread )
user = ( yield Scheduler.SLEEP )
if user is None:
raise Exception( "user %s is unknown" % self.username )
user.email_address = self.email_address
self.database.save( user )
print "email set"
def main( program_name, args ):
print "IMPORTANT: Stop the Luminotes server before running this program."
if len( args ) < 2:
print "usage: %s username emailaddress" % program_name
sys.exit( 1 )
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Setter( scheduler, database, *args )
scheduler.wait_until_idle()
if __name__ == "__main__":
import sys
main( sys.argv[ 0 ], sys.argv[ 1: ] )

53
tools/setplan.py
View File

@ -1,53 +0,0 @@
#!/usr/bin/python2.5
import os
import os.path
import sys
from config.Common import settings
from controller.Database import Database
from controller.Scheduler import Scheduler
class Setter( object ):
def __init__( self, scheduler, database, username, rate_plan ):
self.scheduler = scheduler
self.database = database
self.username = username
self.rate_plan = rate_plan
self.password = None
threads = (
self.set_rate_plan(),
)
for thread in threads:
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def set_rate_plan( self ):
self.database.load( u"User %s" % self.username, self.scheduler.thread )
user = ( yield Scheduler.SLEEP )
if user is None:
raise Exception( "user %s is unknown" % self.username )
user.rate_plan = int( self.rate_plan )
self.database.save( user )
print "rate plan set"
def main( program_name, args ):
print "IMPORTANT: Stop the Luminotes server before running this program."
if len( args ) < 2:
print "usage: %s username rateplan" % program_name
sys.exit( 1 )
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Setter( scheduler, database, *args )
scheduler.wait_until_idle()
if __name__ == "__main__":
import sys
main( sys.argv[ 0 ], sys.argv[ 1: ] )

97
tools/updatedb.py
View File

@ -1,16 +1,16 @@
#!/usr/bin/python2.5
#!/usr/bin/python2.4
import os
import os.path
from config.Common import settings
from controller.Database import Database
from controller.Scheduler import Scheduler
from model.Note import Note
from model.User_list import User_list
from new_model.Notebook import Notebook
from new_model.Note import Note
from new_model.User import User
from tools.initdb import fix_note_contents
class Initializer( object ):
class Updater( object ):
HTML_PATH = u"static/html"
NOTE_FILES = [ # the second element of the tuple is whether to show the note on startup
( u"about.html", True ),
@ -25,101 +25,58 @@ class Initializer( object ):
( u"advanced browser features.html", False ),
]
def __init__( self, scheduler, database, navigation_note_id = None ):
self.scheduler = scheduler
def __init__( self, database, navigation_note_id = None ):
self.database = database
self.navigation_note_id = navigation_note_id
threads = (
self.create_user_list(),
self.update_main_notebook(),
)
for thread in threads:
self.scheduler.add( thread )
self.scheduler.wait_for( thread )
def create_user_list( self ):
# if there's no user list, create one and populate it with all users in the database
self.database.load( u"User_list all", self.scheduler.thread )
user_list = ( yield Scheduler.SLEEP )
if user_list is not None:
return
self.database.next_id( self.scheduler.thread )
user_list_id = ( yield Scheduler.SLEEP )
user_list = User_list( user_list_id, u"all" )
for key in self.database._Database__db.keys():
if not key.startswith( "User " ): continue
self.database.load( key, self.scheduler.thread )
user = ( yield Scheduler.SLEEP )
if user:
user_list.add_user( user )
self.database.save( user_list )
self.update_main_notebook()
self.database.commit()
def update_main_notebook( self ):
self.database.load( u"User anonymous", self.scheduler.thread )
anonymous = ( yield Scheduler.SLEEP )
read_only_main_notebook = anonymous.notebooks[ 0 ]
main_notebook = anonymous.notebooks[ 0 ]._Read_only_notebook__wrapped
startup_notes = []
anonymous = self.database.select_one( User, User.sql_load_by_username( u"anonymous" ) )
main_notebook = self.database.select_one( Notebook, anonymous.sql_load_notebooks() )
# get the id for each note
note_ids = {}
for ( filename, startup ) in self.NOTE_FILES:
title = filename.replace( u".html", u"" )
note = main_notebook.lookup_note_by_title( title )
note = self.database.select_one( Note, main_notebook.sql_load_note_by_title( title ) )
if note is not None:
note_ids[ filename ] = note.object_id
# update the navigation note if its id was given
if self.navigation_note_id:
self.database.next_id( self.scheduler.thread )
next_id = ( yield Scheduler.SLEEP )
note = main_notebook.lookup_note( self.navigation_note_id )
self.update_note( "navigation.html", True, main_notebook, read_only_main_notebook, startup_notes, next_id, note_ids, note )
note = self.database.load( Note, self.navigation_note_id )
self.update_note( "navigation.html", True, main_notebook, note_ids, note )
self.database.save( note, commit = False )
# update all of the notes in the main notebook
for ( filename, startup ) in self.NOTE_FILES:
self.database.next_id( self.scheduler.thread )
next_id = ( yield Scheduler.SLEEP )
title = filename.replace( u".html", u"" )
note = main_notebook.lookup_note_by_title( title )
self.update_note( filename, startup, main_notebook, read_only_main_notebook, startup_notes, next_id, note_ids, note )
note = self.database.select_one( Note, main_notebook.sql_load_note_by_title( title ) )
self.update_note( filename, startup, main_notebook, note_ids, note )
for note in startup_notes:
main_notebook.add_startup_note( note )
if main_notebook.name != u"Luminotes":
main_notebook.name = u"Luminotes"
self.database.save( main_notebook, commit = False )
main_notebook.name = u"Luminotes"
self.database.save( main_notebook )
def update_note( self, filename, startup, main_notebook, read_only_main_notebook, startup_notes, next_id, note_ids, note = None ):
def update_note( self, filename, startup, main_notebook, note_ids, note = None ):
full_filename = os.path.join( self.HTML_PATH, filename )
contents = fix_note_contents( file( full_filename ).read(), read_only_main_notebook.object_id, note_ids )
contents = fix_note_contents( file( full_filename ).read(), main_notebook.object_id, note_ids )
if note:
main_notebook.update_note( note, contents )
note.contents = contents
# if for some reason the note isn't present, create it
else:
note = Note( next_id, contents )
main_notebook.add_note( note )
main_notebook.remove_startup_note( note )
if startup:
startup_notes.append( note )
next_id = self.database.next_id( Note )
note = Note.create( next_id, contents, notebook_id = main_notebook.object_id, startup = startup )
self.database.save( note, commit = False )
def main( args ):
print "IMPORTANT: Stop the Luminotes server before running this program."
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
initializer = Initializer( scheduler, database, args and args[ 0 ] or None )
scheduler.wait_until_idle()
database = Database()
initializer = Updater( database, args and args[ 0 ] or None )
if __name__ == "__main__":

24
tools/verifyconvertdb.py
View File

@ -3,7 +3,7 @@
import os
import os.path
import psycopg2 as psycopg
from controller.Database import Database
from controller.Old_database import Old_database
from controller.Scheduler import Scheduler
@ -34,8 +34,8 @@ class Verifier( object ):
def verify_database( self ):
inserts = set()
for key in self.database._Database__db.keys():
if not self.database._Database__db.get( key ):
for key in self.database._Old_database__db.keys():
if not self.database._Old_database__db.get( key ):
continue
self.database.load( key, self.scheduler.thread )
@ -92,6 +92,16 @@ class Verifier( object ):
assert row[ 1 ] == notebook.object_id
assert row[ 2 ] == read_write
if notebook.trash:
self.cursor.execute(
"select * from user_notebook where user_id = %s and notebook_id = %s;" % ( quote( value.object_id ), quote( notebook.trash.object_id ) )
)
for row in self.cursor.fetchmany():
assert row[ 0 ] == value.object_id
assert row[ 1 ] == notebook.trash.object_id
assert row[ 2 ] == read_write
self.verify_notebook( notebook )
elif class_name == "Read_only_notebook":
@ -105,9 +115,9 @@ class Verifier( object ):
)
for row in self.cursor.fetchmany():
assert row[ 0 ] == value.email_address
assert row[ 1 ] == False
assert row[ 2 ] == value.object_id
assert row[ 0 ] == value.object_id
assert row[ 1 ] == value.email_address
assert row[ 2 ] == False
elif class_name == "User_list":
pass
else:
@ -167,7 +177,7 @@ class Verifier( object ):
def main():
scheduler = Scheduler()
database = Database( scheduler, "data.db" )
database = Old_database( scheduler, "data.db" )
initializer = Verifier( scheduler, database )
scheduler.wait_until_idle()