Persevere

Persevere is an open source persistent object mapping framework for JavaScript in the browser. Persevere allows programmers to access, traverse, and manipulate persisted objects graphs easily with standard JavaScript syntax and intuitive Persistent JavaScript (PJS) API. Persevere implements the PJS API and maps JavaScript objects to persistent storage objects remotely through JSON based RESTful web services following the JSPON (JavaScript Persistent Object Notation) data interchange specification. Persevere accesses persisted object graphs provided through JSPON data sources which can represent various underlying sources such as database tables, XML files, web services, and object repositories, and such persisted graphs can even span domains. Persevere allows application code to be persisted and exist within these object graphs, to facilitate improved object oriented design and organization. Persevere can provide orthogonal persistence and lazy loading support with standard JavaScript property access and modification through JavaScript pre-processing, or it can provide these capabilities natively with JavaScript 1.7. Persevere is built on top of Strands and therefore supports coroutines to provide continuations and threading. With Persevere, you can use access objects that are mapped to persistent stores on a server with standard JavaScript. In this example we can see access and manipulation of an object:

var purchaseOrder = pjs.load(poId); // load an object by id
var customerName = purchaseOrder.customer.name; // retrieve the customerís name
purchaseOrder.quantity = 5; // change the quantity

In this example, Persevere loads the object from the server, and retrieves the customer name. If the customer object and the name string have not been loaded from the server, then it can automatically retrieve this information from the server using JSPON. Persevere handles all the Ajax interaction with the server transparently so that you donít need to worry about the remote requests. In the last line in the example, the persistent field for quantity is changed. Persevere can automatically send the data change back to the server as well for the appropriate field to be updated. In this example, the data could be representing database tables, such as a purchase order table that is related to a customer table. You can visit the Persistent JavaScript site for the full PJS API documentation. Additional Persevere functions are documented here.

Modes of operation

Persevere supports three different coding methods. For the first two methods, files with persistence capabilities should be loaded by calling persevere.loadScript(scriptName). The loadScript will use the appropriate loading and processing mechanism. Here are the methods:

Persistent JavaScript is standard JavaScript that is processed/compiled to have persistence awareness and coroutine support for suspending while retrieving lazy loaded data. Using this method is a good choice when objects are frequently be changed and code needs to be kept as simple as possible. Persistent JavaScript files should have .pjs suffix. In Persistent JavaScript we can write functions that might access lazy loading properties with normal JavaScript syntax. The lazy loading is performed transparently (no special syntax needed):
```
function getCustomerName(purchaseOrder) {
        var customerName = purchaseOrder.customer.name; // if customer or name is not loaded yet, it will be loaded automatically
        return customerName;
}
```
example-pjs.html has examples of using Persevere in this mode.
Persevere can run natively (without compilation) in JavaScript 1.7 as implemented in Firefox 2 and higher. In order to utilize coroutine support for suspending execution for lazy loading, the JS 1.7 yield operator is used. The Persevere compiler can compile JavaScript 1.7 code in to 1.5 code for other browsers. This allows you to write and debug code in Firefox, but still retain support for other browsers. When we use the yield operator we must wrap the function with the strand wrapper. We would now write the getCustomerName function:
```
var getCustomerName = strand(function (purchaseOrder) {
        var customerName = yield purchaseOrder.customer.name; // if the name is not loaded yet, it will be loaded automatically
        returns(customerName); 
});
```
The yield operator only needs to be used when accessing a property that may need to be lazy loaded requiring suspension, or when calling another function that might require suspension. JavaScript 1.7 files should have .js17 suffix. example-js17.html has examples of using Persevere in this mode. This is a good choice to combine the power of inline lazy loading, and still control of what is delivered to the browser (in Firefox) and allows flexibility and power of controlling which methods can suspend execution and when to save properties. With this level of control, higher performance can be acheived than the first approach.
The Persevere API can be used without pre-processing/compilation and consequently without transparent lazy loading and object modification detection. However, Persevere still retains persistent object mapping capabilities, as well lazy loading and object modification through the API. Please see the section on utilizing the PJS API from uncompiled code for more information. example-api.html has examples of using Persevere in this mode. This is a good choice if you want to avoid any pre-processing/compilation.

Using Compiled JavaScript

Compiled in persisted objects

With the Persevere JSPON/Object Browser you can build persistent objects and persistent methods directly into your persistent storage. Using the browser, your code is automatically compiled, and developing directly in the object graph enables type checking to work properly and encourages organization, object oriented design, and inversion of control (dependency injection). With this approach you can use the full persistence and coroutine support. Compiled JavaScript files The second approach is create .pjs or .js17 files that use PJS or JavaScript 1.7 syntax. Persevere can automatically process .pjs files and .js17 files on the client to add persistence and coroutine support. When using this approach ensure that pjs.compilerURL has the correct URL for the compiler.js file. This provides a very simple way to use persistence, however processing on the client adds significant overhead and is therefore not recommended for production use. Instead, .pjs and .js17 files should be compiled beforehand with the provided Persevere compiler before maximum in production environments. If your JavaScript files are precompiled (versus dynamic compilation in the browser), you should set

persevere.precompiled=true

There are two ways to compile .pjs files before they are sent to the browser:

To use the PJSCompiler in a build. Use the following syntax from the build directory: >java -classpath lib/js.jar;lib/persevere.jar com.xucia.persevere.PersevereCompiler js/myfile.pjs Use the provided PJSCompiler Java servlet. If you are using a J2EE application server, this is easiest approach. Add the persevere.jar and persevere-compiler.js to your classpath and add the following entry to your web.xml for your web application:

  
    PersevereCompiler
    com.xucia.persevere.servlet.PersevereCompilerServlet
  
  
    PersevereCompiler
    *.js

With these settings .pjs and .js17 will automatically be compiled for you when they are requested from the server

Utilizing the Persevere PJS API from Uncompiled Code

Uncompiled code can not utilize transparent lazy loading and automatic object modification detection, nor coroutine capabilities. However, uncompiled code can utilize the PJS API in Persevere, and access all the persistent object mapping capabilities of Persevere. By default, Persevere uses asynchronous requests for certain methods and without pre-processing you must provide a callback handler if you want to resume execution after the completion of the method. To use asynchronous pjs library functions from uncompiled code looks like this:

pjs.load(poId,function(purchaseOrder) { // load purchase order
        pjs.get(purchaseOrder,"customer",function(customer) { // if the customer property is lazy loading, we must use the pjs.get method 
                var customerName = customer.name; // assuming that name property is not lazy loaded  
                purchaseOrder.quantity=5;
                pjs.save(purchaseOrder);                
        });
});

Synchronous mode

The simplest way to use the PJS library from outside compiled code is by using synchronous mode. In synchronous mode, all requests to the server are done with synchronous calls and therefore there is no need for the functions to suspend and resume using continuations. However, in synchronous mode all calls to server cause the browser to lock up until the call is completed. This is generally a bad experience for users and therefore this approach is generally not recommended. To set synchronous mode:

pjs.synchronousMode = true;

In synchronous mode you can utilize the pjs library functions as you would normally. For example to load a purchase order and get the customers name and set the quantity like in the earlier example we could write:

var purchaseOrder = pjs.load(poId);
var customerName = pjs.get(purchaseOrder,"customer").name;
...

Note, you should not mix synchronous and asynchronous mode. You must stay in one mode and not switch back and forth.

Persevere API

Most of the core functionality of Persevere is defined in the Persistent JavaScript API. The API is copied here for your reference:

Persistent JavaScript also defines an API for accessing and manipulating persistent objects. The API is contained in the pjs namespace.

pjs.load(id:String[, callback:Function]):*

Returns an object with the given id. All persisted objects should have an id, and this functions provides the ability to retrieve an object by its id. callback is an optional parameter if an asynchronous non-blocking operation is used to load the id. If available, callback will be called when the load is finished, and will be called with a single parameter, the value loaded.

pjs.set(target:Object,key:String,value:):

This will set a persistent property of the given target object.

pjs.get(target:Object,key:String[, callback:Function]):*

This will retrieve a property from the given target object. This should act essentially the same as target[key]. callback is an optional parameter if an asynchronous non-blocking operation is used to get the property (lazy loading). If available, callback will be called when the get is finished, and will be called with a single parameter, the value retrieved.

pjs.commit([callback:Function])

Commits all the changes that have been made in the current transaction. callback is an optional parameter if an asynchronous non-blocking operation is used to commit. If available, callback will be called when the commit is finished, and will be called with a single parameter, a boolean indicating the success of the operation.

pjs.rollback()

Rolls back all the changes that have been made in the current transaction.

pjs.changing(object)

Prepares an object to be modified, this is not necessary if you using orthogonal persistence.

pjs.save(object)

Saves the changes made to a persistent object in the current transaction.

pjs.getAccessLevel(object:Object):Number

Returns the access level of the current user on the given object. The access level should be an integer denoted a permission according to this table:
    * 0 - none - can not access any of the fields on this object
    * 1 - browse - can only read from a limited set of fields (usually name and basis)
    * 2 - read - can read all the information from this object, but can not make any modifications to this object
    * 3 - append - for list objects, entries can be appended, but no property modifications can be made
    * 5 - write - any modifications can be made to this object
    * Note that Persistent JavaScript is garbage collection based so there is no concept of deleting an object, only properties. Objects are deleted when all references to them are removed.

pjs.getId(object:Object)

Returns the object id for the given object.

pjs.isPersisted(object:Object)

Returns true if the object has been persisted or is in the act of being persisted.

pjs.newInstance(basis:Object):Object

This function is called whenever the new operator is applied to an object (not a function). This returns a new object with a basis of the provided basis parameter.

Array support

Persevere also adds persistence capabilities to the core JavaScript array methods. Because Persevere supports lazy loading, Persevere adds an additional optional callback parameter to all the array methods to support situations where lazy loading is needed. For example if an array object had items that had not been initialized, one could call join this way:

myArray.join(',',function(returnValue){
        alert('Here is the joined array: ' + returnValue); 
}

In addition, provides several static methods on the Array constructor object. In particular, it is recommended to use the Array.forEach or Array.some for iterating through arrays. The Array.forEach automatically handles lazy loaded arrays properly, so it can easily be used in any programming mode (pre-processed or not) to iterate through an array. The Array.prototype.forEach offers the same functionality, and can be used in Firefox, but it is not available in IE. It also recommended that use Array.remove to remove elements from arrays that represent sets, like query results and database table data.

Array.forEach(array:Array, func:Function(value:*,index:Number,array:Array)[, callback:Function()])

Array.every(array:Array, func:Function(value:*,index:Number,array:Array)[, callback:Function()])

Array.some(array:Array, func:Function(value:*,index:Number,array:Array)[, callback:Function()])

This function can be used to iterate through an array. For example:

Array.some(myArray,function(value,i) {
        ... // processing for each item
        return i >= 50; // when it is gets to 50 stop iterating
},done); // done will be called when it is finished iterating

Array.remove(array:Array, valueToRemove:*)

Removes the given value from the array (the first occurence if it exists multiple times)

Persevere and Strands API

Persevere is built on Strands, and therefore includes the full Strands API for coroutine and threading support. However these additional functions are also available:

persevere.loadScript(scriptName)

This function should be used to load scripts for Persevere. If a .pjs file (JavaScript with persistence support) or .js17 file is loaded than client side compilation is performed if necessary (if it was compiled on the server, or and the browser supports doesn't support JavaScript 1.7).
persevere.compilerURL should be set to the correct URL for the compiler.js file. This defaults to "js/compiler.js" which should be correct for files that are in the root of the installation directory.
persevere.precompiled should be set to true if the compilation is done the server. persevere.setAutoSave(autoSave) This sets whether or not Persevere should automatically save changes made to persisted objects, without needing to call save (orthogonal persistence). This is on by default with Persistent JavaScript files (.pjs) and off by default with others (.js17 and .js files). This option is not supported in pre JavaScript 1.7 environments without pre-processing. If setAutoSave is called, it should be called after loading persevere.js and prior to loading any persistent objects or loading any other pre-processed scripts (.js17 or .pjs files):

If this is not enabled, one must call pjs.save(object) to save changes made to a persistent object. If this set, all scripts should be loaded with loadScript to ensure that any necessary compilation takes place. If setAutoSave is set to true and scripts are loaded without loadScript, they will behave differently in Firefox and IE. Firefox supports auto save natively without compilation, and IE does not, so property changes could be persisted in Firefox and not persisted in IE. If setAutoSave is set to false, scripts do not necessarily need to be loaded with loadScript for Firefox and IE to behave the same. persevere.request(url,params,operation) This makes an Ajax call to the server for the given url, with the given parameters. The parameters can be any objects, JSPON referencing and serialization is used to transfer the values. The operation parameter defines the status of what is happening for the UI. persevere.stringify(value) This method converts a value to a JSPON string representation. pjs.rpc(thisObj, params, methodName [, callback]):Object This will fire remote RPC call to the server.

Persistent and Transient Properties

Persevere supports mixing persistent and transient properties on objects. However, it is important to understand which properties are treated as persistent. Persistent and transient properties can be explicitly in the object structure. By creating an object structure in the JSPON browser, you can define for each property if it is persistent by setting the persistent property in the corresponding structure property. You can see Persistent JavaScript specification for more information. If the property is not explicity defined in the structure, properties will be treated as persistent if they were loaded from persistent information on the server. If any changes are made to these properties while in auto save mode, the changes will be automatically be persisted. Any new properties that are added will be treated as transient properties (no changes will be saved, even if in auto save), until pjs.save is called for the object. If pjs.save is called on an object, all properties, new and old, that are not explicitly defined as transient in the structure will be persisted.

JSPON Object Browser

Persevere comes with the JSPON persistent object browser, which is a free, open-source tool for browsing and manipulated JSPON persisted object graphs/data sources. The JSPON browser can be found at www.jspon.org/browser.html or downloaded at www.jspon.org/files/browser.zip . JSPON is a JSON extension for robust access and manipulation of persisted objects that includes definitions for how to support object and array identification, referencing, lazy loading/partial graph transfer, object modification, prototype definition, and more. The JSPON specification can be found at www.jspon.org.

You can start the browser by opening browser.html in your browser. There is sample JSPON data that comes with the browser. When the browser asks what object you want to open, enter "SampleData". You can then see some examples of the expressive capabilities of the JSPON. There are further examples that can be access on the jspon website. Using the JSPON browser, you can load the object http://www.jspon.org/browser.html?id=dyna%2F100788 (or https://xucia.com/browser.html?id=dyna%2F100788 if you are using Adblock/filterset G which blocks jspon for some reason). The following properties of the root object from the example url illustrate some of the major aspects of JSPON:

types - Just the different types of values that can be transferred with good 'ol JSON: string, boolean, number, array/list, object, and null. This includes a real date object (defined by JSPON).

referencing - This should hows identification/referencing allows circular referencing and multiple references to single objects. This is a graph of the Doe family, and the properties refer to the family members by relation, but since the objects are referenced not just copied, the graph can be infinitely recursively drilled down, and it is still the same objects that are being accessed.

lazyLoading - This can be seen when drilling into any part of the persisted graph that has not been loaded yet, but this property references a larger set of the graph, and you can see how values are loaded into the graph as needed. JSPON provides definition for the identification and lazy endpoints so this client browser can provide lazy loading for any JSPON data source.

inheritance - This is an object that is defined to inherit properties from another object. This object is inheriting from the object referred by the type property. In other words the object with the id of dyna/107650 has a prototype or delegate that is the object with the id of dyna/107653. If a property is changed or added in the delegate object, they will be reflected in the instance object, unless the instance object defines that property. In this case the stringField property is defined in the instance and all the other properties are inherited from the delegate object. The basis property refers to the delegate/prototype object. The browser provides coloring to help know what is inherited and what is not.

modifiableData - This object has been defined by the server to be publicly modifiable, so you can create and modify objects on my server. You can go ahead and add new properties and modify values in this object. You can delete properties if it looks like there is plenty of other ones, but please refrain from deleting all the properties, we want to keep it fun for everyone. You can modify properties and watch your HTTP traffic to see how the object modifications are sent back to the server. You can view the headers (the Access-Level header) to see how the definitions for access levels is transferred to the client from the server. Of course, a server shouldn't and this server doesn't count on the client to enforce the access levels, but maintains it's own security as well.

structuredObject - This object has a structure definition, in the structure property (provided that no one has deleted it, it is publicly modifiable), and it is defined to allow only a string value in the stringOnly field, and only a number in the numberOnly. You can try to change value of the numberOnly property to have letters in it, and it should complain.

crossDomain - The browser supports the JSON-P protocol for cross domain transfer, so this persisted object graph actually spans domains, and references objects from another host within the graph. Of course, the usual warnings about using cross-domain dynamic scripts applies, you don't want to use the browser to connect your high-security objects with an untrusted source using JSON-P, but rather use a proxy, but in this case the foreign host is one of my own domains (www.authenteo.com), so it is trustworthy. This property refers to a list persitent objects that represent the pages on my www.authenteo.com site. Having persisted object graphs that span domains can make mashups have a much more consistent object model.

differentSources - This object references to JSPON objects that represent different source mediums. There is a SQL database (HSQL), an XML file, and an RSS source. The server can expose all of these data sources as JSPON, and they can even be modified using the JSPON browser, and server can make the appropriate SQL updates, or XML file changes (OK, the RSS isn't modifiable, and you can't modify any of these data sources, because I haven't given you that access level). This demonstrates the flexibility of JSPON as persistence protocol for different mediums of data (and it also demonstrates the capabilites of the Jsponic server that is being used). If you want access to modify the SQL tables, or the XML files, let me know, and I can set you up with an account.

perseverePreview - This is all built with my soon to be released Persevere Ajax persistence framework which maps remotely persisted data to JavaScript objects in the browser with orthogonal persistence. This property refers to a persisted function object that gives a little preview, and uses the persistence mapped objects to access a persisted property (this.referencing.wife.name). When accessing properties, the framework will transparently perform lazy loading through JSPON of any data that has not been transferred. Then the function sets a new value in this.modifiableData.randomNumber. This value is automatically persisted to the server and should be available if you come back to the page later (if someone else hasn't changed it). Since the data is mapped to JS objects, you can also look at this object graph in JS debugger (like Firebug). Add a watch for the global variable "rootObject". The debugger won't be capable of lazy loading, but you can see everything that has been downloaded.

The server is the Jsponic server (JSPON server) that will also be released soon, but the purpose of this tool is to make the communication protocol visible and open so others can review and hopefully implement it in other technologies.

If anyone would like their own account on my object server, and their own access controlled persistent object graph, just let me know and I will set you up ().

License

BEGIN LICENSE BLOCK
Version: MPL 1.1/GPL 2.0/LGPL 2.1

The contents of this file are subject to the Mozilla Public License Version
1.1 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.mozilla.org/MPL/

Software distributed under the License is distributed on an "AS IS" basis,
WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
for the specific language governing rights and limitations under the
License.
END LICENSE BLOCK