My Current Model Framework, Part 1

Sun Nov 17 12:19:18 EST 2013

Tags: xpages mvc
  1. Nov 17 2013 - My Current Model Framework, Part 1
  2. Feb 21 2014 - My Current Model Framework, Part 2: An Example

As I do from time to time, I've recently been taking another stab at a standard model framework for my XPage apps. My latest one has been proving its worth in a couple apps I've been writing lately, and I'd like to go over the general goals and advantages of the way it works.

The framework is focused on a couple main ideas:

  • Low Overhead. The Java language requires a certain amount of overhead to get anything done, but I want to minimize that. The task of defining a new model object consists of two classes: one for the object itself and one for the collection manager (e.g. connecting to Domino views), and each is geared towards only writing the code required to describe the necessities.
  • Embracing XSP. The framework is intended for writing XPages apps, and so I want to make sure it works smoothly with EL and standard controls like inputs, repeats, and view panels.
  • Embracing document databases. Since I use almost entirely a document database for storage, I want to make sure to take advantage of that, and that primarily means flexibility in data modeling. Accordingly, data objects allow arbitrary field access, with any strictures from your model class layered on top of that.
  • Embracing Domino. Since the document database I use is Domino, I want to make sure its peculiar features are retained as well: reader/author fields, arbitrary object storage via MIME, (ideally) RT and attachments, full-text search, categorized views, and so forth.
  • Conceptually simple. Though there's some ugly code involved in parts like the Domino view wrapper, the conceptual layout of the framework is kept very simple with few moving parts, making it easy to recognize the function of each aspect at a glance, whether when building it, when looking at another's code, or when returning to your own code months down the line.

For this post, I'll give an example of a model class, while later I'll go into the collection managers and some real-use examples. This is what a basic model class looks like:

package model;

import java.util.Arrays;
import java.util.Collection;
import java.util.List;

import org.openntf.domino.*;

import frostillicus.model.AbstractDominoModel;
import frostillicus.model.DominoColumnInfo;

public class Request extends AbstractDominoModel {
	private static final long serialVersionUID = 589766180414699322L;

	public Request(final Database database) {
		super(database);
		setValue("Form", "Request");
	}

	public Request(final ViewEntry entry, final List<DominoColumnInfo> columnInfo) {
		super(entry, columnInfo);
	}

	public Request(final Document doc) {
		super(doc);
	}

	@Override
	protected Collection<String> nonSummaryFields() {
		return Arrays.asList(new String[] { "Body" });
	}

}

There's more Java overhead than I'd like, but that's the name of the game. Fortunately, each method has a role and can be used as hooks for differing behavior.

The first constructor is the one used to create a new document of the appropriate type in the provided database, so it sets the form field appropriately - it would also set any other appropriate defaults. The second constructor is used when traversing a view - model objects pull values from view entries when available, but also transparently pass along requests to the document when the requested field isn't in the view. The final constructor wraps an existing document.

The "nonSummaryFields" method is one of the hooks available to define the Domino data model, in this case providing a list of fields that should be flagged as non-summary when written, to help work around Domino limits. I also have hooks, not needed here, for authors/readers/names fields and form-style query/post events.

Conspicuously absent are any field definitions. By default, model objects act much like XPage DominoDocuments, passing setValue and getValue calls on to the underlying document more or less directly. However, the framework allows for hooks here by writing getters and setters in the standard format, and their presence changes the behavior of getValue and setValue. When just a getter is present, the field becomes read-only; when a setter is present, the field can be written, but now does data-type validation. The getters and setters allow for arbitrary validation and additional behavior for specific fields (say, changing related fields when one changes) without having to write out giant blocks of "getFoo() { return foo; }" and "setFoo(String foo) { this.foo = foo; }", and this has been a huge win for me. Even though Eclipse helps with the initial creation, the code still has to exist, and it takes a cognitive toll.

I also use these overriding methods to create relations between models through their collection managers. For example:

public Client getClient() {
	String clientId = (String)getValue("ClientID");
	if(clientId == null || clientId.isEmpty()) {
		return null;
	}
	return (Client)JSFUtil.getClientManager().getValue(clientId);
}

This allows for XPage El bindings like #{task.client.name} - no extra panels with data sources, no inline lookup code.

Next time, I will go into the collection-manager side, which provides fairly high-performance access to views while remaining simple and flexible.

I Am Terribly Excited About Custom Renderers

Tue Nov 12 19:56:45 EST 2013

Tags: xpages java
  1. Nov 12 2013 - I Am Terribly Excited About Custom Renderers
  2. Nov 24 2013 - Building My App On Custom Renderers

It's much to my chagrin that it took me until this past weekend to properly dive into custom renderers. I had seen them before, primarily when working with mypic, but always avoided building my own, primarily because of what a hassle they are to write. However, I'm now convinced that the hassle is worth it, at least for some primary uses.

If you're not familiar with custom renderers, they're sort of an odd beast, but they're the kind of thing where the concept eventually "clicks" in your mind at some point. In XPages, a component (such as a link, a panel, or an application layout) consists of two main aspects: the structural code that defines the properties of the component and the renderer that handles actually outputting the HTML for the browser. What adding your own custom renderers allows is for you to continue to use the same XSP markup to define the control, yet have very different results.

The best example of this (and the source of most of my education) is the recent Bootstrap4XPages project. It takes a standard OneUI applicationLayout-based app and styles it using Bootstrap markup instead of OneUI, even though the XSP markup is the same. In the normal case, this just means that you can reuse some of the basics when building a Bootstrap-targeted app. And that's good enough on its own: it saves you tons of work.

But the more important aspect is that it lets you focus much more on solving the task at hand and much less on writing to the layout framework.

This is a huge advantage for XPages (and any framework with this separation). As low-key as Bootstrap is (and other modern CSS frameworks are), you still have to structure your markup and sprinkle class names around to match its expectations. With a really fleshed-out set of custom renderers, though, you could potentially not care at all about the resultant UI framework when writing your app and instead focus on using stock or ExtLib controls to solve the problem directly, letting the theme+renderers do the work.

In reality, there'll be a bit of leaking (for example, using Bootstrap-friendly FontAwesome classes instead of images doesn't really fit with standard IBM controls), but I aim to minimize that as much as possible in my future projects. My goal is to use applicationLayout and standard controls for as much as I can, and instead writing a set of custom renderers for each primary theme I want to apply.

I'll have more to write about this later, but for now, the TL;DR version is: custom renderers have the potential to dramatically improve the way XPages apps are written, as long as you're willing to do the initial Java legwork.

How I Want To Use Domino, Take 2

Wed Oct 30 16:10:50 EDT 2013

  1. Mar 07 2012 - How I Want To Use Domino
  2. Oct 30 2013 - How I Want To Use Domino, Take 2

A while back, I wrote a post about how I wanted to use Domino. The gist of that was that I was enamored with the idea of using Domino as a back-end database, but not necessarily as a app-dev platform on its own - basically, how you would use a SQL or NoSQL database. Since then, I've doubled down on my use of XPages as an app-dev platform with many advantages, but I still find it very useful to imagine Domino not as "Notes apps on the web, now with a modern coat of paint", but as a collection of related but not mutually-required components competing with other web-dev stacks.

I started working on an updated take on that post, inspired by some recent posts and chat conversations I've had (and I may return to it), but then I ran across Grand Decentral Station, which is a vision of an OS/app-dev platform taking the best of the lessons of the last decade and turning them into a coherent platform. It's a compelling vision, and reading it made me realize something: most of those goals are describing Domino, or, more accurately, the Domino that could be. Take a look down the list and see how many points Domino goes about 80% towards:

  • App Installer & Updater: though Domino doesn't really handle app versions, the deployment strategy is nonetheless quite good, with all app code contained in a distinct NSF, not a bunch of files strewn in a couple directories.
  • Sandboxed apps: again, Domino doesn't quite sandbox apps, but appropriate use of ACLs can bring you close.
  • Mail server: I hear tell that Domino is capable of acting as a mail server.
  • Calendar server: with proper CalDAV support, Domino could act as a real calendar server for non-Notes clients like OS X and iOS.
  • Addressbook server: like with the calendar server, this is just a matter of adding CardDAV support.
  • Asset handling: CSS and JS optimization in XPages is a huge step in this direction.
  • Avatar server: the Directory can already act as a profile-picture server for Sametime, and something like mypic could standardize this use.
  • vCard server: well, it already serves LDAP.
  • Unified sessions: done.

And to cap this off, the prescribed per-app database is CouchDB, which is already modeled on Domino. And, of course, it already has a standard API for email, which conveniently doubles as a method of cross-app messaging in some cases, and its replication and clustering are top-notch. It's not, itself, an OS, but its fairly-cross-platform nature means that that problem is already "solved": install Domino on the server platform you're most comfortable with and it acts basically the same.

Of course, that final "20%" is, as always, the crux of the problem. Domino is only really a fully-integrated mail/contacts/calendar server when you use Notes or iNotes, XPages and legacy Domino dev are really the only games in town if you want to maintain the benefits of the NSF package, agents aren't integrated with the XSP environment and DOTS isn't a real replacement yet, there are still a number of items on the list not at all touched on by the existing stack, and licensing basically removes Domino from consideration for app development for anyone not already mentally invested in it. But hey, one can dream, no?

Thanks, Bruce

Wed Oct 02 10:55:21 EDT 2013

Tags: thanksbruce

Though I'm not as eloquent on the matter as the many others who have shared their appreciation for what Bruce Elgort has done for the community, I wanted to be sure to pitch in as well. I have been active in this community for a relatively short amount of time, but even before that Bruce's was one of the names I knew looking in from outside, from his personal contributions to the Taking Notes podcast and to his leadership of OpenNTF. And since I started being actively involved in the community, all of my interactions with him have been not only pleasant, but encouraging and inspirational.

So: thanks, Bruce!

URLs in XPages

Tue Oct 01 20:28:59 EDT 2013

Tags: urls xpages

The topic of URLs came up today in a chat and in the most recent NotesIn9, and I think this is an area that deserves some particular attention. When designing any web app, it's easy to fall into some traps with URLs, and writing XPages code juggles things around a bit - some things are much easier than usual, some are insidiously harder.

At a base level, there are four main types of URLs you'll use when writing a web app, distinguished by how they begin (don't quote me on the names, or even whether or not they're technically called "URLs"):

  • Full-protocol URLs beginning with a protocol followed by a colon, such as https://frostillic.us/f.nsf/Home.xsp
  • Protocol-relative URLs beginning with a double-forward-slash, such as //frostillic.us/f.nsf/home.xsp (these are fairly rare)
  • Server-relative URLs beginning with a forward-slash, such as /f.nsf/home.xsp
  • Page-relative URLs beginning with basically anything else, such as home.xsp

As a general rule, page-relative URLs are only a good idea if you are absolutely certain of the structure of your app relative to the URL the browser is using. The tricky part here is that it's very easy to write such a URL that works when you test it (say, pointing from one XPage to another with a URL like <xp:link value="otherXPage.xsp" text="Click me"/>), but fails in other situations. For example, see how such a URL would work when visiting each of these URLs, which all point to the same XPage:

Yes, the last one is legal - you can get to the stuff after the ".xsp" via facesContext.getExternalContext().getRequest().getPathInfo().

Fortunately, the XPage environment provides a nice solution to this problem: when using XSP elements, it converts the third type (server-relative URLs) from being relative to the server to be relative to the app. This is almost always1 what you want. So if I write a link as <xp:link value="/otherXPage.xsp" text="Click me"/>, it will work correctly in all of those situations.

Note the opening caveat: you have to be using XSP elements. Fortunately, this covers all WYSIWYG use and pretty much everything you're going to do elsewhere. It's rare that you actually want a stock-HTML <a/> instead of an <xp:link/>, and this includes resource URLs in an XPage, custom control, or theme. So, as a rule, you should almost always preface your in-app links with a forward slash to piggyback on this feature.

There are two cases that need special treatment: when you want to link out of your app from an XPage element and when you want to link into your app from a non-XPage element. Fortunately, there are two bits of non-obvious code that will do these jobs easily and reliably:

  • To break out of the app-relative "jail" in an XPage element, prefix your URL with "/.ibmxspres/domino/". For an explanation as to what this stands for and a list of similar prefix URLs, read Mastering XPages. For now, just know that that prefix means "get me out of here". So to link to, say, the directory, you could write a link like <xp:link value="/.ibmxspres/domino/names.nsf" text="Domain's Directory"/>.
  • To reliably reference the current app from a non-XPage link, use this bizarre EL formulation: ${facesContext.externalContext.requestContextPath}. What that gets you is basically like @WebDbName prefixed with a slash. So, for example, if you wanted to reference a video attached as a file resource, you could do something like <video src="${facesContext.externalContext.requestContextPath}/someVideo.mp4" />.

Keep in mind that the special XPage behavior applies only to when you're writing XSP markup (or otherwise pulling URLs into XPage elements); it doesn't apply to, for example, rich text or other HTML brought into the page.

And on a final, self-aggrandizing note, the "mentoring" offer in that NotesIn9 applies to me and my company. Feel free to contact us or get in touch with me directly if you're ever in need.

1. Except when writing CSS files. In CSS, URLs are relative to the location of the CSS file. This is usually just fine, since you, not the browser, control the URLs you use to refrence the CSS file. It gets tricky when you use resource aggregation, though.

Domino Wishlist, September 2013

Fri Sep 20 13:06:48 EDT 2013

I joined in a tweet from Paul Hannan earlier about desired Domino features (that's a very worthwhile thread to read - lots of great ideas from others) and it's had me thinking about more on my list. Fortunately, I have a blog for this kind of thing! I'm leaving off a couple big-ticket items like Eclipse-4.x-based Designer on the Mac because IBM may not be required for that.

  1. Make $WSIS not trigger crummy SSL behavior (really, first-class support for proxies in front of Domino, not just IHS/Windows)
  2. Direct access to note item data via Java
  3. Newer JRE
  4. Direct creation/update of NSFs from git server-side
  5. New XSP model-object framework
  6. New views, though that may be our job
  7. Aggressive, small-company-friendly licensing to compete with OSS stacks
  8. NSF-based SSL keychains
  9. NSF-based jvm/lib/ext distribution
  10. "Sensible defaults" mode for clean-install servers to get appropriate memory/cache/performance settings on modern setups without learning dozens of secret notes.ini settings
  11. apt-get-based distribution and updates for Domino on Debian/Ubuntu
  12. Improved XPages file-upload and -download controls to handle modern file uploading and provide better hooks for non-dominoDocument bindings
  13. Bootstrap theme for XPages. Done - thanks!
  14. Built-in default support for authentication fields in LDAP
  15. init.d startup script installed with Domino, because seriously, who ships server software without one?
  16. OS X build of Domino 64-bit
  17. Built-in Firefox sync server, because that would be a nice overture to general open-source use
  18. Support for HTTP authentication filters in Java
  19. Built-in beer database to compete with Couchbase's substantial lead in the area
  20. Easier configuration of multiple TCP/IP adapters on a server and assignment of services to bind to them
  21. Full control over URL routing inside a database (e.g. "no legacy" mode where a Java class can map requests to resources)
  22. Keep up the ODS improvements - 8.x was a great set of releases on this front
  23. CalDAV and CardDAV support - Domino should be a close-to-drop-in replacement for Gmail on OS X and iOS
  24. Along the lines of #7, a general push to establish Domino as a top-flight NoSQL database and app-dev platform, not just "your old company's mail system"
  25. A real focus on standard-API performance, with less XSP-specific cheating

I'm sure this list is not exhaustive.

Progress on the org.openntf.domino Design API

Fri Sep 06 18:57:39 EDT 2013

Tags: java openntf

The primary focus of my recent work with the OpenNTF Domino API has been the implementation of a fleshed-out design API. I've done a lot of work in my Domino-programming career manipulating design elements, usually via DXL (such as with my Forms 'n' Views app), and my aim is to codify all of that into the API itself, eventually growing it to cover almost every design element that Designer does (we'll see about Navigators, Composite Apps, and the like).

One of the big hurdles in recent years when manipulating design elements has been dealing with XPages-related elements - XPages themselves, Java design elements, Jars, faces-config.xml, etc.. Though XPages by their nature have a very nice XML representation, that doesn't carry over to DXL - all of those elements are dumped out in the "raw" data format that DXL uses when it doesn't have a better idea of how to handle what you're trying to export. And more specifically, though the XSP markup and the compiled Java code are present in the result, attempts to actually manipulate that were stymied by a "checksum" block of bits at the start of the data - you couldn't just drop the Java bytecode into the doc and expect it to work.

Fortunately, it turns out that those bits aren't a weird "checksum" at all: the data is just stored as Composite Data, the internal format of rich text in Domino. Specifically, all of those elements share the same internal representation as File Resources and Style Sheets, being composed of a CDFILEHEADER and list of CDFILESEGMENTs. When DXL uses the "raw" format, it exports that data as an array of Base64-encoded bytes directly. After discovering this, I wrote the start of a CD implementation in the API and got it to work: the API is now able to read/write the attached data of any one of those file types. Now, there are a few steps that would have to be implemented between this and "write a new XPage with a few lines of code", but this is a step in that direction (and you could theoretically modify your faces-config.xml, xsp.properties, etc. on the fly now, though I haven't tested much).

More immediately, it's also a huge step in the direction of interesting read-focused uses. One of these uses in particular has caught my fancy: loading and using XSP-related Java classes in other contexts. To go along with my work, I wrote a DatabaseClassLoader that gives you access to all of those class types from anywhere using the API (including Jar design elements). The possibilities for this have made me giddy: re-using data-model objects in external scheduled tasks, sharing classes with Java agents (...with the caveat that you have to use reflection or a scripting language), "loading" other XPage apps from inside another, treating NSFs as replicating, access-controlled Jars, and so on. I have my own notions for some practical uses I may want to explore, and I'm very interested in seeing what kind of things this lets others do as well.

Materials From MWLUG 2013

Sun Aug 25 12:54:32 EDT 2013

Tags: mwlug

I gave two presentations at MWLUG 2013 in Indianapolis, and here are the slide decks and resources for those:

AD110: Write Cleaner, More-Manageable XPages Applications With Java

AD108: The org.openntf.domino API: Modernized Domino Programming for Java and XPages

Expanding your use of EL (Part 2)

Wed Jul 24 12:37:12 EDT 2013

Tags: el java
  1. Jul 23 2013 - Expanding your use of EL (Part 1)
  2. Jul 24 2013 - Expanding your use of EL (Part 2)

Last time, I said that my next post would delve into operations and comparisons in EL, but I figured first it would be good to have a real-use example of what I talked about last time.

In several of my apps lately, I've taken to using properties files to store app configuration, primarily the server and file paths for the databases that store the actual data. The standard way of doing this is to use the platform's built-in bundle-handling abilities. However, that'd leave me stuck with either putting the bundle reference in the XSP markup somewhere or in a theme - the former would muddy things up and the latter is not available at page-load time. So instead, I wrote a quick class to use as an application-scoped bean, making it available everywhere via EL and Java:

package config;

import java.io.IOException;
import java.io.Serializable;
import java.util.Locale;
import java.util.ResourceBundle;
import javax.faces.context.FacesContext;
import com.ibm.xsp.application.ApplicationEx;
import com.ibm.xsp.model.DataObject;

public class AppConfig implements Serializable, DataObject {
	private static final long serialVersionUID = -5125445506949601097L;

	private transient ResourceBundle config;

	public Class<?> getType(final Object keyObject) {
		return String.class;
	}

	public Object getValue(final Object keyObject) {
		if(!(keyObject instanceof String)) { throw new IllegalArgumentException(); }
		return getConfig().getString((String)keyObject);
	}

	public boolean isReadOnly(final Object keyObject) { return true; }

	public void setValue(final Object keyObject, final Object value) { }

	private ResourceBundle getConfig() {
		if (config == null) {
			try {
				ApplicationEx app = (ApplicationEx) FacesContext.getCurrentInstance().getApplication();
				config = app.getResourceBundle("config", Locale.getDefault());
			} catch (IOException ioe) {
				ioe.printStackTrace();
			}
		}
		return config;
	}
}

This is an example of a simple read-only DataObject implementation. There are only four methods to implement: getType, which returns the class of the value for the key; getValue, which returns the actual value; isReadOnly, which determines whether a given key is settable (always false here); and setValue, which would set the value for the key if I wasn't making it read-only.

It's pretty straightforward to take this idea and apply it to any of your Map-like objects to ease their use in EL without having to implement or stub-out the dozen or so methods demanded by the actual Map interface.

Expanding your use of EL (Part 1)

Tue Jul 23 13:07:43 EDT 2013

Tags: el java
  1. Jul 23 2013 - Expanding your use of EL (Part 1)
  2. Jul 24 2013 - Expanding your use of EL (Part 2)

One of the main aspects to cleaning up the code of an XPage is the heavy use of Expression Language (EL), which is XPages/JSF's shorthand language for binding properties to values and methods. The most common kind you'll run across is the kind you get for "free" when working with document data sources:

<xp:inputText value="#{doc.FirstName}"/>

This is the essence of EL in a nutshell: the actual code behind the scenes can be pretty complex, but EL handles that for you. It goes far beyond this, though, in two important ways: accessing custom and non-Domino data sources and writing complex operations.

Accessing Custom Data Sources

In addition to accessing "built-in" data types like Domino documents and the scope variables, EL is generic enough to deal with several kinds of data access (technically, it ONLY does these, and the standard bindings are examples of it):

  • Lists (and probably arrays). EL can use bracketed-subscript notation to reference individual elements of a Java list/array, e.g. #{names[3]}. I've found this to be the rarest use, but it could be useful if repeating over two parallel lists.
  • Maps. This is how EL accesses the scope variables (requestScope, viewScope, etc.). It translates code like #{sessionScope.cart} to sessionScope.get("cart") and sessionScope.put("cart", ...).
  • DataObjects. This is basically a "Map-lite" interface from IBM that is meant for making generic key/value objects easily-accessible via EL - this is what Domino document objects implement. It translates code like #{doc.FirstName} to doc.getValue("FirstName") and doc.setValue("FirstName", ...).
  • Generic bean-style methods. If (and only if) your object doesn't implement any of the above interfaces, EL falls back to seeking out bean-style getters and setters. For example, if you have a class that implements no special interfaces, EL will attempt to match #{myObject.foo} to myObject.getFoo() (or myObject.isFoo()) and myObject.setFoo(...).

The first important thing to remember is that EL applies these rules to your own and third-party objects just as much as it does to IBM's stock objects. If your code is just #{doc.foo}, you can switch "doc" from a Domino document to a HashMap, a custom DataObject class, a bean with getFoo, or an entirely-different third-party data source that implements one of those and it will continue to work smoothly. If you instead wrote it as #{javascript:doc.getItemValueString('foo')}, your experience would be... a bit rougher.

The second important thing to remember is EL's equivalence between dot and bracket notations. What I mean by this is that these three are all equivalent (assuming "barHolder" is a variable on the page containing the string "bar"):

  • #{foo.bar}
  • #{foo['bar']} or #{foo["bar"]}
  • #{foo[barHolder]}

Note that those are equivalent for every supported object type, including generic beans (okay, but not Lists, because #{foo.1} is illegal). This equivalence allows you to use EL to reference keys that would not be legal in dot notation (e.g. #{someProperties['us.iksg.setting']}) and to use other EL expressions to determine the field name. Say you're keeping some user-specific info in the application scope; you could end up with a chain like this to access it via EL:

<xp:text value="#{applicationScope.userCache[context.user.name].ticketCount}"/>

The more you understand the different ways of accessing your objects via EL, the more you can remove SSJS from your XPage markup and make your apps cleaner, (probably) faster, and more portable.

In my next post, I will cover the second major topic: complex operations and comparisons in EL.