Syntactica Solutions for XRX Developers

Dan McCreary

The XRX Web Application Architecture (XForms, REST, XQuery) is a new "no-translation" software architecture that can be driven by precise metadata-registry semantics. By "no-translation", we mean that XML is used in all components on both client and server. Since XRX is new, there are few mature frameworks for building XRX applications and few metrics measuring developer productivity. This presentation discusses early finding of the use of a rapidly evolving semantic-driven-development (SDD) XRX approach and architecture and some initial findings of increased developer productivity compared to four-step translation architectures.

Today over half of information system costs are frequently related to integration costs. Semantic driven-development (SDD) attempts to address integration costs in early stages of application development. SDD begins the development process by the selection of data elements that are already pre-defined by external sources. This process ensures the creation of artifacts that already have a high semantic footprint: their meaning or semantics has been standardized over time and accross organizations.

SDD is very different from the popular Rails web development frameworks. Rails begins with the creation of middle-tier business objects that are frequently created by the whims of the software developer without reguard to external semantics. Unlike Rails, SDD begins with the selection of data elements taken from carefully controlled ISO/IEC-11179 metadata registries. These data elements are then “composed” into various XML Schemas using web-based authoring tools. The users are then guided through web form dialogs to create fully functional web applications that create, read, update, delete and search for versioned XML data.

This SDD approach has many benefits over traditional web application development where each object designer frequently creates their own objects with inconsistent interfaces. These older approaches postpone the creation of XML web services till late in the development cycle. By placing semantics at the beginning of the software-development-lifecycle applications are easier to integrate with other systems that use these same vocabularies.

This article will describe the XRX SDD process, describe its benefits and discuss early metrics that show increased developer productivity with the use of XRX. We will outline of the XRX SDD architecture, the artifacts created by this process and overall developer workflow. We will also describe the tools needed to support XRX SDD and the patterns found in our work. We end with a roadmap for future efforts and links to resources to get more information on XRX SDD.

XRX Terminology and Early Design Patterns

In our work integrating XForms with native XML systems we have begun to use the following terms. These terms are starting to evolve into a set of "best practices" and we are starting to give them names and document their use in an XRX pattern registry. As our early XRX frameworks evolve we continue to seek other XRX developers to build a shared knowledge base so that XRX design patterns can easily be communicated with other developers.

1. XRX – XRX is the name of the meme we use to describe the elegant coupling of XForms on the client, REST interfaces, XQuery on the server. These three technologies all work together like a hand-in-glove. The fit is very good.
2. Zero translation – A Zero translation system is one where there is no conversion from HTML to middle-tire objects and then to RDBMS databases. There are no objects, there is no SQL code, and there is no XSLT transformation. Everything is done with a consistent XML and XPath architecture and rules and structures can be elegantly be moved between the client and the server without rework of the XPath expressions.
3. XQuery Style Modules – All shared user interface elements (header, footer, navigation, breadcrumbs) around the central content is isolated in a central XQuery style module. This module works like a Server-Side-Include (SSL) library but with much more powerful functionality. Our style module has a single function called style:assemble-page($content) that is used to wrap all the content pages with site-specific headers, footers and navigation. Each of these structures can conditionally import CSS files, JavaScript files or navigation components based on the context of the application. Context includes items like the person's role, team, status, training or personal preferences.
4. URL Rewriting Controllers In our XRX frameworks, a central controller.xql file manages all URL rewriting and passes the context to individual applications and pages. Information about the site home collection, the application home collection, the default data collection etc. are automatically passed as parameters to each XQuery. This keeps our templates very small and clean and yet the templates each can override the behavior of these inputs if the business rules require it. We gain, in effect, an inheritance behavior that is common in object-oriented systems. Much of the elegance and power of these rewriting controllers were developed by Wolfgang Meier and greatly extended and standardized by Joe Wicentowski. We are indebted to them for these powerful contributions.
5. Site, Application and Page Configuration Files In our XRX framework each site, application and web page has an XML file or XML fragment that defines the function and rules. XForms can be used to change the file information. This makes it easy for people that are not familiar with XML to change the status or availability of a site, application or page as new resources are added to the system.
6. Encapsulation – XQuery functions and modules separate subscribers from the actual data within the application's data collection. Subscribers can be protected from changes in the data structures by using an interface modules. Currently these interface modules are standardized and in the future we will be investigating the ability to auto-generate the interface modules directly from and XML Schema and resources in a metadata registry.
7. Application Modularity – Each XRX application is considered a "package" that can be migrated to new systems with a minimal amount of change. Each application has its own resources and interface module that can be used by other applications. Applications all use relative module imports so that the import module functions do not need to be modified.
8. XForms Composition – XForms applications are generated by an XQuery program. There are many composite RESTful web services. For example all the code-tables in an application are dynamically generated from code tables that can be stored with a code-tables area of that application or they can draw on interface modules of other applications that provide the data for a code table. Any artifact of a XForms application can also be customized.
9. Standardized Views – In our XRX framework, many views have standardized functions, names and files for consistency across all applications. This allows us to use standardized templates where only the XPath expressions within the templates need to be changed. This can frequently be done by non-programmers.
10. Search Services – A search form and search services is defined in each XRX application. Each application can participate in site-wide search by just changing a single element in the application information file (app-info.xml)
11. Code Table Services – The efficient management of code tables is critical to developer productivity. The XRX frameworks always uses the ISO/IEC 11179 metadata registry standards when a value of an element has enumerated values. We are very careful to differentiate between in-database "values" and on screen "labels". This is totally consistent with the way that XForms manages selection lists using the <xs:select> and <select1> controls. ISO Values are stored in all XML files for enumerations but are stored as mnemonic strings that are readable by humans to ensure that standalone XML fragments are always self describing. ISO Labels are natural language-specific(English, Spanish) elements that are used in all presentation layers. Comparisons, indexes and REST services always use the value of a code. Views and forms always use the label of a code. All code tables are managed in centralized code-table collections and forms can dynamically and conditionally load selection lists based on the users project, group or role. For example if a user does not have "publishing" rights the document status code selection will never have "publish" code on it.
12. Field Validation Tradeoff – Either client or server-side structures can be used to validate any field in a form. The decision to do in-browser as-you-type validation vs. server-side validation upon a form save can be easily changed at any time.
13. Role-Based Access Control (RBAC) Views – The Edit/Publish User interface controls can be automatically hidden from uses that do not have the edit or publish roles in the system.
14. Managed Metadata – XML Schemas are not sufficient to capture presentation elements such as field labels and code labels. This need to be done in a central location so all XRX generated applications have consistent user interfaces and to allow managers to change a single location and have all XRX applications be updated. Much like the US Federal NIEM.gov standards, our frameworks use ISO/IEC 11179 metadata concepts whenever possible.
15. Model-Driven – Changes to a central model (Metadata Registry + XML Schemas) drives can be configured to drive aspects of the application. We think of all XForms or views as a series of queries that are all dynamically executed when the form or view is rendered. This avoids having to pre-compile static rules in the system.
16. Non-Programmer Empowerment – XForms allow non-programmers to change almost any XML system configuration and status. Our goal is to XForms-enable any areas that need to me modified by non-programmers. We feel there are fewer reasons in XRX to prevent non-programmers from viewing and changing business logic when it becomes necessary. Our goal is to continue to make XRX application frameworks easier for non-programmers to use and maintain.

XRX Architectural Diagrams

Figure 1: Four Translation Rails Architecture

We first contrasts Semantic-Driven XRX with the Rails development methodology. Rails is designed with the assumption that there are three different datatype representations in a web application, each with their own language, datatypes, rules and structures. To build a working application, four translations steps need to be created:

1. HTML to Objects
2. Objects to RDBMS
3. RDBMS to Objects
4. Objects to HTML

In addition, if XML Web services are created then Objects to XML and XML to Objects translations are also required (six translations) are needed.

Rails attempt to address the four-translation problem head on by allowing the web application developer to create the middle-tier objects and then Rails attempts to automate the generation of the client HTML and server RDBMS artifacts. With Rails a developer creates a new web application by first creating as specification of mid-tier objects and then Rails automatically generates the client and server components. This architecture makes perfect sense if you have three different representations of your data (HTML, Objects and RDBMS systems). This architecture is simply not appropriate if the translation problem is not present. With XRX and zero-translation we have an opportunity to rethink the way we build web applications and we have a chance to look carefully at the total cost of ownership of building web applications.

Figure 2: Workflow of Semantic Driven Design (SDD)

With Semantic-Driven XRX, a shopping cart is first used to select data elements from a metadata registry. This metadata registry is usually modeled on the concepts in the ISO/IEC 11179 metadata registry specification. It is important to note that a registry is not a "dumping ground" for data elements. These dumping grounds are called repositories and they typically do not have formal change control processes in place to removed duplicate items. Registries on the other hand have carefully controlled, human-centric processes to ensure that only data elements with unique semantics are used in the software development process.

Many early XRX frameworks attempted to auto-generate web applications directly from XML Schemas. And although there is a great deal of information within an XML Schema that is important, XML Schemas were designed primarily to validate date sets but not capture user interface requirements such as field labels. This information is best stored in a central location for consistency accross many applications.

Our initial fears were that building metadata registries as part of the XRX development process was going to create too much of a heavyweight burden for our customers. And indeed, the costs of metadata registries in the past using traditional RDBMS systems was very prohibitive. But we have found that building an XRX metadata registry can be as much as one tenth the cost of using a traditional RDBMS storage system. We also see that as XRX metadata registries become part of the open-source community's solution space these cost will continue to decline.

Nest, the non-programmer can then place these data elements in structure that adds application-specific constraints. This composition process adds data element ordering, structure and cardinality to the specification. An XML Schema-like structure is then generated and from these structures and from the data in the metadata registry all of the components of a fully XRX application can be generated. These are generated using XQuery transformations of the XML Schemas that define the structure of the XRX application. Our frameworks do not use XSLT since XSLT is not designed to use the indexes in modern native XML systems.

It is important to note a few items that differentiate SDD XRX from traditional Rails development. Rails targets the software developer. Rails assumes that the person defining middle-tier objects is familiar with the core concepts in object-oriented design namely Encapsulation, Inheritance and Polymorphism.

Developers is not the only intended audience of the SDD XRX user. SDD-XRX also targets the business analysts or the subject matter expert that is familiar with the data and would like to build web applications that manage this data. No assumptions of object-orientation is assumed. The focus is what data do you want to use, what order do these elements appear in your application, what are the rules of gathering this data (constraints such as required fields and cardinality) and how should this data be entered, edited and reported.

Figure 3: Generated XRX Application Package Components

This figure describes the application package structure. Note that XQuery functions and queries surround the data set, similar to how Encapsulation works in object-oriented systems.

Figure 4: XRX Collection Structure

This diagram gives a more detailed explanation of each of the components generated by the Semantic-driven process are grouped into collections.

Figure 5: XRX Application Deployment Stack

This figure describes how applications are packaged within a series of XQuery layers that are built on top of a native XML system. Note that each application has its own interface module which is implemented as a series of XQuery functions that insulate the subscribers from changes to the internal XML data structures.

These figures document how we have seen increased developer productivity from using these frameworks.

This figure describes how we have seen good training programs have a significant impact on the rates that developers increase their productivity. We feel that the more automated an framework becomes the more important it is to train our users in how the framework can be used to automate developer productivity.

Roadmap for Future Work

Although XRX has clearly shown increased developer productivity, there is not yet a mature SDD framework that allows us to compare SDD XRX with non-SDD XRX development. We hope to continue to build these frameworks and continue to gather more information about developer time, development costs and integration costs as these systems mature.

Note, much of this material was taken from material presented at a poster session at the 2010 XML Prague conference.