August, 2006

Aug 06

XQOM project

Frank Cohen who hosts site, has volunteered to host our the XQOM (XQuery Object Mapping) project on his site.  This is great, being that it will be surrounded by other great XML/XQuery resources.  He’s also going to join the project, and we welcome his great expertise.

The XQOM project is now hosted at  There is nothing there yet, as we’re still preparing for our first release, which will be in a few weeks.

Aug 06

XML Persistence Part 1

So currently we’re looking into native XML persistence.  I’ve been doing a lot of reading/research in the last few months.  Although I’m pretty versed in XML technologies, XML persistence is a completely new beast.  I’ll explain…

I think most enterprise developers today look at XML as an intermediary transport format, configuration file format, and other applications where readability of data and interoperability is a must.  It’s been embraced more as an integration technology and for good reasons of course.  There has always been a feeling that persisting application data can be done in XML, being that your domain model fits into the benefits criteria of XML Schema storage.  Just like Object Oriented databases, native XML databases have not caught up and/or even made a dent in the relational database market, though relational vendors are boosting XML features on top of their relational schema.  Why is that the case?  I think it’s the fact that most developers are comfortable with relational databases and there are magnitudes of technologies and frameworks to support their application endeavors.

Unlike OO databases, which I think have missed their calling and now only have a place in a niche market, XML persistence mechanisms have a chance to change the way we persist data.   I don’t think relational models are going anywhere, but I think that many application domain models do not fit very well into the rectangular view of the relational schema.

I think the biggest benefit of persisting data in XML is if your schema is dynamic and requires constant, and at times, unpredictable change.  Yes, there are ways around it with relational schemas, like having generic tables that store field name/value pairs, etc…  But that’s an ugly way of covering up a true limitation of the relational schemas.  XML schema is a dynamic hierarchical model that perfectly fits that particular use case.

So being that our storage model is a perfect candidate for native XML persistence, we’re greatly experimenting with it with hopes to use it in production.

Because most enterprise applications today are written using some OO language/framework (i.e. Java, C#, etc…), developers are always looking to abstract and eliminate various concerns from their applications.  The bottom line is, applications developers are hired to write business logic not framework code.  In the current environment, where OO/Relational technologies dominate the application development domain, there are many popular frameworks that allow you to ease the impedance mismatch concerns of both.  ORM (Object Relational Mapping) frameworks are the most talked about topic in the J2EE and .NET communities (followed closely, if not tied with SOA and Spring:-).   There are many proven/robust solutions to ease the OO/Relational integration pains, out there.  Some offer more features/flexibility, some closely follow proposed standards, etc…, but either way, developers have a choice and they feel comfortable going with such framework, due to it’s popularity and proven successes.  This is yet another reason why relational models dominate the persistence world.

There seems to be a lack of a full fledged XML Persistence framework.  There are many very robust Object/XML mapping technologies (i.e. JAXB, JiBX, Castor, etc…), but they don’t truly abstract the actual persistence concerns that come with storing your data in a native XML database.  Most of the mapping technologies were developed to marshal/unmarshal XML to and from Objects.  Most are used as a part of a bigger picture, which is web services stack, as they facilitate the mapping of objects for SOAP and other remote invocations proxies that use XML as their transport protocol.

There is a need for a true XML persistence framework, that is tightly coupled with XQuery.  Why XQuery?  Well, just as the ORM frameworks are tightly coupled to SQL and one can’t really talk about one without the other, XQuery is the standard (although relatively new) for querying XML.  The day is coming, when XQuery is going to be the true standard for querying XML.  Another question is what will happen to XPath, XSLT, etc…, we’ll see.

The XQuery/OO framework needs to fill the space of transparently persisting your objects that are mapped to an XML schema.  There is the question of how much should the framework cover, and how much flexibility should it allow.  Due to the dynamic nature of XML and the fact that XQuery result sets are not structured, as are SQL result sets, it becomes a lot harder to make certain assumptions about the structure of the results sets, dynamic XQuery generation, etc…  Though, I think a fine balance lies in allowing the flexibility of writing XQuery in the most optimized way, and mapping it’s results to an object graph.  I don’t think that taking XQuery out of the equation, as some ORM frameworks take SQL out of the equation, would be a practical short term approach.  A lot of this is due to the nature of XQuery as well as the fact that the standard is relatively new and is still questionable as far as it’s implementations are concerned.  Though true optimizations of XQueries behind the scenes might not be as practical at this time, as they are in the SQL world.  I think the word iBatis comes to mind somewhere here.  No, I’m not talking about iBatis for XML, but conceptually the idea of flexibility of writing XQuery expressions and unmarshal it’s result set into an object graph.  There is of course a lot more too it, as with any other robust framework.  Enteprise features like caching, thread safety, transparent access to backend XML sources through a common interface (maybe XQJ, if only they’d add it to JSE already), etc…

In the next few posts I will explain in more detail the functionalities and benefits of native XML persistence stores, their limitations, roadblocks we encountered, my new endeavor of creating an XML persistence framework, etc…  I hope that these experiences will allow others to shorten their time for evaluating similar technologies and actually spread the word about how to effectively use native XML in your applications.