Good web services make webapps programmable

Just as exposing COM automation interfaces on desktop applications made Excel and Visio more powerful by enabling people to write a Visual Basic script to build composite applications from the two, exposing your webapp as a web service enables people to integrate your webapp with others.

Recipe for a good web service: Start with a good webapp like Amazon or Google or Travelocity. Expose the functionality in the webapp as POSTing and GETing self descriptive messages. That's about it. Now anyone can script your application and coordinate reserving an airline ticket with ordering a book.

Link resolvers

PURL -- persistent URL -- has been around a while, but I only just discovered I could make one myself: http://purl.oclc.org/net/hughw/blog. That URL will forever redirect to http://hughw.blogspot.com/, they think. If I move the blog, I have to update the pointer.

(TinyURL is another useful service for link resolution. Its purpose in life is to shorten long URLS to a manageable length you can actually type; if you move your resource you can't update the tiny URL to point to the new location. http://tinyurl.com/62dn9 gets you here too.)

Why don't we embed a unique identifier into each web page or searchable resource? As I have pointed out (also here), a search engine will always turn up the moved resource, eventually. Couldn't we use META tags to do that? Embed this in your file:

<META scheme="UUID" name="identifier" content="cce89bf0-92e6-11d9-9669-0800200c9a66">

Then a persistent link to the resource might be

http://www.google.com/search?q=cce89bf0-92e6-11d9-9669-0800200c9a66&btnI=I%27m+Feeling+Lucky

NetKernel: RESTful Application server

I have discovered NetKernel at a very late date and I played with it about half a day last weekend. These guys have taken uniform semantics to a new level. Every service, local or remote, is represented by a URI, and you compose services using a declarative * scripting language. You invoke verbs like SOURCE and SINK (analogous to GET and POST) on these URIs, and the kernel understands dependencies and can cache results intelligently.

You create your own services and map them to addresses in the URI namespace.

They also put a lot of emphasis on their available XML pipeline service, and if used properly, there's your engine of application state.

I hope this product can get some attention from REST architects. It's really a unique, REST friendly way to build services.

*My first brush with that language leaves two impressions: a) looks kind of procedural to me, and b) XML is really too verbose to use as a framework for scripting language.

MDA is so February

I'm riding the crest of the Ontology Driven Architecture wave that's sweeping the industry. Grady Booch et al. say your legacy appserver identifies you as from the last century... you need an Ontology-based application server. Joseki anyone? (Thanks Andrew Newman).

The perfect distributed application

The perfect distributed application: The client GETs an URL returning an enhanced RDF Form -- enhanced so that some RDQL accompanies each parameter. The RDQL describes how the client is to retrieve data from its own data model and populate the "form" parameters. The RDQL references RDF types and instances defined in the service's own ontology. Now a) we cannot require the client's data repository be physically stored as RDF, and b) even if it were, we cannot require that the client's ontology be the same as the one described in the service's ontology. So there is a model mapping problem ahead of us. But for the moment, presume we have solved that problem -- our client understands, somehow, the service's ontology, and can honor RDQL requests made using terms from that ontology.

So devise a client agent analogous to a web browser, and furnish it with with a reference to a callback interface it uses to satisfy RDQL requests. The callback interface is analogous to the human user, who, reading an ordinary HTML form page, knows how to populate the fields. This callback interface accepts RDQL queries, and honors them from your data repository. Voila! Instant, resilient distributed application engine. The service is free to change even the parameters it needs to satisfy any request. Say the airline reservation service evolves, and now, due to new TSA requirements, must have the passenger's Social Security Number to complete the reservation. No problem for our engine. The service simply adds a new element to the RDF Form, and supplies the RDQL to populate it. Our client agent passes the aditional RDQL to the callback, and obtains the SSN.

This system is a poor man's mobile agent. We're not going to use ObjectSpace Voyager, as cool as that is. Instead, our limited mobile agent performs local RDQL queries and sends messages back to the server. The enhanced RDF Form is the code for this agent. You download the form to your client agent and run it.

Now about that elephant in the room. How to map the service's ontology to our own? And don't we have to map everyservice's ontology to our own? Well, yes. Let's investigate how you'd attack that problem. (sound of dozens of shuffling feet leaving the room).

If your data model is an RDF store, the problem is mapping your ontology to the service's. The airline reservation service's ontology defines terms like Flight, Seat, City. Your own business's ontology has no concept of Flight or Seat. And it has an idea of City that's maybe a lot different from the airline's.

So part of creating this mapping is augmenting your ontology with terms required by the airline. Sure, you had city names in your model, but you didn't have airport codes like AUS for Austin. So wherever possible, you use constructs like owl:equivalentClass to map the airline's classes to existing classes in your ontology; elsewhere, add properties like airline:airportCode to your classes. You had to do all this work to invoke the service anyway, you know -- this is just a methodology for organizing it.

Now, if your data model is not stored as RDF, consider exposing an RDF interface to it. It would be straightforward to map SQL rows to RDF statements and column names to OWL predicates.

And in passing I'll mention that Service Data Objects are pretty cool. Programs operate on abstract object graphs, and you supply a Data Mediator Service that maps graphs to your own data model, be that in SQL, an XML repository, RDF repository, the file system, some remote EIS... wherever. If you've got a DMS, you've already mapped an arbitrary graph to your data model. So you can map an RDF graph to your data model. I'm not saying that work's done for you; just saying this problem is isomorphic to that one -- literally.

(I notice Patrick Logan remarking, "...if your restful interpreter and mine can understand some of the same state then they can cooperate. This is the idea behind these interpreters sharing partial ontologies or even being able to translate parts of one to another." Right on.)

More on using RDF Forms to maintain application state in machine to machine services: If RDF forms contained RDQL to constrain parameters, then the client would know more completely how to fill in the "form". Say the form required submitting two flight numbers: departure and return. Both have rdf:type FlightNumber, but you better put the right one in the right field. A little RDQL, drawing on terms from the service's ontology, could constrain the destination city of the departure flight to be the same as the origination city of the return flight.

When you sit down to write a description language for REST services (a IDL or WSDL for REST), you discover that doing so is unnecessary. "Hypermedia as the engine of application state" means that the service, not the client, constructs the URLs the client needs to invoke via GET, POST, etc. The heart of an IDL or WSDL is that they are instructions for clients invoking the service.

Example: a reservations system. In the RPC-style case, an IDL or WSDL might declare a method "NewReservation", and it would tell you some parameters: NewReservation (name, flight number, airline, date); you write client programs that collect that information from a user, and invoke the RPC.

An HTTP/HTML reservation system, however, constructs a form with input elements named "name", "flightnumber", "airline", and "date"; the user fills in the form and pressing the Submit buttons sends the information to the service. So the client program, a web browser, never knew the semantics of the service. The human operator did, of course: he read the descriptive text in the form and put his name in the proper box.

Now, that's fine for human operators. But what about machine to machine operations? In the RPC case, the client program is essentially the same as in the human case, only now the program collects the parameters from a database rather than from a live user. The client program, as before, is compiled against the IDL or WSDL. In the RPC case, we presume we can just invoke the service "out of the blue." We needn't have obtained any information from the service before hand; we just connect and invoke.

In a REST-style service, we want an analogy to the self describing hypermedia we have in the HTML scenario. First let's assume we'll use some form of XML as our hypermedia. It's easy to imagine a XML document. Maybe even more machine friendly would be an RDF document -- a bunch of RDF statements. Your client invokes GET on the well known URL of the service, and receives an RDF form. The RDF form describes the names of parameters and how to serialize them. So just as in the user driven HTML case, the client needs no foreknowledge of what the neccessary parameters do, or even what their names are. But we still need an automated "user" to fill out those parameters. Since the RDF form describes the parameters in RDF, your client can map the RDF types of those parameters to elements in its data model. Your client has to "understand" the service's ontology, sure. But that is a one-time mapping of ontology elements to, say, SQL queries.

Could you have done all of this using an RPC-style architecture? Maybe. You could have retrieved WSDL from some well known service. You would do that each time you want to invoke the service, to emulate the self-describing part. Then you could dynamically construct the RPC call -- the serialization bit wouldn't be hard -- if you also had a mapping of the RPC parameters to your data model. How would you do that? There would need to be some semantic description of the service parameters and you would need a mapping of that description to your own data model. Could you use some RDF to describe these semantics? Probably. But it's not a system designed from the ground up to be self describing.

A "key" difference between a primary key and an object identifier is that the primary key is part of the table row -- it's part of the information content of the thing itself. An object identifier is metainformation, information about the object.

A reference to information in the row moves around wherever you move the thing. A reference to an object identifier has to be updated when you move the thing.

A query "Find me the page containing terms 'Hughw' and 'blog'" searches the information content of the thing itself. ('Hughw' and 'blog' are not primary keys of course, just ordinary "column information"). A URI to this page is like an object reference. If I move the page to another server, I have to update all links to it. And so do you.

Which do you think is a better technique for persisting references?

Someday all links on the web will look like this link :

Hughw's Blog


The URL above is
http://www.google.com/search?hl=en&q=hughw+blog&btnI=I%27m+Feeling+Lucky; i.e. it's the result of a query. Google returns 302 FOUND with the redirect to http://www.hprscc.org/ in the Location header.

What's the big deal? Well, the link above will never break until Google does. This feature is precisely the reason Codd avoided handling pointers in the relational model. You never obtain a reference to a row in the relational model (and Oracle REFs are decidedly not relational). Instead, you can only specify rows by requiring column values to be equal, or in some other relation.

Someday query URLs we use may be more structured. We'll pass RDF queries to Google.

To treat the web as an information retrieval mechanism, these query URLs will be the only sensible way to store references in web pages! Today, every site has broken links, even to pages under its own control.

Hmm, Google will crawl pages and discover links that call its own search function... weird... there is a recursive issue there, for them to solve :)

The most interesting thread on the rest-discuss list in a long time begins with a simple request for assistance: How to implement a query with a long query string? Most of the RESTigentsia weighed in before it was all over, and Roy Fielding gave Mark Baker the business, though it wasn't deserved.

And I learned something. As Roy pointed out, I had not appreciated the role of "hypertext as the engine of application state".

This idea illuminates a whole world actually.

The HTTP application protocol -- the type section for a REST application framework -- enables lots of different applications all to run on the same fabric. Roy points out, in applications using HTML, clients know something about how to process the GET, based on the context. Both the <a> and <img> tags retrieve resources using GET. Browsers know how to render the returned entity differently depending on whether the HTML referenced the resouce using <a> or <img>. REST folks use "self-descriptive" to refer to entities like HTML documents, that indicate to knowing agents how to process the resource references in them.

And lots of other agents are out there on the web, non-HTML agents, knowing nothing about how to process <a> and <img> tags. And these agents will ordinarily never encounter those tags. If you've written an agent to process some XML dialect, your agent normally navigates among documents of that type.


So all kinds of applications out there are blissfully navigating via HTTP, and mostly they don't know how to make sense of one another's resources.

(For the case of POST, every application -- heck every resource -- can have wildly different semantics. On his blog, Mark Nottingham gives an excellent discourse on the topic).

So the REST world is this world of a few verbs, and zillions of nouns. And the nouns form clusters around the semantics they honor. In the case of HTTP, you have clusters of nouns that understand HTML. You have other clusters that understand XML+Xlink. Still others understand XML+RDF. And a jillion more understand their own custom media in whatever idiosyncratic way the application authors wanted. The apps in each cluster all play on the web, oblivious to the apps in the other clusters.




`