Messages not Models

Inflexible, unmaintainable, fragile code in Clojure

2009-07-16T17:56:00.009-05:00

I constructed a subsystem in Clojure. We had a nice, compact problem to solve. The rest of our app talks to this Clojure bit through a small interface. The attraction of Clojure was that I could enjoy all the nimbleness of hacking Lisp interactively. Using such a powerful language, I could adapt to emerging requirements, experiment, twiddle, and workaround faster than using Java. And early on, that largely proved true.

But I am here to tell you: It is possible to write inflexible, unmaintainable, fragile code in Clojure.

As they say in bicycle racing, the riders make the race. I never said I was a good programmer.

It's my own fault. I dived into Clojure and learned key concepts on the run. And these key concepts are fundamental to how you should structure your program. FP has a learning curve. Managing shared state using Refs, Agents and STM -- better to understand these concepts up front. Even now, I'm unsure how I'd restructure my code to use them properly. I'm very glad not to have to worry about deadlocks. But there are parts of this code I still am unsure how to to test for consistency under multithread access.

Consequently, I'm slogging through this bowl of spaghetti of my own making, cursing the author.

[Edited to add "fragile" ... as I made one more change and my tests came tumbling down... ]

Wolfram Alpha isn't sure what to do with your input

2009-05-30T15:10:00.002-05:00

I have yet to get any answers to any interesting, real question I have had, from Wolfram Alpha.

If RMS has his way, the GPL is dead

2009-04-25T17:17:00.012-05:00

The GPL has never made me feel particularly free. Only in a kind of newspeak can you say that constraining the kinds of agreements I can make with people who buy my software enhances my freedom .

I'm not saying I don't like and use GPL software, or that the GPL is unfair. Just don't tell me that handcuffs are extra freedom.

Only copyright law enables GPL to constrain my actions from beyond the grave. In a world without copyright, GPL would be toothless. I would be able to use any freely available software I could get my hands on, incorporate it into my own, and distribute only binaries.

Richard Stallman proposes reducing copyright protection to three years for software. And he admits "It would be necessary to prohibit the use of contracts to apply restrictions on copying that go beyond those of copyright." That prohibition would apply to FOSS too, of course. No copyright after three years; no contract allowing the author to constrain your actions after that.

That would mean that you could link to three year old GPL libraries and ship binaries!

The REST Hypothesis

2009-04-18T16:17:00.018-05:00

I'm glad Joe Gregorio frankly assessed Atompub's meager adoption.

Atompub served as an experiment to confirm the REST hypothesis: If you construct distributed systems that look like the world wide web, the world will adopt them broadly and quickly -- the way we adopted the web itself. *

Lesson learned: the world's most successful distributed applications run on the web's architecture. That doesn't mean that by designing around web architecture, you will build the world's most successful distributed application.

Atompub came along at the time a critical mass of thought was building in favor of adopting the web architecture for designing new systems, rather than the ill-named "web services", SOAP and RPC model. Early drafts of Atompub allowed for a SOAP envelope.

The RESTful style prevailed, on its many merits. We got caching, and a resource oriented model, and a small, uniform interface.

But what the real, browser plus HTML web has, that RESTful systems don't, is the user agent. The human in front of her browser. An intelligence that reads and understands the meaning of "Author name" and "Title", and fills in an HTML form using queries against her personal database, stored in her brain.

As Joe put it, the problem is that Atompub clients aren't web browsers.

RESTful systems that aren't web browsers try to substitute understanding of media types for that intelligence. Compare Atom clients to HTML browsers.

An "Atom application" is some sort of content management system: a system that understands the semantics of feed documents. An Atom agent populates <author> and <title> elements. Only machines understand the meaning of <link rel="edit">. It's a world constrained to a narrow range of meanings.

But an HTML application can do anything. A human reads some text next to a form field, labelled "Author name", or "Preferred airline", and enters meaningful answers. If you are writing the kind of content management system where "Preferred airline" is a meaningful concept, HTML might be the way to go. There's no such concept in an Atom feed document. You get to present your own user interface, too.

RESTfully designed systems might profit from superior evolvability, cacheability, and interoperability over RPC systems. They are of the web. But they are not the web. The web is in your browser. The web is HTML.

*The hypothesis was not part of Roy Fielding's thesis. It's a hypothesis that many REST proponents, including me, have deployed.

Cheney Wheelchair Horror

2009-01-20T11:01:00.003-06:00

Best comment so far:

If only he had a small cat to pet as they rolled him to the car. You know something that he could stroke until he snapped its neck to suck the blood from its spine.

(by "Brian")

Clojure vs JavaFX Script: only 5x slower!

2009-01-03T01:42:00.006-06:00

Chris Oliver compares his JavaFX script to JRuby and Groovy and finds JavaFX 25 times faster.

I failed to get JavaFX compiler running on my Ubuntu, but I easily ran Chris's JRuby 1.1.6 test; so for calibration, my laptop comes in at 3:52, vs 4:22 on Chris's machine. Presumably we can scale results on my machine by 1.13.

Here's my Clojure version of the test:


(defn tak [#^Integer x #^Integer y #^Integer z]
  (if (>= y x)
    z
    (recur (tak (- x 1) y z)
    (tak (- y 1) z x)
    (tak (- z 1) x y))))

(dotimes [n 1000] (tak 24 16 8))

It runs in about 50 seconds, or 56 after normalizing to Chris' machine speed. This puts Clojure north of JRuby and Groovy, but still 5 times slower than JavaFX.

If you remove the type hints, it runs in 4 minutes, or about the same time as JRuby -- and about 4 or 5 times slower than with type hints.

Clojure uptick

2009-01-01T12:48:00.004-06:00

Needing to build a standalone piece of server software as an add on to our Java product, I seized the opportunity to see how far Clojure has come. Getting my dev system set up, and learning the language, I noticed the results Google was returning for my queries were mostly very recent... and lots from just last month.

Could Clojure be approaching critical mass?

I've only written a hundred lines or so of code, but here are some random things I love:

1. Shorthand for maps: {:a 1 :b 3}.
2. sorted-map and hash-map -- thanks for letting me choose. Same shorthand syntax for either.
3. Shorthand to select a value from a map: the map is like a function name (my-map :a).
4. Shorthand for lambdas: #(...code...)
5. Shorthand for generating temporary symbols in macros: my-var#. No need to call (gensym).
4. No looping required -- use recur.
5. Java libraries are easy to use.

I haven't yet fiddled with modifying state.

It's free, it's functional, it's got a hardened standard library, it's got a REPL. You can write a program and deliver it on any platform. What could hold it back?

Flying the F-22

2008-12-12T18:52:00.006-06:00

My gut reaction to the internet chatter about distributed version control systems, like Git, was, "What advantage could it possibly offer us, that would justify me and developers spending so much time re-learning how to manage source code?" CVS kinda sucked, but was good enough, and Subversion was better... what more could you want? It's just version control.

But we had a branching requirement, and a customer requiring a branch we'd maintain for a few years.

From the Subversion manual:

Tracking Merges Manually

Merging changes sounds simple enough, but in practice it can become a headache. The problem is that if you repeatedly merge changes from one branch to another, you might accidentally merge the same change twice. When this happens, sometimes things will work fine. When patching a file, Subversion typically notices if the file already has the change, and does nothing. But if the already-existing change has been modified in any way, you'll get a conflict.

Ideally, your version control system should prevent the double-application of changes to a branch. It should automatically remember which changes a branch has already received, and be able to list them for you. It should use this information to help automate merges as much as possible.

Unfortunately, Subversion is not such a system....It means that until the day Subversion grows this feature, you'll have to track merge information yourself. The best place to do this is in the commit log-message.

OK! So, relearning version control might not be a bad idea!

Git is all about changesets, and it can merge changes from branch to branch painlessly. A few months ago, we imported a year or so of changes from SVN to a Git repo (our tree structure was way too f***ed up for git-svn), and never looked back.

I'm a Piper Cub pilot in the cockpit of a F-22 Raptor. There are a lot of switches. I kind of understand some of them. I'm afraid to touch most of them. Every now and then we'll try a new trick -- hey, I think I can just push this changeset to you, from the branch I'm working on to yours... SHIT, THAT WORKED! We haven't crashed yet, and we have the rumbling sense of untapped power.

For reasons unrelated to version control, it turned out we could manage the customer requirements without a long running branch. But we're not going back on Git.

REST Blessed

2008-11-21T12:29:00.000-06:00

Gartner has blessed REST architecture by giving it a three letter acronym and a $495 report. But you can get the gist of the paper for free in Nick Gall's blog post: WOA: Putting the Web Back in Web Services.

Commercially, it's a big help that I can now point a customer's solution architect to a Gartner report, rather than to a guy's thesis and 20,000 posts to the REST-discuss mailing list. I hope that doesn't sound cynical -- it's not. It's just that architecture arguments are hard to win just on the merits. You need a little mindshare too. Thanks, Nick!

Electronic Vote Fraud

2008-10-23T20:01:00.007-05:00

I think I know how we can use electronic voting to increase our confidence in vote counts -- as opposed to what we have.

What we have is a mad world where you can construct the most extreme hypotheses about what is happening to your vote, and they could be true. You can't prove they're not.

When I complete my vote, I want to receive a printed receipt with a) my vote in clear text, b) a hash of my identity, and c) my digital signature of the above, for non-repudiation.

I should be able to log onto the county's web site, and verify that my vote was recorded for my candidate:

Voter	Candidate
38907389343666	McCain
84567867435432	Obama
45878963063630	McCain
92573532235672	Obama
...	...

Anybody can look at this table and count the votes for McCain and Obama ,and confirm that they match the announced totals. They can confirm, too, that their own vote was counted for the proper candidate.

My voting receipt matches one row of the table. The voting machine prints it out, and I review it. I should be able to confirm the hash of my identity by some public algorithm. It could be a hash of my voter registration number plus some secret password I make up.

My receipt also has a digital signature that I apply only after I review its accuracy. Yep, I voted for that guy. I insert the paper into the machine and it prints a string that's the digital signature of my identity and my vote. That way, I can't claim the machine got my vote wrong -- I reviewed the clear text vote, and signed it.

Voting this way would be lots more verifiable than paper ballots and hanging chads. You'd be able to verify your vote was counted.

Since early voting began Tuesday in Travis County, Texas, the email chain below has grown virally (local news story). And how can you prove these fears are wrong?

From: redacted
Date: October 22, 2008 11:48:50 PM CDT
To: redacted
Subject: DOUBLE CHECK YOUR BALLOT before pushing VOTE

no matter who you are voting for:
A friend of a friend of mine has this horror story about voting this morning
in Austin, TX.

He voted at Fiesta Mart on 38th and IH-35.

He voted a straight Democratic ticket.

When he was reading the 'voted for' listing at the end of his
ballot, all of those listed were Democratic candidates EXCEPT FOR
PRESIDENT. The list showed that he had voted for John McCain!!!

And he voted the straight Democratic ticket.k

He reported it to the election official and that person was as
shocked by it as my friend. They corrected the vote BEFORE he hit
the CAST BALLOT button.

My friend said the experience made him sick to his stomach. He said
he was the youngest person in the voting area and all he could
think is that the older people around him may not proofread their
ballot before pressing the CAST VOTE button. They may believe that
they voted for Obama but the voting machine may have registered a
vote for McCain.

He called the Travis County voting office and they said they would
look into this. When he called me I gave him the telephone number
for the Democratic Party and he then called them to report what had
happened and they said they would look into it, also.

Please PROOFREAD your ballot choices BEFORE hitting the CAST BALLOT
button. This is vitally important. (For EVERYONE who votes,
whichever way you vote.)

Leaps

2008-10-10T20:41:00.003-05:00

Dear Lazyweb,

1. In 1929, how many Wall Streeters actually did leap to their deaths from their office windows? I'd be surprised if it were as many as 10. Growing up during the sixties, I had the impression it was raining stockbrokers on Black Tuesday; but now I find it hard to believe.

2. They had names. Who were they?

3. Why did they kill themselves? In 1929, was there a greater sense of personal responsibility for the margin calls they wouldn't be able to make? A greater fear of the shame that would attach to them personally?

4. Has any Wall Streeter killed himself or herself during the current financial crisis?

5. Have any of the current super wealthy attempted to guarantee the system using their personal wealth, as Rockefeller did in 1907?

I pray the answer to 4 is no.... yet I wonder about what level of personal responsibility we should expect, and why it's different than 1929 and earlier.

Dell Precision M2400 + Ubuntu Hardy

2008-09-14T00:57:00.008-05:00

Welcome, Googlers. You are here because you have put Ubuntu Hardy (or perhaps Intrepid) on your shiny new M2400 (or other Precision laptop). And you've run into issues with your wireless; to wit, it don't work.

I've got the nVidia Quadro 370M, and Intel wireless 5100 cards. A fresh install of Hardy on a Dell Precision M2400 supports the nVidia (howto). But the 5100 wireless card support is non-existent.

To get both of these devices working, you're going to have to build a new kernel, and also compile a beta nVidia driver interface. It's actually an automated and fairly bulletproof process, so don't lose your nerve.

I think there are some other approaches that could work. I tried the compat-wireless strategy -- it's a technique using the recompiled Intel 4965 driver plus new firmware (oh yeah... you're going to need that 5000 series firmware from the Intel site anyway, so go get it). That strategy actually worked... as long as I ran "make load" on every boot... and occasionally my keyboard would freeze, requiring a reboot. I never could get that approach to feel stable, and my googling left with the impression that nobody had taken it much further than I had.

But to focus on the strategy I got working:

The problem is that the new Intel iwlagn drivers are part of the very latest kernel. I do not believe they are in even the Ububtu Intrepid kernel, so I did not attempt to promote to that alpha version of the distro. You can google to find lots of people dist-upgrading to Intrepid only to find they still don't have support for the 5100.

But there is support in the 2.6.27-rc6 kernel, the latest stable development pre-patch.

Build the 2.6.27-rc6-ultimate kernel

If you have X + nVidia set up the way you like, now would be a good time to save a copy of /etc/X11/xorg.conf.

You want to build this kernel the Ubuntu way. I suggest you follow the command line instructions on that page, and avoid the KernelCheck GUI program. I tried KernelCheck first and it is a really handy way to build a kernel. But it seems not to have left artifacts you will need later, when you build the nVidia driver interface. I admit there could have been pilot error in my case -- but I know the command line method works correctly.

When you get to the "make xconfig" step, choose Find from the Edit menu and search for "iwl". Turn on all those options.

Don't forget to select 64 bit options.

I emulated KernelCheck's naming pattern and identified my kernel with the name "ultimate", so my make-kpkg command was

make-kpkg --initrd --append-to-version=-ultimate kernel_image kernel_headers modules_image

So my uname -r is "2.6.27-rc6-ultimate".

Follow the instructions all the way and install your kernel. Don't worry, your old kernel is still there and in case of trouble, you can always choose it in the boot menu.

Remember that firmware you downloaded above? Put it at /lib/firmware/2.6.27-rc6-ultimate/iwlwifi-5000-1.ucode. Then reboot, and you'll have wireless... but no nVidia.

Build the nVidia driver

When you reboot into your new kernel, the system will drop you into a default 800 x 600 X session. Don't attempt to configure X yet if it prompts you.

There's no pre-built nVidia driver for this kernel. I also found that only the latest beta nVidia driver interface compiled against this kernel.

In order to build and install the driver, you need to kill X and make it stay killed. Ctrl-Alt-F1 to get a console, then "killall gdm". Then, following the instructions on the nVidia site, run the install script you downloaded. It should even edit the xorg.conf file to install itself, but in my case I still had to go back and set the resolutions -- I just replaced xorg.conf with the one I had saved at the beginning.

Our only departure from the Ubuntu way was using the nVidia install script -- it puts nvidia kernel module in a place of its choosing, and it won't be managed by apt for us. But we're not going to be getting new kernels from apt anyway -- we built our own, and we're now going to take responsibility for upgrading kernels (as needed) and nVidia at the same time.

Tidying up

You want to be able to suspend and hibernate, and it just takes a couple of edits. To hibernate, you'll need as much swap space as you have RAM. With this computer, unlike my last one, I've done that, and hibernate now works after the edits.

The NetworkManager annoyingly asks you to enter your keyring password whenever you log in. Googling around, I found some solutions. One of them worked for me. I'm not sure I used the best solution, so I'm punting and letting you find your own answers.

Hey -- this backlit keyboard is pretty nice!

Update Sept 17: Using the Hardy config file as your initial config (config-2.6.24-19-generic), simply enabling iwl* as above leaves you missing two modules that Ubuntu installs only as part of the linux-ubuntu-modules package. Your sound card and webcam require extra modules. In make xconfig, be sure also to select "USB Video class" (if you have the webcam) and "Intel HD Audio".

Check your DNS

2008-07-24T23:08:00.004-05:00

Dan Kaminsky has published a widget you can use to test your DNS server for the cache poisoning design flaw. You know, the one that allows a malefactor to send your PayPal requests to their own fake servers? The one that doesn't require any vulnerability on your computer, just the standard, unpatched DNS server that you use?

Both my home and business ISPs failed the test. I followed Dan's advice and pointed our routers to OpenDNS. I guess it's a good day to be an investor in OpenDNS. Except I can't figure how they're going to execute on their plan to serve me ads via DNS. Anyway, this is their moment of glory, and they seem to be holding up under the load. If Dan has any stock in OpenDNS, that would be clever of him. I am not suggesting he does. Unless it turns out that he does. Then in that case, I am suggesting it retroactively, and don't say I didn't tell you so.

Get with it, Pops

2008-07-01T20:57:00.004-05:00

Dare gives evidence that older Googlers are coming away dissatisfied from the experience.

Here are a few disapproving quotes:

It was obvious that they do not care that I had 12 years software engineering experience
orientation towards cool, but not necessarily useful or essential software
don’t have a career path for their employees
There is no legacy code
As all organizations mature they tend to add PROCESS

Isn't this the static vs dynamic argument? Old models of software development vs new? On premise software vs software as a service? Will legacy code even be important at Google? Shouldn't our goal be disposable code?

Aren't perspectives like this the reason Microsoft is perceived as a falling star while Google is a rising one?

Asynchronous HTTP POST

2008-06-28T17:26:00.014-05:00

I've got to process a huge POST asynchronously. If it were small enough I'd just return 201 Created, with the URL of the new resource in the Location header. But this is a massive file upload that requires a bunch of processing, and checking, before I can create the resource. It takes so long to process, your TCP connection will time out.

202, right?

This is a job for 202 Accepted, right? I had always understood 202 was how to respond ansynchronously to a POST. I return some hypertext with a link you can follow to see the status. You follow that link, load the status page, and hit refresh until it shows "100% done" and displays yet another link to the resource you created. That's what the RFC says to do.

In a machine to machine case, e.g. Atompub, I have to define meaningful content types so that clients can follow the hyperlinks to learn how the POST came out. If you're thinking of using the Location header: It isn't blessed for 202; even if it were, we couldn't guarantee that the resource at the returned URL will ever exist.

Why not 303?

I prefer to respond with 303 See Other. Clients follow that Location header to a status page, eliminating the extra manual step required by 202. Retrieving that page would itself return 202 Accepted, until I've created the resource, or failed to. Then that URL would return 201 Created, or whatever the result would have been in the synchronous case.

This way, user agents just follow the semantics of HTTP, and never need to understand any application entities.

Summarizing:

Any user agent will knows to follow the 303 to the status page automatically. This is the URI giving "the response to the request", which is a status page. Any time you want the response to this POST request, go to that URI.

GET on that status URI, for some time, returns 202 and an entity giving "an indication of the request's current status".

Finally, at some time when you check that URI, you get the final "response to the request".

The surprising part might be encountering 202 or 201 in response to GET. But nothing says you can't, and in fact that is the "response to the request".

(Ben Ramsey discusses returning 202 from the POST. It gets a little messy.)

Ruby: DSL for Writing Programs

2008-06-14T23:05:00.004-05:00

Ruby:



PEOPLE = [{:first=>'scott', :last=>'guthrie', :age=>32},
              {:first=>'susanne', :last=>'guthrie', :age=>32},
              {:first=>'bill', :last=>'gates', :age=>50}]

def avg_age
  guthries = PEOPLE.select {|p| p[:last] == 'guthrie'}
  guthries.inject(0) {|s, p| s + p[:age]} / guthries.length 
end

Common Lisp:



(defvar  +people+ 
  (list 
   (list :first "scott" :last "guthrie" :age 32)
   (list :first "susanne" :last "guthrie" :age 32)
   (list :first "bill" :last "gates" :age 50)))

(defun avg-age ()
  (let ((guthries  (remove-if (lambda (p) (not (equal "guthrie" (getf p :last)))) +people+)))
    (/ (reduce #'+ (map 'list  (lambda (p) (getf p :age)) guthries)) (length guthries))))

These snippets calculate the average age of the people having last name "guthrie". The functions are both two lines long, but the Ruby one is more readable.

S-expressions give Lisp powerful ways to build code in code. But writing programs using S-expressions seems cumbersome compared to Ruby's syntax. The syntax of Ruby shortens the code you have to write to do everyday tasks. It's like a domain specific language for, er, writing computer programs.

You give up some programming power -- a fair trade if you rarely need that power.

Self-appointed Guardians of Truth

2008-05-29T15:17:00.003-05:00

SiteTruth has given my company's web site a big red .

Even though we don't sell products electronically, they wish we had a certificate. And we've never put our office address on that site -- an oversight when we moved, not a scam. So they give us a big red "Do Not Enter" sign, indicating our site is dangerous to enter. From their "about" page:

Every on-line commerce web site must display the name and address of the business behind the site. That's the law in much of the developed world. SiteTruth tries to identify that business, then find information about it. That check is used to influence search rankings. That's SiteTruth. (emphasis mine)

We're not an "on-line commerce web site", but their system can't detect that, so they're selling technology that will lower our search rankings?

We'll fix our site to please them, of course; why not? But their technology doesn't seem to increase the safety of the web. And is likely to piss off other legit site owners, some of whom may even feel litigious. Could you blame them?

Hardly any near death experiences

2008-04-25T12:03:00.006-05:00

The Hardy upgrade went... ok. Better than the upgrade to Gutsy, at least --the initial boot this time at least got me to an X, if only in 600 x 800 mode.

So you don't have to read to the end, here's my helpful hint to you this upgrade cycle. If you're having trouble with your nVidia: after you upgrade, use Synaptic to uninstall your old restricted drivers packages (in my case, for kernel 2.6.22), and select the new ones (for kernel 2.6.24) which, in my case at least, were not selected after the upgrade.

Only after you do that, and reboot, can you see the new nVidia driver in the System/Administration/Hardware Drivers applet. This used to be called the Restricted Drivers Manager and all the online docs still refer to that. But Hardware Drivers is what you want. Go there and do what comes naturally.

Last night I was finally able to get Update Manager to connect and start downloading files. I let that run overnight -- it took 6 hours or so to download everything, presumably because of the tremendous load on the U.S. Ubuntu server.

This morning, I awoke to find it paused in a dialog. Just a warning that it was about to clobber my modified mime.types. I answered OK. It proceeded a little, displaying the progress in a little terminal window. I noticed some interesting progress output, and started editing some notes. At one point, I selected some text in the terminal window, and copied it. Using Ctrl-C. Who would have thought the goddamn terminal window was accepting keyboard input and processed the interrupt. So immediately I get three alerts warning that it could not complete the emacs/ede/eieio installations becuase it had received an interrupt. Yesterday I predicted the emacs upgrade would fail, but I didn't mean to fulfill my own prophecy.

It seemed to continue from there almost without trouble -- but at the end it complained it could not upgrade the update-manager. Ironic, isn't it? And at the end of the install it displayed a scare alert: "Your system may be unusable". Nice. Bravely, I rebooted, and found myself in 800 x 600, but at least with an otherwise stable system.

After some flailing I discovered the secret sauce to getting nVidia working again. And all systems are now go!

I wonder how Mark Pilgrim's mom and dad are doing?

[updated to fix link]

Ubuntu Hardy is out and I'm a sucker

2008-04-24T15:32:00.003-05:00

I managed to hold off about 6 hours before giving in and pressing the "Upgrade" button to get Hardy Heron. Yes, I'm in the middle of a project. Yes, I expect I'll have nVidia issues. My emacs will probably fail upgrading as usual. But: it's too tempting. I'm a sucker. I know it! No discipline whatsoever.

Hm... I guess the servers are mighty busy today. My first attempt timed out: "Could not download the release notes/ Please check your internet connection".

Lisp a casualty of its time

2008-04-12T11:38:00.015-05:00

This sad editorial remark from the CLIPS architecture document, describing perceptions about Lisp within NASA around 1984:

...Despite extensive demonstrations of the potential of expert systems, few of these applications were put into regular use. This failure to provide expert systems technology within NASA’s operational computing constraints could largely be traced to the use of LISP as the base language for nearly all expert system software tools at that time. In particular, three problems hindered the use of LISP based expert system tools within NASA: the low availability of LISP on a wide variety of conventional computers, the high cost of state-of-the-art LISP tools and hardware, and the poor integration of LISP with other languages (making embedded applications difficult).

Would they encounter the same barriers today, twenty-four years later?

1. Low availability of LISP on a wide variety of conventional computers: Now we have commercial implementations and open source Lisps, working on most computers and operating systems.

2.High cost of state-of-the-art LISP tools and hardware: The hardware issue has faded away since Lisp machines gave way to general purpose computers. And Lisp is no more memory intensive than Java or .NET. Of the commercial Lisps, LispWorks is affordable for normal mortals, and certainly the Allegro products are within reach of NASA. And there are several good, free, open source CLs.

3. Poor integration of LISP with other languages: An issue only if your mindset is 1984. Nowadays, to integrate with a Lisp program, you'd treat it like any other network resource -- like an RDBMS, or a web service.

I wonder what is the state of the Lisp renaissance within NASA?

[updated to expand remark on item 2]

Rule of Least Power: Bah!

2008-03-03T13:01:00.002-06:00

The Rule of Least Power, a W3C TAG finding, posits: "Powerful languages inhibit information reuse." They're observing that it's easy to scrape documents written declaratively using HTML. The problem with using more powerful languages like Javascript, they say, is that "you typically cannot determine what a program in a Turing-complete language will do without actually running it."

So? As long as the output is a DOM, just run the program and inspect the DOM.

You already have to use a good HTML parser, right? Now, just run all the script elements on the page too -- obviously, in a restricted environment.

I'm sure Google and friends must do this. They're not going to leave valuable information on the table.

Yet another reason to use NoScript

2008-02-05T21:10:00.000-06:00

Niall Kennedy tells us all how he grabs your browser history "for improved user experience".

1. put links on the page to all the interesting sites you want to identify in the user's history. Have they visited Google? Have the visited certain porn sites?
2. put some css for those links that colors them some weird color when they have been a:visited.
3. add some script that crawls the dom looking for links styled that color.

Voilà.

Yet another reason I'm glad I've turned off most scripting using NoScript.

[Updated to remove a made up porn site domain name that actually is a porn site domain name.]

Eli can't get no respect

2008-02-04T19:02:00.000-06:00

[Update Feb 5: they've fixed the headline, but I just noticed the caption at the right still says "Super Bowl XLI". So that explains it. To build yesterday's Super Bowl results page they just grabbed last year's. Then they attempt the replacements s/Peyton/Eli/g and s/XLI/XLII/g. Probably they left off the 'g'!]

One Manning's as good as another to the British:

Clojure

2008-01-20T21:16:00.000-06:00

Clojure is a Lisp that runs on the JVM. I agree with everything on this page describing the design rationale. Objects are overrated; functional and immutable are good. I can't wait to get a breather and figure out whether and where Clojure fits into our technology arsenal.

The real measure will be how well it can leverage the Java platform. There are plenty of great languages, but they have inconsistent support for, say, sockets, or window systems. Can I override a protected method of a Swing base class? Can I connect to an HTTPS server? JRuby has knocked Java integration out of the park, so you get all that capability for free, plus a powerful language.

Update: I think the bullet point "Extend Java in Java, consume Java from Clojure" on the rationale page suggests you can't easily write a Swing app in Clojure. Shoot.

RIA using JRuby and Web Start

2008-01-20T14:21:00.000-06:00

I went dark over the last eight weeks, building a rich web application to integrate with our product. It's been years since I built a desktop GUI application, but I was pretty good at it once. It's not always a lot of fun. But some things a man just has to do himself.

The application does tons of vector drawing.... it draws hundreds of thousands, even millions, of line segments. Yet it needed to be responsive... scrolling, zooming, panning. It's data intensive, too. It downloads dozens of megabytes of data for each view. I needed to develop this program fast, so I needed a programming system and platform allowing me to iterate and adapt to new requirements recursively revealed. I preferred a declarative graphics language where possible, and for any procedural code, a powerful dynamic language. Ideally the program should run on all OS platforms; but the business could probably tolerate a Windows only solution, if that offered superior results. I made tradeoffs among these requirements.

I was not constrained by download footprint, a concern which eliminates Java from consideration in lots of other scenarios. Anyway, that problem is going away soon, they tell us.

So here's the spoiler: I chose Java Web Start + JRuby. And now a short dissertation on how I arrived at that choice:

I looked to browser and browser++ technologies like SVG, Flex, AIR, Silverlight, Java applets/Web Start/JavaFX. (Open Laszlo deserves a look too, but I didn't eval it). It would be great for this app to run within the browser -- that would be the most seamless experience. But using an out-of-browser technology like AIR or Web Start isn't a deal breaker if you do it right.

SVG was very promising. It would be a pure browser solution. I really valued the declarative model and the standards alignment. But it does not work in IE. I also did some performance experiments with Opera, Safari, and Firefox. Drawing polylines of a hundred thousand points or so, Opera and Safari performed well, but Firefox performed poorly. It's conceivable, if unlikely, that I could distribute the app as a FF-only solution -- even if I bottle it up as a a XULRunner app. But the FF performance was inadequate. And distributing only for Safari or Opera is not realistic.

I made a significant evaluation effort with Flex/AIR. Surprisingly I learned that the Flex 2-D graphics model is procedural, not declarative. You can, however, bottle up your procedures and use them declaratively from mxml, the Flex XML language. I felt I was going to spend a significant amount of time learning Flex -- time I needed to spend on my app -- and that I needed the assistance of tools like Flexbuilder. I also became concerned that I was developing a program away from the design center of Flex. Would I encounter design/ performance constraints?

Silverlight performed very well, and it's got an XML declarative model just like SVG, with the power of a bunch of WPF components to boot. I was confident that all the capabilities of the Windows platform would be available to my program. Microsoft distributes Silverlight for Windows and Mac, but not Linux, which they are leaving to Novell. I read that as: No Linux. But Windows and Mac are enough coverage, even if I would be sorry to leave aside my Ubuntu dev machine. My fate on the Mac would be in Redmond's hands, of course. Does Microsoft have my best interests at heart ?

Java applets or Web Start, using Java 2D, I also knew to be capable of these rendering tasks. I'm satisfied that I can make Java applications now that look pretty sweet. The down sides were the procedural approach and the static Java language. It's the 1999 state of the art. Layout code so impenetrable you need GUI builders.

JavaFX deserves a separate mention. The capabilities of the entire Java platform are there for you. Yet you program in a declarative/dynamic language optimized for building Java 2D apps. Perfect! Unfortunately, it's pre-alpha. Sun hasn't yet released it under a license allowing you to put your work into a customer's hands.

I wanted the wide reach of Java. I think that the 5% of customers on non-Windows platforms may be more important than their numbers.

I also wanted the enormous hardened Java platform. It's as capable as the Windows platform. You won't encounter many problems you can't address or workaround.

Couldn't I have all that, plus a dynamic language to stitch it all together?

Guess what. JRuby is stable and 1.x. It's dynamic, yet it compiles on the fly to JVM byte codes. It's got great Java integration, meaning you can invoke any Java API. Profligacy is a terse little Ruby lib for stitching up Swing components. JRuby's just a jar I download with my Web Start app, and launch with the main Ruby script from my Java entry point. The rest of the program is in Ruby, occasionally calling into Java libraries. No, there's no declarative GUI language here. It's one of the tradeoffs.

My work environment: Fire up NetBeans for its good Ruby editor. In a command window kick off jirb and execute a script to load and run my main code. Edit code and reload it into the running program in jirb. Lather, rinse, repeat.

I would argue that JRuby accelerated my application's performance, because I iterated and applied two major architectural optimizations that would have introduced impenetrable spaghetti in Java. The bottleneck for speed and responsiveness has never been JRuby. It turned out that even Java2D + acceleration could not keep up acceptably with the level of interactivity I wanted. The SVG or any other pure declarative approach would have failed (although I think JavaFX's scene graph might be able to optimize the drawing). I had to go in and surgically rearrange the innards of the program. The changes were effective but had low impact on stability, thanks to the flexibility and power of Ruby.

If Sun can focus more energy on these dynamic languages built on the JVM, that would be a pretty powerful story: The deep platform capabilities, the broad reach, the rich development environment.

And Speaking of Paul Graham...

2008-01-15T23:46:00.000-06:00

Arc to be open sourced this winter. It's going to be hard to top the expectations.

The Truth about Lisp

2008-01-14T19:24:00.000-06:00

Majorly funny. Leon Bambrick wrote (back in 2006):

If you're good enough to use lisp, you'll soon be frustrated with lisp. Lisp is not an adequate lisp. By the time my bus had made it two blocks I'd written some simple lisp macros that were so powerful they made lisp completely obsolete and replaced it with a new language. Fortunately, that new language was also called lisp. And i was able to prove, mathematically, that the new lisp i'd created was both far superior to lisp in every conceivable way, but also exactly equivalent to lisp in every possible way. I was very excited by this. But also found it very boring....Paul Graham himself was completely written in lisp, by an earlier version of himself, also written in lisp, by an earlier version of lisp. It's lisp, paul graham, lisp, paul graham, all the way down.

Why objects suck

2008-01-10T20:22:00.000-06:00

class Curve {
String name;
float[] data;
Color color;
}

class Plot {
Curve[] curves;
void addCurve(Curve c);
void removeCurve(String curveName)
}

"No, no, no... you're mixing in style information. Pull the style out of Curve and separate the concerns." Oh. OK...

class Curve {
String name;
float[] data;
}

class CurveStyle {
String curveName;
Color color;
}

class Plot {
Curve[] curves;
void addCurve(Curve c);
void removeCurve(String curveName);
void setCurveStyle(String curveName, CurveStyle style);
}

Well, all right then! I'm sure this in a patterns book somewhere. I feel warm satisfaction with the pure, objectified minimum entropy of my factorization. Until I implement Plot.removeCurve... oh shit... I have to search for all the dangling CurveStyles referring to the curve, and delete them. As OO programmers we do loads of that every day. You come to think of it as natural. The best OO languages have list comprehensions that make it easier to do. That's nice, but it's not enough. The problem is that "objects" aren't a logical model.

Wouldn't it be more "natural" for the dangling CurveStyles to delete themselves when no longer relevant? There's a logical model for that, the relational model:

Style

CurveName	Color
MSFT	Red
GOOG	Blue

Curve

CurveName	Data
MSFT	1,4,-9
GOOG	13,2,22

Logic tells you that the style row cannot exist when the Curve row has been deleted. A foreign key relationship forces the system to delete the style when you delete the curve.

I want my programming language to do that for me, and that's part of what I was wishing for two years ago.

FIQL: Query language for feeds

2007-12-18T20:10:00.000-06:00

My first reaction to Mark Nottingham's FIQL draft for querying Atom and RSS feeds was: Do we really need to bake in a prescription for constructing query URLs? Section 4 seems to suggest that. It seems to take the control over the URL namespace away from the server.

Here's Mark's example HTTP FIQL (I'm already saying "fickle") query:

http://example.org/feed.rss?title==*great*;ex:rating=gt=4

Element names become the query keys. So if your feed language has element <ex:rating>, you can use that as a query key.

FIQL describes rules for constructing URLs based on your content type. Contrast that to HTML forms which prescribe how to construct URLs based on content.

Is that so bad? It's a little too API-like for me. Any Atom feed now has that query namespace imposed on them: If you want to honor queries, these terms become part of your URI space. Amazon, Blogger, you, I all have this URI subspace imposed on us.

If, instead, we define a FIQL content type, application/fiql, and you POST a FIQL document, the server can respond with the query result, or can construct a URI of its own design and redirect to it.

The server retains complete control of its URI namespace.

I'm interested in how we work through this issue because I just suggested a similar strategy for WITSML queries a few days ago.

What is wrong with this picture?

2007-11-16T19:28:00.000-06:00

Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
content-disposition: attachment; filename=Hill%20Country%20water%20issues[1].ppt
Content-Type: application/unknown

IIS servers don't come provisioned with the .ppt extension mapped to application/vnd.ms-powerpoint? Microsoft server? Microsoft application?

What hope can there be for authoritative metadata?

My Firefox browser figures it out just fine, and launches Open Office. Presumably FF went through all this first, just to determine it to be application/octet-stream; then ran a further sniffer to identify it as a Power Point.

It's the hyperlinks, Stupid!

2007-11-06T18:41:00.000-06:00

Henry Story thinks Echo2 is "Web 2.0 in Java", and they do have a killer demo. But it's yet another example of incredibly brilliant developers going to great lengths to bring the desktop app to the browser, and ignoring that the value of the web lies in hyperlinks.

So navigate that demo: Click the "Next" arrow. Your address bar doesn't change even though you have navigated to what most people would call a clearly identifiable different resource.

I can't give you the link to "page 2" because it isn't addressable on the web. Sorry. Go to page 1, then click to go to page 2.

Under the hood, it's the usual collection of RPCs masquerading as URLs -- some GET, some POST -- using port 80 since we know it's open, after all. Every single GET uses all the usual tricks to make sure that nothing -- not even JPEGs -- gets cached.

(Why don't we just give Javascript in browsers an API to open up a socket and execute any protocol you like? Seriously: Wouldn't that be better than abusing HTTP? If you don't want to use the HTTP protocol, you shouldn't have to.)

I love the look of that demo, and I think their technology is clever. With a little effort I am sure they can make the framework webby.

I was just thinking that!

2007-11-03T21:08:00.000-05:00

I've been working on the technology requirements for a "rich web application". All the pieces are in place for a cross platform solution. With Javascript, HTML, SVG, and CSS, you can build a pretty rich application without resorting to JavaFX, AIR, Flex, Silverlight, Click Once, or Web Start.

All the browsers support JS, HTML, SVG and CSS. Except one. Per Rob Sayre:

If Microsoft were really interested in making life easier for web developers, they could do so, without a standards committee. They would need to fix the (nasty) bugs in IE’s JScript engine, implement SVG, implement canvas, implement more of CSS, support a standard event model, and on and on. Then, the behavior of IE would be a lot closer to Firefox, Opera, and Safari.

It's no secret Microsoft doesn't see an advantage in a web built on cross platform technologies. I'm not very sure about Adobe, either.

Why should we, as a a software company, invest in technology from companies that are actively working to subvert what we want to do?

Rich Web Applications

2007-10-26T21:45:00.000-05:00

Here's the five step test to determine whether your "rich internet application" is a rich web application:

You click on a hyperlink in a web browser.
Your RWA opens up, and updates its code if necessary
Your RWA renders the document, and you edit it.
You save the file -- to its URL
You e-mail the hyperlink to someone; they open up the document and see your edits.

You should be able to click a link to a Word document, edit the document, and save it -- to the URL you got it from. The RWA is a rich, articulate way to render and edit particular kinds of documents.

If saving isn't a feature of your app, fine: But honor (1) ,(2), and (3).

You can do this with .Net Click Once, Java Web Start, and for all I know, Adobe AIR. But the design of the first two, at least originally, amounted to a way to install and launch your application by clicking a hyperlink, and update the code automatically. Worthy objectives. But no actual Web Start or Click Once apps I have used can do this: Click a document, edit it, save on server. You couldn't possibly do it with Web Start until JDK 1.5. And to do it with Click Once you have to sign the application and party in the registry.

The first thing is to register your RWA as the helper application for your document's mime type and file extension. Web Start enabled doing so when they added <association> to the JNLP 1.5 format. Web Start registers itself as the handler for your file extension; the browser launches javaws to open your file, and javaws looks at the file extension and invokes your application. Or: sign your Click Once application and you can set your application as the mime type handler in the registry.

That makes your application launchable when you click a link.

You will have to design the URL of the document into the document itself. Browsers download documents and save them to temp files; they don't tell you the URL of the document. When the browser launches your helper app, your app needs to know where to HTTP PUT updates, or to DELETE the resource. You can also design in other URLS your client understands: a base URL under which you can POST to implement "File/New", for example. Notice Atom has <link rel="self"> and Atompub service documents tell you where to POST things; you could build a good RWA around that scaffolding. Make your own documents describe how to navigate the states of your application by traversing hyperlinks.

Your RWA should have an address bar. Show the users where they are. Let them type in any URL they want to. If they type in one for a mime-type you don't understand, hand off the download to the registered helper application, or back to the browser.

All those buttons desktop apps use to navigate? Menu commands? Keyboard shortcuts? Back them with URLs you GET from or POST to. Display each URL in the address bar as the user traverses your application. After clicking any button that does GET, whatever you see in your RWA should be what your friend will see when you email him the URL in the address bar.

Automatic code update (step 2) is a necessary feature for a RWA. It's not enough just to have a link launchable application. You can evolve your schema, and change the meanings of elements in your document format, as long as you know that only the most modern code, that understands the new meanings, will be interpreting the document.

Make the extra effort to leverage the web architecture, and I won't blog about your beautiful but shitty application.

That killer web platform

2007-10-02T17:27:00.000-05:00

Here we go again with this specious argument that the Web isn't rich enough. Joel thinks there's a new killer platform out there to be invented, that will seize control of web applications as Windows seized the desktop.

Ain't gonna happen. Or, if you prefer: Already happened.

Right there in his own essay is the reason.

And that’s exactly where we are with Ajax development today. Sure, yeah, the usability is much better than the first generation DOS apps, because we’ve learned some things since then. But Ajax apps can be inconsistent, and have a lot of trouble working together — you can’t really cut and paste objects from one Ajax app to another, for example, so I’m not sure how you get a picture from Gmail to Flickr. Come on guys, Cut and Paste was invented 25 years ago.

See, Ajax gives you the capability to turn a perfectly good hypertext application into a miserable facsimile of a 1980's PC. And you're not going to fix Ajax by adding a bunch of new APIs. Applications need more constraints, not fewer.

Think how absurd it is that you can't copy a picture from GMail to Flickr. The tools are right there, but the application designers do not leverage them. a) Right click photo in GMail. b) "Copy link location". c) Paste hyperlink into Flickr. d) Flickr either downloads photo from GMail or references it. No new APIs needed -- it's all just hyperlinks.

It's great, and necessary, to extend HTML with rich widgets. We'll never capture them all, declaratively, in a common HTML. I am, even as we speak, constructing a Flash widget. But the web is the platform. Any time I push information deep into my widget -- text that could be searchable, graphics that could be linkable -- and hide it from the web, I've failed to leverage the platform.

Whois spam

2007-09-13T20:18:00.000-05:00

I ran whois on microsoft.com. My whois by default returns a substring match for the search term, and a lot of names have microsoft.com in them. It's hard to pick a favorite:


hughw@gx150:~$ whois microsoft.com | sed -n -e "s/Server Name: \(.*\)/\\1/p"
   MICROSOFT.COM.ZZZZZZ.MORE.DETAILS.AT.WWW.BEYONDWHOIS.COM
   MICROSOFT.COM.ZZZZZ.GET.LAID.AT.WWW.SWINGINGCOMMUNITY.COM
   MICROSOFT.COM.ZZZOMBIED.AND.HACKED.BY.WWW.WEB-HACK.COM
   MICROSOFT.COM.ZZZ.IS.0WNED.AND.HAX0RED.BY.SUB7.NET
   MICROSOFT.COM.WILL.LIVE.FOREVER.BECOUSE.UNIXSUCKS.COM
   MICROSOFT.COM.WILL.BE.SLAPPED.IN.THE.FACE.BY.MY.BLUE.VEINED.SPANNER.NET
   MICROSOFT.COM.WILL.BE.BEATEN.WITH.MY.SPANNER.NET
   MICROSOFT.COM.WAREZ.AT.TOPLIST.GULLI.COM
   MICROSOFT.COM.TOTALLY.SUCKS.S3U.NET
   MICROSOFT.COM.STINKT.NACH.WIESBADEN.VISIT.CALITEC.NET
   MICROSOFT.COM.SMELLS.SIMPLECODES.COM
   MICROSOFT.COM.SHOULD.GIVE.UP.BECAUSE.LINUXISGOD.COM
   MICROSOFT.COM.RAWKZ.MUH.WERLD.MENTALFLOSS.CA
   MICROSOFT.COM.OHMYGODITBURNS.COM
   MICROSOFT.COM.MORE.INFO.AT.WWW.BEYONDWHOIS.COM
   MICROSOFT.COM.LOVES.ME.KOSMAL.NET
   MICROSOFT.COM.LIVES.AT.SHAUNEWING.COM
   MICROSOFT.COM.IS.NOT.YEPPA.ORG
   MICROSOFT.COM.IS.NOT.HOSTED.BY.ACTIVEDOMAINDNS.NET
   MICROSOFT.COM.IS.NOT.AS.COOL.AS.SIMPLECODES.COM
   MICROSOFT.COM.IS.IN.BED.WITH.CURTYV.COM
   MICROSOFT.COM.IS.HOSTED.ON.PROFITHOSTING.NET
   MICROSOFT.COM.IS.GOD.BECOUSE.UNIXSUCKS.COM
   MICROSOFT.COM.IS.A.STEAMING.HEAP.OF.FUCKING-BULLSHIT.NET
   MICROSOFT.COM.IS.A.MESS.TIMPORTER.CO.UK
   MICROSOFT.COM.HOSTED.BY.CALITEC.NET
   MICROSOFT.COM.HAS.ITS.OWN.CRACKLAB.COM
   MICROSOFT.COM.HAS.A.PRESENT.COMING.FROM.HUGHESMISSILES.COM
   MICROSOFT.COM.GOT.OWNED.BY.CALITEC.NET
   MICROSOFT.COM.FILLS.ME.WITH.BELLIGERENCE.NET
   MICROSOFT.COM.DRINKS.LISTERINE.NET
   MICROSOFT.COM.CAN.GO.FUCK.ITSELF.AT.SECZY.COM
   MICROSOFT.COM.ARE.GODDAMN.PIGFUCKERS.NET.NS-NOT-IN-SERVICE.COM
   MICROSOFT.COM.AND.MINDSUCK.BOTH.SUCK.HUGE.ONES.AT.EXEGETE.NET

For completeness here's the whois version:


hughw@gx150:~$ whois --version
Version 4.7.11ubuntu1.

Report bugs to <md+whois@linux.it>.

Oh crap, I just invented Prolog

2007-09-13T19:07:00.000-05:00

Following a link trail that started with a discussion of CouchDB, I just found this old comment posted by Bill de hÓra:

Perhaps we can go top down - write smart analysers to dynamically denorm data based on usage patterns; indeed database optimisation is an industry sector. But another, dumber, option is bottom up - avoid the initial structural 'typing' step and normalise where necessary....Normalisation can be done later on, based on demand.

I don't even want to design database schemas. To hell with modeling. I just want a system that takes a great, undifferentiated pile of facts, and infers entities based on the statistics, primarily of the content, but also on the access patterns. Actually I don't even care that it infer entities; that's an implementation detail I don't need to know about. It would just optimize for query time, or insert time, or some combination; inferring some table structure will probably help it do that.

Why do I need to tell the computer the relations in my data? All I should have to do is insert facts, as triples. When I ask it a question, I want those facts returned (or maybe other conclusions).

Is a really tweaked Prolog the ultimate DBMS?

Using Atom to manage lists of Atom feeds

2007-09-09T18:27:00.000-05:00

As Phil Wilson does, I have come to rely on Planet Intertwingly as a substitute, largely, for maintaining my own list of interesting feeds. But now Phil wants to filter Sam's picks; he wants to select favorites from the PI OPML.

That raises the interesting prospect that we now need Atom feeds for OPML updates. When Sam adds a new feed, I would want to check it out. Since OPML content is, by definition, a list, I guess Sam could mark up the feed with Simple List Extensions. I'm temporarily suspending my suspicion of SLE, because PI's growing and shrinking list of feeds is clearly unlike an ordinary, infinitely growing feed.

Using Atom to manage lists of Atom feeds demonstrates that Atom captures a powerful abstraction.

Acura ITX

2007-08-15T18:57:00.000-05:00

A year and a half ago, I started following this (anonymous?) guy's blog about his project to install an ITX computer in his Acura. It covers everything: selecting hardware including touch screen and GPS, configuring software (Windows + misc. media toys), installation, and he's recently updated it with new photos.

I'd do things a little differently: Carbuntu (hey... carbuntu.org seems to have disappeared...), and I'd want a HSDPA or EVDO card and a dual core Intel... but there's little doubt mine would come out a mess. This fellow's really put in the polish.

Restful Servlets + JSP: My framework

2007-08-15T10:46:00.001-05:00

Stefan and Bill asked, so here's how I do it (with a tip of the hat to Django).

It is a micro framework. You subclass RestServlet and declare some URL patterns to match, and handlers for them. The base class parses the URI, sets attributes in the ServletRequest object based on the URI pattern, and invokes your handlers.

So here's how a simple BlogServlet would look:

[updated: fixed path to JSP, added a note above about attributes].
[update: added Apache license]


Copyright 2007 Wellstorm Development, LLC

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.


public class BlogServlet extends RestServlet{

  
   /**
    * GET an entry. The base class will populate the "entryId" attribute before calling invoke.
    * We told it to do so below, when we defined entryIdentifier.
    */
   RequestHandler entryGetHandler = new RequestHandler(){

       public void invoke(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
           try {
               // The base class has parsed the URI and populated the request with
               // the "entryId" attribute
               String entryId = (String)request.getAttribute("entryId");
               String forward = "/WEB-INF/entry.jsp";
               request.setAttribute("entryHTML", getEntryHTML(entryId));
               request.setAttribute("entryTitle", getEntryTitle(entryId));
              
               request.getRequestDispatcher(forward).forward(request, response);
           } catch (Exception e) {
               response.sendError(404);
           }
       }         
   };
  
   
   /**
    * POST a new entry to the collection URI
    */
   RequestHandler collectionPostHandler = new RequestHandler(){

       public void invoke(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
           try {
               String entryId = saveEntry(request, response);  
              
               response.setHeader("Location", buildEntryUri(entryId));
               response.setStatus(201);
               //write out some HTML or maybe the entry itself.
               String forward = "/WEB-INF/entry.jsp";
               request.setAttribute("entryHTML", getEntryHTML(entryId));
               request.setAttribute("entryTitle", getEntryTitle(entryId));
              request.getRequestDispatcher(forward).forward(request, response);
           } catch (Exception e) {
               response.sendError(404);
           }
       }         
   };
  
   //Stubbing these in...
   RequestHandler entryPutHandler =null;
   RequestHandler entryDeleteHandler = null;
   RequestHandler collectionGetHandler = null;

   //
   // One ResourceIdentifier per URI pattern. 
   // Each one tells us how to parse the pattern into attributes, and handlers
   // for HTTP methods on the resources it identifies.
   //
   ResourceIdentifier entryIdentifier = new ResourceIdentifier(       
           "^/(\\d+)$",                // URI pattern
           new String[] {"entryId"},   // match pattern and insert named request attributes
           entryGetHandler,            // GET handler for entry URIs
           null,                       // no POST handler for entries
           entryPutHandler,            // PUT an entry
           entryDeleteHandler);        // DELETE an entry
                                      
  
   ResourceIdentifier collectionIdentifier = new ResourceIdentifier(
           "^/$",                      // URI pattern
           collectionGetHandler,            // GET handler for collection would list entries
           collectionPostHandler);          // POST handler for collection will add and entry
                                       
  

   @Override
   /**
    * Here's how we tell our base class how to map URIs to handlers:
    */
   protected ResourceIdentifier[] resourceIdentifiers() {      
       return new ResourceIdentifier[]{entryIdentifier, collectionIdentifier};
   }

   // these are just stubs... exercise for the reader.
   private String buildEntryUri(String entryId) {
       return null;
   }

   private String saveEntry(HttpServletRequest request,
           HttpServletResponse response) {
       return null;
   }

   private Object getEntryTitle(String entryId) {
       return null;
   }
   private Object getEntryHTML(String entryId) {
       return null;
   }
  
}

Here's the base RestServlet class. In real life this class also has convenience methods to send redirects, and other standard HTTP stuff.



public abstract class  RestServlet extends HttpServlet {

   private static final long serialVersionUID = 1L;

   private static Logger logger = Logger
           .getLogger(RestServlet.class.getName());

   protected RestServlet() {
       super();
   }

   protected abstract ResourceIdentifier[] resourceIdentifiers();

   /** try calling doGet, doPost, or whatever, on each ResourceIdentifier, until one succeeds. 
Uses reflection to reduce bloat.
*/

   private void doMethod(String methodName, HttpServletRequest request,
           HttpServletResponse response) throws ServletException, IOException {
       boolean did = false;
       try {
           // The method must be like e.g.
           // boolean doGet(HttpServletRequest request, HttpServletResponse response)
           Method method = ResourceIdentifier.class.getMethod(methodName, new Class[]{HttpServletRequest.class, HttpServletResponse.class});
           Object [] params = new Object[]{request, response};
           for (ResourceIdentifier rid : resourceIdentifiers()) {
               if (did = (Boolean)method.invoke(rid, params)) {
                   break;
               }
           }
           if (!did) {
               response.sendError(HttpServletResponse.SC_NOT_FOUND);
           }
       } catch (Exception e) {
           logger.log(Level.SEVERE, "Exception in doPost", e);
           throw new ServletException(e);
       }

   }

   @Override
   protected void doPost(HttpServletRequest request,
           HttpServletResponse response) throws ServletException, IOException {
       doMethod ("doPost", request, response);

   }

   @Override
   protected void doGet(HttpServletRequest request,
           HttpServletResponse response) throws ServletException, IOException {
       doMethod ("doGet", request, response);
   }

   @Override
   protected void doPut(HttpServletRequest request,
           HttpServletResponse response) throws ServletException, IOException {
       doMethod ("doPut", request, response);
   }

   @Override
   protected void doDelete(HttpServletRequest request,
           HttpServletResponse response) throws ServletException, IOException {
       doMethod ("doDelete", request, response);
   }

Here's the ResourceIdentifier class. It tells us how to map URI patterns to handlers, and maps matched pattern groups to attribute names


public class ResourceIdentifier {

   private final Pattern pattern;
   private final String[] attributeNames;
   private final RequestHandler getHandler;
   private final RequestHandler putHandler;
   private final RequestHandler postHandler;
   private final RequestHandler deleteHandler;

   public ResourceIdentifier(String regex, String[] attributeNames, RequestHandler getHandler, RequestHandler postHandler,
           RequestHandler putHandler, RequestHandler deleteHandler) {
       this.pattern = Pattern.compile(regex);
       this.attributeNames = attributeNames;
       this.getHandler = getHandler;
       this.postHandler = postHandler;
       this.putHandler = putHandler;
       this.deleteHandler = deleteHandler;
   }


   public ResourceIdentifier(String regex, RequestHandler supportsGet) {
       this(regex, new String[] {}, supportsGet, null, null, null);
   }

   public ResourceIdentifier(String regex, String[] attributeNames, RequestHandler supportsGet) {
       this(regex, attributeNames, supportsGet, null, null, null);
   }

   public ResourceIdentifier(String regex, RequestHandler supportsGet, RequestHandler supportsPost) {
       this(regex, new String[] {}, supportsGet, supportsPost, null, null);
   }

   public ResourceIdentifier(String regex, String[] attributeNames, RequestHandler supportsGet, RequestHandler supportsPost) {
       this(regex, attributeNames, supportsGet, supportsPost, null, null);
   }

   public boolean doGet(HttpServletRequest request, HttpServletResponse response)
           throws Exception {

       return doMethod(this.getHandler, request.getPathInfo(), request, response);
   }

   public boolean doPost(HttpServletRequest request, HttpServletResponse response)
           throws Exception {

       return doMethod(postHandler, request.getPathInfo(), request, response);
   }

   public boolean doPut (HttpServletRequest request, HttpServletResponse response)
           throws Exception {

       return doMethod(putHandler, request.getPathInfo(), request, response);
   }

   public boolean doDelete(HttpServletRequest request, HttpServletResponse response)
           throws Exception {

       return doMethod(deleteHandler, request.getPathInfo(), request, response);
   }
 
   /**
    * Test the uri against our pattern. If matched, dispatch to the handler.
    */
   private boolean doMethod(RequestHandler handler, String uri, HttpServletRequest request, HttpServletResponse response) throws Exception {
       if (uri == null){
           uri = "";
       }
       Matcher matcher = pattern.matcher(uri);
       boolean bDid;
       if (matcher.matches()) {
           if (handler == null) {
               response.sendError(HttpServletResponse.SC_METHOD_NOT_ALLOWED);
               bDid = true;
           } else {
               dispatch(matcher, request, response, handler);
               bDid = true;
           }
       } else {
           bDid = false;
       }
       return bDid;
   }

  
   private void dispatch(Matcher matcher, HttpServletRequest request, HttpServletResponse response,
           RequestHandler listener) throws Exception {

       // The regex matched. Extract all the named attributes from the URL and
       // set them as attributes on the request. Then invoke RequestHandler.
       int n = matcher.groupCount() ;
       if (n != attributeNames.length) {
           throw new RuntimeException("must have same number of matches as attribute names");
       }
        for (int i = 0; i < n; i++) {
            request.setAttribute(attributeNames[i], matcher.group(i + 1));
        }      
        listener.invoke(request, response);
    }
}

The RequestHandler interface is the merest slip of a thing:


public interface RequestHandler {
    void invoke(HttpServletRequest request, HttpServletResponse response) throws Exception;
}

We don't need PATCH

2007-08-08T02:20:00.000-05:00

John Panzer asks:

....if PUT can be used to send just the part you want to change. This can be made to work but has some major problems that make it a poor general choice.
* A PUT to a resource generally means "replace", not "update", so it's semantically surprising.
* In theory it could break write-through caches. (This is probably equivalent to endangering unicorns.)
* It doesn't work for deleting optional fields or updating flexible lists such as Atom categories.

We could PUT an instance of a new MIME type, atom-update+xml, and the meaning of that document could be: Please selectively update fields of the entry resource, including optional fields and Atom categories.

RIA can really suck

2007-06-12T17:24:00.000-05:00

Rome Reborn is somebody's Flash wet dream. The content is spectacular (if you're into Roman history, as I am). But some lame brain has encapsulated all the content behind that one URL. I can't give you links to the interesting parts. I have to give you verbal instructions for navigating to different parts of the site. "Wave your mouse over the initial image until the Colosseum is mostly in view, then click, then...." In other words, it's like any desktop application -- which seems to be the idea behind rich internet applications -- "improve" the web until it sucks as much as Windows.

It's maddening, because the site contains tons of useful information -- but none of it is on the web. It's a black hole from which no information can escape.

If new RIA tools don't encourage authors to expose linkable resources, they're just going to continue to be irrelevant to the web.

Microsoft vs TestDriven.Net Express

2007-06-09T00:36:00.000-05:00

A sad Microsoft drone named Jason is getting famous for the worst reasons. He's a program manager for MS Visual Studio and he's hassling a well meaning fellow named Jamie who wrote a unit testing plugin for VS 2005.

Jamie wrote the plugin using VS 2005 Express, the "free" version, and got enough traction that MS made him an MVP. But then Jason noticed that the plugin supports VS Express. Bad Jamie -- no plugins allowed for Express.

Jamie has published the fatuous correspondence Jason directed his way. They took away his MVP, and MS lawyers are sending cease and desist letters.

Um, the guy is developing free software enhancing the value of all the VS products? Did I mention that part?

Great coverage from The Register.

Hyperbolism of the Month

2007-06-08T11:36:00.000-05:00

...goes to (the envelope please)... Elliotte Rusty Harold!:

Java’s exception handling is the single best error handling and reporting mechanism ever built into a programming language.

I personally don't think you're going to beat the Common Lisp condition system. You can emulate Java-style exception handling, but it's so much richer to be able to restart in the context where the error occurred, without unwinding the stack.

What is Lisa Simpson Doing to that Poor Man?

2007-06-06T20:05:00.001-05:00

Once someone has pointed out that interpretation of the London 2012 Olympics logo, it's forever ruined, isn't it? Sorry. (via DPF).

Fifty-four forty or fight!

2007-05-21T17:08:00.000-05:00

Today I flunked this quiz question asked of registrants for a Dell sweepstakes:

Where in the U.S. did Dell recently build a Customer Contact Center?

Roseburg, Oregon
Oklahoma City, Oklahoma
Edmonton, Alberta
All of these

Yep -- I missed the correct answer: "All of these." (When did we get Alberta?)

And I had never heard of Doug Engelbart!

2007-05-07T19:50:00.000-05:00

Since Reginald B. and Alan Kay have urged us to google Doug Engelbart, I reckoned it must be important, so I did. And I found this amazing demo from 1968 of, basically, my computer.

RIA Not Advancing the Ball

2007-05-04T13:03:00.000-05:00

Rich widgets obscure the semantics of hypertext. Only the code behind the form knows what the widget really does. Contrast to HTML 5, and in particular Web Forms 2. These guys are extending HTML to capture what we really do on the web. As a consequence, client programs can (or, have a chance to) understand the meanings of hypertext documents from the web.

Example: you want to write a script to automate some remote bookmark service, as part of your mashup. But this service, unlike del.icio.us, has no documented "API". So you have to download its form, complete it programmatically, and POST an entity.

Case 1: The form uses Plain Old HTML. You're golden. All the semantics are right there for you to parse, or read. You identify the name of the text box where you stick the URL, and the name of the text box where you add a description. You compose the URL encoded form data, and POST it to the action URI.

Case 2: The "form" uses Javascript to modify the DOM on the fly: the onload() method adds text boxes, and a submit button, to an empty DOM. In fact, it might not even use the submit button as a form element; when you press the button, its onclick() might send a custom XMLHttpRequest back to the server. Your code will never automate this interaction.

Case 3: The "form" uses XAML + Silverlight plugin. An exacerbated case of (2).

In contrast, Web Forms 2 attempts to capture the semantics of what we do with forms. Because browsers will understand more of the semantics of the form, we can do declaratively what we now have to do in Javascript. For example, lots of HTML forms now have to use script to add a row of controls to a form on the fly ("Click here to add another bookmark"). WF2 captures that as repeating control groups, and the browser can handle it.

(Then again, I am a documented forms nut.)

Mike Dierken justly analogizes: "RIA is to user interfaces as RPC is to messaging interfaces". And notice, it's Rich Internet, not Rich Web Applications. These technologies pay lip service to the web, but they're not advancing the ball toward building more and better links throughout the web information space.

P.S. Wonderful rant by Mark Pilgrim!

Update: fixed a link.

RIA -- Fill 'er Up!

2007-05-03T18:35:00.000-05:00

I'm having an ongoing email exchange with my friend Peter. He's convinced MS and Adobe herald a new age of Rich Internet Applications. He pointed me to this guy who's backed up a tanker to the Kool-Aid trough.

Sure, MS and Adobe have to sell something as the Next Thing -- what else have they got? But we've had RIA ever since Java 1.1 applets. We have Flash. We have <embed> and <object>. Do you really think what's been holding RIA back is the technology?

Users have voted with their mice, and they've voted for the web experience -- exploring the web information space using hyperlinks -- as far more important than whizzy UI. Ask eBay. Ask MySpace.

Flash, applets, Silverlight, Javascript -- the more you use them, the suckier your web apps are at exploring the web information space. I don't think it has to be this way, but it takes a design discipline few seem to have. These programming models are from the 80s. They have web APIs, but they're not web oriented. Programs end up as little desktop applications, not web apps. I don't see Silverlight changing that. It is good to have super expressive widgets -- hear hear. But if you're not pushing a bunch of hypertext down to my browser, you're not helping me explore the space.

The Penny Drops

2007-04-27T12:20:00.000-05:00

It's enjoyable, and instructive, to watch the penny drop for venerable DCOMster/SOAPster Tim Ewald: I finally get REST. Wow.

Instructive, because coming from a strong RPC perspective, Tim illuminates the distributed application problem with slightly different shades. I like this graphical model:

Every communication protocol has a state machine. For some protocols they are very simple, for others they are more complex. When you implement a protocol via RPC, you build methods that modify the state of the communication. That state is maintained as a black box at the endpoint. Because the protocol state is hidden, it is easy to get things wrong. For instance, you might call Process before calling Init....The essence of REST is to make the states of the protocol explicit and addressible by URIs. The current state of the protocol state machine is represented by the URI you just operated on and the state representation you retrieved. You change state by operating on the URI of the state you're moving to, making that your new state. A state's representation includes the links (arcs in the graph) to the other states that you can move to from the current state.

Austin to Paris in 30 days

2007-04-17T17:38:00.000-05:00

Since I'm traveling to Paris soon, thought I'd get directions from Google Maps. It's going to take 30 days, 9 hours. Notice item 28. (via Peter Flanagan).

Twenty miles with the Tour de France Champ

2007-04-16T00:32:00.000-05:00

Today I rode 20 miles with 2006 Tour de France Champion Floyd Landis. And I don't care who knows it.

Microsoft is Dead

2007-04-10T16:52:00.000-05:00

Ha! I've been telling people Microsoft has become irrelevant. And now Paul Graham crystallizes the thought. Especially rich:

...I'm now surprised when I come across a computer running Windows. Nearly all the people we fund at Y Combinator use Apple laptops. It was the same in the audience at startup school. All the computer people use Macs or Linux now. Windows is for grandmas, like Macs used to be in the 90s. So not only does the desktop no longer matter, no one who cares about computers uses Microsoft's anyway.

An irascible colleague at a large software company used to say, "Hugh, you have to understand: XYZ isn't really a software company. It's an old folks home for software." XYZ had the same problem PG describes:

Microsoft's biggest weakness is that they still don't realize how much they suck. They still think they can write software in house. Maybe they can, by the standards of the desktop world. But that world ended a few years ago.

The cure is worse than the disease

2007-04-03T13:17:00.000-05:00

This paper from Fortify makes the case that sending sensitive information using JSON exposes it to cross-site maliciousness. GMail sent your contact list down as JSON and evaled it. Turns out, any old site could do the same: just put a <script> tag referencing that contact list, and install some interceptor code that overloads setting e.g. the "email" property on any object: That enables the malicious code to see the values in the JSON.

Here are a couple of their proposed measures:

1. "Add the session cookie to the request as a parameter." Knee-slapper, that. See, the exploit only works because vulnerable sites put your identity into the cookie, and use a single URL for all users to download the object; the server uses the cookie to send you your personalized contact list. So the attacker just has to hardcode <script src="http://yoursite.com/contact-list">. The paper proposes uniquifying the URL. Here's an idea: design your app so that each user's info is at a unique URL in the first place!

2. Send all legitimate requests for JSON data using HTTP POST! That way you know any GET requests are malicious ones from <script> tags. They do concede that "The use of GET for better performance is encouraged by Web application experts from Sun and elsewhere". There's no use for this measure if you use unique URLS, of course.

So yeah, this is a serious problem, but not for apps using best web architecture practices. Millions of web developers read papers like that and then crap all over the web.

Life imitates art

2007-03-31T22:07:00.000-05:00

During the Atom Publishing Protocol process, posting your cat pictures was a recurring use case. Now there's a whole site devoted to social cat picture publishing. John Panzer, is that you?

Update: I guess it's not John -- they only expose an RSS 2.0 feed.

Glitch Undercuts the Dow

2007-02-27T19:30:00.000-06:00

The Dow fell over 400 points today. And some IT guy's ass is fired. From Stocks Have Worst Day Since 9/11 Attacks:

The Dow's decline accelerated at a faster than normal pace during the afternoon after a computer glitch kept some trades from being immediately reflected in the index of 30 blue chip stocks. Dow Jones & Co., the media company which manages the flagship index, said the problem occurred after it was discovered computers were not properly calculating trades, prompting a switch to a backup computer.

The result was a massive plunge in the average in the seconds it took Dow Jones to switch to its secondary computers.

SOA Facts

2007-02-14T01:42:00.000-06:00

I came across this hilarious list of way more than ten SOA Facts, including

SOA is an anagram for OSA, which means female bear in spanish. It is a well-known fact in the spanish-speaking world that female bears are able to model business processes and optimize reusable IT assets better than any other hibernating animal

SOA actually stands for SOA Oriented Architecture

SOA is also a yoga posture that consists of performing all other yoga postures simultaneously

Submit your own fact. (This site really really needs a feed).

leet.google.com

2007-02-09T12:35:00.000-06:00

Google s34rCh(via Phil Windley)

Forms Language Use Case

2007-02-06T08:18:00.000-06:00

Tim Bray on the Atompub list:

If I fetch a service doc with a collection with no <app:categories>, does that mean the server is suggesting that I can post any category I want, or that I can't post any category at all?

This question would a non-issue if APP used a forms language. A form for submitting an entry would have a) an enumerated list of choices, b) a free form text field, or c) no field at all to submit the category. No ambuiguity.

Web 0.9

2007-02-05T01:44:00.000-06:00

Sure took a long time to pay the car note this evening. I wish the developers had read Mark's caching tutorial (or rather that their J2EE framework developers had). Below are headers representative of about a hundred .gif, .css, and .js resources used on the page:


GET /navigation/images/global/company.gif HTTP/1.1
Host: www.financecompany.example.com

...

HTTP/1.x 200 OK
Server: IBM_HTTP_Server/2
Last-Modified: Tue, 20 Sep 2005 18:24:36 GMT
Etag: "31e467-450-2c11c100"
So far, so good
Accept-Ranges: bytes
Content-Length: 1104
Content-Type: image/gif
Expires: Mon, 05 Feb 2007 07:31:41 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Mon, 05 Feb 2007 07:31:41 GMT
Huh? It's a GIF that hasn't changed in a year.
Connection: keep-alive

Floyd Fairness Fund

2007-01-30T22:52:00.000-06:00

During the final week of the Tour de France, I published my initial doubts about the "science" accusing Floyd Landis of doping. All evidence since then reinforces my doubts.

Ultimately it all comes back to character: Do you believe the man or not? The evidence of doping that they have, alone, isn't convincing, but it would be consistent with doping. Lots of guys have gone to the mat lying. Floyd could just be one more.

I just don't think he is.

And now I know it. Because I don't think it's possible anyone, even Richard Virenque, would form a legal defense fund taking money from elderly cycling enthusiasts like me just to reclaim a ride in the pro peloton. If Floyd had doped and had created this fund... well, nobody is that low. If you doubt his word now, you are saying, Yes Hugh: He is THAT LOW.

I gave. I think it's important. It's really important that the innocent get off, if we're going to have faith in the convictions of the guilty.

Wait, my new publishing technique is unstoppable

2007-01-29T08:35:00.000-06:00

[updated to correct an omission in POST description]

Bill outlines the conventional Atompub publishing pattern. I'm pitching this one:

GET Introspection URI: scan the list of workspaces for the collection you want to post the blogpost to. This list is not an Atom service document, but a microformat outline e.g. XOXO. (The Atompub WG considered and dropped using XOXO).
GET to Collection URI: read the nice atom feed
GET the "New Entry" URI from either the service doc or the feed doc.
: Retrieve an XHTML microform describing how to construct a POST message
POST to the blogpost's collection URIthe action URI of the form.: push a blogpost formatted as a nice atom entryas URL-encoded form data.
GET or HEAD to blogpost URI: grab the blogpost
PUT or DELETE to blogpost URI: change or delete the blogpost

Web Service of the Future: Comment Spam

2007-01-24T12:18:00.000-06:00

Guess what web technology, right now, most successfully uses microforms a.k.a. forms-driven web services: Comment spam. Spambots crawl the web, looking for form fields they recognize, completing them, and submitting them. There's no one universal blog comment document format. Each site tells the bot, by using typical names in form fields, how to complete the form.

Imagine you had some service you actually want agents to be able to understand: a fare search site. Farebots would query your site for fares from Austin to Bangalore leaving Thursday. They'd know how to fill out the form field because you'd annotate each one with some agreed class attribute. Doesn't this sound a lot more realistic than getting everyone to agree on a common fare search document?

Forms-driven Atom

2007-01-18T11:38:00.000-06:00

Here's how forms-driven Atom publishing would work. Let's drop the Xforms idea, and take a page from the smart microformats people: We'll overload XHTML forms with semantics, so that blogging clients can parse them, fill them in correctly, and then POST or PUT them. Call the idea "microforms".

Want an example? Your browser, right now, can auto fill lots of form fields based on the field name. And your browser retrieves that information from its own little database. Let's just generalize that idea, and furnish client programs with the ability to understand lots more about the service it's exercising.

Where do we get the semantics for Atom? The Atom syntax document goes a long way. Henry Story has constructed an OWL ontology for Atom. This is a good starting point. We can use those property and class names in XHTML class attributes.

Remember the motivation: We want web services to spread as powerfully as the real WWW has. The Atom group believes RESTful web services will do it. But little in Atom, or any other document driven protocol, resembles the actual web we experience. In Atom, they agree on a document format that carries semantics that all servers and clients have to understand, or at least handle gracefully. That concept has no analogy on the web. When you order airline tickets, do you complete the One Universal Reservation form? No. Each airline or agency has its own form, because each business has its own differentiators.

It's the same in web publishing. Consider Moveable Type and Blogger. To create an entry for MT you fill out this form:

They have two places to enter content! One is the lead, the other is the "Read more...." section. There's no concept like that in Atom. I guess the MT guys will propose an extension. But then every other blogging service on the planet will have to handle the case where somebody submits an "mt:extendedContent" extension element. Blogger doesn't have that concept:

I marked up those screen captures with tags grabbed loosely from the Atom ontology. We can make up more. Here's a microform MT could present (modulo GUI labels and layout):


<form method="post" action="/feed/">
  <input type="text" name="title" class="atom:title"/>
  <select name="category" class="atom:label">
    <option value="Programming"/>
    <option value="Politics"/>
    <option value="Personal"/>
  </select>

  <textarea name="entry" class="atom:htmlContent"/>
  <textarea name="extended" class="atom:htmlContent"/>
  <input type="checkbox" name="draft" class="app:control" value="app:draft"/>
</form>

Later we can hash out the details of these class attributes. Maybe we do need an MT specific extension for that extended content. But only MT servers, or any service supporting extended content, ever will have to deal with clients that submit that field.

So the microforms idea is: You describe the semantics of the form elements in their class attributes. You program clients to GET forms, understand form elements, and fill them out correctly, getting the information from some kind of database, or from a live user.

If we do it well enough, we can build agents that syndicate stuff to all kinds of servers automatically.

(updated: fixed omitted class attribute in form example).

We're Writing Hardware

2007-01-17T12:16:00.000-06:00

Using statically typed languages, we're constructing hardware, not software, says Steve. A great rant. Let me extend the analogy. Think about the bullet points they used to sell object oriented programming back in the 80s: "Hardened components", "Reusable". These qualities are the opposite of dynamic.

Wait! There's an alternative!

Common Lisp is in many ways really ideal: it's a dynamically typed language with optional type annotations (i.e. you build software, then selectively turn it into hardware), lots of great tools and documentation, all of the essential features of living software I've enumerated in this essay... However, it has stopped growing, and programmers can sense momentum like a shark senses blood.

Oh man. Don't give up on the old girl. The Lisp Renaissance is happening all around you. We have the popular book; now we just need the Django/Rails thing.

The Volkswagen Lisp

2007-01-11T23:02:00.000-06:00

I've been working in Lisp for about six weeks now, and I am far enough along to
get this.

Quote of the day 01/08/2007

2007-01-08T22:34:00.000-06:00

"I think smell certainly can be a terrorist mechanism, whether it harms people or not."

Is Atompub Superfluous?

2006-12-27T15:52:00.000-06:00

I hate to suggest this at such a late stage -- Bill and Joe just published the twelfth rev of Atompub -- but do we really need Atompub?

All we need is RFC 4287, plus maybe the service description doc in Atompub.

Atompub tells you how to identify URLs where your application can POST documents of certain types. That's not a whole lot better than some so-called "RESTful" API that tells you how to construct a URL. At design time, I'm learning more about your application than I need to know.

My proposal: Once you've located the URL of a what they call an "edit" link, you should be able to GET a form from that link, describing what you can POST there. Since we already have well defined semantics for the XML elements of Atom entries, an edit URL could return an XForm model:


<model>
  <entry xmlns="http://www.w3.org/2005/Atom">
     <link/>
     <updated/>
     <summary/>
     <content type=""/>
  </entry>
  <submission id="form1"
                 action="entries"
                 method="post"/>
</model>

1. This pattern --- getting a form, filling it out, and submitting it -- differentiates the web, REST style from RPC. In RPC, the programmer learns an API at design time; on the web, clients discover the "API" at run time.

2. We sidestep lots of arguments on the APP mailing list that are driven by different ideas about expected behavior; here, the server tells you its behavior. This server could not honor an atom:id should you submit one. So, it doesn't ask you for one.

3. Extensions understood by the server? Don't worry: if the server doesn't ask you for it, it won't honor it. Example: Atom Threading Extension. A very nice server could offer a catchall element (to be invented) where an APP client might stuff all the custom extensions it really wants preserved.

4. We can evolve the Atom syntax without argument. Old servers will never request newly defined elements so will never have to deal with them. I'm guessing this is where Mark is going with reference to semweb.

MVC Considered Harmful

2006-12-23T11:12:00.000-06:00

Struts 2, the popular Java webapp framework, like other frameworks, advertises a Model-View-Controller architecture. MVC seems to be an item on a checklist that frameworks think they have to have. Now, MVC is a great architecture for building desktop applications, with a user interface posting messages through a controller, to query or update a model. But the skeptic would ask, Does the web really feel like a desktop application?

What is wrong with this picture?

Notice the central concept of a Struts application is the Action. Building a Struts app is constructing a bunch of Action classes. The Struts controller accepts requests and maps the URL to an Action. Ahem. The web is not built upon Uniform Action Locators, is it? An Action implies a verb, and that is the way developers model applications under Struts and other MVC webapp frameworks. In fact, Struts typically identifies Action URLs by the the extension ".do" -- the verb "to do".

Struts misses the concept of Resource. You can't really model a web application without conceiving it as a collection of resources, identified by URI. I think Django is on to this. In Django, you furnish the framework with a map of regular expressions to callback functions. The Django docs call these callback functions your "views", but of course, if done correctly, they are really your resources. (Interestingly, the Django FAQ addresses the question Is Django an MVC framework and has evolved the answer, from this to this. They can't bring themselves to acknowledge a complete break with some form of MVC).

A well architected webapp will organize around Resources and Representations. There are typically four methods on a Resource. Omitting some details, you have

Representation Resource.get()
Representation Resource.post (Representation)
Representation Resource.put (Representation)
void Resource.delete()

To handle a request, you

1. Identify the resource given its URI, and instantiate it
2. Call the method, passing the request representation if any.
3. Respond with the returned representation.

Resources are probably not much like your domain model. You'll have to map Representations to Resource instances, and Resource instances to, say, database queries. A good architecture abstracts out dispatching to methods based on types of Resources and Representations, and affords a loose coupling between your domain model and the Resource layer. These are sound principles that also will help your webapp play well with REST web architecture. But all dispatch and loose coupling is not MVC.

Google Boomerang

2006-12-20T10:03:00.000-06:00

Blogger, the Google-owned engine behind this blog, just went final with a major new rev. But it continues to generate massively invalid XHTML, and there's nothing authors can do -- it's bugs in the new templating system. This page you're reading, for example, gets 864 validation errors as of today.

Arne Bleijs made this pithy observation:

I suddenly remember some quotes from the annual conference of the
American Association for Artificial Intelligence (AAAI), this summer in Boston. After Sir Tim Berners-Lee held a keynote speech about the Semantic Web. Peter Norvig, Director of Research at Google challenged him afterwards:
"What I get a lot is: 'Why are you against the Semantic Web?' I am not against the Semantic Web. But from Google's point of view, there are a few things you need to overcome, incompetence being the first," Norvig said. Norvig clarified that it was not Berners-Lee or his group that he was referring to as incompetent, but the general user."
"We deal with millions of Web masters who can't configure a server, can't write HTML. It's hard for them to go to the next step. "
Somehow this sounds like a Google Boomerang hitting Blogger Beta. How can it be possible that Google complains about incompetent webmasters and is not able to produce valid XHTML?

Reverse Spam

2006-12-20T08:51:00.000-06:00

This isn't a new phenomenon but it's happening ten times more frequently to me this week: I am receiving bounced mail notifications from mail I never sent. I funnel all mail sent to my domain, no matter the recipient, to my main account.

Spammers send mail all over the planet using random domain names and generated account names. As spam filters get better, they are bouncing these messages. But they are bouncing them back to me. So I'm getting other people's spam!

gbiv.com

2006-12-18T23:26:00.000-06:00

Roy Fielding authored RFC 2616, the HTTP spec. I've seen his personal URL for years: http://roy.gbiv.com/. What is this gbiv.com?

My wife recently left her job of 11 years assembling books for a book production company. She's acquired quite a bit of esoteric knowledge about printing. Tonight she pronounced, out of nowhere: "Roy Gee Biv". She does this sometimes. Her neurons have been reordered so that occasionally she blurts out mnemonic devices she used in that past life.

This mnemonic device helps you remember the colors of the rainbow: ROYGBIV

red
orange
yellow
green
blue
indigo
violet

Message Level Security in HTTP: How hard can it be?

2006-12-13T11:29:00.000-06:00

A recent flurry of interest in message level security for Plain Old HTTP: Gunnar makes valid criticisms (though he confounds REST with HTTP); and Robert takes the point. And the issue pops up again on www-tag.

The criticism is this: Using TLS, each intermediary decrypts the message, and re-encrypts it to send along to the next leg of its journey. The message is vulnerable to prying eyes at those transition points.

How hard can it be to graft MLS onto HTTP without the baggage of WS-Security and XML Encrypt, and XML DSig? Somebody must have done this long ago, right? Couldn't find it with a quick google, but it has to look like this:

Define a content transfer encoding for PK, and pass the cert bits, or a URL:

Content-transfer-encoding: RSA; cert="...base64 bits here..."
or
Content-transfer-encoding: RSA; certref="http://foo.bar/cert"

You'd also use these values in "Accept-Encoding" headers.

And we're done, as far as HTTP is concerned.

Applications are in charge of trust. Maybe HTML5 could enhance the
<form> element so you could have:

<form method="POST" action="action" certref="http://mybank.com/cert">

and user agents would then encode the entity using PK encryption, and deliver the cert bits in the header as above. As with TLS, the UAs could generate temporary one-off local certs for this purpose.

To GET a secure page, you'd have to have sent the proper Accept-Encoding with your cert as a parameter. If you failed to send this header you'd just get 406, and you'd have to restry with your cert.

This has all been done before somewhere, right?

Lisp Is Too An Acceptable Lisp (with caveats)

2006-12-07T10:04:00.000-06:00

To get my hands dirty with Lisp, I've been putting together a RESTful HTTP framework. It runs with Apache 2.0 and mod_lisp. Over the past year, I've been researching Lisp: absorbing Practical Common Lisp, reading Paul Graham and Bill Clementson and lemonodor and others. There's a consensus out there that Lisp rules, but there's a permathread lamenting the balkanized libraries many think prevent widespread uptake of Lisp. As Steve Yegge puts it in Lisp is not an Acceptable Lisp:

Every single non-standard extension, everything not in the spec, is "wrong" with Common Lisp. This includes any support for threads, filesystem access, processes and IPC, operating system interoperability, a GUI, Unicode, and the long list of other features missing from the latest hyperspec.

Point taken, to some degree. But that problem doesn't directly affect you if you're an architect building an application or web service -- you just pick Allegro or LispWorks or CMUCL or SBCL or CLISP, and you're done. Each Lisp has its own APIs for those services. It's true your code won't port trivially from one platform to another. That's usually not much of a concern, if the platform you choose runs on all the hardware and OSs you need.

But the balkanization indirectly affects you. It impedes the development and distribution of free libraries. If you're like me, you've become addicted to free libraries. A jabber component, or a message queue, or task scheduler, is just a Google away, if you're developing in Java.

To build a cross platform library, a Lisp developer has a fair amount of work to do. At a minimum, if you use any extensions, like sockets or Gray streams, you have to find or write an abstraction that may do little more than hide the package name of your implementation. You may not be able to abstract away some issues -- does each sockets implementation know how to send out of band TCP (MSG_OOB)? Which methods do the underlying Gray streams really require you to override?

Common Lisp has been frozen since 1994, and as best I can tell, there is no process by which Gray streams, sockets, or IPC, or anything could be added to the language now. Contrast to Java Community Process, which for whatever warts[1], has enabled the language to evolve uniformly across implementations. You never have to worry that your Java 5 code that you developed using Sun's JDK won't run on Jrocket or IBM. It just works.

Lisp needs some JSR-like, ongoing process -- not another eight year, big bang ANSI revision -- to standardize changes we take for granted in all other environments. And a benevolent dictator would help.

[1] How ironic that Peter Seibel contrasts the long JSR process against the capability Lisp gives you to extend the language with macros.

Edgy + Beryl = Love

2006-11-06T11:36:00.000-06:00

I spent the weekend upgrading my laptop from Ubuntu Dapper to Edgy. It was painful, relative to the usual seamless Ubuntu intra-release updating process. Painful, but surmountable. To reward myself after the pain, I installed the Beryl desktop. I LOVE BERYL! It renews my love affair with my computer! If you're unfamiliar with Beryl/Compiz, check out some of these videos... but they don't do the user experience justice, trust me. It's worth the effort to try out Beryl.

The rest of this posting is really for Googlers trying to solve upgrade issues from Dapper to Edgy, and also a little Beryl guidance. Welcome, visitors!

After I upgraded using apt-get distupgrade, my system failed to boot into the new kernel. If you are booting and it seems to hang, wait three and four minutes, and you may see something like this:

ALERT! /dev/disk/by-uuid/ed4395c8-e15c-4b20-8716-76ceff89614e does not exist. Dropping to a shell!

BusyBos v1.1.3 (DEbian 1:1.1.3-2ubuntu3) Built in shell (ash)
Enter 'help' for a list of built in commands.

Don't panic. Since you're upgrading, you still have your old kernel you can boot into. Boot it, change to your /boot directory, and do this:

dpkg-reconfigure linux-image-2.6.17-10-generic

(use the same name suffix, e.g. "2.6.17-10-generic", as the new kernel you upgraded to). This command builds you a new initrd image file, and you will be able to boot.

My second issue was particular to my laptop, a Dell Precision M70 with an Nvidia card. You really need your Nvidia card to work if you're going to use Beryl. Something in my default upgrade linked to the latest Nvidia driver (1.0-8776), but that driver did not come in the upgrade; X failed to start. Thanks to Alberto Milone for these excellent notes on upgrading the driver without disabling your wireless card.

I had a fair amount of trouble getting Compiz/Beryl to work. I had tried Compiz on Dapper, and never got a working install. After upgrading to Edgy and getting it stable, I investigated Beryl, and had similar trouble: Lots of garbage bits on the screen, windows with no titlebars or borders -- unusable and depressing.

Finally I fixed all my problems by twiddling my xorg.conf. Here are the relevant twiddles (ymmv):

Section "Device"
Identifier "NVIDIA Corporation NV41 [Quadro FX Go1400]"
Driver "nvidia"
BusID "PCI:1:0:0"
Option "RenderAccel" "true"
Option "AllowGLXWithComposite" "true"
Option "Triplebuffer" "true"
Option "AddARGBGLXVisuals" "True"
EndSection
...
Section "Screen"
Identifier "Default Screen"
Device "NVIDIA Corporation NV41 [Quadro FX Go1400]"
Monitor "Generic Monitor"
# depth had been 16:
DefaultDepth 24

...

Tim Bray on Ruby and Python vs Lisp

2006-08-27T22:36:00.000-05:00

Tim Bray: "If Lisp’s audience had been harried sysadmins rather than AI researchers, it’d rule the world by now."

The Ruby Conspiracy

2006-08-07T20:22:00.000-05:00

The Ruby Conspiracy: "Who are those who are benefiting from Ruby on Rails? Answer: O'Reilly Publishing, the authors Bruce Tate and Dave Thomas and a handful of consultants....We have two production applications running on Ruby. And how is it. Well, despite being perhaps no more than 5% of the functionality of our applications, Ruby on Rails is the number one consumer of Oracle CPU and logical gets....After all, the productivity benefits of Ruby are so much greater than Java you will save all of the money in development. Or do you. Our experience was that Ruby on Rails took longer than Java would have. And what about maintenance. Well we just refactor as things change. Or do we? There are no Ruby tools that support refactoring. And nor are they are expected due to the difficulties of implementing refactoring tools for Dynamic Languages, or so I am advised."

Interpreted... tons of magic^H^H^H^H^H abstraction. Right.

Voidstar - Desktop clients

2006-08-03T10:34:00.000-05:00

Julian Bond references Martin Geddes on desktop clients -- the browser, the messenger, the "manager":

All the portals are focussed on collecting everything you might read in one place. The "My Page". Nobody focusses on the reverse, collecting everything I create in one place *for other people* to read about me.

Analyzing along this line is getting closer to what I had in mind.

YouOS

2006-08-02T16:14:00.000-05:00

My first reaction to YouOS: Cool. A webtop that looks the same from wherever I log in.

Second reaction: Hey, we had this in the 80's! Log in to any Sun on the LAN and your NFS shares are all in the same structure, your desktop just as you left it.

Third reaction: Did we invent the web only to repro the 1984 Mac desktop yet again?

Fourth reaction: There isn't hyperlink number one on this desktop. Shouldn't I be able to get the hyperlink of an object and send it to someone so they can see it or get it?

I'm not sure what I'm looking for, but it needs to be more webby. It needs to enable something I can't do now with a desktop. There's some new metaphor we're groping for. I'll know it when I see it.

Understand the measurements

2006-07-30T19:14:00.000-05:00

I'm sort of a cycling nut. The events of the last week, and the last couple of years, inspired me to put down these critical thoughts on anti-doping hysteria:

Bayesian Analysis for Dummies
My training is in geophysics. I have no expertise in biology. But as a geophysicist, I have worked with measurements a lot, and I know how to assess them.

Here's a simple example. Suppose a dangerous disease affects one of every 100,000 people. A lab develops a test that is always positive if you have the disease, but that gives a false positive in one percent of cases. Your test returned positive... do you have the disease?

Well, you might. But chances are, you don't. In fact, your chances of having the disease are about one in a thousand. The “prior probability” of 100,000 to 1 dominates the test result. The test would have to show far fewer false positives to be a useful tool in diagnosing the disease.

Recent accusations against cyclists – notably Floyd Landis, Lance Armstrong, and Tyler Hamilton, have been based on biological measurements. The measurements are valuable and largely trustworthy. But the meanings of all measurements need to be assessed in light of errors and uncertainties surrounding them. A newspaper publishes an article that Landis's testosterone to epitestosterone ratio (T/E) exceed the allowed limits of 4. Let's examine the measurements.

First, go back to my earlier example. Substitute “testosterone abuse” for “deadly disease”, and assume one out of every ten cyclists is an abuser. Is Landis guilty? Probably... but not certainly: His probability of guilt would be less than 92%. So eight times out of a hundred we would be wrong to take away his victory.

But wait: the T/E test doesn't always give a positive result for abusers. Lots of abusers can pass that test. If we think 50% of abusers can scrape past, does that affect Landis's odds? Yep – now he's only 85% likely to have abused. Still want to apply a two year exile from the sport?

Here's the article that details that argument: Inferences about Testosterone Abuse among Athletes. They make this point: “Conclusions about the likelihood of testosterone doping require consideration of three components: specificity and sensitivity of the testing procedure, and the prior probability of use. As regards the T/E ratio, anti-doping officials consider only specificity. The result is a ﬂawed process of inference.” In other words, the WADA procedures assume the test catches all abusers, and don't account for the known prevalence of abuse, so they're wrong.

Landis's Eleven
Now how about those lab results, anyway? So far, we've just accepted the lab's numbers as golden. I have heard a ratio of 11:1 for Landis. But all measurements are uncertain. How certain is that 11:1? We want error bars around that number, 11. Is there some non-zero chance the ratio is 10? 15? Even 4? If you ask WADA, it is just: Eleven. (As if Dick Pound would understand the question, or even hear you out.)

How tall are you? Can you tell me to the 32nd of an inch, or to the millimeter? How about to the nanometer? At some level of granularity, you just can no longer resolve a difference in distance. And how about that yard stick you used to measure? Pretty sure it's accurate to a millimeter? So instrument resolution is one source of uncertainty.

Uncertainty is OK! We just have to know how large the potential errors are.

In the case of the T/E ratio test, there are a lot of systems involved. Gas chromatography is well understood, and there are uncertainty estimates available for the systems they use at LNDD (Laboratoire National de Dépistage du Dopage, the Lab testing Landis's samples). The process is temperature sensitive, so we'd really want to know the uncertainty bounds on the actual temperature program they used. The instrument documentation might give us some idea how to translate temperature variations into variations at the mass spectrometer output. There might also be some pressure control program pushing material through the column; how accurately do we understand the effect of uncertainty in the pressure? The mass spectrometer itself, only a subsystem of the whole, has its own uncertainty analysis.

Below is an example of the mass spectrometer output for a similar experiment, taken from a recent paper on screening for steroids. If these peaks were epitestosterone and testosterone, this would be a picture similar to the analysis of Landis's sample.

I think they get the E/T ratio by calculating the area under each of the two peaks, and dividing one by the other. So first of all, any uncertainty in the temperature and pressure would affect these areas. Secondly, the process has to separate the peaks far enough apart so that the two “hills” don't bleed into one another. Thirdly, somebody has to decide where the hills “start and stop”. See that little bump at 12.30 above? Is it part of hill 2 or not? Judgment call.

So if we know how variations in the temperature and pressure affects the shape of the picture above, and if we can estimate how the uncertainty in the temperature and pressure during this test on Landis's sample, then we'd have some decent error bars on Landis's “Eleven”.

Once you have error bounds on Landis's T/E ratio, you revisit the Bayesian analysis. Any appreciable uncertainty will decrease the likelihood that Landis abused testosterone.

The analysis of Landis's sample won't stop with the T/E ratio test, of course. The next step evidently may be an IRMS analysis for the ratios of two carbon isotopes. As the news emerges, you should ask how the test works, and what are the uncertainties.

Armstrong and Hamilton
And so with the charges against Armstrong made by l'Equipe last year, and with those against Tyler Hamilton in 2004. In Armstrong's case, the uncertainty begins with the chain of custody. Then you have an experimental test, applied to six year old samples. In the Hamilton case, the anti-doping agencies placed full faith and credit in a new test – only published months earlier -- with documented repeatability problems. The actual procedures in executing these tests are more complex and sensitive than the procedures for T/E. Yet, all we get from the lab is, “positive”. These positives are the results of judgments, made in good faith presumably, but not subject to review by the athletes or the public (see e.g. the Vrijman report).

My point isn't that these men are innocent. My point is that the probability of their guilt is far less than the public assumes from news reports. We can pardon the public, and even the press, for putting more faith in the numbers than they deserve. But the bureaucrats at the anti-doping agencies irresponsibly accuse athletes. They pay lip service to “science” but know nothing of it. They make great displays to be the most earnest of witch burners, lest they endanger their jobs for lack of vigor pursuing dopers. Cycling pays the price.

Additional Reading
Can we handle the truth? We are our own worst enemy -- Knowledgeably explores the madness and injustice of the anti-doping regimes, from someone inside the sport.

Deep Quote

2006-07-29T18:13:00.000-05:00

I've wanted a service like this for a long time!

We'll never get O/R mapping right

2006-07-25T19:06:00.000-05:00

Ted Neward calls O/R mapping a "quagmire" in The Vietnam of Computer Science. He's right (although the Vietnam analogy is a distraction).

He considers approaches to licking the problem, and he almost gets it right with "Integration of relational concepts into the languages". Yeah -- but he limits his consideration to tweaks to mainstream languages (dismissing "fringe" languages like Ruby... not that Ruby's doing it right).

We're not going to get this O/R mapping thing right... ever. It's the "O" in "O/R" that's the problem. We need languages that think in terms of tables or relations. The object languages have us whirling in a Sapir-Whorf spiral. Relations -- tables -- are the language of data. Objects attempt to deal with data but in an ad-hoc way, with no grounding theory that enforces integrity and consistency; you enforce all that in procedural code. Objects may be a handy idiom for writing simulation programs. But the 99% of other programs out there that traffic in data, even transiently, need to be written ground up in a relation oriented language.

Relation Oriented Programming: I've written about it before. I believe the right answer is going to be to construct a relational idiom in the fringe language Common Lisp. I've already begun! Expect results about the time Arc is ready.

But is it enterprisey class?

2006-06-18T16:23:00.000-05:00

Via Alex Russel, who calls attention to Sun's Project Phobos:

The goal of Project Phobos is to show that Java is an excellent platform for server-side scripting, allowing dynamic-language developers to leverage the power of Java SE and EE. The initial focus for Project Phobos is JavaScript, but the design supports the use of other dynamic languages as well....Project Phobos attempts to learn from Rails, but is not limited to the use of any particular programming language and may prove to have a different sweet spot

I checked... the word "enterprise" does appear on that page, but only in the breadcrumbs (" Projects > java-enterprise > enterprise-incubator > phobos).

Rich JavaScript Apps

2006-06-18T01:33:00.000-05:00

Two developments are really going to accelerate rich Javascript web apps -- they sure have around here...

1. Dojo is a framework for building js + html + css widgets. A widget ends up looking like this in your html:

<div turboalign="left" dojoType="TurboSplitter"></div>

That is a splitbar widget from TurboAjax, who extend the Dojo framwork, and it really is that simple to put a sliding splitbar on a web page.

2. Google has open sourced Excanvas. This is a one line include that lets you script the <canvas> element in IE. <canvas> is an element available in Firefox, Safari, and Opera, and now standardized by whatwg, that lets you script vector drawing primitives. Excanvas just implements all the Canvas apis in VML. Finally, a unified model for vector drawing!

IT Consultants Underestimate the Long Tail. Again.

2006-06-06T18:46:00.000-05:00

From Marketplace's story on Google Spreadsheets:

I.T. consultant Gerald Murphy won't recommend it to his clients....

GERALD MURPHY: The only people I think this would have any applicability at all to are very small companies that have extremely limited budgets who wanna pay no money at all.

Um, yeah. Sorta like the way Adwords lets all these very small companies advertise for pennies.

2006-05-28T00:13:00.000-05:00

Getcher Web 2.0 Certification here.

Requirements:

Big fonts
Oversized input fields
Silly or misspelled name
"Beta"
AJAX
Community content
"Something"-sharing
Bright colors and/or pink
Rounded corners
Use of Google maps
Founder has a blog
RSS
Tagging
Creative commons
Wiki
Podcast/video/mobile content

Peter Seibel Video

2006-05-27T18:40:00.000-05:00

Peter Seibel, author of the excellent Practical Common Lisp, gave this hour long presentation at Google, evangelizing Common Lisp. And here are the top three reasons CL rules and Java sort of sucks:

3. He gives a good worked example of using CL generics to replace miles of Java code implementing the visitor pattern. The key is the "double dispatching" in the pattern: You visit each node, calling some polymorphic method; you pass the method a reference to the calling object, which may itself be polymorphic. So the number of cases to dispatch is m * n, where m and n are the number of possible types for each of the two objects involved in the call. In CL, you just create a generic with m * n implementations; in Java, well, you have to touch a lot more code.

2. He contrasts ordinary try/catch exception handling with CL's conditions mechanism. Java unwinds the stack, losing all the state; CL notifies handlers up the stack without yet unwinding the stack, so that a handler up the stack can ask a function lower down to restart.

1. CL Macros let you abstract out syntax paterns.

It's the kind of video you can mostly listen to and only watch occasionally, as you get other work done -- just the way you'd do if you were at the presentation.

80/20 REST at Amazon

2006-05-01T18:23:00.000-05:00

The Register has short interviews with Tim O'Reilly and Amazon's Jeff Barr on WS-* vs REST, and unsurprisingly, REST "wins". Barr says they see "20 per cent SOAP, 80 per cent REST."

The Enterprisey Web Style

2006-04-27T20:01:00.000-05:00

Oh this is too funny: SOA integration with Flickr and del.icio.us.

We picked a messaging bus that uses the industry standard WS-HTTP and the emerging WS-Blog protocol.

But you know this is how the consulting crowd are going to sell web style, starting very soon.

GData Optimistic concurrency

2006-04-24T22:42:00.000-05:00

Google Data APIs beat Atompub to the punch and stamped legitimacy on REST. I hope Atompub will end up looking as nearly like GData as possible.

Of note: this is a Hi-Rest API, and now, we can safely say, it's the way the web actually works. Not just the way some very dedicated people aspire for the web to work.

So enough rejoicing. Let's begin analyzing GData to death.

First technical question: optimistic concurrency: What's wrong with plain old HTTP if-unmodified-since/if-match? That's how you guard against concurrent updates in HTTP right? Why invent a new protocol for that?

SOAP+REST: Why?

2006-04-06T23:55:00.000-05:00

mnot declares

I’m a little confused by Mark Baker’s stance regarding SOAP; he seems to encourage the Web services world to use SOAP on top of HTTP in a fashion compatible with HTTP.

MarkB responds he's offering a face-saving out for SOAP vendors. Is that the best reason? I can't think of one reason to wrap the messages I traffic in, in a SOAP envelope. What are the use cases for RESTful SOAP, please?

Slightly More Complex List Extensions

2006-04-06T09:55:00.000-05:00

Using the MS Simple List Extensions for RSS and Atom, you can really make a "feed" be anything you want it to be. Anything you can put between the angle brackets of a <cf:treatAs> tag, you can represent in a feed. I call it SMCLE (Slightly More Complex List Extensions).

Here is how it will work:

We'll define several new values for <cf:treatAs> element. Currently the only defined value is "list". Now we will have:
<cf:treatAs>Bag</cf:treatAs> : the items form an unordered collection.
<cf:treatAs>Queue</cf:treatAs> : the items form a fifo queue.
<cf:treatAs>Stack</cf:treatAs> : the items form a lifo queue.
<cf:treatAs>CircularQueue</cf:treatAs> : when you get to the end of the collection, start over again at the top.
<cf:treatAs>OneBigString</cf:treatAs> : the items aren't items at all, in the RSS/Atom sense. The content of items really should be concatenated to form one big string.

Additionally, we'll allow extension namespaces so you can define your own "treatAs" values, e.g.:


<cf:treatAs ns="http://my.example.com/smcle">JavaScriptStatements</cf:treatAs>

Now, I know your aggregator is going to suck if it can't process all these variations correctly. Not to worry: the MS Feeds API will handle it all for you! In fact, the MS Feeds API will pretty much become the definition of a feed. If it doesn't work with the API, you don't need it!

Simple List Extensions: Invidious?

2006-04-05T13:14:00.000-05:00

The Microsoft Simple List Extensions for RSS + Atom seem like a great idea. You declare attributes for your entries -- like Artist, Date Added, Price, Sales Rank -- and then set the attribute values on each entry. It's a tiny little schema language, with a type system. IE7 can then sort and filter the entries based on those attribute values.

The problem is that once you have that capability, you come to rely on it. If your site has a thousand entries, you can serve it up as a single feed with attributes allowing the client to filter. But you'd organize your site differently if clients didn't have that capability. So, for feed readers not supporting SLE, you'd want to break up your site into several feeds, maybe one for each Artist.

I'm sure every feed reader will have to support these extensions. Otherwise they would appear to suck on feeds that rely on SLE.

IE7 Transforms Atom to RSS 2

2006-03-30T23:35:00.000-06:00

I put a little mileage on IE7 feeds today.

First this Wellstorm Atom feed. Of note: a) it validates on Feed validator; b) it uses an extension namespace; c) it specifies a CSS stylesheet.

I've pasted in the relevant files below.

Observations:

IE7 uses its own stylesheet, ignoring mine. That's ok I guess; it's a smart client, so it should be styling to suit its own purposes.

Before you subscribe, View Source: It displays the Atom xml content.

After you subscribe, View Source: IE7 has transformed the Atom to RSS 2.0.

Unfortunately, within IE7 you can never again view the original atom file! You can't enter the URL and View Source... it always shows it transformed to RSS. Weird.

Here is the Atom feed:


<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet href="http://www.hughw.net:8080/witsml/styles/atom.css" type="text/css"?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:wixp="http://www.wixp.org/ns/atom">
  <id>http://www.hughw.net:8080/witsml/atom/Default%20Repository/!well</id>

  <title>WITSML Repository</title>

  <updated>2006-03-17T14:01:14.935-06:00</updated>

  <link rel="self" href="http://www.hughw.net:8080/witsml/atom/Default%20Repository/!well"/>

  <entry>
    <id>http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12</id>
    <title>6507/7-A-42</title>
    <link href="http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12.rby" rel="http://www.wixp.org/rel/contains"/>
    <updated>2001-05-31T08:15:00.000000+00:00</updated>
    <published>2001-04-30T08:15:00.000000+00:00</published>

    <author><name>John Smith</name></author>
    <wixp:ObjectType>well</wixp:ObjectType>
    <wixp:LastModified>2006-03-17T14:01:14.935-06:00</wixp:LastModified>
    <wixp:Name>6507/7-A-42</wixp:Name>
    <wixp:uidWell>W-12</wixp:uidWell>
    <wixp:nameSource>John Smith</wixp:nameSource>

    <wixp:dTimStamp>2001-05-31T08:15:00.000000+00:00</wixp:dTimStamp>
    <wixp:dTimCreation>2001-04-30T08:15:00.000000+00:00</wixp:dTimCreation>
    <wixp:dTimLastChange>2001-05-31T08:15:00.000000+00:00</wixp:dTimLastChange>
    <wixp:itemState>Plan</wixp:itemState>
    <wixp:comments>These are the comments associated with the Well data object.</wixp:comments>
    <summary type="xhtml"><div xmlns="http://www.w3.org/1999/xhtml"><a href="http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12">well 6507/7-A-42</a> <a href="http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12.rby">[contents]</a></div></summary>

    <content src="http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12" type="application/x.witsml+xml"/>
  </entry>
</feed>

Here is the IE7 persisted version:

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"
    xmlns:cf="http://www.microsoft.com/schemas/rss/core/2005">
    <channel>
        <guid isPermaLink="false">http://www.hughw.net:8080/witsml/atom/Default%20Repository/!well</guid>
        <title cf:type="text">WITSML Repository</title>
        <pubDate>Fri, 17 Mar 2006 14:01:14 GMT</pubDate>
        <atom:link href="http://www.hughw.net:8080/witsml/atom/Default%20Repository/!well"
            rel="self"/>
        <item>
            <guid isPermaLink="false"
                >http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12</guid>
            <title xmlns:cf="http://www.microsoft.com/schemas/rss/core/2005" cf:type="text"
                >6507/7-A-42</title>
            <atom:link xmlns:atom="http://www.w3.org/2005/Atom"
                href="http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12.rby"
                rel="http://www.wixp.org/rel/contains"/>
            <pubDate>Thu, 31 May 2001 08:15:00 GMT</pubDate>
            <atom:published xmlns:atom="http://www.w3.org/2005/Atom">Mon, 30 Apr 2001 08:15:00 GMT</atom:published>
            <author>John Smith</author>
            <atom:author xmlns:atom="http://www.w3.org/2005/Atom">
                <atom:name>John Smith</atom:name>
            </atom:author>
            <wixp:ObjectType xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">well</wixp:ObjectType>
            <wixp:LastModified xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">2006-03-17T14:01:14.935-06:00</wixp:LastModified>
            <wixp:Name xmlns="http://www.w3.org/2005/Atom" xmlns:wixp="http://www.wixp.org/ns/atom"
                >6507/7-A-42</wixp:Name>
            <wixp:uidWell xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">W-12</wixp:uidWell>
            <wixp:nameSource xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">John Smith</wixp:nameSource>
            <wixp:dTimStamp xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">2001-05-31T08:15:00.000000+00:00</wixp:dTimStamp>
            <wixp:dTimCreation xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">2001-04-30T08:15:00.000000+00:00</wixp:dTimCreation>
            <wixp:dTimLastChange xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">2001-05-31T08:15:00.000000+00:00</wixp:dTimLastChange>
            <wixp:itemState xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">Plan</wixp:itemState>
            <wixp:comments xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom">These are the comments associated with the
                Well data object.</wixp:comments>
            <description xmlns:cf="http://www.microsoft.com/schemas/rss/core/2005" cf:type="html"
                ><div><a href="http://www.hughw.net:8080/witsml/atom/Default
                Repository/well.W-12">well 6507/7-A-42</a> <a
                href="http://www.hughw.net:8080/witsml/atom/Default
                Repository/well.W-12.rby">[contents]</a></div></description>
            <content src="http://www.hughw.net:8080/witsml/atom/Default%20Repository/well.W-12"
                type="application/x.witsml+xml" xmlns="http://www.w3.org/2005/Atom"
                xmlns:wixp="http://www.wixp.org/ns/atom"/>
            <cf:id>0</cf:id>
            <cf:read>true</cf:read>
        </item>
    </channel>
</rss>

Lisp for Entrepreneurs

2006-03-27T21:56:00.001-06:00

Yet another Lisp is for Entrepreneurs link -- this one from Bill Clementson. Programming languages are converging on Lisp, so I'm bypassing the deep dive on Ruby...

I've gone deep on Lisp recently and I have to tell you: maybe I'm not smart enough to be a Lisp programmer, at least soon enough. I've written a functioning, and reasonably complex, program (a Gibbs sampler for a Bayesian network) but I can hardly read the thing. It's all (car(cdr(assoc('xxx ...)))) ...that design is nobody's fault but my own, but it's just the path of least resistance as I'm writing the thing. I have begun to see a few glimmers of possibilities I don't have in other languages. I've written a macro, and I get it. I've passed lambdas to functions. This code is a a lot terser than a Java version I wrote years ago. Terse is good, right? But I have to comment the code more extensively because it's not really "self describing."

Maybe I'm the sort of weak-natured guy who needs an early-binding language enforcing the discipline up front. 'Cause I have to admit, strong typing makes me write better programs. It's a character flaw.

Microcontent Three-Way Processses

2006-03-22T21:46:00.000-06:00

We're all going to have to package our services as microcontent (via Sean McGrath). For those of us in the business of delivering "two-way" data services, our webapps will wither. Good!

Now we have to figure out how to get paid. I can deliver some critical information to you via Atom or RSS. And you have some client subscribing, consuming, and processing that data on your behalf. I need to get paid by subscription, or by the piece. I'll no longer have a webapp where you subscribe or agree to purchase single items, and where I meter your consumption. Aggregators, and successor services like them, will have to do that. So we need extensions to APP to compose three-way transactions.

So you will initiate a subscription via your aggregating service. I'm not neccessarily talking about a feed aggregator; this might be some specialized service in a vertical, like stock quotes. It will collect your personal and payment information. It will contact me to subscribe you. It will send me your PayPal, or whatever, payment authorization. I will respond with approval or denial. When retrieving information on your behalf, your aggregator definitely has to tell me you are retrieving that information, so I can authorize it. Contrast to current practice: aggregators retrieve a feed on behalf of hundreds of anonymous users. And we have to address the issue of whether I trust the aggregator service to tell me the truth, or whether we can design protocols preventing it paying for one subscription when it has sold 20.

To make that happen, we'll need a microcontent business process specification that can compose three-way transactions. And it has to be as simple and transparent as RSS and Atom. I'm not talking about BPSS, although there's a lot to be learned there.

Spooky Rojo Mojo

2006-03-22T19:34:00.000-06:00

Alerted by Mark's post that he switched to Rojo, I just now signed up to give it a try. After going through the wizard, I land on my new home feeds page. And my very first "Rojo Feed Recommendation" is: Mark's other blog, Integrate This. And no, I haven't imported my feeds from Bloglines yet!

URIs are opaque, except when they're not

2006-03-16T21:10:00.000-06:00

Roy jumps in to clarify the law about URI opaqueness:

The key is that the server is free to tell the client that there does exist structure in a given URI-space, and then the client is free to make use of that knowledge in future requests. That is how server-side imagemaps worked -- HTML says that the URI is structured such that appending "?X,Y" to that URI, where X and Y are non-negative integers, corresponds to points on a map that can respond to future GET requests.

Thus, one way for the server to tell the client that a given URI is structured is to provide the URI in a standard element of a standard media type that has been defined as such. Another is to include the URI in a response header field.

I may be having my own, belated moment of total, violent, zen clarity.

Patterns Problem

2006-03-04T22:17:00.000-06:00

Peter Seibel's book has a nice analysis of the "patterns problem":

The reason both these functions start the same way is because they're both test functions. The duplication arises because, at the moment, test function is only half an abstraction. The abstraction exists in your mind, but in the code there's no way to express "this is a test function" other than to write code that follows a particular pattern.

Unfortunately, partial abstractions are a crummy tool for building software. Because a half abstraction is expressed in code by a manifestation of the pattern, you're guaranteed to have massive code duplication with all the normal bad consequences that implies for maintainability. More subtly, because the abstraction exists only in the minds of programmers, there's no mechanism to make sure different programmers (or even the same programmer working at different times) actually understand the abstraction the same way. To make a complete abstraction, you need a way to express "this is a test function" and have all the code required by the pattern be generated for you. In other words, you need a macro.

Lisp, again.

IE7 Quick Take

2006-02-22T09:04:00.000-06:00

Curious about IE7, I installed the beta preview.

The browser itself looks OK. Basically it's Firefox look and feel. Tabs. Wow. I'm interested in exploring the RSS/Atom support but did not get the chance to exercise much. They do have a little "subscribe" icon that looks just like Firefox's. It makes you wonder: If it takes the equivalent of a space program to catch up to dinky little Firefox and feed aggregators, how is MS going to keep up with whatever comes next? They'll be starting out a nose behind. The little guys will blazing new continents before MS ships Vista.

About ten hours later, I had to do System Restore to get back to IE6. Principally, that's because I was working on a project using somewhat unfamiliar tools ( Nullsoft Installer and VS 2003) and needing the online help a lot. After installing IE7, almost no pages were visible in the Nullsfoft CHM, and all the pages lost their styling in VS online help.

Demonstrating the tightly coupled Microsoft world: Install IE7 and you'll be "upgrading" several other products that used to work just fine.

Nokia, Ericsson gonna slap you upside the head

2006-02-15T16:21:00.000-06:00

Fear, uncertainty, and doubt being spread by Iron Mountain to get webheads to enroll in their seminar:

In May 2006 the new global .MOBI top-level domain extension is expected to arrive. It’s not the usual "yada-yada" new domain extension. It’s backed by major players in the mobile world, like Ericsson, GSM Association, Hutchison, Microsoft, Nokia, Samsung Electronics, Syniverse Technologies, Telefonica Moviles, TIM, T-Mobile and Vodafone. These investors are serious about bringing a much better and consistent mobile browsing experience to all users no matter what device they may be using. They plan on enforcing certain web site formatting requirements. Failure to comply may mean you are shut out reaching mobile users, even if you meet all legal requirements for registering a .MOBI domain name.