Hugh Winkler holding forth on computing and the Web

Thursday, May 29, 2008

Self-appointed Guardians of Truth

SiteTruth has given my company's web site a big red do not enter sign.

Even though we don't sell products electronically, they wish we had a certificate. And we've never put our office address on that site -- an oversight when we moved, not a scam. So they give us a big red "Do Not Enter" sign, indicating our site is dangerous to enter. From their "about" page:

Every on-line commerce web site must display the name and address of the business behind the site. That's the law in much of the developed world. SiteTruth tries to identify that business, then find information about it. That check is used to influence search rankings. That's SiteTruth. (emphasis mine)


We're not an "on-line commerce web site", but their system can't detect that, so they're selling technology that will lower our search rankings?

We'll fix our site to please them, of course; why not? But their technology doesn't seem to increase the safety of the web. And is likely to piss off other legit site owners, some of whom may even feel litigious. Could you blame them?

Friday, April 25, 2008

Hardly any near death experiences

The Hardy upgrade went... ok. Better than the upgrade to Gutsy, at least --the initial boot this time at least got me to an X, if only in 600 x 800 mode.

So you don't have to read to the end, here's my helpful hint to you this upgrade cycle. If you're having trouble with your nVidia: after you upgrade, use Synaptic to uninstall your old restricted drivers packages (in my case, for kernel 2.6.22), and select the new ones (for kernel 2.6.24) which, in my case at least, were not selected after the upgrade.

Only after you do that, and reboot, can you see the new nVidia driver in the System/Administration/Hardware Drivers applet. This used to be called the Restricted Drivers Manager and all the online docs still refer to that. But Hardware Drivers is what you want. Go there and do what comes naturally.

Last night I was finally able to get Update Manager to connect and start downloading files. I let that run overnight -- it took 6 hours or so to download everything, presumably because of the tremendous load on the U.S. Ubuntu server.

This morning, I awoke to find it paused in a dialog. Just a warning that it was about to clobber my modified mime.types. I answered OK. It proceeded a little, displaying the progress in a little terminal window. I noticed some interesting progress output, and started editing some notes. At one point, I selected some text in the terminal window, and copied it. Using Ctrl-C. Who would have thought the goddamn terminal window was accepting keyboard input and processed the interrupt. So immediately I get three alerts warning that it could not complete the emacs/ede/eieio installations becuase it had received an interrupt. Yesterday I predicted the emacs upgrade would fail, but I didn't mean to fulfill my own prophecy.

It seemed to continue from there almost without trouble -- but at the end it complained it could not upgrade the update-manager. Ironic, isn't it? And at the end of the install it displayed a scare alert: "Your system may be unusable". Nice. Bravely, I rebooted, and found myself in 800 x 600, but at least with an otherwise stable system.

After some flailing I discovered the secret sauce to getting nVidia working again. And all systems are now go!

I wonder how Mark Pilgrim's mom and dad are doing?

[updated to fix link]

Thursday, April 24, 2008

Ubuntu Hardy is out and I'm a sucker

I managed to hold off about 6 hours before giving in and pressing the "Upgrade" button to get Hardy Heron. Yes, I'm in the middle of a project. Yes, I expect I'll have nVidia issues. My emacs will probably fail upgrading as usual. But: it's too tempting. I'm a sucker. I know it! No discipline whatsoever.

Hm... I guess the servers are mighty busy today. My first attempt timed out: "Could not download the release notes/ Please check your internet connection".

Saturday, April 12, 2008

Lisp a casualty of its time

This sad editorial remark from the CLIPS architecture document, describing perceptions about Lisp within NASA around 1984:


...Despite extensive demonstrations of the potential of expert systems, few of these applications were put into regular use. This failure to provide expert systems technology within NASA’s operational computing constraints could largely be traced to the use of LISP as the base language for nearly all expert system software tools at that time. In particular, three problems hindered the use of LISP based expert system tools within NASA: the low availability of LISP on a wide variety of conventional computers, the high cost of state-of-the-art LISP tools and hardware, and the poor integration of LISP with other languages (making embedded applications difficult).


Would they encounter the same barriers today, twenty-four years later?

1. Low availability of LISP on a wide variety of conventional computers: Now we have commercial implementations and open source Lisps, working on most computers and operating systems.

2.High cost of state-of-the-art LISP tools and hardware: The hardware issue has faded away since Lisp machines gave way to general purpose computers. And Lisp is no more memory intensive than Java or .NET. Of the commercial Lisps, LispWorks is affordable for normal mortals, and certainly the Allegro products are within reach of NASA. And there are several good, free, open source CLs.

3. Poor integration of LISP with other languages: An issue only if your mindset is 1984. Nowadays, to integrate with a Lisp program, you'd treat it like any other network resource -- like an RDBMS, or a web service.

I wonder what is the state of the Lisp renaissance within NASA?


[updated to expand remark on item 2]

Monday, March 03, 2008

Rule of Least Power: Bah!

The Rule of Least Power, a W3C TAG finding, posits: "Powerful languages inhibit information reuse." They're observing that it's easy to scrape documents written declaratively using HTML. The problem with using more powerful languages like Javascript, they say, is that "you typically cannot determine what a program in a Turing-complete language will do without actually running it."

So? As long as the output is a DOM, just run the program and inspect the DOM.

You already have to use a good HTML parser, right? Now, just run all the script elements on the page too -- obviously, in a restricted environment.

I'm sure Google and friends must do this. They're not going to leave valuable information on the table.

Tuesday, February 05, 2008

Yet another reason to use NoScript

Niall Kennedy tells us all how he grabs your browser history "for improved user experience".

1. put links on the page to all the interesting sites you want to identify in the user's history. Have they visited Google? Have the visited certain porn sites?
2. put some css for those links that colors them some weird color when they have been a:visited.
3. add some script that crawls the dom looking for links styled that color.

VoilĂ .

Yet another reason I'm glad I've turned off most scripting using NoScript.

[Updated to remove a made up porn site domain name that actually is a porn site domain name.]

Monday, February 04, 2008

Eli can't get no respect

[Update Feb 5: they've fixed the headline, but I just noticed the caption at the right still says "Super Bowl XLI". So that explains it. To build yesterday's Super Bowl results page they just grabbed last year's. Then they attempt the replacements s/Peyton/Eli/g and s/XLI/XLII/g. Probably they left off the 'g'!]

One Manning's as good as another to the British:

Sunday, January 20, 2008

Clojure

Clojure is a Lisp that runs on the JVM. I agree with everything on this page describing the design rationale. Objects are overrated; functional and immutable are good. I can't wait to get a breather and figure out whether and where Clojure fits into our technology arsenal.

The real measure will be how well it can leverage the Java platform. There are plenty of great languages, but they have inconsistent support for, say, sockets, or window systems. Can I override a protected method of a Swing base class? Can I connect to an HTTPS server? JRuby has knocked Java integration out of the park, so you get all that capability for free, plus a powerful language.

Update: I think the bullet point "Extend Java in Java, consume Java from Clojure" on the rationale page suggests you can't easily write a Swing app in Clojure. Shoot.

RIA using JRuby and Web Start

I went dark over the last eight weeks, building a rich web application to integrate with our product. It's been years since I built a desktop GUI application, but I was pretty good at it once. It's not always a lot of fun. But some things a man just has to do himself.

The application does tons of vector drawing.... it draws hundreds of thousands, even millions, of line segments. Yet it needed to be responsive... scrolling, zooming, panning. It's data intensive, too. It downloads dozens of megabytes of data for each view. I needed to develop this program fast, so I needed a programming system and platform allowing me to iterate and adapt to new requirements recursively revealed. I preferred a declarative graphics language where possible, and for any procedural code, a powerful dynamic language. Ideally the program should run on all OS platforms; but the business could probably tolerate a Windows only solution, if that offered superior results. I made tradeoffs among these requirements.

I was not constrained by download footprint, a concern which eliminates Java from consideration in lots of other scenarios. Anyway, that problem is going away soon, they tell us.

So here's the spoiler: I chose Java Web Start + JRuby. And now a short dissertation on how I arrived at that choice:

I looked to browser and browser++ technologies like SVG, Flex, AIR, Silverlight, Java applets/Web Start/JavaFX. (Open Laszlo deserves a look too, but I didn't eval it). It would be great for this app to run within the browser -- that would be the most seamless experience. But using an out-of-browser technology like AIR or Web Start isn't a deal breaker if you do it right.

SVG was very promising. It would be a pure browser solution. I really valued the declarative model and the standards alignment. But it does not work in IE. I also did some performance experiments with Opera, Safari, and Firefox. Drawing polylines of a hundred thousand points or so, Opera and Safari performed well, but Firefox performed poorly. It's conceivable, if unlikely, that I could distribute the app as a FF-only solution -- even if I bottle it up as a a XULRunner app. But the FF performance was inadequate. And distributing only for Safari or Opera is not realistic.

I made a significant evaluation effort with Flex/AIR. Surprisingly I learned that the Flex 2-D graphics model is procedural, not declarative. You can, however, bottle up your procedures and use them declaratively from mxml, the Flex XML language. I felt I was going to spend a significant amount of time learning Flex -- time I needed to spend on my app -- and that I needed the assistance of tools like Flexbuilder. I also became concerned that I was developing a program away from the design center of Flex. Would I encounter design/ performance constraints?

Silverlight performed very well, and it's got an XML declarative model just like SVG, with the power of a bunch of WPF components to boot. I was confident that all the capabilities of the Windows platform would be available to my program. Microsoft distributes Silverlight for Windows and Mac, but not Linux, which they are leaving to Novell. I read that as: No Linux. But Windows and Mac are enough coverage, even if I would be sorry to leave aside my Ubuntu dev machine. My fate on the Mac would be in Redmond's hands, of course. Does Microsoft have my best interests at heart ?

Java applets or Web Start, using Java 2D, I also knew to be capable of these rendering tasks. I'm satisfied that I can make Java applications now that look pretty sweet. The down sides were the procedural approach and the static Java language. It's the 1999 state of the art. Layout code so impenetrable you need GUI builders.

JavaFX deserves a separate mention. The capabilities of the entire Java platform are there for you. Yet you program in a declarative/dynamic language optimized for building Java 2D apps. Perfect! Unfortunately, it's pre-alpha. Sun hasn't yet released it under a license allowing you to put your work into a customer's hands.

I wanted the wide reach of Java. I think that the 5% of customers on non-Windows platforms may be more important than their numbers.

I also wanted the enormous hardened Java platform. It's as capable as the Windows platform. You won't encounter many problems you can't address or workaround.

Couldn't I have all that, plus a dynamic language to stitch it all together?

Guess what. JRuby is stable and 1.x. It's dynamic, yet it compiles on the fly to JVM byte codes. It's got great Java integration, meaning you can invoke any Java API. Profligacy is a terse little Ruby lib for stitching up Swing components. JRuby's just a jar I download with my Web Start app, and launch with the main Ruby script from my Java entry point. The rest of the program is in Ruby, occasionally calling into Java libraries. No, there's no declarative GUI language here. It's one of the tradeoffs.

My work environment: Fire up NetBeans for its good Ruby editor. In a command window kick off jirb and execute a script to load and run my main code. Edit code and reload it into the running program in jirb. Lather, rinse, repeat.

I would argue that JRuby accelerated my application's performance, because I iterated and applied two major architectural optimizations that would have introduced impenetrable spaghetti in Java. The bottleneck for speed and responsiveness has never been JRuby. It turned out that even Java2D + acceleration could not keep up acceptably with the level of interactivity I wanted. The SVG or any other pure declarative approach would have failed (although I think JavaFX's scene graph might be able to optimize the drawing). I had to go in and surgically rearrange the innards of the program. The changes were effective but had low impact on stability, thanks to the flexibility and power of Ruby.

If Sun can focus more energy on these dynamic languages built on the JVM, that would be a pretty powerful story: The deep platform capabilities, the broad reach, the rich development environment.

Tuesday, January 15, 2008

And Speaking of Paul Graham...

Arc to be open sourced this winter. It's going to be hard to top the expectations.