Ross Judson: Java Etc.

Mon, 19 Jan 2004 20:03:43 GMT

Protocols Plugin.

I've begun organizing some of my source code with Sten-Erik Bergner's Protocols Plugin for Eclipse. What it does is provide a decent and clean way to organize the members of a Java class. You declare section names and give it a sort order. Then, in the Java Browsing perspective (not the Java perspective), you can drag and drop from the member types window into the list of member categories ("protocols").

The plugin organizes your source code as you go. What you end up with are pretty nice looking source files that have all members that correspond to a given category ("accessing", "creation", "event handling", or whatever) all bunched together, in sorted order.

When you're working with your code, you can click on any Protocol in the Protocols view and have your list of members filtered to include only members of that category. When you're working with larger classes or more mature systems, this is a godsend. You can quickly zero in on public stuff, property handlers, and whatever else you want.

You can make up your own protocol names...I wrote a paper a while back that describes a series of method classifications. I've made some protocol names that correspond to those.

Sun, 09 Nov 2003 05:05:24 GMT

The Relentless March Of Stupid Technology.

Here in the DC area there's a story getting some buzz: Glitches Prompt GOP Suit Over Fairfax Tabulations. The GOP is absolutely right to protest this vote; it's not about anything other than the validity of the process.

The machines in question are the WinVote model, from Advanced Voting Systems. Just listen to their pitch: The functionality linchpin of the WINvoteTM system is its wireless LAN (IEEE 802.11b) system - called the Wireless Information Network (WIN) -- that enables the user to communicate remotely with the major components of the voting system.

Does anybody else see a problem with enabling wireless communications to voting machines? This is just f'ing stupid, beyond belief. Encryption can help ameliorate the situation, but what I see is the possibility of ne'er-do-wells attempting to disrupt elections with laptops. All they have to do is hang around the building within a few hundred feet, and they can screw with the election machines to their heart's content.

Who wrote the IP stack on the machines in question? We can't tell from the web site, which means in all probability it isn't open source. That means we have no idea what stack is running in there and what its vulnerabilities are. Even encrypted stacks can be vulnerable to certain kinds of attacks.

Voting companies have been pretty underfunded. Why are we trusting our democracy to these stupid machines? The only function of a computer in the democratic process should be to help print a legible vote.

Count me, as a computer guy, concerned.

Sun, 09 Nov 2003 05:00:40 GMT

Fixing Patents.

Great reading here...the FTC is releasing findings about the patent system. Reading this, it's just common sense to do so, and everything in here squares with what I've experienced.

Tue, 04 Nov 2003 20:59:42 GMT

On the 1.5 Java Language Extensions.

I was reading the (old) chat at the Javasoft site. It's got some insight from the developers...here are a few comments...

On Generics: Nice addition to the language, but for the love of God why aren't primitive types included? After all this time, I still cannot make an efficient map of integers to doubles, or any other primitive type combination. What's missing is a generic-type capable implementation of a BTree...this will lead to some serious compaction in atomic data storage...

Typesafe Enums: Nice to have, like'em, but Delphi did it better 47 million years ago. Bit mask sets were perfect for tons of different things, and they're still perfect today. A tiny little bit of syntactic sugar is all that's necessary to make them work. Why should I incur hundreds of bytes of overhead to store a set of enums? It makes no sense. And if you want to treat them like real objects, the compiler can autobox into a typesafe enum object. Just keep the representation in the field structures as compact as possible. Remember, there might be several millions sets. One of the most important uses for enums are flag sets, which just aren't going to be efficient in the proposed scheme.

JFarre: What about supporting "references to references"? I know it's one of the hardest things to learn in C, but it allows us to almost do magic...

Neal Gafter: This is sometimes called "hooks", and I do miss this ability in the Java language. The closest thing we have is java.lang.Ref. We haven't considered adding anything like pointers-to-pointers in the Java language.

You don't need pointers to pointers. You need a functional type, which can be used to do pretty much the same thing, with much more flexibility. A proper functional type doesn't need an "object" to call a method on -- you can just call a "thing", which can be atomic, a method call, a function call, or a composition of the above, and it returns a result of that type. Of course, you can throw exceptions and they are declared in the functional type's signature.

With the generic system you can create containers full of functional objects, and the possibilities are quite endless. Functional type compatibility should be automatic with the compiler -- if the call signatures are compatible, it can be used...

This is, in my opinion, the single biggest missing thing in Java. If we had proper functional types, our code would be so much simpler, and we'd all be doing cool higher-order functional programming.

Mon, 03 Nov 2003 18:21:29 GMT

Sandboxing Outlook's HTML Display.

One of the perfidious things out there on the web right now are the nicely formatted "scam" emails that purport to be from places like PayPal. Sometimes it can be pretty hard to distinguish these from the real thing -- the problem is that the HTML displays inside of common programs, like Outlook, don't show you where they're getting their information from. The email program cracks the email apart, extracts the HTML content, then hands it to a browser component for display.

What we need to do is have the email program pass the domain of the reply-to or message-source field to the browser component, which is then instructed to not display anything that is not from that domain. HTTP requests for that content in that component will not be sourced from anywhere but the indicated domain. What's more, if any part of the page tries to hit something outside that domain, the entire message will not display.

This will pretty much eliminate HTML-formatted spam email. It won't work because the pages can't access anything other than the server that sent the email. Forge that, and your email doesn't display.

Note that we can extend this concept to include regular email as well. If the email contains a web address that comes from anywhere other than the domain that sent the address, we can elect to not display it. Of course, this messes up having one friend send a link to another...so it's not particularly workable as is.

Thu, 23 Oct 2003 22:38:59 GMT

Spam Throttle.

I've written on this before, but it's worth repeating:

My feeling is that the internet as a whole is going to evolve towards a trusted/untrusted scenario. Big vendors like AOL and MSN will be "in the club", so to speak -- their mail systems will "trust" each other, and accept email at full speed.

Other systems, particularly those connecting for the first time, will be untrusted. Those systems will be severly limited in the number of emails that will be accepted. A Bayesian filter can be applied to the email coming in from any new connection. The filter doesn't stop the email; rather, it builds an approximation of how much spam is coming in over that particular connection. The odd false positive isn't going to hurt, that way.

As the system gains trust, which can happen only by having time elapse and by having a low spam percentage, trust will gradually accrue, and email traffic will be permitted to have progressively higher volumes.

The beauty of this situation is that you still accept email from just about anywhere, but you don't _trust_ that source, until that source proves itself.

It doesn't take very much to implement a solution like this. Any organization that failed to implement it or have a strong anti-spam policy would find itself being on a lot of untrusted lists, and unable to send very much email. Simply being a transfer point for spam will also get you untrusted, which means that you'll have to pay attention.

Sun, 12 Oct 2003 21:01:23 GMT

Eliminating NullPointerExceptions.

Every reference in Java is either null, or valid. In the development stage of many applications, and even during production phase, we can still see a lot of null pointer exceptions being thrown. This is usually due to failure to check a return value from a function call.

The null result is often returned from a function to indicate "I couldn't", or "that object was not available". It is very common to want to return this result, and exceptions shouldn't be used to signify it. We don't want to incur the overhead of generating an exception.

It would be interesting to be able to mark a method as "returns null", in the same manner that we can throw an exception. The compiler can then check to see if the value returned is checked for null, before being used. If the value is returned from the function, that function must be marked the same way, forcing callers of it to check, or declare to throw.

Yes, we can always put the null value into a class, and there are more convolutions of this principle that could get at that...

It is an interesting idea, nonetheless. Extending it further, we can note that we might declare multiple return paths for any function, and the calling of that function becomes like a switch statement, where the type of the result directs the path of execution, much like a type-inferring language like Haskell or ML.

And maybe that is what I really want. For now, though -- enforced null checks!

Fri, 10 Oct 2003 19:49:51 GMT

Anders Hejlsberg is Wrong, Part II.

And this time it's not me saying it. James Gosling is saying it. Excellent. Bottom line is, it's fun and fast to ignore exceptions or catch them in some pointless top-level loop. That's fabulous for, I don't know, applications that don't matter? Now you're writing an app that has to stay up. This is what most of us are doing now, in the Java world. It has to stay up for months, or years. Do you think your top-level exception loop is going to be able to handle the multiple strategies that are necessary to deal with failure in that environment? You're going to activate retry-logic at the top level?

And what the heck is the top-level in any modern Java app anyway? It doesn't exist.

Java helps you build robust code. That is now, and has always been, its most important feature. It does a better job of this than any other commonly used language. We have a couple of a million lines of Java code in our app, and it's pretty damn bulletproof. Trying to get to that level of reliability in C could take decades. And no, we are not crappy programmers. If you think you could do it in less time, you think you're smarter than a lot of very smart people out there who have been writing system-level stuff in C for longer than you have been alive. Here's a clue: You're not. You need the help; you just don't know it yet.

Fri, 10 Oct 2003 06:54:35 GMT

At Play in Piccolo.

Something unexpected...(this likes a fast machine).

Thu, 09 Oct 2003 06:27:48 GMT

Piccolo Clock Action.

This is something I played with this evening. Note that if you are running IE and you haven't installed the Java 2 stuff, you'll need that...this uses the default APPLET tag, only the recent Javas support that on Windows. Go to JavaSoft and pull the latest runtime and you'll be set.

Ain't it cool? Check out Piccolo!

Thu, 09 Oct 2003 01:36:28 GMT

The Other Eclipse.

The Haley Project is Paul Haley's commercial exploration of rule system technologies and advances. If you haven't paid much attention to rule-based programming, you should become familiar. I've written on the topic before, and you didn't listen, did you? You should check out Jess, from Sandia. You should also check out Eclipse, The Haley Project's powerful rules engine. One of the best things you can do for yourself once you get past the initial familiarity phase of CLIPS and rule programming is read the downloadable manual for Eclipse. It is chock full of advice for the rule systems programmer; serious advice, on how to structure facts, information, and requests. It was an unusual read for me -- I didn't think that anything like that existed. I have been painfully creating my own structures for performing many of these tasks, and I would have had a huge head start if I'd read the Eclipse stuff first.

Eclipse also comes with a powerful debugging system that allows you to really pull apart the rule network and understand what's going on inside of it. Being able to observe these things gives you the ability to fine-tune your rule network. Eclipse's variant of the CLIPS language provides additional mechanisms for specifying network structure that can come in really handy for tuning. It also provides very powerful agenda handling; this can be used to avoid a lot of the salience games you normally play in CLIPS.

Haley also has the AuthoRete product, which allows you to build rule systems from plain english statements. And it understands quite a bit of english, if you follow some reasonable structuring guidelines.

I'll be examining this one closely over the next few days.

Wed, 08 Oct 2003 21:11:01 GMT

I Like Mozilla.

Kicking mozilla to the curb. ...in a manner of speaking. The Firebird and Thunderbird Mozilla projects have both been working very well for me. In particular Firebird gets rid of the vein-poppingly infuriating UI decision to put 'Close Tabs' and 'Close Other Tabs' right next to each other, and when looking at your 'Sent' folder Thunderbird can actually display the recipient rather than the sender, which is always you. Thunderbird in particular seems a bit snappier when marking junk mail and deleting messages. And both seem stable on Win32 (XP) and Linux. [cwinters.com]

You know, I've never really understood why people keep complaining about it. It's way cool, stable, fast, everything. The usual complaints are too big, too bloated...what kind of machine are you running on, anyway? On my box Moz sits in resident mode, and it's up and browing in a fraction of a second, whenever I want it. It's a small download. It installs painlessly, without requiring reboots. Popups are a thing of the past. It has comprehensive newsgroup handling, chat, and lots of other cool stuff. You can customize the heck out of it.

What's not to love? Where's the love, man?

I am anti-dissing Mozilla.

Mon, 06 Oct 2003 01:08:39 GMT

H1-B Realities.

Chiara has recently written on this subject. You should read what she said, and take the time to read the report she refers to. There are a number of forces at work, and they're not working in favor of us programmers here in the US...

Cheating is rampant. Companies abuse the H1-B process to find cheap labor. They falsely certify that they are unable to find appropriate employees already able to work in the US.
Corruption is present, and may be common. There are people out there who "broker" H1-B positions; they're in a position to hire and fire, falsify certifications, and so forth. Employees are often required to pay to get their position, or give part of their pay.
L1 abuse is becoming more frequent. This does an end run around the entire visa process; consulting front companies use the L1 process to import additional consultants, who are farmed out at cheap rates.
Fear means silence. Current H1-B holders are often scared to complain about long work weeks, low pay, paying a "toll" to get their job, or giving part of their pay to some organizer.

What we've got is a system with good intentions (to allow American companies to import the specific, skilled expertise they need), that's resulted in a lot of cheating and imbalances.

In the interests of full disclosure, I'll point out that I am a Canadian working in the US. Fortunately, Canada and the US have a bidirectional visa program, whereby Americans who work in certain professions can easily get a visa to work in Canada, and vice-versa. Our two cultures are very comparable in just about every way, so that program has worked well for both countries, and given the citizens of each substantial additional freedom.

Thu, 02 Oct 2003 20:19:44 GMT

My Eclipse Wish List.

Move member function somewhere outside its hierarchy.
Export/import breakpoint set (I have complex ones with filters I would like to share).
Common LISP support.
Scheme support.
Jess support.
Splitting of editor windows
Fast numbered bookmarks like JBuilder (ctrl-shift-# to set, ctrl-# to go to).
Better decorators to show cvs modification status (they're too small).
A "reconfigure" mode for large projects, where I can turn off updating until I'm finished making project changes. Then, it refreshes and rebuilds everything.
Better compaction of the debugger view -- filter out regular, running threads.
Hotkey-based pop-out object inspection (nice big window for complex objects).
Less finger-painful hotkey for Quick Fix. Control-1 is a stupid key for anything, dammit. OK, I guess I can remap this myself, but really, man...what were you thinking?
Never auto-open imports in outline.
Resource History should be full window, not a tab.

Wed, 01 Oct 2003 04:29:39 GMT

Overseas Job Exodus.

I was thinking about yet another call I got today from yet another offshore outsourcing company. There has to be some way of stopping some of this flow of jobs, in the high tech industry. Here's my thought -- drop any tax write-off for overseas services work. That is, disallow as a business expense money given to overseas outsourcing shops. I have no problem with a bunch of hard-working guys in India doing software. How about they develop their own software economy, over there? Figure out some products, design'em, build'em...it's not like they can't...lots of great companies.

We just need to find a way to stop the blood-flow of jobs. What's happened in the manufacturing sector could happen in high-tech. I think it's worth mentioning that the playing field isn't really level -- American companies must provide a lot of benefits to their workers (health care, etc)...the costs just aren't there overseas. It's not really protectionism...

Maybe I'm really wrong-headed, here...

Tue, 23 Sep 2003 16:14:50 GMT

Fixing Electronic Voting.

Interesting article on Slashdot about Electronic Voting...read through the links, especially this one. It strikes me as incredible that the "technical" people writing these emails are engaged in such Mickey Mouse chatter, and so interested in just cranking out something, anything that will work. I just don't see how electronic voting is really all that hard to engage in...as long as you have your priorities straight.

There are two primary things we want to accomplish with EVotes -- first, we want to make the voting process easier to engage in. Second, we want to make the counting process more efficient (less costly). We would also like to reduce the error rate, to the extent that we are able.

A touch screen voting interface, big and clear and nice, is exactly what we need to help walk people through the process. We can't, though, rely on the software in these machines. One read through the memos above should convince you as to why -- these people just have no idea what they're doing. Basic? Access databases? Windows? My god.

What this says to me is that we simply cannot get away from paper. So what we want is a system that makes paper easier to use, leaves a paper trail for auditing and verification purposes, and provides ample opportunity for error checking by the voter and by election officials.

We use the touch screen to answer questions. At the end of the voting session, the system prints a "vote" and electronically tabulates the results. The voter verifies that his printed vote matches what's on the tabulation screen. The voter then folds his paper vote and deposits it with election officials in a good old fashioned ballot box.

We can then use the electronic tabulation to check quickly on the results -- this is quite efficient. We will also engage in a substantial amount of verification, by counting the paper votes by hand and verifying this against totals learned electronically. The paper always wins, in this system. We do not necessarily need to count all of the paper votes -- we can use random sampling.

It seems like a win in both directions, for me. Risks include unacceptable printout quality (printer wear), and insufficient random verification.

Tue, 23 Sep 2003 02:03:04 GMT

H1-B Visa Cut.

It doesn't affect me because I'm on a different visa as a Canadian, but it's interesting nonetheless. The Washington Post is reporting that the number of H1-B visas is going to be cut from 195,000 to 65,000. I know a fair number of people who have been out of work (although a lot of them have found jobs now), so I can certainly sympathize with this. What I am worried about, though, is that we're just going to drive companies into overseas outsourcing. In my position, two years ago I might have gotten one call a year asking me if I wanted to outsource. Now I get at least two or three a month...outsourcing has become huge. Sending those dollars overseas is the kind of thing that just might be contributing to our "jobless" recovery at the moment.

In the long run, the only thing that has value in our business is change. Get used to it. You need to be able to efficiently do those changes. As a domestic worker, you have serious advantages over the competition. Your communication loop is tight. You have a better cultural fit. You can be a known quantity. You can provide references. Some of these advantages are going to fade over time, so you need to keep them up.

Something that can't be outsourced, though, is raw creativity. If you come up with something that hasn't been done before, it's your idea. Creativity builds markets, companies, jobs, and wealth...it's something that this country does incredibly well, and can't be outsourced.

Sat, 20 Sep 2003 19:53:42 GMT

Technical Notes.

Trackback is now enabled (I think) for this Radio Userland -powered blog. I've been playing a bit with Movable Type and may switch, but for now...we'll proceed with this. The pictures in the previous entry might look a little better if you expand your browser window.

I wrote a quick Java program to read in my 1MB+ source images (from my Canon S50 camera)...It uses the 1.4 SE imaging API. I've done a lot of image work in the past, and I don't mind telling you that the imaging API is, on the face of it, pretty crappy. You simply have to do too much work to do some simple things, and there are way too many formats and options on most of the classes. Some of it is infrastructure that's dedicated to handling asynchronous image work and I understand the complexities of it are necessary, but for God's sake, provide simple ways to do the simple things.

Resizing an image turns out to be a pain in the ass. You can easily load your image in with ImageIO, which provides you with a BufferedImage. BufferedImage inherits a getScaled-type call, which looks like exactly what you need, until you realize that it returns an Image. If you check the class of the image, it's some weird AWT thing. In order to use ImageIO's writing, you need a RenderedImage (of which BufferedImage is one). So how do you get there? The answer is, you can't.

What I ended up doing is creating a new BufferedImage, getting a Graphics2D from it, then rendering the source image onto the target. I set the rendering hints, but frankly the image quality absolutely sucks. I'll need to go back and tack on the interpolation hint directly and see if that makes a difference.

The main reason I ended up writing a little program to do this is that the batch resizing program I have (Adobe Photoshop Elements) is just plumb stupid about portrait vs. landscape images. There's no way to convince it to do the right thing.

Fri, 12 Sep 2003 21:20:23 GMT

Seeing-Eye Tongue.

Clive Thompson finds the damndest things. Who woulda thunk it? I can't see any particular need for a yucky thing in your mouth, though -- why not just have a headband that divides the space around you into quadrants (or finer). Tingle the appropriate quadrant of the with a strength roughly approximating distance (via ultrasonic bounce) from obstacles in that direction.

That way we can have blind people who will have "eyes" in the back of their head.

Fri, 12 Sep 2003 05:08:14 GMT

Takedown.

One of our clients had a good chunk of their network taken down by one of the trojan emails today -- probably blaster or something like that. Let's look at the simple equation:

C + Unrestricted Access + Buffer Overflow = You Will Never Be Secure.

Windows machines are fundamentally flawed, at the core. It can never be fixed. You either need a VM (which provides a relatively secure environment), or an operating system that uses a variety of techniques to prevent a process from doing anything it's not supposed to do.

COM and ActiveX were built on the notion that by certifying something, you assure that it is secure. Today's additional DCOM patch, and the worms that are out there, show us otherwise. This isn't the last one. There will never be a "last one", because new code keeps landing on machines, and it's all vulnerable.

It's time for a do-over, people.

My weekend reading: Bruce Schneier's Beyond Fear. I hope to come out educated.

Thu, 11 Sep 2003 07:13:08 GMT

Chapter 1.

I have been writing (and there's a lot to do):

This is not a book of answers. It’s a book full of questions – hard-won observations about enterprise-class software and its implementation. My intent is to put some new thinking in front of you, the reader. I would never claim to have invented many of the techniques that you will read about, but I do hope that this book is bringing some of them together for the first time.

The set of keywords I used professionally used to include reliability, encapsulation, normalization¸ and so forth. These are still great words! We need what they imply. But there are more words that are important – resilience, partial information, self-repair, aspects, deployment…enterprise-class software needs these.

I’ve spent 20 years following the winding evolution of programming. Like most programmers my age, I began my learning process playing with early microcomputers like the Apple II and the Atari 800. We have come a long way since POKE 101, 25. Computer languages have sparked and grown in an ecosphere full of needs. Assembly language still has its place today, but for most usages, there are easier and more appropriate dialects to work in. We’ve seen evolution from assembly to procedural languages, structural to object-oriented, and a myriad of functional and logical languages. Each has its place, its strengths and weaknesses.

As of 2003, most applications are constructed with an object-oriented basis. A number of mature methodologies exist for creating object models; they have produced some excellent results. One of the noted strengths of good object-oriented design is the ease with which change can be accommodated by altering the model. Change is at the heart of almost every software enterprise. Any software that is actually used has constantly evolving requirements.

We also find a growing trend towards special purposes interfaces. Rarely is software constructed in a vacuum: We code to the reality of the internet, to the sharing of data, and to many users. With such a myriad of information available, we often wish to focus the information available to a particular class of user. Specialized interfaces meet this need, deployed against particular platforms such as web browsers, Java, or a traditional client. We also see hybrid user interfaces gaining in popularity, where the best of traditional user interfaces design is coupled with web capabilities.

The only constant in all of this is change. New methodologies, generally referred to as Agile Development, have arisen to help manage change within the development process. Object-orientation in general is capable of doing a good job of handling change at the design level. We can refactor a design to handle new tasks, achieve better organization, expand on capabilities, and so forth. This represents a significant evolution from the early days of programming, where applications were excessively brittle and hard to evolve once written.

Modern object-oriented programming environments make it easier to deploy these applications. The various incarnations of the Java platform and of Microsoft’s .NET architecture are prime examples. These systems come with vast libraries of functionality, and many tools to effect the sophisticated deployment of applications. Java achieves a high level of portability as well.

For all this coding and development flexibility, we have made little progress in creating applications that are flexible at runtime. We need applications that can evolve, adapt, and even reason. We need to structure our information in a way that allows this complex processing to take place. At the same time, we don’t want to lose the structuring capability that a traditional object model provides. Object models still give us a good mechanism for dealing with some of the real world things we want to model, and achieving comprehension of the result.

What we want to create is a blend of conventional application design and techniques from the worlds of knowledge management and artificial intelligence. We want a common way of expressing this information, of transforming it, of querying it, and displaying it. Aha! You may think at this point…this sounds like XML! XML is an interesting way of expressing information, but is difficult to work with and not really appropriate for the kinds of applications we want to construct. XML has problems with scale and querying, as the specifications stand – very few tools can really deal with large amounts of XML (say, gigabytes worth).

There’s something more fundamental than XML’s tree of information.

Instead, we want to break things down to the basics. Statements (or facts) are at the base of the information pyramid in computer science. We can construct complex entities by binding together statements. Consider the following:

Ross has a last name of Judson.

Here we indicate that a person named Ross has a certain last name. If we decide to be a little more “computery” about this, we might rephrase as follows:

Ross has_last_name Judson.

There are three symbols involved here: Ross, has_last_name, and Judson. Symbols are extremely useful; we’ll be delving into them extensively later on, and talking about efficient ways to implement them and work with them. Our symbol-based example doesn’t quite express what we want, though. Let’s add the following:

Ross is_a Person.

Now we know that Ross has a last name of Judson, and Ross is a person. The examples given so far are in a triple form. Triples are very useful things; they form the basis of RDF, an important standard we will discuss later. A triple is a special form of a tuple.

These atomic statements can be used to construct just about anything. Consider:

Employee subclass_of Person.

Here we have declared that employee is a subclass of person. In a conventional object language, we might do something like the following:

class Person {

String name;

}

class Employee extends Person {

String department;

}

What I am trying to convey here is that for any given object-oriented program, we can encode both the data and metadata of that program in simple statements. There can be any number of such encodings; RDFS is one example that we will discuss at length later. Symbols and other identifiers can be used to join the statements together into a complex model. Concepts such as generalization, aggregation, and association can easily be mapped into statements. Once we have done this, we have created a fully fluid meta-model. A fluid meta-model easily copes with change at runtime by altering the statements that control the metamodel. This characteristic is key to the flexible applications we seek to develop.

Conventional object-oriented languages execute a compilation phase that creates a runtime version of the application’s metamodel. This information is sometimes available, at least in part, to programs written in the language. A Java program can, for example, elicit a good deal of information about itself from the runtime system. Most C++ programs cannot, though – the C++ runtime type standard is primitive at best. A C++ program is not privy to the information the compiler has about it.

Instead of creating an object model, we create a knowledge base. We structure the knowledge base with statements that correspond to object-oriented concepts. Knowledge bases are highly amenable to logic processing with systems such as Prolog (for backward chaining) and CLIPS (for forward chaining). Using these tools we can easily react to change in our knowledge base (run rules), and also perform a significant level of ad-hoc querying. Because we are expressing our data and metamodel in the same way, we can easily react to changes in our metamodel, as well as changes in the data.

As important as a flexible metamodel is, it is equally important to be able to distribute that knowledge and work with it in applications that span large areas, geographically or otherwise. When we condense our metamodel and model down into statements, we only need to develop mechanisms that can distribute the statement information. Once we have successfully done that, all our knowledge and metamodels are easily transferred. Distributed propagation of a complex object model is very difficult and error prone. Numerous point mechanisms are usually constructed and the “hidden corners” are what bites back in the deployment of such a system. We choose instead to propagate our statements, and the edits to those statements. This is a substantially easier problem, as a quick recitation of Linda tuplespace history will show (covered later).

Finally, straightforward mappings of knowledge to XML are available (RDF is one such mapping). Once the knowledge base has been queried and transformed into XML, a whole host of capabilities become available for generating pages and other kinds of displays that can be presented to a user (in XUL, HTML or XHTML form).

We do not need to transform into XML before presenting to a user, though – we can create widgets that are able, through adapters, to interoperate directly with the knowledge base the application contains. When a user works with the application, we make appropriate changes to the knowledge base. Our rules engines and other user interfaces react appropriately.

Ted Nelson (of hypertext fame) said this:

``Intertwingularity is not generally acknowledged -- people keep pretending they can make things deeply hierarchical, categorizable and sequential when they can't. Everything is deeply intertwingled.''

I can’t vouch for the presence of the word intertwingularity in a dictionary, but it should be there! What I am trying to do in this book is describe are some of the ways of creating this intertwingularity and working with it to create real applications. At the least, I hope to change the way you view object models, and open your mind to the simpler and more powerful techniques that pure knowledge representations offer.

In other words, everything you know is wrong. Well, not everything, but enough of it that I hope you will have at least a few lightning bolts strike in your mind as you engage it on what follows…I know that as I learned of these ideas for the first time a few brain cells were singed.

Wed, 10 Sep 2003 02:52:29 GMT

Interdependency vs. Reuse.

The proliferation of pseudo-standard Java libraries, such as the various commons libraries, has me a bit nervous. On one hand we've got some extraordinarily useful stuff out there -- byte code engineering libraries, XML handling, mathematics, you name it. We Java programmers have riches that no other environment has, right now. We even have handy tools like maven, which are designed to have a look at what's being built and automatically download dependent libraries. I've never personally used Maven other than to build someone else's project; I'm not really a command line kind of guy, on a day-to-day basis. In my day job, we use an ant build system for the baseline and release builds, but nobody uses it incrementally (we all use Eclipse, except for this one guy who insists on using JEdit, and we all laugh at him but he's really good so we don't laugh too much).

It's just way too easy to pluck one or two bits of functionality out of a huge library, and drag a huge mess of dependencies and the accompanying work into the VM. Most of the time those .class files are going to sit there out of reach and you won't have to worry about them -- the VM isn't going to be doing much extra work. But are you sure? Have you watched the classloader to see what it is doing?

The internal dependencies of libraries are a funny thing. You never really know what you're going to be bringing in, when you touch something that you think it quite simple and quite isolated. Back in the Day, we used to have these funny things called linkers. You old guys know what I mean. You'd compile your stuff down into object files, link with library files, and voila, you'd have your nice tight executable. All is well. The thing is, the main function of a linker is the throw away the things you don't need (well, there are other things too, but it's still a main point). When the compiler and linker know what's being used, there's a lot of optimization that can take place.

Java, with its dynamic linking capability, is another story. You never really know what's going to be used. We have optimizers (like yGuard) that can do some of this optimization for us, throwing away some of what we need. Still, it's an uncomfortable thing. Too much gets dragged in; things just get too complicated.

The bottom line is, you've got a huge library of base functionality sitting there in J2SE. Consider using it as much as you can. Logging? Unless you have some pretty specific requirements, just use what's in there. Same goes for XML, regular expressions, collections, and so forth. Come up with a damn good reason before going outside the base kit.

Second, before pulling in something huge, evaluate your level of usage. Do you really need the whole thing? Can you just pull out a small part of it and use that? Seriously consider a little cut-and-paste reuse. You might just be able to get by quickly and easily, and not create additional dependency problems.

Third, be sparse in your use of code from another library. Restrict yourself to the smallest set of interfaces, and the smallest set of methods that you possibly can. Cutting these dependencies will pay dividends later, when that library morphs in a way you didn't expect, which happens with distressing frequency. I don't begrudge library authors their right to change; they can and should. It's simply a matter of balancing the cost of the various methods of adapting, copying, or reusing library code.

Keep in mind all three methods, and make conscious decisions.

Sun, 07 Sep 2003 08:28:08 GMT

On Checked Exceptions.

Significant disagreement in the blogosphere -- what a surprise! I've read a number of entries disagreeing with my approach to checked exceptions. Let's look at a few of the objections:

1. An exception subclass doesn't tell you what's wrong. Last time I checked, every exception supports the getMessage() call. There's nothing stopping people from implementing that and ensuring that good, accurate messages are present in string form. In fact, it's pretty much a requirement that they be there. So if you have a subclass you're able to divert control based on being a subclass, but also able to simply deal with a more general case of having a string-based error message.

2. It's all just a goto. I don't think it is. Exceptions do a whole bunch of things that goto simply doesn't do. Yes, we could simulate them. No, we shouldn't do that. The whole point behind checked exceptions is to ensure that things are cleaned up correctly when something goes wrong.

The main point is this: It is very rare that we actually care what has gone wrong. We just know that something has. What we really care about are the categories of responses we can make to a given kind of error. I think there is a significant case to be made for throwing separate exception families for a given response category. For example -- if you are iterating through a loop that is performing remote calls and you encounter a remote communications error, you probably want to exit the loop. You're not going to be getting anywhere further, and more calls aren't necessary. You need to engage a certain logic pathway to deal with that. If your iteration call encounters some kind of localized, "I can't deal with this object" error, then you might want to skip over that specific object and move on to the next one.

There are various forms of error response, each divided by category. One size does NOT fit all. And in many cases, we want to ensure that the appropriate handling has take place. Checked exceptions are really the only static way to get at this problem. On projects of any significant size, static methods are the only way to succeed. Putting an app into production and then hoping you have all your RuntimeExceptions caught is a guaranteed way to fail.

I use RuntimeExceptions for one thing -- a problem that should NEVER happen in a production run. Sometimes we will choose to ignore the problem and move on, but it signifies a bug in the software.

Checked exceptions are for everything else -- a RARE event, something that the developer acknowledges COULD happen in a normal production run. These are preferably categorized (via subclassing or some other mechanism) by essential response characteristic.

Thu, 28 Aug 2003 02:07:44 GMT

Public Commentary on Patents.

Yeesh. These guys, bless their hearts, are out there attempting to patent LISP! Well, not exactly -- they're trying to patent the use of XML as a LISP machine. Essentially what they've done is dumb down s-expressions into an XML form. It's amazing sometimes that when you navigate your way through a problem you end up right back where you started. At each step of the game you found something new, something incredible -- an excellent new thought or path. And then it all led back to the beginning.

Thus it is with LISP. Now there are things I don't like about LISP. It's syntax, while incredibly clean, simply doesn't excite too many pattern recognizers in the brain. Everything is a parenthesis, and that's tough when you're reading. I wrote a while back on cognitive signposts in computer languages, and boy could LISP ever stand to have a few more of them. A sort of "pretty LISP" if you will.

There's another way to solve that, of course -- instead of doing our editing with fixed column width editors that just colorize what we're looking at, do actually desktop-publishing style layout, on the fly, on the LISP code. That would be cool. It would be a kind of literate programming...more like art programming. But I digress.

The point of the patent post is this: The patent office has already managed to put pretty much everything online, so the public can at least view these stupid patents. Why not go the extra step and allow the public to comment on what they see? Why shouldn't the public be able to point out prior art, make commentary on the patent, and basically try to thoroughly invalidate these things early in the process?

We should be able to get involved. Companies lie about their patents all the time, during the application process. The patent process presumes the honesty of the applicant, and that is simply not the case, much of the time. A company will deliberately hide the origin of the patent, fail to point out prior art to the examiner, and other such shenanigans. We've got to start using the internet to put things back in balance.

Thu, 28 Aug 2003 01:58:58 GMT

Anders Hejlsberg is Wrong on Checked Exceptions.

Boy, I can't believe I just wrote that. He's got the quote of the day on Cafe au Lait, from something called "The Trouble with Checked Exceptions". He gives two arguments on why checked exceptions are a bad idea. Versioning is first -- he says that if you need to throw a new exception from your library, it has to go into the signature, making it incompatible. Second, when combining together multiple libraries, you end up with a kind of cross-product situation for catching exceptions. If your librarty calls another, you need to add the other library's exceptions to your own throws list.

He's wrong on both points. First, most libraries will declare a general exception class, like LibraryException, and build all their more specific exceptions using subclassing. Most methods will throw a LibraryException, and specialize from there if necessary. Problem solved. You can pay attention to the messages and concrete subclasses if you want to, but you don't have to.

Second, chained exceptions are an effective way of dealing with the library combination issue. You capture a sub-library's exception, then wrap it with one of your own. As a library, your use of a sub-library is something that should be transparent to the calling system. Chained exceptions allow you to convey the right information back in a general form.

Java provides good tools for handling exceptions. Languages that don't check them (which is basically everything else out there) just kind of scare me right now. How do you know you've got everything handled? The answer is...you don't.

And centralized error handling in a server-based application isn't necessarily practical.