Wednesday, January 15, 2003

Distributing Prevayler-like Commands. I built something like Prevayler about a year ago, except that it maintains distributed knowledge stores.  It's kicking around statement edits, basically.  I can have a bunch of tuplespaces locate each other through JINI...they are able to relay commands to each other when other processes do things (like write or remove statements).  The commands are kept in a buffer.  The trick is, how do you guarantee that each instance executes the same commands in the same order? The answer is what I called a time ordering protocol, but I'm sure other people have named it something else.  Basically I try to get my peers to agree on the current time, but I really don't care if they do -- they just have to be reasonably close to each other.  Each peer has its own unique id.  I sort the commands/operations by time first, and originating peer id second. Presto -- completely sorted lists of commands, around my entire set of peers.  What happens if I receive a command that needs to be sorted earlier than the latest one I have?  I pop off the commands, one by one, inserting them into my wait list, undoing them, until I can insert the command I've received.  I can do that because my do method keeps enough information to undo if necessary.  I then sort the list of commands I have waiting and execute them all. Works pretty well.  Yes, the checkpointing and all that seems to work. I think that something like Prevayler and what I implemented back then are OK, but they don't really scale all that well.  What happens when you need to have 20 or 40 GB of information?  Can't keep that in memory, but you still need to get the job done.  At that point you're writing a compressed paging store, assigning object ids as your commands are executed, and pushing things out of memory and onto disk when you need to. Still, you need a lot of data before you get to that point. 11:25:03 PM