Conversations Driving Actions

The conversation on a Pile of Lamps continues...

Bob Warfield chimed in with his view on Pile of Lamps, which includes a mapping of the style into the Roy Fielding's assessment framework. Perhaps most importantly, he reminds us that Roy Fielding's contribution is not limited to REST - Roy's dissertation provides a useful tool for thinking about web architectures in general.

Bob's remark about whether a Pile of Lamps was implemented in a grid or cluster struck my fancy. A recent, yet unrelated, conversation on conversations has also been lingering around in my brain. Like two molecules floating around, the two ideas combined.

Techniques for managing large sets of machines tend to either highly centralized or highly decentralized. Centralized solutions tend to come from system administration circles as ways to cope with large quantities of machines. Decentralized solutions tend to come from the parallel computing space where algorithms are designed to take advantage of large quantities of machines.

Neither approach tends to provide much coupling between management actions and application conditions. Neither approach seems well adapted for any form of semi-intelligent dynamic configuration of multi-layer web application. Neither of them seem well suited for non-trivial quantities of loosely coupled LAMP stacks.

I've recently been contemplating a constellated approach, where various subsets of machines engage in conversation amongst themselves.

As an example, a set of application servers might engage in a conversation concerning their general status. A failover heartbeat is a simple form, but the idea encompasses larger classes of interchange and generalizes the model. If the servers start to become overloaded, the conversation might converge on a decision to spawn a conversation between services capable of contributing to a solution. That conversation could, for instance, result in a decision to add another server to the pool.

Once an additional server is made available, the application pool would be responsible for initiating the provisioning of the new server and for integrating it into the pool. The information necessary for proper provisioning are and integration are contained within the application pool rather than some external source.

Basically, it's a pull rather than push model, with localized conversations driving actions.

A fully developed model for inter-machine decisions would most likely become unbearably complex and subject to emergent behaviors, but a basic implementation should be capable of providing an adequate framework for many situations.

I haven't completely thought through the idea, but I though I'd toss it out for comments.