BasePatterns.org

Ok, so it’s been a while since I paid any attention to this blog – not that I have nothing to say, but rather that I couldn’t decide what demands the most attention. I also want to ease up on the hostility expressed in a number of previous posts, as there’s probably enough people complaining about the industry without me making more noise..

So I think what I need to do is focus on a single topic for posting about, and in the spirit of this blog’s title I have decided to document my experiences creating the Next Big Thing in software.

For quite a number of years (perhaps as far back as the beginning of my career about ten years ago) I have been particularly unimpressed with the features provided by the second most ubiquitous software category (the web browser being the first), the email client. Over the years this dissatisfaction has grown to include all so-called “groupware” clients, notably infamous among these being Microsoft Outlook. So in my spare time over the years I have explored and experimented with what it would take to write an Outlook killer (for want of a better term). Being that I have used Java almost exclusively for my entire career, I had confidence in language features and the growing list of third-party libraries to be able to produce a complex rich client.

With an attention span challenged at the best of times, often my focus has spun off on tangents – particularly when Java proved to be too immature for certain critical features of an email client (a situation that may be argued still exists today). Occasionally these tangents have proved to be not a total waste of time, with projects such as iCal4j and mstor resulting from these divergences.

The long-term goal however, remained, and in the last couple of years a number of developments have arisen within the Java community that I believe have been of critical importance to developing complex rich client Java applications:

• First, there have been dramatic improvements with desktop Java performance in Java 5, and even more so in Java 6. Hopefully we can expect even greater performance from the upcoming Java 6 Update N, and Java 7
• Second, I believe modularisation plays a key part in building complex systems, whether they be client- or server-side. Taking modularisation to the next level is the Open Services Gateway Initiative (OSGi), which I think could change the way we develop applications in the future
• Finally, and I really mean finally (!!), we now have a better way to build Swing GUI interfaces in JavaFX Script. Admittedly this technology is still very new, and to be honest the implementation is not even finished yet, however with my limited exposure I am already feeling much more comfortable with writing complex rich client interfaces

And so armed with a new set of technologies, I am once again attempting to make progress on this Next Big Thing. Along the way I will be documenting progress, insights and frustrations, in the hope that something of value may find it’s way into this blog. Stay tuned.

Design, Software

When designing a software architecture it is important to maintain a focus on the problem you are trying to solve. Without this focus you will most likely diverge from the original goals, resulting in a solution that may not represent the best approach to solving the problem at hand. Often is the case that JEE architects lose sight of the problem domain in favour of the technologies employed in the end solution.

The Java (2) Enterprise Edition (JEE) is partly to blame for this loss of focus due to the rigid structure defined in the specification. When a framework such as JEE imposes such tight control over application design, typically you will see every application implemented the same way. The primary goal of JEE was to provide a standard approach to enterprise application development that, whilst particularly useful for those unfamiliar with enterprise architectures, has resulted in a single “flavour” of application design that is not necessarily optimal for all problem domains.

Consider banking systems. Aircraft control systems. All of these problem domains will typically find certain architectures more favourable than others. Embedded systems obviously aren’t interested in JEE-sized applications, whilst enterprise applications need enterprise-quality scalability. From this kind of logic has emerged the different flavours of Java: JSE as the basis for all things Java, the superset JEE, and the subset JME. Unfortunately this model brings an assumption that problem domains only differ in terms of size, power and scalability, which is clearly not the case. If we consider the needs of an insurance company, in terms of size, power and scalability it may not be very different from the needs of an online bookstore, however the architectures most optimal to each domain are bound to be quite different.

If we consider the online bookstore example, you might assume that they may require some kind of supply-chain management software to handle the placing of orders through to delivery of the product. In addition to the typical enterprise-scale requirements, such a system might also be centered around a messaging or ESB technology. Alternatively, if we look at the insurance company’s requirements, we might find a need for a policy management system, responsible for generating premiums and managing renewal periods. While the policy management system may also depend on messaging technologies for communication with legacy systems, the focus of such a system is more likely to be the business logic and underwriting rules necessary for policy management. Now of course a JEE application can be implemented and deployed for each of these particular scenarios, however in doing so we lose focus on what is important to each of these systems. At the same time we are deploying technologies that aren’t even used in the problem domain, simply because they come bundled with the application server, further increasing the “clutter” and distracting us from the end goal.

Without having a clear focus of the goals of a software system, we find ourselves simply “shoe-horning” the implementation into the simplest configuration of a JEE application. Of course some JEE architects do try to model the application on the problem domain, being so bold as to implement multiple EJB modules in a single enterprise application. Multiple web applications are less common, typically because inter-web application communication is so difficult (something that portals tries to rectify, however the fact that it was “taped on” as an afterthought isn’t really optimal), which ultimately means that web applications are huge and unmanageable, even if the architect has the foresight to separate the business logic from the user interface.

In fact many JEE architects have discovered that the most configurable way to implement an enterprise application is as a collection of JEE applications. Each application might contain a single EJB module, or a web application, or web services. This approach will only become more prevalent with the popularity of Service Oriented Architectures (SOA). Assuming this trend does continue enterprise applications will ultimately become redundant, serving only as a defunct wrapper to the actual module contained within. Add this to the list of other failed technologies (Entity and Stateful EJBs), and JEE starts to become made up of more legacy technologies than useful ones.

There is a lesson to be learned from JEE, and that is if you try to be everything to everyone you will end up being not particularly useful to anyone. As this realisation is made more apparent with every new release of the JEE specification, we will ultimately see a rejection of JEE as a whole, instead being replaced with architectures employing only those technologies required to meet the goals of the project. Such architectures will not necessarily resemble a traditional JEE design, but will rather be designed specifically for the problem domain. With the resurgence of such architectures of many different flavours will come a renewed demand for true software architects, and will hopefully result in software designs more suited to the task at hand.

Design, Software

If there is one thing in abundance in software development it is opinions. From which technology to use (languages, frameworks, libraries, etc.), to architectural design (everyone’s an architect!), to the user interface layout, it is never hard to find opinions.

One thing that is rare however, is a firm grasp on reality when it comes to these opinions. It is common for developers to have difficulty conceptualising how long it will take to implement a requirement. As a result they will often “aim too high”, setting goals that are surely not achievable within the given time constraints. The end result will be that even if the requirement is implemented, it will usually be over time, over budget, and of poor quality due to the pressure of looming deadlines and the abandonment of adequate testing (which is, without fail, always the first casualty).

Also common amongst software developers is the resume-friendly implementation. How many times have you seen an implementation use AJAX where it is completely unnecessary? I know that I have seen a few. Sure a technology might be cool, and developers do need to keep their skills up to date, but is it the right tool for the job? Will it realistically integrate with existing infrastructure with the given time-frames? Using the wrong tools to implement production systems is ultimately irresponsible, as it can eventually lead to lower quality software and higher maintenance costs.

Typically you might expect a greater level of responsibility from software architects, however this is not always the case. Unlike developers, architects need to be aware of the entire system infrastructure, and as a result may make safer decisions when it comes to technology choices. However when it comes to the system design, more often than not the architecture will include needlessly complicated configuration options, allowing for the greatest number of scenarios possible. Maybe the architect’s model is sufficiently complex to allow it to be used in many different configurations, but how many configurations do you actually need? If the software is designed properly, shouldn’t you be able to add support for additional configurations at a later time as required? This kind of unnecessary complication is also often a major factor in implementation issues.

One of the biggest failings of the modern software development process is the disregard for Human/Computer Interaction (HCI) experts. Once upon a time people with a good knowledge of interface design were employed to do just that. More recently the system users feel they are qualified to do it themselves (how hard could it be?), leading to glaring inconsistencies and ultimate confusion for the end user (“this isn’t what we wanted” – “but you designed it yourself!”). Or in the slightly less disastrous situation the developers are given free reign to design, and whilst the user interface design may be fine for developers, will it be easy to understand for the end users?

Having access to a diverse range of opinions is ultimately a good thing, but you also need people that can adequately filter these opinions in order to recognise the approaches that are in fact realistic and will achieve the desired goals. Such an ability takes a good deal of self-discipline and objectivity, which is unfortunately not something that is all too common in Information Technology circles.

Build, Design, Software

In the search for methodologies and patterns that help to build better quality software, we find that our methods continue to evolve at a blistering pace. Whether its the leap from procedural to object-oriented design, or the difference between the waterfall and prototyping approaches, all share the common goal of building robust, complex software systems. Whilst many great advances continue to be made, one thing that eludes us in the search for the perfect development strategy is how to future-proof a software architecture. We like to think that we will have all the requirements and intended uses of the software prior to design and implementation. In reality however, this is almost never the case. Requirements change, focus shifts, often to the point that the final requirements bear little resemblance to those the original design was based on. Ultimately a good design should be able to handle such a shift in requirements, however even the best designs will not cope with a major refactoring of the intended uses of a system.

So how can we protect our software designs from such a seachange in required functionality? In addition to accepted practices and methodologies, we should try to incorporate the following principles into our architecture:

• Think Small Usually when we design architectures the components tend to be large and provide support for a multitude of features. It is not uncommon to even have a single component representing the front end, back end, or business rules of a system. Combined with the principle that a good design will explicitly identify each component’s roles and responsibilities, we find our components are too rigid and not really amenable to changes in these roles and responsibilities. The solution to this problem is to build smaller, more specific components. Rather than components handling many different tasks, we should design the architecture in such a way that each component is responsible for handling a single task. Such tasks may include support for a specific protocol or data format, a service used to access an external or legacy system, or even data structures specific to a particular feature in the system. The important thing to remember is that a component should only handle a single task rather than a collection of them as is the popular approach.

• Separate APIs from Implementation Another recognised good practice is the use of Interfaces when designing an API allowing components to communicate with each other. By using interfaces we are effectively defining the contract a component holds with the rest of the system, irregardless of the implementation details. Whilst this allows us to provide alternate implementations of the API without further changes to external components, we still generally need to provide an entirely new version of a component – even if the API itself has not changed. Although the API is not bound by the implementation, usually the entire API (interfaces and implementation) are bundled together anyway. The solution then is to separate the API definition from the implementation and provide them as two distinct components. By doing this the API specification does not require updating where only the implementation requires changes. Additionally, alternative implementations of the API may also be deployed independently, thus maximising the possible re-use of a component API.

• Loose Coupling A major contributing factor to the complexity of software are the dependencies each component has on its peers. Aside from the careful design decisions required to ensure no two components are dependent on each other (i.e. cyclic dependencies), typically where even a single dependency is not resolved the entire system will fail. To avoid architectures easily susceptible to such a catastrophic failure we need to maximise decoupling of components. This does not necessarily equate to minimising dependencies across components, but rather is more concerned with ensuring that such dependencies impose as little restrictions as possible. This is the promise of Loose Coupling, whereby individual components may operate independently or as a part of a larger system constructed of similarly loosely coupled components. One popular approach to incorporating Loose Coupling in software design is the Inversion Of Control pattern (also known as Dependency Injection). A number of frameworks based on the Inversion Of Control pattern are now available, which assist in the “wiring together” of individual components to construct a complete system. This encourages greater encapsulation in components and simplifies other aspects of quality assurance such as unit testing.

By adhering to these principles we are able to design software that can adapt to just about any change in direction required of it. No longer do we need to throw away an entire system and start from scratch, as we will be able to incrementally upgrade and replace individual components without detriment to the rest of the system. This approach also enables us to extend the functionality of the system by “wiring in” new components as required. Best of all, these principles ensure that the software architecture is built from clearly defined design contracts and encourages maximim re-use.

Design, Software