Devoxx highlights

In order to embrace the true Christmas spirit, I thought I’d share a few goodies from the Devoxx conference that took place in Antwerp, Belgium in mid December. Devoxx is the former JavaPolis that has changed name due to trademarking issues with Sun. Nonetheless, it’s still the worlds largest independent Java conference, where the 3200 attendees find themselves being literally showered with news from the broad Java community. Below follow summaries from some of the, IMHO, best sessions from this years conference. Enjoy!

BDD in Java with easyb (John Ferguson Smart)

John started out by arguing that TDD is not about testing, it’s about writing better code. That is, it’s about making your code more maintainable, flexible, reliable and simple. TDD is forces the programmer to think about how to test the class before actually writing the class. BDD, on the other hand, helps to determine what to test and to write more focused code, by starting with the behavior. Instead of thinking that you’re testing your class, you should think that you’re validating your requirements!

easyb is a BDD testing framework for Java, written in Groovy (hence providing full access to all Java API’s). It ensures that tests become clearer and easier to both write and read! It helps the developer to focus on the requirements only. A competitor to easyb is JBehave, which is an extension to JUnit. However, according to John, JBehave is more cumbersome than easyb.

easyb makes use of stories, much like the story cards in the Agile world. In a story, a narrative approach is used to describe a precise requirement. In easyb, a story could look like this:

This scenario is quite clear and readable, isn’t it!? Furthermore, it’s self-documenting – a very appreciated feature =) And the syntax is so trivial that even a non-technical stakeholder would have no problem understanding it. The developer can then reuse the given scenario (pseudo) code and implement the test case, like this:

The shouldBe call is an intuitive and readable way to verify that the requirement holds. Another way to verify the outcomes is to use the ensure syntax. Example:

or

easyb could be extended to use DBUnit (and further plugins for Grails and Excel are on the way). On the IDE support, the only good option is IntelliJ IDEA. It really comes down to the Groovy support, and Eclipse and NetBeans doesn’t shine here. In those cases, you’re better off running the tests in Maven or Ant, which both are well supported. In the upcoming version, easyb will have full support for Continuous Integration (CI), enabling automatic test execution in a CI-server such as Hudson.

Furthermore, easyb comes with a web application – Easiness (cmp. Fitnesse) – where stakeholders can create stories in plain text. The developers can then go ahead and implement the stories defined by the stakeholders. This is an important features, that further improves the cooperation between the customer and the provider. It really seems like a quite nice little framework! If you agree, have a look at it at http://easyb.org/.

From Concurrent to Parallel – Library-based parallelism in JDK 7 (Brian Goetz)

Brian is an expert in the concurrency area, but he’s not a good presenter! He speaks faster than I can read, and his slides are so packed with info that he doesn’t have time to say the half of it. That put aside, in this very interesting session Brian presented some really cool concurrency features to be included in JDK 7.

As you probably know, JDK 5 introduced a set of very useful classes for course-grained parallelism. JDK 7 will introduce a framework for fine-grained parallelism – the “fork-join” framework.

We have now reach a point in time when Moore’s laws isn’t valid anymore. The new reality is not faster CPUs but more CPUs. To adapt to these changes, programmers have to change the way they think about parallelism. The unit of work (UoW) that represents a task to be solved must be split into even smaller parts to be able to keep all hardware busy. Nowadays, it isn’t enough to for ex. spawn a new thread for each service call in a server, the UoW must be split up even further.

Examples of finer-grained parallelism is searching, sorting and filtering a data set. Such a task is preferably solved by dividing it into sub-problems and combining the sub-results at the end. We could use the course-grained concurrency tools (for ex. Executor + Future) introduced in JDK 5 to solve this problem. However, with multiple CPUs available, these tools doesn’t perform optimally. For ex., the shared work queue in the Executor becomes a bottle neck when the number of threads/CPUs increases. Furthermore, these tools provide no form of load balancing, i.e. if one thread finishes before the others, it is never reused.

The divide-and-conquer algorithm could also be applied to solve the problem. This great advantage of this approach is that it’s independent of the number of CPUs used! Using plain threads for this approach is too expensive though, since thread creation costs way too much. However, the fork-join-operation in JDK 7 solves problems like these using the divide-and-conquer algorithm in a much more effective way! Internally, it uses an slim thread pool to overcome the cost of thread creation. Furthermore, no data-copying penalty exist since the data set (for ex. an array) is not divided/copied during the solution. Instead, the indexes within the data structure is used to define each data subset. Another advantage is that the code doesn’t know how many CPUs it executes on, hence it’s portable!

While the Executor (JDK 5) should be used for tasks consisting of both IO and computing, the fork-join-operation (JDK 7) should be used for highly compute-intensive tasks. Examples of such tasks are matrix operations, numerical integrations and game playing. The JDK 7 class to be used for these kind of problems is ParallelArray. It’s a utility class that greatly simplifies the work of solving a task in parallel. You can think of it as an in-memory database for data sets. Typical usage is:

The ParallelArray class provides operations for:

  • Filtering (nestable)
  • Mapping
  • Replacement
  • Aggregation (max, count, sum, …)
  • Application (one action performed for each selected element)

To compare, the ParallelArray class acts much like the Map-Produce concept introduced by Google but it works in a different domain (local system instead of in a distributed system).

The only downside really is that JDK 7 isn’t scheduled until mid 2010, meaning we’ll have to wait quite some time before we can actually use these new concurrency features. But from the looks of it … it may be worth the wait!

The SpringSource DM Server (Joris Kuipers)

The reasons why SpringSource decided to develop a new application server were many. Today, modularity is important and developers spend much time on achieving it in their applications (layering, separation of concerns, etc.). However, when the application is deployed, all modularity is lost – the deployable unit is one huge, monolithic WAR file. That means that sharing code (let alone services) between applications become hard if not impossible. In order to perform the smallest update on the application, whole WAR file have to be redeployed. Furthermore, the servers themselves lack modularity; they come prepackaged with a lot of modules but there’s no way to configure that your application only make use of a couple of them. Hence, they are all left in there, consuming unnecessary CPU cycles and memory.

The DM (Dynamic Modules) server was developed to tackle these issues of non-modularity. In order to achieve true modularity, is has been built on top of OSGi (namely the Equinox container). Since OSGi can be quite hard sometimes, the DM server conceal the complexity of OSGi in the same way as the Spring framework does with for ex. Remoting or JMX.

In order to be able to use a third-party library from within the DM server (and from any other OSGi containers), the standard JAR library needs to be extended with OSGi-specific meta data – the OSGi manifest. SpringSource has therefore created the Enterprise Bundle Repository, where they have started to upload and publish OSGi-ified Java EE libraries. If you’re using Maven or Ivy, the bundle repository can easily be configured as a remote (or local) repository.

The DM server also introduces a couple of new OSGi import constructs, enabling ease of development. One example is the “Import-library” that can be useful if you want to make use of a framework (for ex. Spring) but don’t want to waste your time having to specify every single Import-package (for example Spring-Remoting) within the framework (as you normally would have had to do). This doesn’t affect performance improvement since (thanks to OSGi) none of the pre-installed bundles are loaded before they are actually used! Furthermore, with the DM server it’s now possible to remove the library bloat of monolithic Java EE WARs. Instead of building the external libraries into your WAR, you can instead declare them as dependencies in the OSGi manifest headers. The server will then convert these into plain “Import-package” constructs during deployment. That way, the size of the built/deployed WAR can be reduced significantly!

In the DM server, one can of course deploy plain OSGi bundles but also WARs, EARs etc. Typical scenarios for deploying a plain OSGi bundle are stand-alone libraries, global services and small stand-alone applications. However, being able to deploy a bundle is not enough in most cases … A normal application normally consist of multiple bundles, which becomes hard to un/deploy since there is no single deployable unit. Furthermore, with multiple bundles you don’t get a common log file or a notion of an application scope. To tackle this issue, the DM server adds the notion of an application by introducing the PAR (Platform Archive) format. It’s basically a JAR with a number of ‘Application-*’ manifest headers. You can think of it as an EAR in Java EE. PARs are versioned and the versions apply to all of their bundles. This mean that one can deploy a PAR twice with different versions but with the same bundles, without conflicts.

The DM server comes in several versions. Of course there’s the community version, using a GPL and SpringSource license. Then there’s the commercial license where full support from SpringSource is included. On the extension side, better Maven support will soon be released so that the dependencies can be specified in a single place (not like now when you have to duplicate them in the pom and in the manifest). There’s a Eclipse-plugin available that enables re/deployments from within the IDE.

These are some of the features in the DM server. There are plenty or more cool ones, read all about them here. There, you’ll also find a lot of sample applications that can get you going! With the DM server, SpringSource feel that they are filling the (up until now) empty space of server side OSGi and states that “The DM server is the healthy new way to run your apps!”. I must say it looks very promising! Why don’t you go ahead and try it out!? I know I will!

If you appreciated these posts, you will definitely enjoy next years conference! Pick up your calendar and reserve December 2009 for a great week at Devoxx!

This Post Has 3 Comments

  1. Actually, Moore’s law is defined in terms of transistors and continues to hold strong -> CPU speeds are now pretty flat and likely to remain so, but the extra transistors are going to more cores instead.

    Also, the current proposal is that fork/join would be in Java 7 but the ParallelArray portion would be external to the JDK.

    Nice writeup!

  2. HAPPY NEW YEAR! Brothers ;))

    And good luck to all of you!

Leave a Reply

Close Menu