But lets take it from the beginning.

Part 1: Why builder pattern?

Suppose you want to create a class name Address. And suppose you want it to have fields like protocol, url, port, path and description. Now, you want to give your users the freedom to choose which arguments to pass when creating an instance of the class. Also, you want to make your class immutable, so set methods are out of the question. You might start up with something like:

public class Address {
    private String protocol;
    private String url;
    private int port;
    private String path;
    private String description;

    public Address(String url, int port) {}
    public Address(String protocol, String url, int port) {}
    public Address(String protocol, String url, int port, String path) {}
    public Address(String protocol, String url, int port, String path,
        String description) {}
}

...but you start realizing there is a problem here. First, maybe you need more combinations, like creating an instance given an url, a port and a path, but - both url and path are Strings, and you already have a constructor which takes two String and an int, and, hey, you know you can't have two constructors with the same signature! Secondly, you realize there's simply too many constructors here, and it reminds you of something you read once in a book about code smells. The book suggests solving this by replacing constructors with creation methods. Using this tip, your class might look like this:

public class Address {
    private String protocol;
    private String url;
    private int port;
    private String path;
    private String description;

    // no one should use this outside your class
    private Address() {} 

    public Address createAddressWithProtocolUrlAndPort(
        String protocol, String url, int port) {}
    public Address createAddressWithProtocolUrlPortAndPath(
        String protocol, String url, int port, String path) {}
    public Address createAddressWithUrlPortAndPath(
        String url, String path) {}
}

...but this is also tedious. There's simply too many combinations. And you need to create an endlessly amount of factory methods to allow all of them. It is then that it strikes you! Why didn't you consider using the builder pattern! This way, you don't have to create loads of constructors or creation methods, and still allow the user to create an immutable instance of your class while freely choosing which arguments to use. Your beautiful new class now looks like that:

public class Address {
    private String protocol;
    private String url;
    private int port;
    private String path;
    private String description;

    // only builder should be able to create an instance
    private Address(Builder builder) {
        this.protocol = builder.protocol;
        this.url = builder.url;
        this.port = builder.port;
        this.path = builder.path;
        this.description = builder.description;
    }

    public static class Builder {
        private String protocol;
        private String url;
        private int port;
        private String path;
        private String description;

        public Builder protocol(String protocol) {
            this.protocol = protocol;
            return this;
        }

        public Builder url(String url) {
            this.url = url;
            return this;
        }

        public Builder port(int port) {
            this.port = port;
            return this;
        }

        public Builder path(String path) {
            this.path = path;
            return this;
        }

        public Builder description(String description) {
            this.description = description;
            return this;
        }

        public Address build() {
            return new Address(this);
        }
    }
    // TODO: add some get methods here...
}

...and you can create a new instance of Address by calling:

Address address = new Builder().url("www.google.com").port(80).build();

Happy? Very. But there's just one tiny problem. What happens if the user of this class tries to create an Address without an url or a port? You simply cant allow that now, can you? So you start thinking. And you realize you simply cannot find a good solution. So you ask a question in Stackoverflow and hope someone would give you a good solution. Most of the answers you get there don't help at all, but after a while, someone called pgras gives you this fantastic solution, and, well, you like it so much you decide to write a blog post about it

Part 2: Mandatory methods in builder pattern

The solution is to change you builder into a state machine. Each mandatory method in it leads you to the next step, until you get to the last step which includes both non-mandatory methods and the build method itself. It uses interfaces to define the different steps, and, well, it look like that:

public class Address {
    private String protocol;
    private String url;
    private int port;
    private String path;
    private String description;

    // only builder should be able to create an instance
    private Address(Builder builder) {
        this.protocol = builder.protocol;
        this.url = builder.url;
        this.port = builder.port;
        this.path = builder.path;
        this.description = builder.description;
    }

    public static Url builder() {
        return new Builder();
    }

    public static class Builder implements Url, Port, Build{
        private String protocol;
        private String url;
        private int port;
        private String path;
        private String description;

        /** Mandatory, must be followed by {@link Port#port(int)}  */
        public Port url(String url) {
            this.url = url;
            return this;
        }

        /** Mandatory, must be followed by methods in {@link Build}  */
        public Build port(int port) {
            this.port = port;
            return this;
        }

        /** Non-mandatory, must be followed by methods in {@link Build}  */
        public Build protocol(String protocol) {
            this.protocol = protocol;
            return this;
        }

        /** Non-mandatory, must be followed by methods in {@link Build}  */
        public Build path(String path) {
            this.path = path;
            return this;
        }

        /** Non-mandatory, must be followed by methods in {@link Build}  */
        public Build description(String description) {
            this.description = description;
            return this;
        }

        /** Creates an instance of {@link Address} */
        public Address build() {
            return new Address(this);
        }
    }

    interface Url {
        public Port url(String url);
    }

    interface Port {
        public Build port(int port);
    }

    interface Build {
        public Build protocol(String protocol);
        public Build path(String path);
        public Build description(String description);
        public Address build();
    }

    // TODO: add some get methods here...
}

Using it is simple:

Address address = Address.builder().url("www.google.com").port(80).build();

Try using it without the mandatory methods, and you get a compilation error:

Autocomplete is also helpful here, and will only give you the correct options:

So there you have it. The API user can now use your friendly API to create a robust and immutable class knowing that she'll get all the help needed along the way, using autocomplete and compilation errors if she did something she wasn't suppose to do. Nothing here can go wrong!

Thanks again, Stackoverflow and  pgras for this nice solution.

Have a look at the following to see what happens:


> db.Person.save({"name" : "foo" })
> db.Person.ensureIndex({"name":1})
> db.Person.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "test.Person",
"name" : "_id_"
},
{
"v" : 1,
"key" : {
"name" : 1
},
"ns" : "test.Person",
"name" : "name_1"
}
]
> db.Person.save({"name" : "foo" })
> db.Person.find()
{ "_id" : ObjectId("4ee9ab58e57f00e1f3ca95c7"), "name" : "foo" }
{ "_id" : ObjectId("4ee9ab63e57f00e1f3ca95c9"), "name" : "foo" }
> db.Person.ensureIndex({"name":1},{"unique":true, "dropDups":true})
> db.Person.find()
{ "_id" : ObjectId("4ee9ab58e57f00e1f3ca95c7"), "name" : "foo" }
{ "_id" : ObjectId("4ee9ab63e57f00e1f3ca95c9"), "name" : "foo" }
> db.Person.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "test.Person",
"name" : "_id_"
},
{
"v" : 1,
"key" : {
"name" : 1
},
"ns" : "test.Person",
"name" : "name_1"
}
]


Next, imagine that the Facade class and the Delegate interface below are part of a bigger system:

Create a matching easymock-test-config.xml (or mockito-test-config.xml).

Now, we have all the bits and pieces that are needed to start writing the integration test. We start simple to make sure that we autowiring and Spring context has been configured correctly:

The test is executed and it passes without any problem.

A second test is added to the same file with some added mock behavior:

Excellent, that test also passes.

Now that we have warmed up, we add a third test to the test class:

As expected, this test also passes without any problem.

Unfortunately, that is not always true. A closer investigation reveals that all tests pass when they are executed individually. When executing all three tests (either from the IDE or from a build tool like Maven), the third test fails unexpectedly:

java.lang.AssertionError:
  Unexpected method call Delegate.doSomethingElse():
        at org.easymock.internal.MockInvocationHandler.invoke(MockInvocationHandler.java:44)
	at org.easymock.internal.ObjectMethodsFilter.invoke(ObjectMethodsFilter.java:85)
	at $Proxy8.doSomethingElse(Unknown Source)
        [...]

This is EasyMock telling us that the mock object is in the wrong state. An EasyMock mock object goes through a series of steps:

1. Initialization - Instantiate the mock object.
2. Record - Record the expectations of the mock object.
3. Replay - Call replay() on the mock object so that it can replay the recorded state.
4. Test - Execute the test assertions.
5. Verify - Call verify() on the mock object to certify that the recorded mock expectations are fulfilled.

In an ordinary unit test, the mock object is thrown away after the verify phase, a new mock instance is created before the next test that can be used to record the new behavior. However, since we are creating a Spring integration test, there is one more thing to consider.

Reused beans

In the testing chapter of the Spring manual it is stated that:

By default, once loaded, the configured ApplicationContext and all of its beans are reused for each test method. Thus the setup cost is incurred only once (per test fixture), and subsequent test execution is much faster.

Consequently, all Spring beans, including mocked beans, will preserve their state between different test methods. The mock object will be in the verify state when the last test begins and not in the record state when delegateMock.doSomethingElse() is called. To solve the problem, we need some mechanism to ensure that the mock object is in a known, well-defined state before each test.

Reset mocks

Both EasyMock and Mockito provide methods to reset() the state of the mocked objects. Normally, resetting mock objects in the middle of a test method is considered to be a potential code smell, because it is likely that you are testing too much and that the test could be refactored into two separate tests accordingly. Nonetheless, resetting the mock instance @Before each test allow us to keep the same mock instance through the entire lifetime of the test class and yet put it in the record state when before each test starts:

When executing all tests in the test class, we see that that the problem has been solved and all tests pass.
For reference, click the link for the full source code of the EasyMock test class (and the Mockito test class).

Comments

You may choose to reset the mocks @After each test, the result will be the same.

The reset() method is called between the different test methods (because of the nature of @After and @Before), so the code smell of calling reset within a test is avoided.

Tests written using Mockito mocks are generally more tolerant for this kind of problem. In contrast to EasyMock, Mockito has no replay() method and no notion of steps. Therefore, it is possible to add additional mock expectations and to replace existing expectations, both after the test has been executed as well as after the result has been verified. Hence, you should always reset your mock objects before each test so that you do not get any accidental mocking behavior.

As a side note, another solution would be to annotate the test class (or all of its test methods) with the @DirtiesContext annotation. This will effectively recreate the entire application context after each test has been executed, causing all Spring beans to be in their initial state and thus the mock beans to be in the record state. Although this will work, there will be a significant performance penalty compared to the suggested reset() and @Before / @After solution, because all beans in the application context will be recreated and re-wired before each test method.

Dependencies

• Spring 3.0.6
• Mockito 1.8.5
• jUnit 4.10
• EasyMock 3.1

References

• Spring Reference Manual - Test Context Management and Caching
• Spring Reference Manual - SpringJunit4ClassRunner
• Spring Reference Manual - ContextConfiguration
• EasyMock
• Mockito

Previous post: Spring Integration Tests, Part I, Creating Mock Objects

@Entity
public class MyEntity {
...
    @Autowired @Transient
    private SomeService someService;

}

Notice the @Transient annotation that tells Morphia to avoid persisting this field! I solved this problem by adding an EntityInterceptor that autowires all entities:

public class SpringAutowiringEntityInterceptor extends AbstractEntityInterceptor {
	@Autowired
	private ApplicationContext applicationContext;

    @Autowired
    private Morphia morphia;

	@PostConstruct
	public void postConstruct() {
		morphia.getMapper().addInterceptor(this);
	}

	@Override
	public void preLoad(Object ent, DBObject dbObj, Mapper mapr) {
		applicationContext.getAutowireCapableBeanFactory().autowireBean(ent);
	}
}

Here is the complete source code SpringAutowiringEntityInterceptor.java. Let me know what you think!

(In this post, Mockito has been used for creating mock objects, but the same problem applies to EasyMock as well. You can find the corresponding files if you follow the provided links.)

Test setup

In the following example the SomeClass needs a reference to an instance of the SomeDependency interface:

When writing the integration test, we use the static Mockito.mock(Class classToMock) method to create a mock instance. The straightforward approach would be to let Spring instantiate the mock object with a static factory method in the xml configuration, e.g.

(Corresponding EasyMock config.)

To verify that the beans are created and wired correctly, we write a simple test:

(Same test for EasyMock, just update the @ContextConfiguration with the EasyMock config file.)

To our surprise, the test fails:

[...]
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: org.springframework.beans.factory.BeanCreationException:
    Error creating bean with name 'someClass': Injection of autowired dependencies failed;
nested exception is org.springframework.beans.factory.BeanCreationException:
    Could not autowire field: private com.jayway.example.SomeDependency com.jayway.example.SomeClass.someDependency;
nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException:
    No matching bean of type [com.jayway.example.SomeDependency] found for dependency:
    expected at least 1 bean which qualifies as autowire candidate for this dependency.
    Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
[...]

Explanation

In order to find a solution to the problem, we need to understand Spring's initialization process:

1. Load bean definitions.
   This step includes scanning the application context XML files, scanning packages defined by component-scan, and loading the bean definitions found into the bean factory. (BeanFactoryPostProcessors such as the PropertyPlaceHolderConfigurer may be called to update the bean definitions.)
2. Instantiate beans and store them in the application context.
   The bean factory creates the bean from the bean definitions. Bean dependencies get injected.
3. Bean post processing.
   All initializing methods (e.g. annotated with @PostConstruct, init-methods declared in the XML and the afterPropertiesSet() method will execute. Annotations such as @Required will be validated. Dynamic proxies in an AOP environment will be created and so on.

The problem that we see occurs due to a corner case in the first two steps. When the bean definition is created Spring scans through the application context XML file. It finds the declaration of the mock bean and a reflective call is made to find out the return type of the method defined by the factory-method declaration in an attempt to determine the bean's type. The return type of this method is declared as the formal type parameter <T> (remember that we use the Mockito.mock(Class<T> classToMock) as factory method). Consequently, the bean definition of someDependencyMock will be of java.lang.Object and not SomeDependency as one would think.

In the second step, the someClass bean is instantiated. Springs discovers the @Autowired annotation and tries to fetch an existing SomeDependency bean from the application context. None has yet been created, so the bean factory proceeds and looks for suitable bean definition in order to create an instance of the requested bean on the fly. This operation fails because the bean definition of someDependency is mapped to java.lang.Object. At this point, Spring bails out and throws the NoSuchBeanDefinitionException.

FactoryBean to the rescue

The problem is solved by implementing a custom FactoryBean that can be used whenever we need to mock an object:

(In the EasyMockFactoryBean, I have added an enum to define the type of mock object.)

Next, the Spring test config is updated to use the factory bean:

(Corresponding EasyMock config.)

Spring will still instantiate the beans in the order that they were declared in the mockito-test-config.xml, i.e. starting by creating an instance of SomeClass and then attempt to inject an instance of a SomeDependency into it before any has been created. The good news is that bean factory will scan through all declared FactoryBeans, calls its getObjectType() method, and if the returned type is correct, call its getObject() which should return an instance of the requested type.

Alternative solution

There is another more brittle solution to the problem. By simply changing the order of the tags in the initial application context, e.g. putting the bean declaration that defines the mock above the component scan, will cause the test to pass. The bean definition will still be wrong in the sense that the someDependencyMock will be defined as a java.lang.Object, but there is another important difference. In this case, the bean factory will invoke the Mockito.mock(Class<T> classToMock) method with SomeDependency as parameter causing the someDependencyMock bean to be created and subsequently stored in the application context. Later, the someClass bean is created and the dependency can obtained from the application context when requested by the bean factory and the wiring of the beans will succeed. However, making sure that the order of the bean declarations in the XML is always correct may neither be simple nor obvious, and it will get increasingly more difficult the more beans you add to the configuration.

Disclaimer

As stated in the in the first paragraph, I would only consider this solution for integration tests because of performance reasons. The application context itself adds some overhead, but the big performance hit is to create and wire all beans that are associated with it. When writing unit tests there is no need for this, so you could revert to constructor or setter dependency injection, Spring's ReflectionTestUtils.setField(), PowerMock's Whitebox.setInternalState(), Mockito's @InjectMocks or similar techniques.

Acknowledgements

Thanks to Wilhelm Kleu at stack overflow for suggesting the solution and to Mattias Hellborg Arthursson for valuable discussions.

Dependencies

• Spring 3.0.6
• Mockito 1.8.5
• jUnit 4.10
• EasyMock 3.1

References

• Spring Reference Manual - Static Factory Method
• Spring Reference Manual - FactoryBean
• Spring Reference Manual - Integration Testing
• Mockito
• EasyMock

Next post: Spring Integration Tests, Part II, Using Mock Objects

In this post I want to show you a step by step guide on how you can start your Scala project with Spring Data for MongoDB.

You should have SBT 0.11 installed on your system since I am going to use it as the building tool through out this post.

1. Creating Project

Create the project like below:

$ mkdir springDataMongo
$ cd springDataMongo/
$ sbt
[info] Loading global plugins from /home/amir/.sbt/plugins
[info] Set current project to default-71a5a0 (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> set name := "SpringMongo"
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> set version := "1.0"
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> set scalaVersion := "2.9.1"
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> session save
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)

Open file build.sbt and add the dependencies:

Save the file and run the following in SBT console:

> reload
[info] Loading global plugins from /home/amir/.sbt/plugins
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> update
[info] Updating {file:/data/3-Projects/scala-projects/springDataMongo/}default-71a5a0...
[info] Done updating.

And generate the IDE related files. I am using IntelliJ:

> gen-idea

or if you use Eclipse:

> eclipse create-src

Now you have configured your project and you can start coding!

2. Configuring Spring

There are two ways to configure Spring Data for MongoDB: Annotations and XML. I explain annotations here. You have to extend AbstractMongoConfiguration class as follows to define your database settings:

As you see there are two methods to implement. mongo method should return an instance of Mongo class. If you are using a mongo instance in a network, you should then specify the address and possibly port (if not the default).

3. Data Model

Consider we have the following class for keeping accounts in a system. This class will be mapped to a mongo document by Spring:

4. Interacting with MongoDB

Now let's write a simple program to interact with MongoDB:

mongoTemplate is an implementation of MongoOperation interface. With MongoOperation you can do all CRUD operations. E.g. inserting/saving a document:

"accounts" is the name of collection. Or if you want to query:

Note that JavaConversions is required here since the result of find is a java.util.List[Account] and we need to convert it to Scala list.

Please refer to documentation of Spring Data for MongoDB for more info about what you can do more

1. Bugfix for NPE when calling facebookStore.createTestUser(false,"...").
2. Added possibility of using a provided HttpClient instance in HttpClientFacebookTestUserStore.
3. Added unit test that displays the use of json-path when asserting data.

Here's a quick reminder of how to create test users for an application. The creation process is handled by an instance of FacebookTestUserStore, e.g. The HttpClientFacebookTestUserStore which uses Apache HttpClient as communication method. The class needs the application ID and the application secret for the application in order to be able to create the users and this is specified in the constructor:

FacebookTestUserStore facebookStore = new HttpClientFacebookTestUserStore("<appId>", "<appSecret>"));

Now you have a choice when creating a test user: You may say that the test user should accept the privileges that you specify for it, which is done by:

FacebookTestUserAccount aTestUser = facebookStore.createTestUser(true, "read_stream");

If you want to give a choice to the test user whether the privileges should be accepted or not, you do the following:

FacebookTestUserAccount aTestUser = facebookStore.createTestUser(false, "read_stream");

The accept/decline choice takes part in the web browser, hence you need the login URL for the test user to access the page. This URL is accessible from the FacebookTestUserAccount:

aTestUser.loginUrl();

Specifying a HttpClient instance to HttpClientFacebookTestUserStore

There might be occasions where you need a special HttpClient instance for communication, for example one instance with proxy settings configures, or another instance where you have stubbed all the communication that takes place. To provide the instance to HttpClientFacebookTestUserStore, just specify it in the constructor:

HttpClient httpClient = new DefaultHttpClient();
FacebookTestUserStore facebookStore = new HttpClientFacebookTestUserStore("<appId>", "<appSecret>", httpClient));

Using json-assert to verify application behavior

The json-assert project, which uses json-path can be used to make life a little less verbose when asserting on data from the JSON document representations of the Facebook data streams that are available in the FacebookTestUserAccount. For example, the user details of a test user can be accessed like this:

aTestUser.getUserDetails();

The result is a java.lang.String containing a JSON Document. To make it easier to assert that the user details actually contain the expected data, you may use json-assert and Hamcrest matchers:

String userDetails = account.getUserDetails();

JsonAssert.with(userDetails).assertThat("$.first_name", equalTo("Sue"));
JsonAssert.with(userDetails).assertThat("$.last_name", equalTo("Ellen"));

Feel free to test it out, the code is open source an can be found on Maven central or on the
Facebook Test Java API project page.

/Tobias

In this post I want to show you how you can write your own parallel collection in Scala. The example I use for this post is taken from Alex Prokopec's presentation at Scala eXchange 2011.

Consider we want to have a parallel String in our application. If you convert a String into a parallel collection you get a ParArray:

scala> "I Love Scala".par
res2: scala.collection.parallel.ParSeq[Char] = ParArray(I,  , L, o, v, e,  , S, c, a, l, a)

This is good enough to take advantage of your multi-core CPU architecture, but for some reason you want to write you own Parallel String collection!

Scala Parallel collection is implemented by a mechanism that is called split-combine. As the name implies a sequential collection can be splitted into sub-collections and each one can be stolen by a thread from a thread pool. Later the results of each sub-collection can be combined to make the final collection. You can find details about this implementation in [1].

In order to start with our Parallel String we need to extend scala.collection.parallel.immutable.ParSeq trait. There are 4 abstract methods that we need to implement:

seq, length and apply have obvious implementations. Method splitter returns a new instance of class ParallelStringSplitter which we are going to define next. Note that this instance is mixed-in into SignalContextPassingIterator which is responsible for passing the signal context along iterators. We define ParallelStringSplitter class inside ParallelString class:

Note that we need to extend Splitter trait. There are 6 abstract methods to implement. Method dup returns a new instance of ParallelStringSplitter with current values. Method split splits the iterator into a sequence of disjunct views and method psplit splits the splitter into a sequence of disjunct views.

Now we have the splitter. Let's test this parallel String in REPL:

scala> val pstr = new ParallelString("This is a long string" * 250000)
pstr: ParallelString = ParallelString(T, h, i, s,  , i, s,  , a,  , l, o, n, g,  ...

If you don't provide a combiner, then the default one is used. Sometimes you want to have control over the combination and you want to take the responsibility for it. Let's write our own combiner to show how you can do that. We need to mix in another trait into the definition of ParallelString: ParSeqLike

In order to specify our own combiner we need to override method newCombiner:

Now we need to define ParallelStringCombiner class by extending the Combiner trait:

There are 5 abstract methods that need to be implemented. As the names imply method combine combines the content of another builder to the current builder and produces a new builder. Method result produces a collection from added elements and += adds a single element to this builder.

To summarize, to build a parallel collection class A, one should:

• extend ParSeq trait
• implement all abstract methods from ParSeq trait
• define a splitter class that mixes in with Splitter and ParIterator traits and put it in class A
• extend ParSeqLike trait
• define a combiner class that extend Combiner trait and implement all abstract methods
• override newCombiner method in class A and return a new instance of defined combiner

For the complete code of this example please refer to Alex Prokopec's Github.

References

[1]: Aleksandar Prokopec, Tiark Rompf, Phil Bagwell, Martin Odersky, "A Generic Parallel Collection Framework", Euro-Par 2011

You will need to download and install a couple of things.

Eclipse

I usually go with the Eclipse Classic version. It is available from eclipse.org. Installing Eclipse is simply just unpacking the downloaded file.

Android SDK

The SDK is located at developer.android.com There is a really good installation guide at developer.android.com

Android NDK

The NDK is located at developer.android.com with an installation guide.

Eclipse C/C++ Development Tools

To make it a bit easier we also install the C/C++ Development Tools to get C/C++ support in Eclipse. From the 'Help' menu choose 'Install New Software...'

From the drop down select the update site for your eclipse version, in my case Indigo.

Under 'Programming Languages' you will find 'C/C++ Development Tools'. Select it and click 'next' and follow the install instructions.

Getting started

I will show you how to get started by making a small application called NDKSetup.

Create a new android project for this example I will call it NDKSetup.

First thing we need to do is to create a new folder called 'jni'

The next thing we need to do is to setup Eclipse so we can build the JNI code. Click on the small down arrow on the external tool button and choose the 'External Tools Configuration...'.

Mark the 'Program' and click on the 'new'-icon.

Select a proper name.

Location is the location of the external tool you want to run, in our case the ndk-build file. Click on 'Browse file system...' and locate the ndk-build file located under your NDK folder.

The working directory is the jni folder we created in our project. Click on 'Browse Workspace...' ...

... and choose the jni folder.

You should now have something looking like this:

Press run and you will get this error:

/Users/uncle/ndk-setup/android-ndk-r6b/build/core/add-application.mk:118: *** Android NDK: Aborting...    .  Stop.
Android NDK: Your APP_BUILD_SCRIPT points to an unknown file: /Users/uncle/Workspaces/ndk-setup/NDKSetup/jni/Android.mk

This means two things. Your setup to the ndk-build works and you are missing the Android.mk file.

Create a new file called Android.mk in the jni folder.

In the Android.mk file write this:

LOCAL_PATH := $(call my-dir)
 
include $(CLEAR_VARS)
 
LOCAL_LDLIBS := -llog
 
LOCAL_MODULE    := ndksetup
LOCAL_SRC_FILES := native.c
 
include $(BUILD_SHARED_LIBRARY)

LOCAL_PATH := $(call my-dir)
An Android.mk file must begin defining the LOCAL_PATH variable, this is where the source files are. The macro 'my-dir' is the path where the Android.mk file is located.

include $(CLEAR_VARS)
Since all the building and parsing is done in the same context the variables called LOCAL_XXX is globals and need to be cleared.

LOCAL_MODULE := ndksetup
This is where you set the name used as the identifier for each module. Later used in java when loading the module. The system will add 'lib' before the module name when compiling into the .so file. So ndksetup will become libndksetup.so. The only exception is if you add 'lib' first in your module name then the system will not add it.

LOCAL_SRC_FILES := native.c
Here you add a list of the files you need to compile your module. You do not need to add headers or include files the system will take care of that for you.

include $(BUILD_SHARED_LIBRARY)
The NDK provides you with two make files that parse and build everything accordingly to your Android.mk file. The two once are BUILD_STATIC_LIBRARY for building static library and BUILD_SHARED_LIBRARY for building shared library.

For the example project here we use the BUILD_SHARED_LIBRARY.

If you run the external tool "NDK Build" we get a new error:

make: *** No rule to make target `/Users/uncle/Workspaces/ndk-setup/NDKSetup/jni/native.c', needed by `/Users/uncle/Workspaces/ndk-setup/NDKSetup/obj/local/armeabi/objs/ndksetup/native.o'.  Stop.

So lets att the missing native.c file into the jni folder.

Before we start adding code to the native.c file I think it is easier to write the API starting from java. We start by adding the loading of the library.

static {
    System.loadLibrary("ndksetup"); // ndksetup is the LOCAL_MODULE string.
}

We also need to define the function we are going to implement. By adding the keyword 'native' the system will know it is a function located in the native code.

private native void printLog(String logThis);

Putting in all together give us this:

package com.jayway.ndksetup;

import android.app.Activity;
import android.os.Bundle;

public class NDKSetupActivity extends Activity {

    static {
        System.loadLibrary("ndksetup");
    }

    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
        printLog("Hello!!!");
    }

    private native void printLog(String logThis);
}

Back to the native.c file.

The name structure of the function is important. It is build from the java starting with a java identifier followed by the package name, followed by the class name, followed by the function name. This is one way of converting our java function into the native function. I usually do it by right clicking on the function name ( printLog ) and select 'Copy Qualified Name' paste it in the native.c file:

com.jayway.ndksetup.NDKSetupActivity.printLog(String)

Start by changing all dots (.) to underscores (_).

com_jayway_ndksetup_NDKSetupActivity_printLog(String)

Add the return value and a Java_ identifier to the function:

void Java_com_jayway_ndksetup_NDKSetupActivity_printLog(String)

The input variable is a String in java so lets make it a java string in native as well.

void Java_com_jayway_ndksetup_NDKSetupActivity_printLog(jstring logString)

The last thing we need to add is a reference to the JNI enviroment and a reference to java object that this function belongs to.

void Java_com_jayway_ndksetup_NDKSetupActivity_printLog(JNIEnv * env, jobject this, jstring logString)

Finally our first native function will look like this:

#include <jni.h>
#include <string.h>
#include <android/log.h>
 
#define DEBUG_TAG "NDKSetupActivity"
 
void Java_com_jayway_ndksetup_NDKSetupActivity_printLog(JNIEnv * env, jobject this, jstring logString)
{
    jboolean isCopy;
    const char * szLogString = (*env)->GetStringUTFChars(env, logString, &isCopy);
 
    __android_log_print(ANDROID_LOG_DEBUG, DEBUG_TAG, "NDK: %s", szLogString);
 
    (*env)->ReleaseStringUTFChars(env, logString, szLogString);
}

Just for fun we add another function, a fibonacci function.

private native int fibonacci(int value);

jint Java_com_jayway_ndksetup_NDKSetupActivity_fibonacci(JNIEnv * env, jobject this, jint value)
{
	if (value <= 1) return value;
	return Java_com_jayway_ndksetup_NDKSetupActivity_fibonacci(env, this, value-1)
            + Java_com_jayway_ndksetup_NDKSetupActivity_fibonacci(env, this, value-2);
}

Now you are up and running with your NDK development for Android.

In this post I go through injectors and mostly extractors. You will see that how extractors can be employed for pattern matching.

Consider that we need to hold first names in an application. We can define a case class for Firstname:

we build a value from this case class in REPL:

scala> val fname = Firstname("Amir")
fname: Firstname = Firstname(Amir)

and now a pattern matching on this case class:

scala> fname match {
     |     case Firstname(f) => println(f)
     |     case _            => println("Nothing found")
     | }
Amir

Here fname is matched against its case class by constructor patterns mechanism and its field is extracted and printed. Very powerful and handy.

What if we need to do a pattern matching for a string? The problem is that strings are not case classes.

Scala provides us with a very interesting mechanism called Extractors. An extractor is an object that has a method called unapply as one of its members. Let's clarify this with an example: assume we want to have an extractor object for IP addresses. We define the extractor as the following:

Method unapply receives a string representing a possible IP address and returns an option of 4 strings. If the string is not a valid IP address, the method returns None. Method isValid is added for validating IP addresses. Let's try this in REPL:

scala> val ip = "127.0.0.1"
ip: java.lang.String = 127.0.0.1

scala> val nonIP = "128.-112.ABC."
nonIP: java.lang.String = 128.-112.ABC.

scala> IPAddress.unapply(ip)
res0: Option[(String, String, String, String)] = Some((127,0,0,1))

scala> IPAddress.unapply(nonIP)
res1: Option[(String, String, String, String)] = None

And if we use it in a pattern matching statement:

scala> ip match {
     |    case IPAddress(_, _, _, a) => println(a)
     |    case _                     => println("Invalid ip address")
     | }
1

scala> nonIP match {
     |    case IPAddress(_, _, _, a) => println(a)
     |    case _                     => println("Invalid ip address")
     | }
Invalid ip address

So ip was a valid string representation of an IP Address and it is matched against IPAddress while nonIP was not a valid one. In the example above we were only interested in the last byte of IP Address and we skipped the rest by _ wildcard. Of course you can extract all the four parts if you need them.

In IPAddress object we used 4 variables that is wrapped in a Some in the success case and returned. This can be generalized to N variables.

It is also possible that an extractor pattern does not bind to any variables. In this case the corresponding unapply method returns a boolean (true for success and false for failure). As an example we changed the unapply method to the following:

and we try it in REPL:

scala> "127.0.0.1" match {
     |    case IPAddress() => println("Valid")
     |    case _           => println("Invalid")
     | }
Valid

Remember that although there is no binding in this case but you have to put the parentheses in front of IPAddress.

So far we have seen fixed number of element values. But what if we have variable number of element values? For example if you have a string that can contain arbitrary number of IP addresses? In order to handle this case, Scala allows you to define a different extractor method called unapplySeq. To see how it can be used, assume we want to have pattern matching on a string containing arbitrary number of IP Addresses:

This time the method returns an option of a sequence of string. Now let's see what we can do with it:

scala> val ips = "192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4"
ips: java.lang.String = 192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4

scala> ips match {
     |    case IPAddresses(IPAddress(a, _, _, _), IPAddress(b, _, _, _), _*) => println(a + " " + b)
     |    case _                                                             => println("Invalid IP addresses")
     | }
192 192

In this example we used both IPAddress and IPAddresses objects in the pattern matching. There are 4 IP addresses in ips and we are looking for the first byte of the first two IPs in the string. How powerful and clean!

The unapply method is called extractor because it takes an element of the same set and extracts some of its parts. In our example IPAddress takes a string and validates and extracts all 4 bytes of the address. You can also define a method apply on an object that does vice versa which takes some arguments and yields an element of a given set. This method is called an injection. We can define injection for our IPAddress object like this:

Injections and extractions are often located in the same object because then you can use the object name for both constructions and pattern matching. However, it is also possible to have an extraction object without injection. The object itself is called extractor regardless of whether or not it has an apply method [1].

Remember if you include the injection method, it should be a dual to the extraction method, for example:

IPAddress.unapply( IPAddress.apply(a, b, c, d) )

should return:

Some(a,b,c,d)

Extractors vs. Case classes

There is a section in [1] that compares extractors and case classes which I summarize here:

• There is one shortcoming with case classes: they expose the concrete representation of data. This means that if a case succeeds in a matching, you know that the selector expression is an instance of that case class.
• Extractors do not expose the concrete representation of data.
• Case classes have less code and they are easier to set up. Scala compiler can optimize patterns over case classes much better than extractors.
• If a case class inherits from a sealed base class, Scala compiler checks for pattern matches for exhaustiveness and will complain in case there exists something while Scala compiler can not do the same thing for extractors.
• If you write code for a closed application, case classes are preferable because of their advantages in conciseness, speed and static checking.
• If you need to expose a type to unknown clients, extractors might be preferable.

References:

[1]: Martin Odersky, Lex Spoon, Bill Venners, "Programming in Scala", 2nd Edition

 
def words = ['ant', 'buffalo', 'cat', 'dinosaur']
def wordsWithSizeGreaterThanFour = words.findAll { it.length() > 4 }
 

At the first line we simply define a list with some words but the second line is more interesting. Here we search the words list for all words that are longer than 4 characters by calling the findAll with a Groovy closure. The closure has an implicit variable called "it" which represents the current item in the list. The result is a new list, wordsWithSizeGreaterThanFour, containing buffalo and dinosaur. Pretty nice and simple! There are other interesting methods that we can use on collections in Groovy as well, e.g.

• find - finds the first item matching a closure predicate
• collect - collect the return value of calling a closure on each item in a collection
• sum - Sum all the items in the collection
• max/min - returns the max/min values of the collection

So how do we take advantage of this when validating our XML or JSON responses with REST Assured?

XML Example

Let's say we have a resource at "localhost:8080/shopping" that returns the following XML:

 
<shopping>
      <category type="groceries">
        <item>Chocolate</item>
        <item>Coffee</item>
      </category>
      <category type="supplies">
        <item>Paper</item>
        <item quantity="4">Pens</item>
      </category>
      <category type="present">
        <item when="Aug 10">Kathryn's Birthday</item>
      </category>
</shopping>
 

Let's also say we want to write a test that verifies that the category of type groceries has items Chocolate and Coffee. In REST Assured it can look like this:

 
expect().
         body("shopping.category.find { it.@type == 'groceries' }.item", hasItems("Chocolate", "Coffee")).
when().
         get("/shopping");

What's going on here? First of all the XML path "shopping.category" returns a list of all categories. On this list we invoke a function, find, to return the single category that has the XML attribute, type, equal to "groceries". On this category we then continue by getting all the items associated with this category. Since there are more than one item associated with the groceries category a list will be returned and we verify this list against the hasItems Hamcrest matcher.

But what if you want to get the items and not validate them against a Hamcrest matcher? This is also simple using the XmlPath class:

 
// Get the response body as a String
String response = get("/shopping").asString();
// And get the grocieries from the response. "from" is statically imported from the XmlPath class
List<String> groceries = from(response).getList("shopping.category.find { it.@type == 'groceries' }.item");
 

If the list of groceries is the only thing you care about in the response body you can also use a shortcut:

 
// Get the response body as a String
List<String> groceries = get("/shopping").path("shopping.category.find { it.@type == 'groceries' }.item");
 

Depth-first search

It's actually possible to simplify the previous example even further:

 
expect().
         body("**.find { it.@type == 'groceries' }", hasItems("Chocolate", "Coffee")).
when().
         get("/shopping");
 

** is a shortcut for doing depth first searching in the XML document. We search for the first node that has an attribute named type equal to "groceries". Notice also that we don't end the XML path with "item". The reason is that toString() is called automatically on the category node which returns a list of the item values.

Root path

Let's expand the example by also validating that the category of type supplies contains Paper and Pens:

 
expect().
         body("**.find { it.@type == 'groceries' }", hasItems("Chocolate", "Coffee")).
         body("**.find { it.@type == 'supplies' }", hasItems("Paper", "Pens")).
when().
         get("/shopping");
 

Even though this works you can probably see that the XML path is almost duplicated for the two body expectations and this is of course something we want to avoid. To address this we can use a root path and path arguments:

 
expect().
         rootPath("**.find { it.@type == '%s' }").
         body("", withArgs("groceries"), hasItems("Chocolate", "Coffee")).
         body("", withArgs("supplies"), hasItems("Paper", "Pens")).
when().
         get("/shopping");
 

By specifiying a "root path" we don't need to duplicate the entire path just to change the category type.

JSON Example

Let's say we have a resource at "/store" that returns the following JSON document:

 
{ "store" : {
  "book" : [
          { "author" : "Nigel Rees",
            "category" : "reference",
            "price" : 8.95,
            "title" : "Sayings of the Century"
          },
          { "author" : "Evelyn Waugh",
            "category" : "fiction",
            "price" : 12.99,
            "title" : "Sword of Honour"
          },
          { "author" : "Herman Melville",
            "category" : "fiction",
            "isbn" : "0-553-21311-3",
            "price" : 8.99,
            "title" : "Moby Dick"
          },
          { "author" : "J. R. R. Tolkien",
            "category" : "fiction",
            "isbn" : "0-395-19395-8",
            "price" : 22.99,
            "title" : "The Lord of the Rings"
          }
        ]
   }
}
 

Example 1

As a first example let's say we want to make the request to "/store" and assert that the titles of the books with a price less than 10 are "Sayings of the Century" and "Moby Dick":

 
expect().
        body("store.book.findAll { it.price < 10 }.title", hasItems("Sayings of the Century", "Moby Dick")).
when().
        get("/store");
 

Just as in the XML examples above we use a closure to find all books with a price less than 10 and then return the titles of all the books. We then use the hasItems matcher to assert that the titles are the ones we expect. Using JsonPath we can return the titles instead:

 
// Get the response body as a String
String response = get("/store").asString();
// And get all books with price < 10 from the response. "from" is statically imported from the JsonPath class
List<String> bookTitles = from(response).getList("store.book.findAll { it.price < 10 }.title");
 

Example 2

Let's consider instead that we want to assert that the sum of the length of all author names are greater than 50. This is a rather complex question to answer and it really shows the strength of closures and Groovy collections. In REST Assured it looks like this:

 
expect().
        body("store.book.author.collect { it.length() }.sum()", greaterThan(50)).
when().
        get("/store");
 

First we get all the authors ("store.book.author") and invoke the collect method on the resulting list with the closure { it.length() }. What it does is to call the length() method on each author in the list and returns the result to a new list. On this list we simply call the sum() method to sum all the length's. The end result is 53 and we assert that it's greater than 50 by using the greaterThan matcher. But it's actually possible to simplify this even further. Consider the words example from the beginning of the article again:

 
def words = ['ant', 'buffalo', 'cat', 'dinosaur']
 

Groovy has a very handy way of calling a function for each element in the list by using the * notation, e.g.

 
def words = ['ant', 'buffalo', 'cat', 'dinosaur']
assert [3, 6, 3, 8] == words*.length()
 

I.e. Groovy returns a new list with the lengths of the items in the words list. We can utilize this for the author list in REST Assured as well:

 
expect().
        body("store.book.author*.length().sum()", greaterThan(50)).
when().
        get("/store");
 

And of course we can use JsonPath to actually return the result:

 
// Get the response body as an input stream
InputStream response = get("/store").asInputStream();
// Get the sum of all author length's as an int. "from" is again statically imported from the JsonPath class
int sumOfAllAuthorLengths = from(response).getInt("store.book.author*.length().sum()");
// We can also assert that the sum is equal to 53 as expected.
assertThat(sumOfAllAuthorLengths, is(53));
 

Summary

At a first glance the syntax may seem quite strange and complex but once you get used to it it's extremely powerful and actually quite simple. It's also a good place to start if you work with Java and want to understand the benefit of closures. There are a lot more features that can help you parsing and validating JSON and XML using REST Assured than the ones shown in this article. For further reading refer to the REST Assured web-page and read up on Groovy collections. Enjoy!

Lower bounds

Consider the class I defined in the first post:

Company is covariant in type T as you know by now. It's been a while and we realize it's the time to define a partnership relation for companies to connect them together and expand their co-operations. So we add the following method to Company class:

When we try to compile this code we get:

Error: covariant type T occurs in contravariant position in type T of value y

What happened here? To see what has happened it's better to take a look at Function1 definition in Scala:

This definition tells us that the input of a function is contravariant (negative) and what it returns is covariant (positive). So remember that no matter how many arguments a function has, all of them are in negative positions:

Now it should be more clear why we got the compile error. We were passing an argument of type T which is covariant (positive) and we know that arguments of a function has negative (contravariant) positions. This is a contradiction.

But how can we solve this? We can use lower type bounds to parameterize a method.

Lower type bounds: help you to declare a type to be a supertype of another type. T >: A indicates that type T or abstract type T refers to a supertype of type A.

Let's correct the defined method by using lower bounds:

By this new definition, partnerWith method accepts arguments that can be supertype of type T.

scala> val littleCompany: Company[SmallCompany] = new Company[SmallCompany](new SmallCompany)
littleCompany: Company[SmallCompany] = Company@149bc5a

scala> val bigCompany: Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany: Company[BigCompany] = Company@116ac04

scala> bigCompany.partnerWith(littleCompany)

So partnerWith method is now generalized and polymorphic.

Upper bounds

Assume that we want to restrict the partnership relation somehow in our example. We wanna say that a company can have partnership with another company that is a subtype of Company[BigCompany]. In order to apply this restriction in Scala, one can use upper bounds. So let's change the partnerWith method to reflect this:

Now partnerWith only accepts arguments that are a subtype of Company[BigCompany]. Let's prove this by introducing a new company:

This class does not extend BigCompany so Company[CrappyCompany] is not a subtype of Company[BigCompany] according to covariant type declaration. Now we try to build partnerships in REPL:

scala> val littleCompany: Company[SmallCompany] = new Company[SmallCompany](new SmallCompany)
littleCompany: Company[SmallCompany] = Company@1979eb4

scala> val bigCompany: Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany: Company[BigCompany] = Company@6d1901

scala> littleCompany.partnerWith(bigCompany)

scala> val crappyCompany: Company[CrappyCompany] = new Company[CrappyCompany](new CrappyCompany)
crappyCompany: Company[CrappyCompany] = Company@1d0d313

scala> littleCompany.partnerWith(crappyCompany)
<console>:11: error: inferred type arguments [Company[CrappyCompany]] do not conform to method 
partnerWith's type parameter bounds [U <: Company[BigCompany]]
       littleCompany.partnerWith(crappyCompany)
                     ^

So by upper type bound declaration we could restrict our partnership relation.

Remember that by definition a class is supertype and subtype of itself. Hence you can pass an instance of Company[BigCompany] to partnerWith:

scala> val bigCompany2 : Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany2: Company[BigCompany] = Company@b61e92

scala> bigCompany.partnerWith(bigCompany2)

Note that both lower and upper bound declarations have the colon as their second character so you don't mix the order ( <: >: )

Conclusion

In these three posts I showed how you can avail from Scala variance annotations and lower and upper bounds together to design your application in a type safe manner. Scala provides you with type-driven design where types of an interface guides its detailed design and implementation [1]. Examples provided in these posts were not the best examples but they could still show the purpose of their existence. I hope you have a better understanding of this cool feature in Scala.

References

[1]: Martin Odersky, Lex Spoon, Bill Venners, "Programming in Scala", 2nd Edition

Contravariant Subtyping

Do you remember the definition of covariant? Contravariant is the other way around. I will clarify this by the following example (example is adopted from [1]). Consider we have a trait:

This trait has a write method that is responsible for accepting an argument and writing it in the current network channel. Assume that we have two NetworkChannel of type AnyRef and String:

We just don't care about the write implementation now. There is another class that is responsible for controlling output channels:

The control method accepts an argument of type NetworkChannel[String]. Do you think we can pass NetworkChannel[AnyRef] to this method? Yes we can! NetworkChannel is declared to be contravariant in its type T.

Contrvariant subtyping means: If S is a subtype of T then NetworkChannel[T] is a subtype of NetworkChannel[S]

This seems to be a little bit strange in our example. We know that String is a subtype of AnyRef in Scala. How come NetworkChannel[AnyRef] can be a subtype of NetworkChannel[String] ?

Consider what you can do with each channel. NetworkChannel[String] accepts arguments of type String and write them in network channel. NetworkChannel[AnyRef] accepts arguments of type AnyRef and write them also in network channel. We can substitute NetworkChannel[String] with NetworkChannel[AnyRef] since whatever can be done by NetworkChannel[String] can be done also by NetworkChannel[AnyRef] (remember that NetworkChannel[AnyRef] can also write strings) but not vice versa. We can not use NetworkChannel[String] instead of NetworkChannel[AnyRef] since it only accepts strings.

Remember that contravariant orders types from more genetic to more specific.

How should I know if two types are covariants or contravariants? Read the following section.

Liskov Substitution Principle (LSP)

LSP is a principle in object-oriented programming. It states:

It is safe to assume that S is a subtype of T if you can substitute a value of type S wherever a value of type T is required without violating the desirable properties of the program [2].

So let's say we have S as a subtype of T. If Class of type S is a subtype of Class of type T then there is a covariant subtyping between them. Otherwise if Class of type T is a subtype of Class of type S then there is contravariant subtyping between them. if there is no subtyping relation between Class of type S and T, then there is invariant subtyping relation.

I hope you have understood Scala Types variances. In the next post I will discuss Lower bounds and upper bounds.

References

[1]: Martin Odersky, Lex Spoon, Bill Venners, "Programming in Scala", 2nd Edition
[2]: Barbara Liskov, Jeannette Wing, "A behavioral notion of subtyping", ACM Transactions on Programming Languages and Systems (TOPLAS), Volume 16, Issue 6 (November 1994), pp. 1811 – 1841

In this series of posts I will try my best to clear out type variance annotations in Scala; a powerful tool to handle type orderings from specific to generic and vice verca.

Covariant Subtyping

We start by covariant subtyping. If you don't know what it is, please be patient. First consider the following code in Scala:

We have a Company class that has type parameter T. This class accepts a value of type T for its constructor. We have two companies defined here: BigCompany and SmallCompany. As it is obvious from the code SmallCompany is a subtype of BigCompany. Finally we have a class Investor that accepts only companies of type BigCompany so it invests only on big ones!
Now let's do some constructions in REPL:

scala> val littleCompany: Company[SmallCompany] = new Company[SmallCompany](new SmallCompany)
littleCompany: Company[SmallCompany] = Company@1e70822

scala> val bigCompany: Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany: Company[BigCompany] = Company@9bb740

scala> val bigInvestor:Investor = new Investor(bigCompany)
bigInvestor: Investor = Investor@1004566

So far so good. After a while it is decided that we need investors that can take care of small companies. We don't want to define a new brand of investor thus we use the existing one. So we try to construct:

scala> val smallInvestor:Investor = new Investor(littleCompany)

But we get:

<console>:8: error: type mismatch;
 found   : Company[SmallCompany]
 required: Company[BigCompany]
Note: SmallCompany <: BigCompany, but class Company is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val smallInvestor:Investor = new Investor(littleCompany)
                                                 ^

Bang! What happened? Compile error!

Let's see what happened exactly here. We know by definition that SmallCompany is subtype of BigCompany. But what is the relation between Company[SmallCompany] and Company[BigCompany] ? It is not defined by default and we need somehow to tell compiler about this. In order to solve this we change class Company to the following:

The only thing that is changed is the type parameter that has a + sign in front. This means that subtyping is covariant (flexible) in paramter T.

Covariant subtyping means: if S is a subtype of T then Company[S] is subtype of Company[T].

Remember that covariant orders types from more specific to more generic.

Let's construct a small investor again with new Company definition:

scala> val smallInvestor:Investor = new Investor(littleCompany)
smallInvestor: Investor = Investor@9fad9c

Great! We could solve this by Scala variance subtyping. By a single + sign that is called variance annotation you could tell Scala compiler that you want Company[SmallCompany] be a subtype of Company[BigCompany].

In the next post, we will investigate contravariant subtyping.

Parallel Collections were introduced in Scala 2.9 release which are built on the same abstractions and provide the same interfaces as existing collection implementation. It follows the model proposed by Doug Lea in his Fork/Join framework [1]. They are really easy to use and this is one of the advantages you can see for them. If you are familiar with parallel programming, it is quite difficult to convert a sequential program into a parallel one. This difficulty comes from the fact that each has its own paradigm, algorithm and data structure.

Scala enables you to invoke a method called par on a collection to get a parallel collection. This parallel collection then employs all the available cores on the target system. You can easily try this out in REPL:

scala> List(1,2,3,4,5).par.filter { _ % 2 == 0 }
res1: scala.collection.parallel.immutable.ParSeq[Int] = ParVector(2, 4)

This is a simple filtering on a list. The only new thing I added is 'par' after list definition.

Now let's have a comparison between sequential and parallel versions and see how parallel collection can help us. Consider that we have a Data class with some data as follows:

and a companion object to construct Data instances:

Now we define computation trait to perform some operations on the aforementioned data class:

This trait has two implementations:

In each implementation, method compute applies a map with function f and then calculate the maximum element in the list. The only difference between these two implementations is the use of 'par' method. Now let's run and time each implementation:

Running the code with SBT on a machine with 4 CPU cores:

> run 3
[info] Running com.jayway.parallel.ParallelExperiment 3
Sequential 44 (ms)
Parallel 14 (ms)

> run 100
[info] Running com.jayway.parallel.ParallelExperiment 100
Sequential 1020 (ms)
Parallel 268 (ms)

As expected, we gained speed up in parallel version. Results for a couple of run with different list sizes is depicted in the following fgure:

Now let's change function f to the following:

  def f(data:Data):Int =  {
    data.a + data.b
  }

I removed the Thread.sleep so the method returns immediately. Now let's have a look at results for different runs again:

Interestingly, this time the result is not good at all! It's even vice versa and sequential implementation has lower running time for the first 5 runs. This is because using parallel collection has some overhead for distributing (fork) and gathering (join) the data between cores. Thus one can conclude having heavy computations (which is the case in most applications), parallel collections can be of great performance improvement. But there might be scenarios (like the second scenario we discussed) where sequential collection is faster

For a better understanding of Scala Parallel Collection internals, please watch Aleksander Prokopec's presentation from Scala Days 2010. If you are interested to know more then you can read [2].

References:

[1]: Doug Lea, A Java Fork/Join Framework, 2000.
[2]: Aleksandar Prokopec, Tiark Rompf, Phil Bagwell, Martin Odersky, "A Generic Parallel Collection Framework", Euro-Par 2011

In some environments user's home directory is located on the mapped network drive which in unacceptable for IntelliJ IDEA. You'll notice the huge performance degradation.

What is the problem?

There are some good motives for keeping the home directory mounted to a network folder rather than on the local hard drive. For example, all user settings and user documents can be backed up centrally, and the user could potentially log in to any computer and have the same user environment. However, there are also some drawbacks. Depending on how the home folder is configured, there will either be a remote network call for each file access, or the file changes will be executed as a batch job at scheduled intervals or specific events such as logging off the computer.

The second part of the problem is that IntelliJ IDEA produces quite a lot of data, typically several hundred megabytes, besides the anticipated artifacts. The data consists of IDE plugins, configuration files, log files, but the vast majority is internal cache files. By default, the data is written to a hidden .IntelliJIdea10/ folder in your home directory.

If you combine these two factors, the performance will suffer. How much depends on the bandwidth of your network, something that I became painfully aware of when I logged in remotely to the customer's network via a slow VPN connection.

Solution

Avoid network overhead by configuring IntelliJ IDEA to cache data on your local computer instead of in your home folder. A recent customer used Windows XP as their working environment, the instructions below are based on that experience.

1. Create a new directory on a local harddrive that IntelliJ IDEA can use, e.g. C:/.IntelliJIdea10/. Hint, you cannot create a library starting with a "." from Windows Explorer, you have to use the mkdir command from the terminal.
2. Locate the idea.properties file in the bin directory in IntelliJ IDEA's installation directory. The default location is C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 10.5.1\bin.
3. Change the relevant properties to point to the recently created directory:

   # path to IDEA config folder. Make sure you're using forward slashes
   idea.config.path=C:/.IntelliJIdea10/config
   
   # path to IDEA system folder. Make sure you're using forward slashes
   idea.system.path=C:/.IntelliJIdea10/system
   
   # path to user installed plugins folder. Make sure you're using forward slashes
   idea.plugins.path=C:/.IntelliJIdea10/config/plugins
   

Considerations

Consider not to change the idea.config.path property if you are using several different computers and you would like to have the same configuration on all machines. Additionally, if you are using IntelliJ IDEA Ultimate, i.e. the commercial version of the tool, you should also make a cautious decision about whether or not to keep this property unchanged, because the license file is stored in the denoted directory. On the other hand, if you always use the same computer you might as well change this property together with the other properties, so that all IntelliJ IDEA files are stored in the same location.

References

• IDEA files location: http://devnet.jetbrains.net/docs/DOC-181
• IDEA license key: http://devnet.jetbrains.net/docs/DOC-200

The test outcome shows up as a green or red bar in the bottom of the Eclipse application window. The plugin also gives more feedback. In case of failing test the failing line of code in the unit test shows up with a "problem" icon just as a compilation error would look like. Doing mouse-over on the problem icon you will get the failing assert message. This feature I have noticed in recent versions of Eclipse so try to update to the latest Eclipse if you don't see this happen.

The problem indicator is quite powerful and allows a higher level of TDD. When doing pure TDD you write a failing test first. This would make Infinitest mark your assert statement as problematic until you save a working implementation of your tested code. This allows a very focused write failing test / write implementation-loop.

Infinitest operates independently from the JUnit view in Eclipse. As Infinitest is activated as soon as source code is saved while JUnit is run when initiated by the developer the state shown by either system is often out of sync. I have also experienced that the Infinitest fails to "re-test" but then a project clean brings it back on track. Even if there are quirks I think Infinitest is useful for my daily work and it really promotes TDD.

The plug-in is available on update site http://infinitest.github.com/ and if you are using SpringSource Tool Suite (STS) the plug-in is also available in STS's  extension list.

Example

Let's say you have a simple HTML page that performs file-uploading from a browser like this:

<html>
<body>
	<form id="upload" action="/fileUpload" method="post" enctype="multipart/form-data">
		<input type="file" name="file" size="40">
		<input type=submit value="Upload!">
	</form>
</body>
</html>

We also assume that we have a server that receives the file and return a JSON response like this if everything was ok:

 
{ "fileUploadResult" : "OK" }
 

Now let's use REST Assured to upload a file and assert that the upload was OK:

 
given().
         multiPart(new File("/home/johan/some_large_file.bin")).
expect().
         body("fileUploadResult", is("OK")).
when().
         post("/fileUpload");
 

With so little code you've now managed to POST a large file using multi-part form data encoding to the server and asserted that the expected JSON bundled in the response body was correct.

But how can this work? We haven't even specified a control name? Well if you don't define a control name specifically REST Assured will default to "file". In many cases this is of course not the case. For example let's say we change the control name in the HTML form to "file2" instead:

<html>
<body>
	<form id="upload" action="/fileUpload" method="post" enctype="multipart/form-data">
		<input type="file" name="file2" size="40">
		<input type=submit value="Upload!">
	</form>
</body>
</html>

REST Assured still makes it simple:

 
given().
         multiPart("file2", new File("/home/johan/some_large_file.bin")).
expect().
         body("fileUploadResult", is("OK")).
when().
         post("/fileUpload");
 

Multiple parts

You can even send multiple files, streams, byte-arrays and form parameters in the same request:

 
InputStream inputStream = ..
 
given().
         multiPart("file1", new File("/home/johan/some_large_file.bin")).
         multiPart("file2", new File("/home/johan/some_other_large_file.bin")).
         multiPart("file3", "file_name.bin", inputStream).
         formParam("name", "value").
expect().
         body("fileUploadResult", is("OK")).
when().
         post("/advancedFileUpload");
 

Summary

As you've hopefully seen it's very simple do multi-part form data uploading with REST Assured. Credits should of course also go to Apache HTTP Client for making it possible. To get started with REST Assured today just visit the web-site. Enjoy!

The pom is configured to use two different plugins depending on which operating system that is executing it.
On Windows, a Launch4j project plugin will be used. On OS X the osxappbundle plugin is used, which creates an OS X application directory structure and uses the JavaApplicationStub binary for launching the application.

The <profile> tag is used for deciding which plugin to use depending on operating system.

The pom also points out different application icons for the binaries, OS X requires the icon to be in the icns and windows in the ico format.

The wrapped application has a main class in the file com.jayway.wrappedapplication.Launcher

The pom.xml

  4.0.0
  com.jayway.wrappedapplication
  launcher
jar
  1.0-SNAPSHOT
  launcher
  http://maven.apache.org


      windows-deploy
      
        
          Windows
        
      
      


            org.bluestemsoftware.open.maven.plugin
            launch4j-plugin
            1.5.0.0
            
              
                launch4j
verify
                
                  launch4j
                
                
                  false
gui
                  ${project.build.directory}/relauncher-${project.version}.exe
                  ${project.build.directory}/${project.build.finalName}.jar
                  Launcher
                  
                    1.6.0
                  
                  
                    com.jayway.wrappedapplication.Launcher
                    false
anything
                  
                  ${basedir}/build-resources/win/icons/application.ico
                
              
            
          
        
      
    

      
        
          mac
        
      
      


            org.codehaus.mojo
            osxappbundle-maven-plugin
            1.0-alpha-2
            
              com.jayway.wrappedapplication.Launcher
              ${basedir}/build-resources/osx/icons/application.icns
            
            
              
package
                
                  bundle
                
              
            
          
        
      
    
  

And that's it.

Type conversion is done by Camel either when you explicitly tell Camel to perform conversion or in some cases where Camel detects that a type conversion is needed. For example in this case Camel tries to perform type conversion from SourceType to TargetType:

public class SourceType {
    private final int value;

    public SourceType(int value) {
        this.value = value;
    }
}
public class TargetType {
    private final int value;

    public TargetType(int value) {
        this.value = value;
    }
}
public interface Service {
    public void call(TargetType value);
}
...
from("direct:call").bean(service, "call");
...
template.requestBody("direct:call", new SourceType(17));

This will fail since there is no type converter registered to handle conversion to TargetType. What if TargetType has a constructor taking a SourceType:

public TargetType(SourceType source) {
    this.value = source.getValue();
}

But this will also fail! My solution is to add a FallbackConverter that can handle this scenario, that is the target type having a single argument constructor taking the source type. The code looks like this:

package com.jayway.camel.typeconverter;

@Converter
public class UsingConstructor {
    @SuppressWarnings("unchecked")
    @FallbackConverter
    public static <T> T convertTo(Class<T> type, Object value, TypeConverterRegistry registry) throws Throwable {
        for (Constructor<?> c : type.getConstructors()) {
            Class<?>[] types = c.getParameterTypes();
            if (types.length == 1 && types[0].isAssignableFrom(value.getClass())) {
                    return (T) c.newInstance(value);
            }
        }
        return null;
    }
}

A couple of things to notice:

• The class has the @Converter annotation
• The method has the @FallbackConverter
• The signature of the method

You also need to include file META-INF/services/org/apache/camel/TypeConverter in your classpath that contains the package name. This allows Camel to automatically discover the type converter class.

com.jayway.camel.typeconverter

Here is an integration test showing this example. I have also implemented a type converter that checks if the source type has a no argument method returning the target type. The source code for both type converters are available at GitHub.

Notice that this is just a demo of the capabilities of fallback converters. You have to decide if this approach is appropriate in your project. It might not be desirable to have a direct dependency between the TargetType and the SourceType. The overhead of reflection might be too high or it might have unintended consequences. For example the UsingMethod converter has the unintended consequence of finding and using the toString() method when a conversion to String is needed. The UsingConstructor is probably faster and safer since there are usually fewer constructors than public methods.

See the Camel documentation for type converts for more information.

• Maven URL handler that allows deployment directly from a Maven repository
• Features that allow grouping of OSGi bundles

Although these features are not part of the OSGi standard, it at least makes my life easier and I believe that something similar will become part of the standard since Spring DM and Apache Aries also have similar concepts. I'll just show how to create a blank OSGi project and deploy in Karaf:

mvn org.ops4j:maven-pax-plugin:create-bundle -Dpackage=com.jayway.osgi.helloworld
cd com.jayway.osgi.helloworld/
mvn clean install

Now you need to download Karaf, unzip and then:

cd $KARAF_HOME
bin/karaf
osgi:install --start mvn:com.jayway/com.jayway.osgi.helloworld/1.0-SNAPSHOT

The output will be (it is not a completely blank project!):

STARTING com.jayway.osgi.helloworld
REGISTER com.jayway.osgi.helloworld.ExampleService

To generate an Eclipse project:

mvn org.ops4j:maven-pax-plugin:eclipse

Some links:

• Apache Karaf have a good quick start page
• Apache Camel OSGi tutorial

The webservices are:

• The Airport Information Webservice to find the location of an airport
• The National Digital Forecast Database Webservice provided by http://www.weather.gov/ to find the weather at a location

My goals have been:

• Use web services from different providers
• Be able to test the routes
• Include error handling
• Try various methods for transformation

So far I'm quite happy with the results and impressed with the flexibility of Apache Camel. Have a look at the code available at AirportWeather project @ GitHub and let me know what you think.

Routes

The complete route looks like this:

from("direct:getMaximumTemperaturAtAirport")
	.to("direct:getAirportInformationByAirportCode")
	.to("direct:fromAirportInformationToLocation")
	.to("direct:fromLocationToNDFD")
	.to("direct:fromNDFDToTempInCelcius");

The flow is synchronous and each step is implemented in a separate route to be testable, for example I can stub out entire steps of the route.

A step can be further broken down into even smaller routes, for example separating the call to the web service and the error handling:

from("direct:getAirportInformationByAirportCode")
	.to("direct:invokeGetAirportInformationByAirportCode")
	.choice()
		.when().xpath("count(/NewDataSet/Table)=0").rollback("No Airport found")
		.otherwise();

Calling a SOAP web service

To call the web service I use Spring WS. The reason I didn't use CXF is that the NDFD is using RPC encoding which is an old web standard not supported by the current version of CXF. The XML payload is generated using a Velocity template. This works well in this case, but later I want to compare this to JAXB generated objects from an XML schema.

from("direct:invokeGetAirportInformationByAirportCode")
    .to("velocity:getAirportInformationByAirportCode.vm")
    .to("spring-ws:http://www.webservicex.net/airport.asmx?soapAction=http://www.webserviceX.NET/

Transformation

Camel has very flexible support for transformation and un/marshalling. I have tried a couple of different variations. Registering a object and simply calling a method:

SimpleRegistry registry = new SimpleRegistry();
registry.put("temperatureTransformer", new TemperatureTransformer());
...
.transform().method("temperatureTransformer", "fromFahrenheitToCelsius");

Let Camel figure out which method to call:

registry.put("cdataTransformer", new CDataTransformer());
...
.transform().method("cdataTransformer");

XML unmarshalling using XStream:

XStreamDataFormat xmlToAirPortLocation() {
    XStream xstream = new MissingFieldIgnoringXStream();
    xstream.alias("Table", AirportLocation.class);
    XStreamDataFormat dataFormat = new XStreamDataFormat(xstream);
    return dataFormat;
}
...
.unmarshal(xmlToAirPortLocation())

XPath transformation:

.transform().xpath("/NewDataSet/Table[1]")

Let Camel figure out how to perform the conversion:

.convertBodyTo(Integer.class)

Stubbing routes

When testing I used AdviceWith to stub out a route in the test case. I'm not really sure if this is the best way, but it works.

context.getRouteDefinitions().get(0).adviceWith(context, new AdviceWithRouteBuilder() {
    @Override
    public void configure() throws Exception {
        interceptSendToEndpoint(someEndpoint)
            .skipSendToOriginalEndpoint().to("velocity:AirportResult-empty.xml");
    }
});

To make the stub actually override the endpoint it is important to call skipSendToOriginalEndpoint. I use Velocity to generate a static reply. See the AirportWeatherTest for more details.

Handling bad results

To verify the result of the invoked web service I have used a choice and rollback to signal the error, for example when the airport does not exist:

.choice()
    .when().xpath("count(/NewDataSet/Table)=0").rollback("No Airport found")
    .otherwise();

Future Camel rides

Things I will investigate further:

• Other ways to stub out routes
• Making use of camel-test
• Use CXF and JAXB to avoid creating XML payloads using Velocity (perhaps for some other web service)

- "Clojure combines the expressiveness of Lisp, the agility of a dynamic language, the performance of a compiled language, and the wide applicability of the JVM in a robust, production-ready package. Clojure is a practical language designed to support high-performance, concurrent applications which efficiently interoperate with other software in the JVM ecosystem. All of this combines to make it an ideal tool for the programmer to quickly build robust programs."

And motivate why they chose to support Clojure:

- "Clojure covers a new use case on the Heroku platform: components which demand correctness, performance, composability; and optionally, access to the Java ecosystem."

Ladies and gentlemen, it's time to consider moving the parenthesis from the end of the method name to the beginning of the function call. Try it, you'll like it.

When it was time to implement the requirement for secure JMS (meaning authentication and authorization) we were sure that this would be easy. After all, even if security is not part of the JMS standard, Spring usually provides support for integrating with the application server in a consistent way.

A simple configuration change?

Since we retrieve connections from the connection factory it should be as simple as adding some properties to the JNDI-lookup like this:

    
      java.naming.security.principaluser=jmsuser
      java.naming.security.credentials=password
   

Not so simple after all...

Unfortunately when we tested this we were greeted with: "JMSSecurityException: Access Denied To Resource". This was discouraging but since using secure JMS with Spring and WebLogic should be quite a common use case we were still confident that there would be an accepted solution available.

Indeed, we were not alone. There were at least three JIRA-issues opened for this problem and a number of forum threads:

• Support secure JMS queue access on WebLogic - SPR-2941
• DefaultMessageListenerContainer failover to work with Weblogic JMS and security credentials - SPR-4720
• exposeAccessContext doesn't resolve the issues with secure JMS and WebLogic. - SPR-5869
• Secure-JMS-and-Spring-with-WebLogic
• weblogic-10-standalone-jms-client
• excessive-JNDI-lookups-on-weblogic

It was evident that this was not going to be so easy after all. Basically, the root of the problem seems to be that WebLogic does not allow the InitialContext created in one thread to be used in another thread which is what happens when Spring does the lookup on the connection factory in one thread and then tries to retrieve a connection in another. Although the problem has been known for at least five years, there is still no real solution.

A solution, sort of...

The accepted way (and the only one we found in the forums that would work for us) of getting around this if you are determined to use WebLogic JMS together with Spring is to use a decorator on the connection factory to create the InitialContext with the required security credentials for each call to "createConnection()". This way, the credentials will be bound to the thread that is executing and WebLogic will not complain. The details of the implementation are here.

While the above solution worked well (thanks Honeybunny!) we were not really happy with it. Introducing a decorator felt invasive. Furthermore there is a potential performance issue in creating an InitialContext for every call to "createConnection()" which could of course be solved by introducing a threadlocal cache to avoid unnecessary calls but the real problem lies in the way that users are associated with threads which, as the WebLogic documentation makes quite clear, needs to be handled carefully:

"When you create a JNDI Context with a username and password, you associate a user with a thread. When the Context is created, the user is pushed onto the context stack associated with the thread. Before starting a new Context on the thread, you must close the first Context so that the first user is no longer associated with the thread. Otherwise, users are pushed down in the stack each time a new context created. This is not an efficient use of resources and may result in the incorrect user being returned by ctx.lookup() calls. "

Making sure the context is closed is certainly possible but it seemed to us that we were going in the wrong direction in writing custom code to get around something that should basically "just work" as we wanted to deliver a maintainable solution.

A one-click solution

After trying the different proposed solutions in the JIRA and forum threads we decided to check if there wasn't some way to solve this by configuring WebLogic as it seems to have configuration alternatives for just about everything. In the admin console we noticed something promising. If you click on one of your deployed applications to get to the "Overview"-tab there is a field called "Deployment Principal Name" where you can specify the principal (user) that should be used during startup and shutdown if you need something else than the anonymous principal. This looked like it was worth a try so we set the principal to a user with the privileges for connecting to the JMS destination, restarted the server and - it worked!

Click the image to see the configuration in the console:

Of course there are other ways of doing this configuration and since operations tend not to like the "point and click" deployment model the best way is to incorporate it in your deployment scripts with WLST.

Final words

Some caveats are in order. We are aware that this is still only a workaround and certainly not the final solution to the problem.

• First of all, this was what worked in our case and your mileage may vary. For one thing we are using the commonj.WorkManager API and not a thread pool managed by the Spring container.
• Obviously, this will only work for applications that are deployed in the application server. If you have a JMS-client that uses Spring JMS that is not deployed in WebLogic, you will need some other workaround.
• In our case, we have a simple deployment setup where we are not using a cluster and distributed JMS destinations so we haven't tested if that will work but we think it's likely that there will be issues.

Anyway, we hope that at least someone will find this solution helpful. Personally, we would have appreciated not having to spend nearly a week reading old forum threads figuring out what would work or not.

And as a final note; if all else fails you can always switch to Active MQ

Clojure maps

A map is a data structure that associates keys to values. Maps are very important in Clojure, because they are used to store entity data, much like custom classes with fields and setters/getters are used in traditional object-oriented languages like Java. In the Java world, we have java.util.Map. They are somewhat cumbersome to work with, mostly due to the static, generic typing, but also because there is no literal map syntax, plus the fact that you can't initialize it with elements. You must first create it, then mutate it:

import java.util.Map;
import java.util.HashMap;

Map<String,Object> map = new HashMap<String,Object>();
map.put("somekey", "somevalue");

Clojure maps, on the other hand, have a very concise literal syntax. Zero or more key-value pairs between curly braces. That's it.

(def map {:somekey "somevalue"})

The key :somekey in the map above is a Clojure keyword. Keywords are symbolic identifiers that always evaluate to themselves, and they have a very fast equality check. For those reasons (and others, as we will see), keywords are often used, rather than strings, as keys in maps.

A Clojure map is not only a data structure, it is also a function. When a map is called with a key, it will return the corresponding value (or nil if no key was found). It can look like this when testing it from the REPL:

user> (def animal {:species :lion, :name "Leo", :age 3})
user> (animal :name)
"Leo"

In fact, keywords are functions too. When a keyword is called with a map as argument, it will look itself up in the map and return the matching value:

user> (:name animal)
"Leo"

This form is more common in idiomatic Clojure.

The Ring library

Ring is a Clojure web applications library inspired by Ruby's Rack. Ring abstracts the HTTP request and response as maps, and expects handlers to be functions that take a map as argument and returns a map. The returned map is transformed by Ring into a HTTP response, which is sent back to the caller. This is a simple, but powerful abstraction. It means that web handlers can be written as regular functions, which can be tested in isolation. Web libraries like Compojure build on top of the Ring abstraction and provide more high-level constructs, like routing. In order to not complicate this example, however, we will stick to plain Ring.

Let's say that we have a HTTP request that looks like this:

GET /welcome HTTP/1.1
HOST: www.sayhello.com

Ring transforms this into a Clojure map that looks like this:

{:protocol        :http
 :request-method  :get
 :uri             "/welcome"
 :server-name     "www.sayhello.com"}

Let's further assume that we want to generate a corresponding HTTP response:

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 11

Hello world

In order to produce that HTTP response, Ring expects this map:

{:status 200
 :headers
    {"Content-Type" "text/plain"
     "Content-Length" 11}
 :body "Hello world"}

So all we need to do is to provide a function that returns the above map, like this:

(defn app [req]
  {:status 200
   :headers
      {"Content-Type" "text/plain"
       "Content-Length" 11}
   :body "Hello world"})

Ring provides a response convenience function that returns a skeletal response map with status 200, no headers, and the given argument as body:

(use 'ring.util.response)
(defn app [req]
  (response "Hello world"))

We'll use the response function in our example later.

Leiningen

In order to manage our dependencies, we will use the Leiningen build tool. The simplest way to install it is to download the stable release of the script and place it somewhere in the PATH, like ~/bin. Then run lein self-install:

$ lein self-install
Downloading Leiningen now...
...

$ lein version
Leiningen 1.5.2 on Java 1.6.0_24 Java HotSpot(TM) 64-Bit Server VM

Building the web application

We create a new project and go there:

$ lein new cljheroku
$ cd cljheroku

The newly created directory contains the following files:

cljheroku/
├── README
├── project.clj
├── src
│   └── cljheroku
│       └── core.clj
└── test
    └── cljheroku
        └── test
            └── core.clj

First we need to create a Procfile for Heroku:

$ echo "web: lein run -m cljheroku.core" > Procfile

This will have Leiningen start the web application by running the function called -main in the namespace cljheroku.core, ie the file src/cljheroku/core.clj. We'll change that file to contain this:

(ns cljheroku.core
  (:use ring.util.response
        ring.adapter.jetty))

(defn app [req]
  (response "Hello World"))

(defn -main []
  (let [port (Integer/parseInt (get (System/getenv) "PORT" "8080"))]
    (run-jetty app {:port port})))

Let's look at this code in more detail. We begin with the first chunk:

(ns cljheroku.core
  (:use ring.util.response
        ring.adapter.jetty))

The ns macro will create a namespace cljheroku.core and load all functions in the namespaces ring.util.response and ring.adapter.jetty. Since we used :use and not :require, these functions can be referenced by name, without any prefix or namespace qualifier. This enables us to write (response ...) instead of (ring.util.response/response ...).

Then we define our Ring handler called app.

(defn app [req]
  (response "Hello World"))

As you can see, it's just a regular function that takes one argument: a HTTP request map, and returns the result of the response function: a HTTP response map. If we wanted to set the Content-Type header, we could use the content-type function, another function from ring.util.response. Instead of (response "Hello world"), we would pass the response through the content-type function: (content-type (response "Hello world") "text/plain"). But here we will just use response.

Finally we specify a -main function.

(defn -main []
  (let [port (Integer/parseInt (get (System/getenv) "PORT" "8080"))]
    (run-jetty app {:port port})))

It will serve the same purpose as the Java main method, namely provide an entry point from the outside world. It gets the value of the PORT environment variable, or uses 8080 as default. It can be enlightening to analyze that part in detail. (System/getenv) is simply Clojure's way of calling Java's static method System.getenv(), which returns a java.util.Map. The get function takes a map (yes, it works with Java maps too), a key and a default value, and returns the matching value. So, (get (System/getenv) "PORT" "8080") will get the PORT variable from the environment, and if there is no PORT set there, it will return "8080". The resulting string value of the port is parsed into an integer, and is stored as the local variable port. This value is then passed in to the Jetty adapter as part of a configuration map. All this in two lines. If you're not used to Clojure's lack of ceremony, you might find it overwhelming. I personally find it refreshingly to-the-point.

OK, that was the source code for our handler. We now need to adjust the project.clj file slightly, and provide a dependency to Ring. We'll also add an exclusion of a duplicate dependency that Jetty has. Here is the resulting project.clj:

(defproject cljheroku "1.0.0-SNAPSHOT"
  :description "Example Ring app running on Heroku"
  :dependencies [[org.clojure/clojure "1.2.1"]
                 [ring/ring-core "0.3.8"]
                 [ring/ring-jetty-adapter "0.3.8"]]
  :exclusions [org.mortbay.jetty/servlet-api])

Before we do anything else, though, we should add this baby to Git:

$ git init
Initialized empty Git repository in /Users/john/Source/cljheroku/.git/

$ git add .

$ git commit -m "Initial commit"

We should probably push this to a maintained repository somewhere, but for our purposes, we have now stored this in Git.

Testing locally

Before we test it, we will ask Leiningen to retrieve the dependencies:

$ lein deps

The lib directory now contains a list of jars that we need. Leiningen will make sure they all get on the classpath without you ever having to see it. You can ask Leiningen for the classpath, though, should you really like to see it:

$ lein classpath

You can also have Leiningen create a pom file for you, which can be handy if you need to look at the dependency:tree, import the project into an IDE that only knows Maven; things like that:

$ lein pom

Anyway, we can now start this web application using the same command as we placed in the Procfile for Heroku. It will start Clojure with the correct classpath, find and make available the given namespace (cljheroku.core), and call its -main function with any given arguments (none, in our case):

$ lein run -m cljheroku.core
2011-06-12 18:41:24.927:INFO::Logging to STDERR via org.mortbay.log.StdErrLog
2011-06-12 18:41:24.928:INFO::jetty-6.1.26
2011-06-12 18:41:24.959:INFO::Started SocketConnector@0.0.0.0:8080

Browsing to localhost:8080 shows a page with "Hello world" on it. Good. It works.

We now change the handler function to instead respond with, say, "Hello everyone" and save it. When we reload the page, the new message appears. Excellent. It reloads automatically.

OK. We're happy with our web application and want to deploy it into the cloud. What do we do?

Deploying on Heroku

Heroku is, as I mentioned at the beginning, mainly a Ruby application platform. The command line tool for working with Heroku is a Ruby Gem. We first need to install Ruby and RubyGems. Assuming that has been done, we need to get the heroku gem:

$ sudo gem install heroku

Now we can create an app on the Cedar stack:

$ heroku create --stack cedar
Enter your Heroku credentials.
Email: john.doe@example.com
Password:
Found existing public key: /Users/john/.ssh/id_rsa.pub
Would you like to associate it with your Heroku account? [Yn]
Uploading ssh public key /Users/john/.ssh/id_rsa.pub
Creating furious-sword-794... done, stack is cedar
http://furious-sword-794.herokuapp.com/ | git@heroku.com:furious-sword-794.git
Git remote heroku added

We can list the remote repositories to see what the last sentence above meant:

$ git remote -v
heroku	git@heroku.com:furious-sword-794.git (fetch)
heroku	git@heroku.com:furious-sword-794.git (push)

And finally, in order to deploy our first verison of the web app, we push everything to heroku:

$ git push heroku master
The authenticity of host 'heroku.com (50.19.85.156)' can't be established.
RSA key fingerprint is 8b:48:5e:67:0e:c9:16:47:32:f2:87:0c:1f:c8:60:ad.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'heroku.com,50.19.85.156' (RSA) to the list of known hosts.
Counting objects: 13, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (13/13), 1.27 KiB, done.
Total 13 (delta 0), reused 0 (delta 0)

-----> Heroku receiving push
-----> Clojure app detected
-----> Installing Leiningen
       Downloading: leiningen-1.5.2-standalone.jar
       Writing: lein script
-----> Installing dependencies with Leiningen
       Running: lein deps :skip-dev
       Downloading: org/clojure/clojure/1.2.1/clojure-1.2.1.pom from central
       Downloading: ring/ring-core/0.3.8/ring-core-0.3.8.pom from clojars
       ...
       Copying 10 files to /tmp/build_19juox87ngduz/lib
-----> Discovering process types
       Procfile declares types -> web
-----> Compiled slug size is 11.0MB
-----> Launching... done, v4
       http://furious-sword-794.herokuapp.com deployed to Heroku

To git@heroku.com:furious-sword-794.git
 * [new branch]      master -> master

We browse to http://furious-sword-794.herokuapp.com, and we see the text "Hello world" (if you remembered to change back from "Hello everyone"). Now we make a small change:

-  (response "Hello World"))
+  (response "Hello World, I tell you!"))

All we need to do to deploy the change is to commit it, and then push to heroku:

$ git ci -a
$ git push heroku master

When we reload the page, the new text is there.

Roll back to previous release

We can list the releases that have been pushed:

$ heroku releases
Rel   Change                          By                    When
----  ----------------------          ----------            ----------
v5    Deploy 34516b0                  john.doe@examp..  3 minutes ago
v4    Deploy e471ca1                  john.doe@examp..  5 minutes ago

Let's say there was a problem with the release that was just deployed, and we want to roll back. With Heroku, it's trivial to roll back to the previous release:

$ heroku rollback
Rolled back to v4

When we're done with this web app completely, we can destroy it like this:

$ heroku apps:destroy

It's even possible to roll back to earlier releases, but I won't show that here. This concludes the Clojure-on-Heroku guide. Thank you for your time.

References

1. Heroku: http://www.heroku.com/
2. Clojure: http://clojure.org/
3. Ring library: https://github.com/mmcgrana/ring
4. Leiningen: https://github.com/technomancy/leiningen
5. Compojure: https://github.com/weavejester/compojure/wiki
6. FlightCaster uses Clojure and Heroku: http://www.infoq.com/articles/flightcaster-clojure-rails
7. I got the idea for this blog from this Gist: https://gist.github.com/1001206