In my previous post, I mentioned how AppHarbor provides an online tool for manually testing transformations. Also, the Visual Studio extension SlowCheetah provides support for manually testing and transformation and diffing them against the base configuration from inside Visual Studio. These tools are useful and may be fine for tinkering with getting a transform just right, but being software professionals, we find it highly desirable to reliably automate our transformation testing.

In this article we shall look at one way to exercise our Web.config transformations from our own code, so that we can TDD our transformations and regression test them as part of our continuous integration cycle, like everything else.

Starting Point

The actual Web.config transformation process is normally invoked by MSBuild, as described in detail by Elton Stoneman in this post. (One approach to testing the transformations could, thus, be to fork off MSBuild processes on custom build targets, but we'd like to take a less cumbersome route.)

Being an MSBuild task means that the transformation is implemented as an ITask realization. The concrete implementation is called TransformXml and resides in the Microsoft.Web.Publishing.Tasks.dll assembly. Note that this assembly is not pre-installed with Windows/.NET but rather installed along with Visual Studio 2010, typically to C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web). Given that its location is not fixed, you'd probably want to add it to your dependencies folder (caveat emptor: check license terms before passing it around).

From Elton's post, we can also see 3 arguments being passed to the TransformXml task:

• Source (the XML file to transform)
• Transform (the transformation XML file itself)
• Destination (the target of the transformed XML file)

Armed with that background knowledge, let's move on and try to get the transformation to run outside of the MSBuild environment in which it normally executes. For our experiment, we'll be transforming the following source (app.config):

   <configuration>
     <appSettings>
	   <add key="testKey" value="testValue"/>
     </appSettings>
   </configuration>

The transformation we'll be testing (app.Debug.config) should change the "testValue" of the "testKey" to something transformation specific ("debugValue" in this case):

   <configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
     <appSettings>
       <add key="testKey"
            value="debugValue"
            xdt:Locator="Match(key)" xdt:Transform="SetAttributes" />
     </appSettings>
   </configuration>

Test Approach

Our initial stab at automatically testing the transformation will be to invoke the XmlTransform through its outermost interface; i.e. the ITask interface. First, we need to add a reference to Microsoft.Web.Publishing.Tasks.dll as mentioned above. Additionally, we need to reference Microsoft.Build.Framework and Microsoft.Build.Utilities.v4.0 (these should be available from your usual Project, Add Reference, .NET tab).

Opening up the XmlTransform class in Visual Studio's Object Browser shows us a single method, Execute(), plus a handful of properties, most of which we recognize from Stoneman's previously mentioned post. A little experimentation reveals the following details:

• SourceRootPath
• The base path to which the XML file specified in Source is relative. If not specified, this will  be the current directory (which, when running from a test runner, would typically be your build target directory).
• Source
• The name of the XML source file to be transformed. If this is not an absolute path, it will be relative to the SourceRootPath.
• TransformRootPath
• The base path to which the XML file specified in Transform is relative. If not specified, this will be the same as SourceRootPath.
• Transform
• The name of the XML transformation to be applied to the source. If this is not an absolute path, it will be relative to the TransformRootPath.
• Destination
• The name of the resulting transformed XML file. If this is not an absolute path, it will be relative to the current directory (typically your build target directory, in a test context).

So all we need to do is something like the following, right?

   var transformer = new TransformXml
   {
      SourceRootPath = @"..\..\..\Jayway.WebTransformTesting\",
      Source = @"app.config",
      Transform = @"app.Debug.config",
      Destination = @"app.transformed.config"
   };
   var transformationSucceeded = transformer.Execute();

Well, as the System.InvalidOperationException with the message "Task attempted to log before it was initialized" thrown in our face subtly tells us, this is not quite the case...

Investigating the task implementation with Reflector shows the problem to be due to the fact that a property, BuildEngine, of the Task base class has not been set. It is used as a dependency injection mechanism, giving the task access to some environment services (see IBuildEngine on MSDN) normally provided by MSBuild.

Well, the whole purpose of this experiment was to run outside of MSBuild, so we'll roll our own "build engine" instead:

   public class BuildEngineForTest : IBuildEngine
   {
      public void LogErrorEvent(BuildErrorEventArgs e)
      {
         Console.WriteLine(LogFormat, "ERROR", e.Message);
      }

      public void LogWarningEvent(BuildWarningEventArgs e)
      {
         Console.WriteLine(LogFormat, "WARNING", e.Message);
      }

      public void LogMessageEvent(BuildMessageEventArgs e)
      {
         Console.WriteLine(LogFormat, e.Importance, e.Message);
      }

      public void LogCustomEvent(CustomBuildEventArgs e)
      {
         Console.WriteLine(LogFormat, "CUSTOM", e.Message);
      }

      public bool BuildProjectFile(
         string projectFileName, string[] targetNames,
         IDictionary globalProperties, IDictionary targetOutputs)
      {
         return true;
      }

      public bool ContinueOnError
      {
         get { return true; }
      }

      public int LineNumberOfTaskNode
      {
         get { return 0; }
      }

      public int ColumnNumberOfTaskNode
      {
         get { return 0; }
      }

      public string ProjectFileOfTaskNode
      {
         get { return string.Empty; }
      }

      private const string LogFormat = "{0} : {1}";
   }

This was the piece we were missing to be able to run the transformation outside MSBuild. We are, thus, able to write our coveted test:

...
   [Test]
   public void ApplyingDebugTransformationToConfigurationSetsExpectedKey()
   {
      const string targetFile = @"app.transformed.config";

      var transformer = new TransformXml
      {
         BuildEngine = new BuildEngineForTest(),
         SourceRootPath = @"..\..\..\Jayway.WebTransformTesting\",
         Source = @"app.config",
         Transform = @"app.Debug.config",
         Destination = targetFile
      };

      // Left as exercise for the reader... ;]
      AssertKeyInTransformedFile(targetFile);
   }
...

Our transformation runs successfully, yielding the following console output from our "build engine":

Normal : Transforming Source File: ..\..\..\Jayway.WebTransformTesting\app.config
Normal : Applying Transform File: ..\..\..\Jayway.WebTransformTesting\app.Debug.config
Low : Executing SetAttributes (transform line 6, 35)
Low : on /configuration/appSettings/add[@key='testKey']
Low : Applying to 'add' element (source line 4, 6)
Low : Set 'key' attribute
Low : Set 'value' attribute
Low : Set 2 attributes
Low : Done executing SetAttributes
Normal : Output File: app.transformed.config
Normal : Transformation succeeded

Yay? Yay.

Conclusion

We have shown how Web.config Transformations can be invoked programmatically, outside the MSBuild process by providing a simple custom IBuildEngine realization; a technique that might prove useful for other scenarios as well.

We have applied that knowledge to enable automated testing of our transformations.

In my next post in this series, we'll continue our investigation, aiming to get rid of the whole, nasty detour around the file system.

For me, the most important aspects of Robotium is first most that it is stable and that one can trust the results. Secondly, that it is easy enough to use for non-technical users. A user should not need to be technically inclined to use Robotium or it would have lost its purpose in helping a large user group who know how to write good test cases but lack a bit in the technical area.

Robotium uses a system of waiting for a component before trying to interact with it. The problem with UI testing compared to normal unit tests is that the UI is slow. When writing normal unit tests, the code gets executed in a fast pace by the CPU. That is not the case with the UI. Therefore, UI test cases that are developed without any kind of wait mechanism will sooner or later run into issues. With Robotium 3.0, two completely new waitFor methods have been implemented:

waitForView(View view, int timeout, boolean scroll)

The various waitFor() methods available in Robotium are used automatically by the, is, get, click, search, scroll and assert methods. However it is always recommended to add additional waitFor calls in your test cases if they will be running on a slow computer or a slow hudson slave. An example of a waitFor method in Robotium that can be beneficial to use is waitForActivity(String name). The purpose of it is to wait for a certain activity before continuing with the execution. To find out which activity that is active at the moment, you can always look at the log by using adb logcat.

One major change in Robotium 3.0 is that finalize() is now replaced with finishOpenedActivities(). Anyone that has used finalize() before in their tearDown() to close the activities that have been started will need to update their test cases.

If you have not tried Robotium before, I recommend you to first read the Question and Answers wiki page.

For tutorials please go to the Robotium Tutorials wiki page.

To download the javadoc and the latest release of Robotium please go to the Downloads page.

I hope you will enjoy the new and improved Robotium!

The test outcome shows up as a green or red bar in the bottom of the Eclipse application window. The plugin also gives more feedback. In case of failing test the failing line of code in the unit test shows up with a "problem" icon just as a compilation error would look like. Doing mouse-over on the problem icon you will get the failing assert message. This feature I have noticed in recent versions of Eclipse so try to update to the latest Eclipse if you don't see this happen.

The problem indicator is quite powerful and allows a higher level of TDD. When doing pure TDD you write a failing test first. This would make Infinitest mark your assert statement as problematic until you save a working implementation of your tested code. This allows a very focused write failing test / write implementation-loop.

Infinitest operates independently from the JUnit view in Eclipse. As Infinitest is activated as soon as source code is saved while JUnit is run when initiated by the developer the state shown by either system is often out of sync. I have also experienced that the Infinitest fails to "re-test" but then a project clean brings it back on track. Even if there are quirks I think Infinitest is useful for my daily work and it really promotes TDD.

The plug-in is available on update site http://infinitest.github.com/ and if you are using SpringSource Tool Suite (STS) the plug-in is also available in STS's  extension list.

Example

Let's say you have a simple HTML page that performs file-uploading from a browser like this:

<html>
<body>
	<form id="upload" action="/fileUpload" method="post" enctype="multipart/form-data">
		<input type="file" name="file" size="40">
		<input type=submit value="Upload!">
	</form>
</body>
</html>

We also assume that we have a server that receives the file and return a JSON response like this if everything was ok:

 
{ "fileUploadResult" : "OK" }
 

Now let's use REST Assured to upload a file and assert that the upload was OK:

 
given().
         multiPart(new File("/home/johan/some_large_file.bin")).
expect().
         body("fileUploadResult", is("OK")).
when().
         post("/fileUpload");
 

With so little code you've now managed to POST a large file using multi-part form data encoding to the server and asserted that the expected JSON bundled in the response body was correct.

But how can this work? We haven't even specified a control name? Well if you don't define a control name specifically REST Assured will default to "file". In many cases this is of course not the case. For example let's say we change the control name in the HTML form to "file2" instead:

<html>
<body>
	<form id="upload" action="/fileUpload" method="post" enctype="multipart/form-data">
		<input type="file" name="file2" size="40">
		<input type=submit value="Upload!">
	</form>
</body>
</html>

REST Assured still makes it simple:

 
given().
         multiPart("file2", new File("/home/johan/some_large_file.bin")).
expect().
         body("fileUploadResult", is("OK")).
when().
         post("/fileUpload");
 

Multiple parts

You can even send multiple files, streams, byte-arrays and form parameters in the same request:

 
InputStream inputStream = ..
 
given().
         multiPart("file1", new File("/home/johan/some_large_file.bin")).
         multiPart("file2", new File("/home/johan/some_other_large_file.bin")).
         multiPart("file3", "file_name.bin", inputStream).
         formParam("name", "value").
expect().
         body("fileUploadResult", is("OK")).
when().
         post("/advancedFileUpload");
 

Summary

As you've hopefully seen it's very simple do multi-part form data uploading with REST Assured. Credits should of course also go to Apache HTTP Client for making it possible. To get started with REST Assured today just visit the web-site. Enjoy!

Easy parsing of the response body

JSON

Let's assume that a REST service at localhost:8080/json returns:

 
{
"lotto":{
 "lottoId":5,
 "winning-numbers":[2,45,34,23,7,5,3],
 "winners":[{
   "winnerId":23,
   "numbers":[2,45,34,23,3,5]
 },{
   "winnerId":54,
   "numbers":[52,3,12,11,18,22]
 }]
}
}
 

As a recap let's say you want to validate that the lottoId is equal to 5 and winnerIds are 23 and 54 you can do like this:

 
expect().
         body("lotto.lottoId", equalTo(5)).
         body("lotto.winners.winnderId", hasItems(23, 54)).
when().
       get("/lotto");
 

where "equalTo" and "hasItems" are standard Hamcrest matchers. But what if you want to return the lottoId or winnder id's and do something with the values? For this reason JsonPath has been added to REST Assured. Example:

 
String response = get("/lotto").asString();
 
// Assuming we've statically imported the "from" method in JsonPath we can do:
int lottoId = from(response).get("lotto.lottoId");
List<Integer> winnerIds = from(response).get("lotto.winners.winnderId");
 
..
 

You can also set a so called "root path" so that you don't have to repeat the entire path for every "get":

 
String responseString = get("/lotto").asString();
 
JsonPath response = new JsonPath(responseString).setRoot("lotto");
int lottoId = response.get("lottoId");
List<Integer> winnerIds = response.get("winners.winnderId");
 

XML

The equilvant of JsonPath for XML is called XmlPath and it works in very much the same way as JsonPath but arguably a bit more complex because of the nature of XML. Consider the following XML available at localhost:8080/xml:

 
<records>
   <car name='HSV Maloo' make='Holden' year='2006'>
      <country>Australia</country>
      <record type='speed'>Pickup Truck with speed of 271kph</record>
   </car>
   <car name='P50' make='Peel' year='1962'>
      <country>Isle of Man</country>
      <record type='size'>Street-Legal Car at 99cm wide, 59kg</record>
   </car>
</records>
 

Now let's make a request and parse the XML:

 
// Get the XML from localhost:8080/xml
String xml = get("/xml").asString();
 
// Get the country of the first car (“from” is statically imported from from XmlPath)
String country = from(xml).get("records.car[0].country”);
 
// Get a list of all car names
List<String> carNames = from(xml).get("records.car.@name");

As with JsonPath you can also set a root path to make it a bit more efficient:

 
XmlPath xmlPath = new XmlPath(xml).setRoot("records");
String country = xmlPath.get("car[0].country");
List<String> carNames = xmlPath.get("car.@name");
 

Filters

Another powerful feature is support for filters. A filter allows you to inspect and alter a request before it's actually committed and also inspect and alter the response before it's returned to the expectations. You can regard it as an "around advice" in AOP terms. Filters can be used to implement custom authentication schemes, session management, logging etc. To create a filter you need to implement the com.jayway.restassured.filter.log.Filter interface. Here's an example filter that simply logs the response body to the console:

 
public class SystemOutFilter implements Filter {
 
    public Response filter(FilterableRequestSpecification requestSpec, FilterableResponseSpecification responseSpec, FilterContext ctx) {
        final Response response = ctx.next(requestSpec, responseSpec); // Invoke the request by delegating to the next filter in the filter chain.
        System.out.println(response.asString());
        return response;
    }
}
 

You can then apply your filter like this:

 
given().filter(new SystemOutFilter()). ..
 

REST Assured already comes with two pre-defined logging filters called ResponseLoggingFilter and ErrorLoggingFilter are there's also shortcuts for using these filters (see below).

Logging

When writing a test it's often useful to see what the response body actually looks like. To easily accomodate this REST Assured provides a shortcut for adding the two logging filters introduced in the previous section. So to log the response body regardless of the status code you can just do:

 
expect().
         log().
         body("lotto.lottoId", equalTo(5)).
         body("lotto.winners.winnderId", hasItems(23, 54)).
when().
         get("/lotto");
 

If you only which to log when an error occurs you can do:

 
expect().
         logOnError().
         body("lotto.lottoId", equalTo(5)).
         body("lotto.winners.winnderId", hasItems(23, 54)).
when().
         get("/lotto");
 

Form authentication

Form authentication is one of the most popular authentication techniques on the net today and using REST Assured it's very simple to test services requiring this scheme. Imagine that the request to localhost:8080/json requires form authentication, we can then do:

 
given().
         auth().form("username", "password").
expect().
         body("lotto.lottoId", equalTo(5)).
         body("lotto.winners.winnderId", hasItems(23, 54)).
when().
         get("/lotto");
 

In many cases it's as simple as that! Basically what happens is that REST Assured parses the login page and logs in for you before making the original request. But if you have many tests or if REST Assured is not able to parse the login page you can supply something called a FormAuthConfig to tell REST Assured how to login. What you provide is the form action and the user name and password input tags. This option will also make your test run faster since there's no need for REST Assured to make an additional requests and parse the login page. For example:

 
given().
         auth().form("username", "password", new FilterConfig("/action", "usernameField", "passwordField").
expect().
         logOnError().
         body("lotto.lottoId", equalTo(5)).
         body("lotto.winners.winnderId", hasItems(23, 54)).
when().
         get("/lotto");
 

If you're using Spring Security there's a pre-defined FilterAuthConfig:

 
given().auth().form("username", "password", FilterConfig.springSecurity()).
 

Reusable specifications

Instead of having to duplicate response expectations and/or request parameters for different tests you can re-use an entire specification. To do this you define a specification using either the RequestSpecBuilder or ResponseSpecBuilder.

E.g. let's say you want to make sure that the expected status code is 200 and that JSON the size of the JSON array "x.y" has size 2 in several tests you can define a ResponseSpecBuilder like this:

 
ResponseSpecBuilder builder = new ResponseSpecBuilder();
builder.expectStatusCode(200);
builder.expectBody("x.y.size()", is(2));
ResponseSpecification responseSpec = builder.build();
 
// Now you can re-use the "responseSpec" in many different tests:
expect().
       spec(responseSpec).
       body("x.y.z", equalTo("something")).
when().
       get("/something");
 

In this example the data defined in "responseSpec" is merged with the additional body expectation and all expectations must be fulfilled in order for the test to pass.

You can do the same thing if you need to re-use request data in different tests. E.g.

 
RequestSpecBuilder builder = new RequestSpecBuilder();
builder.addParameter("parameter1", "parameterValue");
builder.addHeader("header1", "headerValue");
RequestSpecification requestSpec = builder.build();
 
given().
        spec(requestSpec).
        param("parameter2", "paramValue").
expect().
        body("x.y.z", equalTo("something")).
when().
        get("/something");
 

Here the request's data is merged with the data in the "requestSpec" so the request will contain two parameters ("parameter1" and "parameter2") and one header ("header1").

Summary

As you've hopefully seen there are many new useful features added to REST Assured. Other new features that have not been mentioned in this post are e.g. improved HTML parsing, very simple XSD/DTD validation, multi-value parameters, setting root path on expectations, improved configuration of default settings etc etc. In the future we'll hope to add support for JSONP, better XML parsing using XmlPath and a more BDD-like (given, when, then) syntax. Please visit our homepage for more information and downloads.

Example

Let's say we have a (very stupid) Spring bean like this:

 
@Component
public class MyBean {
 
    @Autowired
    private CompanyRepository companyRepository;
 
    public Message generateMessage() throws SAXException {
        final String[] allEmployees = companyRepository.getAllEmployees();
        final String message = StringUtils.join(allEmployees, ", ");
 
        final long id = IdGenerator.generateNewId();
        return new Message(id, message);
    }
}
 

and for demonstration we'd like to mock the static method call to IdGenerator.generateNewId(). Let's write a simple test bootstrapped by Spring using the classloader version of PowerMockRule:

 
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:/example-context.xml")
@PrepareForTest(IdGenerator.class)
public class SpringExampleTest {
 
    @Rule
    public PowerMockRule rule = new PowerMockRule();
 
    @Autowired
    private MyBean myBean;
 
    @Test
    public void mockStaticMethod() throws Exception {
        // Given
        final long expectedId = 2L;
        mockStatic(IdGenerator.class);
        when(IdGenerator.generateNewId()).thenReturn(expectedId);
 
        // When
        final Message message = myBean.generateMessage();
 
        // Then
        assertEquals(expectedId, message.getId());
        assertEquals("John Doe, Paul Anderson, Jane Doe", message.getContent());
    }
 

If you run this test you'll get the following error:

 
java.lang.RuntimeException: java.lang.ClassCastException: com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl cannot be cast to javax.xml.parsers.DocumentBuilderFactory
	at powermock.examples.spring.CompanyRepository.getAllEmployees(CompanyRepository.java:42)
	at powermock.examples.spring.MyBean.generateMessage(MyBean.java:31)
	at org.powermock.examples.spring.mockito.SpringExampleTest.mockStaticMethod(SpringExampleTest.java:58)
...
 

The reason is that companyRepository.getAllEmployees() implementation called from MyBean bean above parses the employee list using Java's XML parsing (javax.xml.parsers.DocumentBuilderFactory). Thus we must use the PowerMockIgnore annotation to ignore the framework:

 
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:/example-context.xml")
@PrepareForTest(IdGenerator.class)
@PowerMockIgnore({"org.w3c.*", "javax.xml.*"})
public class SpringExampleTest {
 
    @Rule
    public PowerMockRule rule = new PowerMockRule();
 
    ....
    // Same code as in the previous example
 

Re-run the test and it'll now pass.

Introducing PowerMock Java Agent

To make it easier to use PowerMock for integration testing version 1.4.9 introduces yet another way to bootstrap the framework, this time using a java agent. Instead of using classloaders to bootstrap and perform byte-code manipulation we now use Java's instrumentation API. This means that you won't run into classloading issues as those presented in the previous example. The test looks exactly the same though:

 
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:/example-context.xml")
@PrepareForTest(IdGenerator.class)
public class SpringExampleTest {
 
    @Rule
    public PowerMockRule rule = new PowerMockRule();
 
    @Autowired
    private MyBean myBean;
 
    @Test
    public void mockStaticMethod() throws Exception {
        // Given
        final long expectedId = 2L;
        mockStatic(IdGenerator.class);
        when(IdGenerator.generateNewId()).thenReturn(expectedId);
 
        // When
        final Message message = myBean.generateMessage();
 
        // Then
        assertEquals(expectedId, message.getId());
        assertEquals("John Doe, Paul Anderson, Jane Doe", message.getContent());
    }
 

Even though the test looks exactly the same the internal implementation has a lot of differences. To have a look at the real code behind the to examples have a look in subversion here for the first example and here for the java agent example.

Round-up

Using the Java agent can help you out when you run into hard-to-resolve classloading issues with PowerMock. The implementation is still experimental and we are aware of some bugs and limitations. To know more and find help getting started with the agent please have a look at the documentation.

The solution I have found is not yet another tool, or framework, but a simple pattern. I have not found an existing name for the pattern so I call it Sync-Async Pair. The idea is to hide the complexity of asynchronous calls and call-back behind a facade, and have a straightforward synchronous implementation. An implementation that is easy to write, test and extend.

Sync-Async Pair Pattern

As the name suggests the pattern consist of a method pair, one is synchronous, and the other is asynchronous. The asynchronous method is backed by the synchronous implementation, and is only responsible for erecting a simplified facade in front of the sometimes complex concurrent implementation.

The first priority is to expose a clean API, the second priority is an implementation that is simple, testable and maintainable. Turns out that beginning with a clean API design (how to use the code), steers you towards a clean implementation (how to write the code). Most developers do the mistake to first write the code, and then struggle to try to use their own code.

Sync-Asyn Pair begins by defining a public API that then steers towards a clean implementation. Let's assume we are implementing some kind of a book reader. We need two model objects; CWBook and CWChapter. Both books and chapters are fetched from the network, so we need concurrency in order to not block the UI thread.

For this example let's focus only on fetching the chapters associated with a book. All other operations would be implemented in a similar fashion. First we need a public API, two methods for the sync-async pair on CWBook, and a delegate protocol.

@protocol CWBookFetchChaptersDelegate <NSObject>
-(void)bookDidFetchChapters:(CWBook*)book;
-(void)book:(CWBook*)book failedFetchChaptersWithError:(NSError*)error;
@end
// ...
-(BOOL)fetchChaptersWithError:(NSError**)error;
-(void)fetchChaptersWithAsyncDelegate:(id<CWBookFetchChaptersDelegate>)delegate;

The synchronous method fetchChaptersWithError: is very straightforward to implement.

-(BOOL)fetchChaptersWithError:(NSError**)error;
{
    NSURL* url = [self URLForFetchingChapters];
    NSData* data = [NSData dataWithContentsOfURL:url
                                         options:0
                                           error:error];
    if (data) {
        return [self parseChaptersFromData:data
                                     error:error];
    }
    return NO;
}

How to construct the NSURL, or possibly NSURLRequest is not important. Nor is it important how you actually parse the resulting data sent from the server, that is totally left to our problem domain. Just make it synchronous and super easy to write automatic unit checks for!

Adding Concurrency

Now to the hard part, that is actually quite manageable; adding the concurrency. For this example I will use performSelectorInBackground:withObject:. Your specific implementation might need NSOperationQueue, or GCD if you have special needs such as queues or cancelable operations. The idea will remain the same.

The most basic, and common, implementation for the public async method simply fires off the concurrent operation:

-(void)fetchChaptersWithAsyncDelegate:(id<CWBookFetchChaptersDelegate>)delegate;
{
    [self performSelectorInBackground:@selector(fetchChaptersWithDelegate:)
                           withObject:delegate];
}

In most cases all it does is request a private method to be performed on a background thread at some point in time. This is also the perfect place to abort operations early, for example a no-op if a request such as fetching a thumbnail is already in progress.

The private method previously launched is where most of the work is done. This is where a NSAutoreleasePool is setup, the synchronous method is called, and the delegate is properly called on the proper thread. The basic implementation for fetching books would be:

-(void)fetchChaptersWithDelegate:(id<CWBookFetchChaptersDelegate>)delegate;
{
    NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
    NSError* error = nil;
    BOOL success = [self fetchChaptersWithError:&error];
    if (success) {
        [[NSInvocation invocationWithTarget:delegate
                                   selector:@selector(bookDidFetchChapters:)
                            retainArguments:YES, self] invokeOnMainThreadWaitUntilDone:NO];
    } else {
        [[NSInvocation invocationWithTarget:delegate
                                   selector:@selector(book:failedFetchChaptersWithError:)
                            retainArguments:YES, self, error] invokeOnMainThreadWaitUntilDone:NO];
    }
    [pool release];
}

For this example I am using my NSInvocation additions to perform the delegate callbacks to the main thread, but it could just as easily been done using GCD. The additions are described in detail and available for download in this blog post, or from in the CWFoundation repo on github.

As you can see the boiler plate code for managing concurrency is actually very minimal, much less than 10 statements in total. The boilerplate code is almost reduced to only calling other methods, leaving very very few opportunities for bugs to sneak in.

Core Data in the Mix

The Sync-Async Pair pattern also lend itself beautifully for writing multi-threaded Core Data. The hassle of juggling NSManagedObjectID and NSManagedObject instances from different contexts can be completely left up to the private implementation that calls the synchronous method.

The private method responsible for calling the synchronous method and handling callbacks could be implemented as such:

-(void)notifyDelegate:(id)delegate ofSuccess:(BOOL)success withError:(NSError*)error;
{
    self = [[NSManagedObjectContext threadLocalContext] objectWithID:[self objectID]];
    if (success) {
        [delegate bookDidFetchChapters:self];
    } else {
        [delegate book:self failedFetchChaptersWithError:error];
    }
}

-(void)fetchChaptersWithDelegate:(id<CWBookFetchChaptersDelegate>)delegate;
{
    NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
    self = [[NSManagedObjectContext threadLocalContext] objectWithID:[self objectID]];
    NSError* error = nil;
    BOOL success = [self fetchChaptersWithError:&error];
    [[NSInvocation invocationWithTarget:self
                               selector:@selector(notifyDelegate:ofSuccess:withError:)
                        retainArguments:YES, delegate, success, error] invokeOnMainThreadWaitUntilDone:NO];
    [pool release];
}

Not much extra code, and the ugliness of handling Core Data's managed contexts is neatly hidden for clients at a single point in the code, that is much easier to test, debug, and maintain than most conventional Core Data code.

Conclusions

The simple Sync-Async Pair pattern makes it easy to write robust concurrent code. It is even a pattern that by design yields an elegant public API. It is also very easy to test, extend and even change the internal implementation completely without affecting any clients.

My friend and colleague Christian Hedin gave me tips on iCuke. Cucumber has been widely used for testing web applications, but now it’s also possible to test iOS (iPhone and iPad) apps with help of the iCuke library (https://github.com/unboxed/icuke). iCuke uses AppleScript to drive XCode in order to launch your application into the iOS Simulator. A preloaded library is used to inject a small HTTP server into your application. The HTTP server allows you to see an XML representation of the iOS device screen and to emulate input, such as taps, swipes and pinch gestures.

You can read more about it here:
http://www.unboxedconsulting.com/blog/cucumber-iphone-icuke
http://pragprog.com/magazines/2010-07/bdd-on-iphone-icuke

Really excited about Cucumber on iPhone we decided to give it a try. After installing iCuke and doing some test runs it was clear that some challenges had to be overcome to make this a truly useful tool for Behaviour Driven Development and automated testing of iOS apps.

Challenge 1: Screen returns the previous screen's xml

In our test, we wanted to tap a button, come to another screen and expected to see a text. When the test runs the iPhone is driven to the correct screen, e can see the expected text but the test fails anyway. After little debugging we realized that the method screen.xml returns old xml directly after changing the screen.

Solution:

We needed to refresh screen before checking if the expected text is on the new screen. iCuke has a method to refresh screen but it is private and we could not use it. So we just added a new method to the existing ICukeWorld class.

class ICukeWorld
 def refresh_screen
   refresh
   screen
 end
end

Calling this method before checking for the presence of the text solved this problem.

Later on, I forked iCuke and added this and some other methods.

git://github.com/DavorC/icuke.git

Challenge 2: Timing

Everybody knows that using sleep and delays in code is not so flexible.

sleep 3 # wait 3 seconds

We want to check something on the screen and give it 3 seconds to finish its loading.

Is it enough? Maybe, maybe not. Screen content loading could take 0.1 second or 5 seconds or... - You know what I mean.

In the first case loading is finished quickly and we unnecessarily spend 3 seconds for doing nothing . If we have a lot of delays in our code then our tests would waste a lot of precious time.

In the second case the delay is not long enough and the test fails.

Solution:

We need to write some help functions to wait for different items which are expected: some text, a button, downloading spinner etc.

Example with wait for text:

def wait_for_text(text, timeout = @@timeout)
 puts "#{method_name}(#{text}, #{timeout})" if @@debug
 refresh_screen
 start_time = Time.now
 until(screen.exists?(text)) do
   if Time.now - start_time > timeout
     flunk("#{method_name}: Timed out after #{timeout} seconds")
   end
   sleep 0.1
   refresh_screen
 end
end

As you can see the method is waiting for the text to appear and does checking every 0.1 second. As soon as the text is found the test continues. If, after given timeout, the text is still not found, the test fails.

I prefer to use unit test assertions in my tests (flunk is an assert which always fails). To use assertions with cucumber you need to add assertions to the Cucumber World:

require 'test/unit/assertions'
World(Test::Unit::Assertions)

Challenge 3: Different tappable object on screen can have the same text label.

Identifying objects on the screen only by text is not enough. By default, the first tappable object is tapped. What if we want to tap the second one?

Solution:

I created a set of help functions for:

returning all objects that satisfy some criteria
returning a specific object
waiting for a specific object
checking if a specific object exists

Objects are described in xml by: type, label, traits and index.

Here is an example of getting an array of all elements satisfying given attribute values.

def get_all_elements_by_type_label_and_traits(type, label, traits)
 puts "#{method_name}(#{type}, #{label}, #{traits})" if @@debug
 refresh_screen
 doc = REXML::Document.new(screen.xml.to_s)
 elements = REXML::XPath.match(doc, "//#{type}[@label=#{label.inspect}][@traits=#{traits.inspect}]")
 elements
end

Observe how it is easy to parse xml using ruby's REXML library.

Of course I could write more generic methods and decrease number of code lines - something like:

get_all_elements(type, options = {})

but I like readability so I wrote a set of help functions with more specific naming:

get_element_by_type(type, index = 0)
get_element_by_type_and_label(type, label, index = 0)
get_element_by_type_and_traits(type, traits, index = 0)
get_element_by_type_label_and_traits(type, label, traits, index = 0)
get_all_elements_by_type(type)
get_all_elements_by_type_and_label(type, label)
get_all_elements_by_type_and_traits(type, traits)
get_all_elements_by_type_label_and_traits(type, label, traits)
get_all_static_texts()
get_all_labels_by_type(type)
get_all_labels_by_traits(traits)
element_by_type_exists?(type, index = 0)
element_by_type_and_label_exists?(type, label, index = 0)
element_by_type_and_traits_exists?(type, traits, index = 0)
element_by_type_label_and_traits_exists?(type, label, traits, index = 0)
text_exists?(text)
wait_for_element_by_type(type, index = 0, timeout = @@timeout)
wait_for_element_by_type_and_label(type, label, index = 0, timeout = @@timeout)
wait_for_element_by_type_and_traits(type, traits, index = 0, timeout = @@timeout)
wait_for_element_by_type_label_and_traits(type, label, traits, index = 0, timeout = @@timeout)
wait_for_text(text, timeout = @@timeout)
get_center_of_the_element(element)
tap_coordinates(x, y)
double_tap_coordinates(x, y)
tap_element(element)
double_tap_element(element)
tap_text(text)
double_tap_text(text)

In order to use these functions you need to use iCuke from:

git://github.com/DavorC/icuke.git

Don't forget to use –recursive flag when you clone it:

git clone --recursive git://github.com/DavorC/icuke.git

After building and installing the iCuke gem you need to

require 'icuke/cucumber_ext'

instead of:

require 'icuke/cucumber'

Example of usage

In your feature-file:

Background:
 Given "myApp.xcodeproj" is loaded in the simulator

Scenario Outline: User try to login with different invalid credentials with valid signs
 When I am in "Account" section
  And I paste in username "<user>"
  And I paste in password "<pass>"
  And I tap Login button
 Then I will see alert dialog

 Examples:
 | user            | pass      |
 | test@test.com   | test      |
 | 123456          | qwertyui  |
 | ..@..com        | . . . €<> |

step definitions:

When /I am in "(.*)" section/ do |section|
 wait_for_text(section)
 tap(section)
 wait_for_element_by_type_and_label("UINavigationItemView", section)
end

When /I paste in username "(.*)"/ do |user|
 label = "E-mail"
 refresh_screen
 assert(get_all_static_texts().include?(label), "No text field with label #{label} was found")
 write_to_mac_clipboard(user)
 paste_clipboard_to_text_field("UITextFieldLabel", label)
end

When "I tap Login button" do
 wait_for_text("Login")
 tap("Login")
 wait_for_download_indicator_finish
end

Then "I will see alert dialog" do
 wait_for_element_by_type("UIAlertView")
end

Testing both iPhone and iPad
If you’re testing a universal app that runs on both iPhone and iPad I recommend writing different scenarios for the platforms. iPad in landscape mode is most likely to reuse most of code you have written for iPhone. Place them in different feature files and tag them with e.g. @iphone respective @ipad tags.

In your env.rb file:

$PLATFORM = "iphone"

Before('@ipad') do
 $PLATFORM = "ipad"
end

Before('@iphone') do
 $PLATFORM = "iphone"
end

Use in your feature files:

Given I have started application

Implementation:

Given "I have started the application" do
 # ... some code
 Given "\"myApp\" from \"myApp.xcodeproj\" is loaded in the #{$PLATFORM} simulator"
 # ... more code
end

If you wants to run iPad simulator in landscape mode:

def switch_ipad_to_landscape
 if(get_ipad_orientation == PORTRAIT)
   rotate_simulator_left
 end
end

where get_ipad_orientation is some application specific method to decide if the simulator is in portrait or landscape mode.

Running iCuke tests on Hudson server

It’s really nice to be able to run your test suite at given intervals, or when you commit to the source code repository. To run your iCuke tests on Hudson (which is a popular continuous integration build server) you must start your iPhone simulator from a terminal window. This is easiest to do by launching an AppleScript from Hudson.

Below are two scripts, one AppleScript and one shell script, that I used to run my iCuke tests from Hudson.

- run_cuke.scpt

-- run_cuke.scpt
tell application "Finder"
 set my_folder_path to container of (path to me) as text
 set posixPath to POSIX path of file my_folder_path
 set scriptPath to posixPath & "cuke.sh"
end tell
tell application "Terminal"
 activate
 do script scriptPath
end tell
delay 1200 -- it should be enough to finish all tests
set logscript to "grep -c FAILED " & posixPath & "cuke.log" & " | cat"
set cuke_failed to do shell script logscript
if cuke_failed > 0 then
 error "Cucumber tests failed"
end if
try -- do not leave terminals after test run
 do shell script "killall 'Terminal'"
end try

- cuke.sh

#!/bin/bash
scriptpath=$(cd ${0%/*} && echo $PWD/${0##*/})
SCRIPTFOLDER=`dirname "$scriptpath"`
echo Cucumber script is run in: $SCRIPTFOLDER
cd $SCRIPTFOLDER
cucumber $SCRIPTFOLDER/features --format=html --out $SCRIPTFOLDER/cuke_results.html > $SCRIPTFOLDER/cuke.log

Add these two scripts to your project.
Then, add this Cucumber hook to your env.rb file:

After do |s|
 if(s.failed?)
   puts "Scenario FAILED: <#{s.name}>"
   puts "More info about failure: SCREEN XML"
   puts screen.xml
 else
   puts "Scenario PASS: <#{s.name}>"
 end
end

On Hudson, create a new job and copy the configuration from your project's existing job.
Configure job:
add to description:

<a href="/hudson/job/Helios_Cuke/ws/cuke_results.html">Cucumber Results</a>
<b> | </b>
<a href="/hudson/job/Helios_Cuke/ws/cuke.log">Debug.log</a>

add build step (execute shell):

osascript ${WORKSPACE}/run_cuke.scpt

Of course you can change all scripts according your needs.
At the end you should have your iCuke tests running on Hudson and you will get both a nice test report in HTML and the debug output as a plain text.
There is a lot of potential for doing automated feature tests for iOS using Cucumber and with iCuke and the additions above you’ll hopefully be well on your way for doing BDD in your next iOS project!

I can understand the joys of making sure that every potential bug is removed, and the bragging rights of 100% automated test coverage as well. But I would argue that it is equally fraudulent to claim 100% automated test coverage to a client as a virtue. I am not arguing that automated tests should be abandoned, far from it. I am arguing that automated tests should be applied with the same care as optimizations.

Premature Automated Testing is Bad

We have all learned that premature optimizations are bad. Guessing what could fail is hard, the only safe route would be to write a check for everything! There is even a coding practice encouraging this, test driven development or TDD for short. Which should really be called ADD for assertion driven development. Since what is done is not testing, but checking by asserting conditions against a specification.

Take a moment and answer honestly; for any project with good automated test coverage what is the ratio between time spent writing the checks versus the time these checks have saved you? In my experience the more zealously automated test coverage is applied, the more time is spent maintaining the tests. For any project where automated tests have been written first, and code later, I have yet never experienced that the checks have saved more time than the time they cost to develop and maintain.

When optimizing it is hard to find the bottlenecks. Gut feeling is good many times, but often even your gut feeling is wrong and the performance bottle neck is somewhere you would never expect. It is the same with defects in code, your gut feeling is good for pin-pointing many defects, but often the cause is something you never expected. So instead of wasting time guessing what the problems may be down the road you should inspect and measure where the issues are. And when an issue is found then you write a check for it to ensure it does not break again.

Change the Algorithm

The best optimization is often not to optimize at all, but to change to a better algorithm. Writing a bubble sort algorithm in assembly will never yield a better result than a sloppy merge sort written in C anyway, so do not waste your time.

Same can be said about code quality. If you have many issues, and your code breaks often, adding automated tests will only be bandaid until it breaks the next time. Change the code into something more stable. If needed, write checks to ensure the new implementation conforms to the same public API when you refactor it. Take notice to write checks with as good coverage as needed when you are about to refactor your code not before.

If you architect your application with many small independent units, each with a clearly defined public API, then refactoring is not a big problem.

Perceived Quality

All green lights and a 100% automated test coverage looks good on a report. But it can never catch what is most important of all; is the application valuable to the user? In that regard the 100% automated test coverage is a false safety, it gives the illusion of quality. A dangerous illusion that can be used to blind managers, customers and other stakeholders from seeing the real issues with the application.

There is yet no testing framework made that is as efficient as a human being actually using the application. Not just people blindly following a test protocol, that is even worse. But stake holders using the application as intended. Not just short burst before committing code to source control, but also regularly on longer sessions. So that you get a feel for how the app is used, what works, or just what grinds your gears. Usability issues are also issues and should be dealt with.

Not every issue is created equal. With the limited time every project have fixing, or predicting, each and every one of them is never feasible. So do not waste time fixing the one in a million issue, or writing checks for the bread and butter code. But do make sure the application fails gracefully. Fail as gracefully as you can for the end-user, they are very forgiving if you fail with style and make sure no data is lost. But also fail gracefully for yourself, be generous with logging when you do fail. And remember to add an automated check so that the reason for the failure may never raise again, once you fixed it.

Model View Controller

All application logic can be divided into three categories. Model for the business logic of your product, View for the display and input to and from the user (or other application), and Controller that is the mediator between the two. Any application of any significant size will have smaller MVC patterns within their units as well. Well defined boundaries between what is a Model, View and Controller is not only good for re-usability, maintainability, and replacing parts. It is also great for testing. Have well defined and testable public APIs for the parts, and the private implementations tend to be simpler and of higher quality. Do not let the public API be defined by how something is implemented, let the implementation be defined by how you want to access it from the outside.

Writing unit tests for the Model of your application is seldom a waste of time. Often the Model is what you begin implementing, so using unit tests as the incubator until you have enough logic in place to implement the first draft of your application is only rational; code that is unused is seldom of any quality at all.

Writing unit tests for the View yield much less returns on investment. It is also highly probably that you will have to write these tests over and over again. I would go so far as call them harmful, too many automated tests on the Views may in fact discourage you from doing drastic but needed usability changes. And no automated test in the world can ever flag if a View is usable, looks good, and feels right to the end-user.

Writing unit tests for the Controller should be a waste of time, otherwise your Controller is doing too much. A Controller should only be the mediator between the View and Model, if it does more then it should be split up and the proper parts moved into Model and View layers. So writing unit tests for a good Controller is basically just verifying that your are using the APIs of the Model and Views correctly.

Conclusions

We have learned that premature optimization is bad, often counter productive or wastes our time for minimal gains. Let us all grow up and also learn that premature automated testing is equally bad. The time we have is to precious to waste on overzealous automated testing instead of adding real value to the product by applying tests where it makes real difference to the products quality.

Mocking final classes

Unlike EasyMock and Mockito, JMock doesn't support mocking classes out of the box. For this to work you need to have the jmock-legacy jar file in your classpath. But even if you have you cannot mock classes that are final:

 
public final class FinalClass {
 
    public String helloWorld() {
        return "Hello world";
    }
}
 

But here PowerMock can help. What you need to do is to use the PowerMock JUnit runner and to prepare the FinalClass for test:

 
@RunWith(PowerMockRunner.class)
@PrepareForTest(FinalClass.class)
 

But unfortunately this is not enough. For PowerMock to work it needs something called a "ProxyFrameworkImpl" that helps the reflection utilities in PowerMock to find an unproxied class type of a proxied class. It also needs an "AnnotationEnabler" that's used for injecting mocks. This may sound complicated but since JMock is using CGLib to create class proxies you can use PowerMock's EasyMock implementation of the "ProxyFrameworkImpl" and "AnnotationEnabler". This works because EasyMock is also using CGLib. Thus you need to put the powermock-api-easymock.jar in your class-path (note that you don't need the easymock.jar in the classpath).

Once we have the powermock-api-easymock.jar in the classpath and have added the two annotations we're ready to test the FinalClass. Here's a full example:

 
@RunWith(PowerMockRunner.class)
@PrepareForTest(FinalClass.class)
public class JMockFinalClassTest {
    private Mockery context = new JUnit4Mockery(){{
        setImposteriser(ClassImposteriser.INSTANCE);
    }};
 
    private FinalClass tested;
 
    @Before
    public void setup() throws Exception {
        tested = context.mock(FinalClass.class);
    }
 
    @Test
    public void mockFinalClassWithPowerMockAndJMock() throws Exception {
        // Given
        final String expected = "something";
        context.checking(new Expectations(){{
            one(tested).helloWorld();
            will(returnValue(expected));
        }});
 
        // When
        final String actual = tested.helloWorld();
 
        // Then
        assertEquals(expected, actual);
        context.assertIsSatisfied();
    }
}
 

Mocking Stubbing static methods

Now that we know how to mock final classes, how about static methods? The bad news is that you can't mock them. But the good news on the other hand is that PowerMock provides a MemberModification API that allows you to stub, suppress and replace methods. So to stub a static method regardless of the underlying mock framework you can do:

 
stub(method(ClassWithStaticMethod.class, "staticMethodName")).toReturn(someObject);
 

To demonstrate how this can be used with JMock imagine that we want to test the generateMessage method in this example:

 
public class JMockExample {
    private final FinalClass finalClass;
 
    public JMockExample(FinalClass finalClass) {
        this.finalClass = finalClass;
    }
 
    public String generateMessage() {
        return "Message is: "+ finalClass.helloWorld() + ClassWithStaticMethod.returnString();
    }
}
 

where FinalClass is the same as in the previous example and ClassWithStaticMethod looks like this:

 
public class ClassWithStaticMethod {
    public static String returnString() {
        return "a string";
    }
}
 

A full JMock test with PowerMock can look like this:

 
@RunWith(PowerMockRunner.class)
@PrepareForTest({FinalClass.class, ClassWithStaticMethod.class})
public class JMockStaticMethodTest {
    private Mockery context = new JUnit4Mockery(){{
        setImposteriser(ClassImposteriser.INSTANCE);
    }};
 
    private FinalClass finalClassMock;
    private JMockExample tested;
 
    @Before
    public void setup() throws Exception {
        finalClassMock = context.mock(FinalClass.class);
        tested = new JMockExample(finalClassMock);
    }
 
    @Test
    public void stubbingAndMockingWithPowerMockAndJMock() throws Exception {
        // Given
        context.checking(new Expectations(){{
            one(finalClassMock).helloWorld();
            will(returnValue("Hello "));
        }});
 
        // Stub the static method
        stub(method(ClassWithStaticMethod.class, "returnString")).toReturn("JMock");
 
        // When
        final String message = tested.generateMessage();
 
        // Then
        assertEquals("Message is: Hello JMock", message);
        context.assertIsSatisfied();
    }
}
 

You can check out the examples from our SVN repo.

Notes

JMock also provides its own JUnit runner called "JMock" that automatically calls context.assertIsSatisfied() after each test. Normally you can use PowerMock with other JUnit runners using the PowerMockRule but unfortunatley the JMock runner extends from a deprecated runner which doesn't support rules.

Conclusion

As you can see it's pretty simple to use some PowerMock basics even for unsupported frameworks as JMock. Of course it's possible to create full-blown support for JMock using PowerMock to enable e.g. new instance and static method mocking but until that day it's good to know that alternatives exist. If you're interested to know more about PowerMock please visit our webpage.

Naive example

Let's say we have a very simple Spring bean like this:

 
@Component
public class MyBean {
 
    public Message generateMessage() {
        final long id = IdGenerator.generateNewId();
        return new Message(id, "My bean message");
    }
}
 

and we want to test the generateMessage method in an integration test. In this test we assume that it's really important for us to mock the static call to IdGenerator.generateNewId(). But in order to run the test as a Spring integration test we need to use the SpringJUnit4ClassRunner which prevents us from using the PowerMockRunner. So by taking advantage of the PowerMockRule we can still benefit from PowerMock's functionality:

 
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:/example-context.xml")
@PrepareForTest(IdGenerator.class)
public class SpringExampleTest {
 
    @Rule
    public PowerMockRule rule = new PowerMockRule();
 
    @Autowired
    private MyBean myBean;
 
    @Test
    public void mockStaticMethod() throws Exception {
        // Given
        final long expectedId = 2L;
        mockStatic(IdGenerator.class);
        when(IdGenerator.generateNewId()).thenReturn(expectedId);
 
        // When
        final Message message = myBean.generateMessage();
 
        // Then
        assertEquals(expectedId, message.getId());
        assertEquals("My bean message", message.getContent());
    }
 

If you're used to PowerMock you can see that the only difference between this test and a standard PowerMock test is that we use the PowerMockRule instead of the PowerMockRunner. You can find the complete example in our SVN repo.

Limitations

For trivial tests like this the Objenesis deep-cloner works just as well as the X-Stream version but in more complex situations X-Stream is more robust. There may still be bugs lurking around and we're thankful for contributions. For example it's not possible to mock system classes using any of the implementations yet.

Conclusion

We'll continue to work on the Objenesis deep-cloner but until it's working better you can use the X-Stream version to combine PowerMock with frameworks that require a JUnit runner. Please refer to this page to get started with the rule.

Example 1 - JSON

Assume that a GET request to http://localhost:8080/lotto returns JSON as:

 
{
 "lotto":{
 "lottoId":5,
 "winning-numbers":[2,45,34,23,7,5,3],
 "winners":[{
   "winnerId":23,
   "numbers":[2,45,34,23,3,5]
 },{
   "winnerId":54,
   "numbers":[52,3,12,11,18,22]
    }
 ]}
}
 

REST assured can then help you to easily make the GET request and verify the response. E.g. if you want to verify that lottoId is equal to 5 you can do like this:

 
expect().body("lotto.lottoId", equalTo(5)).when().get("/lotto");
 

or perhaps you want to check that the winnerId's are 23 and 54:

 
expect().body("lotto.winners.winnerId", hasItems(23, 54)).when().get("/lotto");
 

Note that equalTo and hasItems are standard Hamcrest matchers.

Example 2 - XML

XML can be verified in a similar way. Image that a POST request to http://localhost:8080/greetXML returns:

 
<greeting>
   <firstName>{params("firstName")}</firstName>
   <lastName>{params("lastName")}</lastName>
</greeting>
 

i.e. it sends back a greeting based on the firstName and lastName parameter sent in the request. You can easily perform and verify e.g. the firstName with REST assured:

 
with().parameters("firstName", "John", "lastName", "Doe").expect().body("greeting.firstName", equalTo("John")).when().post("/greetXML");
 

You can also verify XML responses using X-Path. For example:

 
expect().body(hasXPath("/greeting/firstName[text()='John']")).with().parameters("firstName", "John", "lastName", "Doe").post("/greetXML");
 

Authentication

REST assured also supports some authentication schemes, for example basic authentication:

 
given().auth().basic("username", "password").expect().statusCode(200).when().get("/secured/hello");
 

Other supported schemes are OAuth, Digest and certificate authentication.

Conclusion

As you can see it can be really simple to test REST services in Java. REST Assured supports the POST, GET, PUT, DELETE and HEAD http methods and includes specifying and validating e.g. parameters, headers, cookies and body easily. Visit our homepage for more info and downloads.

Testing asynchronous systems is hard. Not only does it require handling threads, timeouts and concurrency issues, but the intent of the test code can be obscured by all these details. Awaility is a DSL that allows you to express expectations of a asynchronous system in a concise and easy to read manner.

Simple example

So let's assume that we send an "add user" message to our asynchronous system like this:

 
publish(new AddUserMessage("John Doe"));
 

In your test case Awaitility can help you to easily verify that the database has been updated. In its simplest form it may look something like this:

 
await().until(newUserIsAdded());
 

Awaitility will now block the test execution and wait until the new user is added to the database. newUserIsAdded is a method that you implement yourself in your test case. It specifies the condition that must be fulfilled in order for Awaitility to stop waiting.

 
private Callable<Boolean> newUserIsAdded() {
	return new Callable<Boolean>() {
		public Boolean call() throws Exception {
			return userRepository.size() == 1; // The condition that must be fulfilled
		}
	};
}
 

By default Awaitility will wait for 10 seconds and if the size of the user respository is not equal to 1 during this time it'll throw a TimeoutException failing the test. If you want a different timeout you can define it like this:

 
await().atMost(5, SECONDS).until(newUserWasAdded());
 

Better reuse

Awaitility also supports splitting up the condition into a supplying and a matching part for better reuse. The example above can also be written as:

 
await().until( userRepositorySize(), equalTo(1) );
 

The userRepositorySize method is now a Callable of type Integer:

 
private Callable<Integer> userRepositorySize() {
	return new Callable<Integer>() {
		public Integer call() throws Exception {
			return userRepository.size(); // The condition supplier part
		}
	};
}
 

equalTo is a standard Hamcrest matcher specifiying the matching part of the condition for Awaitility.

Now we could reuse the userRepositorySize in a different test. E.g. let's say we have a test that adds three users at the same time:

 
publish(new AddUserMessage("User 1"), new AddUserMessage("User 2"), new AddUserMessage("User 3"));
 

We now reuse the userRepositorySize supplier and simply update the Hamcrest matcher:

 
await().until( userRepositorySize(), equalTo(3) );
 

Reduce verboseness

Since Java is so verbose Awaitility also provides a way achieve the same result by building up the supplier part without defining a Callable:

 
await().untilCall( to(userRepository).size(), equalTo(3) );
 

to is method in Awaitility which you can use to define the suppling part inline in the await statement. Which option you like best depends on the use case and readability.

More features

Awaitility uses polling to check if a condition is fulfilled. You can easily specify a polling interval and poll delay (the delay before the first poll commits):

 
with().pollInterval(400, MILLISECONDS).and().pollDelay(2, SECONDS).await().forever().until(somethingHappens());
 

In some cases it's also useful to create named awaits for example if you need multiple awaits in one test. A naive example:

 
publish(new AddUserMessage("User 1"));
await("user 1").until(userRepositorySize(), is(equalTo(1)));
publish(new AddUserMessage("User 2"));
await("user 2").until(userRepositorySize(), is(equalTo(2)));
 

So if the first await statement fails the TimeoutException will indicate that it was the "user 1" statement that failed and so on.

Conclusion

We hope that you find Awaitility easy to use when you need to synchronize asynchronous behaviour in Java. However be aware that since it uses polling to verify that the condition is fulfilled it's not recommended to use it for precise performance testing. In these cases it could be better to use an AOP framework such as AspectJ and leverage on its compile-time weaving.

Please visit our homepage for more info and downloads.

This is the second part of the blog post I wrote about TDD in XCode. Make sure you've followed that tutorial before continuing here, or at least have a project of your own with OCUnit tests ready.

The server which we'll use is called Hudson and is very popular in the Java community. It does not readily support XCode projects but it does allow shell commands, so we'll use that to make our Calculator project compatible. We also want to fetch new versions of the source code as soon as it's checked in. We'll be a bit fancy and use Git as source control software. Here's a summary of what we want to do:

1. Download and install the Hudson continuos integration server
2. Download, install and setup the Git source control software as well as the Git plugin for Hudson
3. Download and install a script that converts OCUnit test results to JUnit format
4. Setup a new job in Hudson that checks out our XCode project and builds and tests it
5. Set up a notification and try it out

Downloading and Installing Hudson

In order to keep the instructions in this tutorial simple, we will have the source code repository and the build server itself on localhost. In real life you'll probably want to setup a dedicated machine.

Head over to the Hudson web site and download your copy of the server. Follow their included installation instructions, it's quite simple. Once you have Hudson up and running you can visit http://localhost:8080 to see the welcome screen. By the way, when you're ready to install Hudson on a real remote server I recommend running it as a Servlet inside Apache Tomcat instead.

Downloading and Setting Up Git

Next we want to install Git. There are several ways to install it. The simplest might be to use the OS X installer, but I personally prefer MacPorts. Either way, go ahead and install it and make sure that you can use the "git" command from Terminal.app when you're done.

In the Terminal, change directory to your XCode project folder and type:

		git init
	

You now have a working source code repository in your project folder. Neat. Before you continue create a text file called .gitignore and let it contain:

		build/*
		*.pbxuser
		*.mode1v3
		.DS_Store
		test-reports/*
	

That's a listing of the files and folders that should not be version controlled, since they are temporary or just used for XCode. Feel free to add any other files that you think only should be present locally. Now add all files (that aren't ignored) using:

		git add *
	

and finally commit them to a first version using:

		git commit -m "Initial import"
	

Fetching the Script

We need to fetch one more thing, ocunit2junit.rb, which is a little Ruby script that I wrote. Since you now have git, you can also download it using git (or clone the repo, which is the correct term). Change directory to /tmp and then type like this:

git clone git://github.com/ciryon/OCUnit2JUnit.git

Copy the script from /tmp/OCUnit2JUnit/ anywhere, like to /usr/local/bin/. Make sure it's executable by typing chmod +x ocunit2junit.rb.

The script will parse output from xcodebuild (the console command for building your XCode project) and look for output from OCUnit. The test results (success, failure, time passed etc) will be saved in the "test-reports" folder in the same XML format that JUnit uses. This also happens to be a format that Hudson understands.

Setting Up Hudson

Now head back to your Hudson welcome screen. Select Manage Hudson, Manage plugins and check "Hudson GIT plugin" under Available. Hudson will automatically download and install the plugin. Afterwards just restart Hudson.

We're ready to create a job in Hudson for our project. From the main screen select "New job", Build a free-style software project and call it "Calculator". There are some settings which you should fill in:

• Select to use git as source code management tool. The URL should be the file path to your project, like:

  /Users/youruser/XCodeProjects/Calculator/

• Select to poll SCM with Schedule:

  * * * * *

  That means it'll check every minute if there are changes. If you want to check less often, do that.

• Add new build step and select to execute shell. Use this command (make sure to point to the version of the iPhone SDK you want to use):

  xcodebuild -target "UnitTests" -sdk /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator3.1.2.sdk/ -configuration "Debug" | /usr/local/bin/ocunit2junit.rb

• Select to also publish a JUnit test result report and the Test report XMLs reside at

  test-reports/*.xml

Trying it out

This is the moment of truth. Try to change your CalculatorTest.m so that one of the test cases fail. Type:

	git -a commit CalculatorTest.m "Broke the test, on purpose"

and see what happens. If all goes well, and you have entered everything correctly, Hudson should automatically notice there's a new version available, build it and try to run the test suite. Check the results and how it reports the test failure. Nice, eh? Fix the error and commit the test class again to see how Hudson reacts.

There are many settings, plugins and cool things to try out in Hudson. Go into Manage Hudson-> Configure system and setup your email settings, to allow Hudson to send mail. Then go back to configure your Calculator job and at the bottom select E-mail Notification. Enter your own email address and see how it works after you commits a file that breaks the build. If you don't want a mail, you can get an instant message, or perhaps have the server play a loud annoying sound? Everyone in your team should be aware of when a build fails, so it can be fixed right away. This is one of the great advantages of having a continuos integration server.

I soon came to understand that I would not be able to take advantage of all the wonderful possibilities that instrumentation offers. The reason for that is quite simple; the application that we are in the process of developing is not only extensive but also complicated with multiple activities, self-defined intents, and hundreds of views that also include scrollable lists. It would not make sense for me to spend a month writing one single test case that would only take 20 seconds to test manually. That is how Robotium was born. I needed a test framework that would help me write good and powerful test cases that emulated real users. The test case should be able to do what a real user does: click on anything that is clickable, look for irregularities, automatically move from activity to activity, etc. More importantly, I should not have to spend more than 10 minutes writing a test case that involves more then one activity.

With the help of Robotium a test case spanning over multiple activities could look like this:

public void testTextIsSaved() throws Exception {
   solo.clickOnText("Other");
   solo.clickOnButton("Edit");
   assertTrue(solo.searchText("Edit Window"));
   solo.enterText(0, "Some text for testing purposes")
   solo.clickOnButton("Save");
   assertTrue(solo.searchText("Changes have been made successfully"));
   solo.clickOnButton("Ok");
   assertTrue(solo.searchText("Some text for testing purposes"));
}

As you can see, I don’t have to specify any technical details or tell Robotium where to look for something, such as scrolling down a list when needed. It handles the above and more all on its own.

If you are interested in writing test cases of similar nature have a look at http://www.robotium.org. It makes writing powerful test cases a breeze.

Background

PowerMock is intended for developers who want to unit test code in isolation from its environment. The environment may be for example other classes or
objects, other systems such as a database or a file-system. To isolate the unit you
can use a technique called mocking. Mocks let you record and verify behavior
according to your needs in order to get a deterministic behavior when executing
your test, for example provide a certain return value. A similar technique is stubs
that simply remove functionality and you don’t care about the return value.

Why?

There already exists frameworks that provide both mocks and stubs, so why
create another one? First of all PowerMock is not a new framework built from
scratch, instead it extends existing mock frameworks such as EasyMock and
Mockito. In most cases these frameworks are still sufficient. However, there are
cases when more power is needed!

To be able to use mock objects your code needs to be designed in a way that
makes it possible to replace collaborators, for example by using the Dependency
Injection pattern. Most often this is a good thing, but sometimes it leads to more
complexity and a more complicated production code that is more difficult to
understand. Another problem is legacy code which is seldom designed with unit
testing in mind. When trying to introduce Dependency Injection you are often
forced to perform a gigantic refactoring touching almost every class. Depending
on how brave you are this can either be a problem or an opportunity.

Another common situation is when there is some initialization routine in your
code or some third party library that does something that prevents you from
unit testing your code. This can both be a constructor or a static initializer. For
example, did you know that an interface can have a static initializer? This means
that just implementing an interface can execute code beyond your control!

Goals

Since there are many good mock frameworks out there already we think that
instead of creating something entirely new we want to reuse the API’s that
people already use and extend them with more capabilities. This means that
PowerMock consists of extension APIs to other popular mock frameworks.
EasyMock was the first API that we supported since we have been using it a lot.
In version 1.1 we also have some basic support for Mockito and we have serious
plans for a JMock extension. PowerMock is fully extendable and you can create
an extension on your own as well.

A key goal of PowerMock is to allow you to unit test your code without having to
think about the testability aspects. We would of course never encourage you to
write inflexible code that is hard to maintain, but we leave the design decisions
up to you and not to the technical limitations of a test or mock framework.

Another important goal is that PowerMock should be just another jar file. As a
user you should not have to care how PowerMock does its magic. You just put it
in your classpath and then you’re good to go. You don’t need to install any IDE
plugin, change the JVM settings or mess with your build environment.

When to use it

Using a 3rd party or legacy framework

• Sometimes communication with a particular framework is done through
  static methods (for example Java ME). PowerMock allows you to setup
  expectations for these static methods and simulate the behavior you need
  to test.
• Other frameworks require you to subclass classes that contain static
  initializers or constructors doing something that prevents you from unit-
  testing your code in isolation. PowerMock allows you to remove static
  initializers and suppress constructors. It also allows you to mock classes
  inside a signed jar files even though a class is package-private.

Design

• You really want to enforce the fact that some methods are private. PowerMock allows mocking and testing of both private and final methods..
• You want to read or write private state from your test without adding unnecessary methods to the production code that are used only for test purposes.
• In some cases you want to create a new instance of an object in your production code but still be able to replace this instance with a mock object from your test.

Examples

All examples in this article are written with the EasyMock extension API.
The examples are written in a way that easily demonstrates the capabilities of
PowerMock and are sometimes very naive. In the examples both the code being
tested and the actual test are completely pointless and is not a recommended
way of coding! The reason why we use such examples is of course that code that
typically need PowerMock is often quite complex and it would be hard to focus
on what we are trying to demonstrate.

When writing tests with PowerMock you basically always need to supply
two annotations at the class-level of your test as you will see in the examples
below. The first is the RunWith annotation which tells JUnit to run the test
with a specific JUnit runner. In PowerMock’s case you always specify the
PowerMockRunner, i.e. @RunWith(PowerMockRunner.class). This is what
bootstraps the entire PowerMock framework and this is how you can see that
a test is using PowerMock. The PowerMockRunner initializes PowerMock’s
classloader which starts to load all classes relevant for your particular test. Some
of these classes need to be prepared in order to allow for testability. These are
the classes that you wish to mock, for example those containing final or static
methods. You supply these classes to the @PrepareForTest annotation. What
PowerMock does then is to modify the byte-code of these classes at run-time so
that they become testable. Let’s look at a couple of examples to clear things up.

Mocking final classes or methods

Imagine writing a unit test for this fictive class:

public class SomeService {
    private final SomeDao someDao;
    public SomeService(SomeDao someDao) {
        this.someDao = someDao;
    }
    public String getFormattedData() {
        String data = ”Data is missing”;
        final String actualData = someDao.getData();
        if (actualData != null) {
            data = actualData;
        }
        return data;
    }
}

For those of you who are used to unit testing it doesn’t look like any problems
at first since it’s easy to inject a mock implementation of the SomeDao class to
an instance of the SomeService at construction since it’s using the Dependency
Injection pattern. But let’s look at the actual implementation of the SomeDao
class.

public final class SomeDao extends JdbcDaoSupport {
    public final String getData() {
        /* Query a database for data,
            this is not something we want to do in a unit test */
        getJdbcTemplate().queryForObject(”some sql”, String.class);
    }
}

As you may see this class communicates with a database which is typically
something we want to avoid in a unit test. The problem is that both the class
and the method is final so we wouldn’t be able to mock this class with standard
EasyMock. But PowerMock makes this possible using the standard EasyMock
syntax:

@RunWith(PowerMockRunner.class)
@PrepareForTest(SomeDao.class)
public class SomeServiceTest {
    @Test
    public void returnedValueIsActualData() throws Exception {
        final String expectedData = ”data”;
         // We use PowerMock.createMock(..) to create
         // the mock object since it’s final.
        SomeDao someDaoMock = createMock(SomeDao.class);
        expect(someDaoMock.getState()).andReturn(expectedData);
        replayAll();
        SomeService tested = new SomeService(someDaoMock);
        final String actualData = tested.getFormattedData();
        verifyAll();
        assertEquals(expectedData, actualData);
    }
}

Looking closely you may see that we’re using the replayAll() and verifyAll()
methods of the PowerMock API. These methods will replay and verify all mocks
created by PowerMock so that you don’t have to keep track of them yourself.
Of course PowerMock provides the standard replay(..) and verify(..) methods as
well if you’d rather want to use those.

Mocking and testing private methods

Sometimes you may want to enforce encapsulation and use private methods.
There’s a debate going on on whether one should test private methods by
themselves or not but we’ll leave that decision up to you. Mocking a method
of the class being tested is known as partial mocking. PowerMock lets you do
partial mocking and also invoke private methods during a test. Let’s say we have
the following class:

public class DataService {
    public boolean replaceData(final String dataId,
                      final byte[] binaryData) {
        // ...do something... then
        return modifyData(dataId, binaryData);
    }
    public boolean deleteData(final String dataId) {
        // ...do something else... then
        return modifyData(dataId, null);
    }
    private boolean modifyData(String dataId, byte[] binaryData) {
        // Imagine this method doing something
         // complex and expensive.
    }
}

DataService has two public methods that both calls the private modifyData
method and you may want to unit test these methods. You may want to test
replace replaceData without invoking modifyData or maybe just test the private
method itself. PowerMock lets you do both. To mock the modifyData method
you create a partial mock using PowerMock.createPartialMock(..) and to setup
expectations for a private method you use the PowerMock.expectPrivate(..)
method. For the replaceData method the test could look like this:

@RunWith(PowerMockRunner.class)
@PrepareForTest(DataService.class)
public class DataServiceTest {
    @Test
    public void testReplaceData() throws Exception {
        final String methodName = ”modifyData”;
        final byte[] expectedBinaryData = new byte[] { 42 };
        final String expectedDataId = ”id”;
        // Mock only the modifyData method
        DataService tested =
             createPartialMock(DataService.class, methodName);
        /*
         * This is the simplest way to expect
         * a method call to a private method.
         */
        expectPrivate(tested, methodName, expectedDataId,
                expectedBinaryData).andReturn(true);
        replay(tested);
        assertTrue(tested.replaceData(expectedDataId,
                 expectedBinaryData));
        verify(tested);
    }
}

PowerMock also provides utility methods located in the Whitebox class that can
assist you with accessing internal state, executing private methods or constructors
etc. So if we’d like to test the modifyData method instead of mocking it we can
do so by using the Whitebox.invokeMethod(..) functionality:

boolean success = (Boolean) Whitebox.invokeMethod(tested, ”modifyData”,
”myDataId”, new byte[]{42}); 

As you can see you just supply the instance containing the method, the method
name itself and the arguments that you want to pass to the method.

Note that in general you should avoid testing and mocking private methods
as these tests are often fragile and tightly coupled with the implementation.
During refactoring, when you most need your test cases, you may be forced to
simply throw away tests that are too tightly coupled to the old implementation.
Therefore, when writing tests you should always focus on testing expected
behavior and not check implementation details.

Mocking new object constructions

Let’s say we have a class that looks like this:

public class Directory {
    public boolean createDirectoryStructure(String directoryPath) {
        File directory = new File(directoryPath);
        if (directory.exists()) {
            throw new IllegalArgumentException(”\”” +
                directoryPath + ”\” already exists.”);
        }
        return directory.mkdirs();
    }
}

When unit testing this class we want to isolate ourselves from the file-system
environment since what’s interesting for us to test whether a directory already
exists or not. So what we want to do is to create a mock object of the File class
and replace the file instance with this mock object. This is something that is
normally not possible since we’re creating a new File instance in our code. But
with PowerMock you can indeed do so:

@RunWith(PowerMockRunner.class)
@PrepareForTest( Directory.class )
public class DirectoryTest {
    @Test
    public void givenThatDirectoryDoesntExistAssertThatItIsCreated() {
        final String path = ”directoryPath”;
        File fileMock = createMock(File.class);
        expectNew(File.class, path).andReturn(fileMock);
        expect(fileMock.exists()).andReturn(false);
        expect(fileMock.mkdirs()).andReturn(true);
        replayAll();
        Directory tested = new Directory();
        assertTrue(tested.createDirectoryStructure(path));
        verifyAll();
    }
} 

As you can see we’re using the expectNew functionality of PowerMock to
intercept the call to new File and return the mock object instead. Note that we
prepare the Directory class for test and not the File class. The reason for this is
that it is the Directory instance that creates a new File instance and thus the File
class itself doesn’t have to be prepared for test.

Suppressing unwanted behavior

Sometimes you want to suppress a method, constructor or static initializer
of your production code when executing your test because they are of no
importance to the test or they even prevent you from testing your unit. Let’s
take a look at the following example:

public class ExampleWithBaseClass extends ThirdPartyFrameworkBaseClass {
    private final String message;
    public ExampleWithBaseClass(String message) {
        this.message = message;
    } 

    public String getMessage() {
        return message;
    }
}

This seems like an easy class to unit test (so easy in fact that you probably
shouldn’t test it, but let’s do it anyway for demonstration purposes). But wait,
let’s look at what the ThirdPartyFrameworkBaseClass class looks like:

public class ThirdPartyFrameworkBaseClass {
    public ThirdPartyFrameworkBaseClass() {
        System.loadLibrary(”framework.dll”);
    }
} 

This class tries to load a dll file which may not be present when you run
a unit test for the ExampleWithBaseClass class and thus will lead to
an exception. With PowerMock you can just suppress the constructor
of the ThirdPartyFrameworkBaseClass so that you can unit test the
ExampleWithBaseClass class in isolation from the native framework dll.
Sometimes this may also save you a lot of time not having to bootstrap an entire
framework. You can achieve this by using the suppressConstructor method of
the PowerMock API. In this case we would do:

@RunWith(PowerMockRunner.class)
@PrepareForTest(ExampleWithBaseClass.class)
public class ExampleWithBaseClassTest {
    @Test
    public void assertThatConstructorMessageDoesntChange() throws
    Exception {
        // This is how we suppress the constructor
        suppressConstructor(ThirdPartyFrameworkBaseClass.class);
        final String message = ”myMessage”;
        ExampleWithBaseClass tested = new
        ExampleWithBaseClass(message);
        assertEquals(message, tested.getMessage());
    }
}

That’s it! PowerMock can also suppress static initializers, methods and fields.

Access private state

For mutable objects internal state may change after a method has been invoked.
When unit testing such objects it’s good to have an easy way to get a hold of this
state and see if it has updated accordingly. PowerMock supplies several useful
reflection utilities specially designed for unit testing. All of these reflection
utilities are located in the org.powermock.reflect.Whitebox class.

For demonstration purposes let’s say we have a class that looks like this:

public class ServiceHolder {
    private final Set