Next, imagine that the Facade class and the Delegate interface below are part of a bigger system:

Create a matching easymock-test-config.xml (or mockito-test-config.xml).

Now, we have all the bits and pieces that are needed to start writing the integration test. We start simple to make sure that we autowiring and Spring context has been configured correctly:

The test is executed and it passes without any problem.

A second test is added to the same file with some added mock behavior:

Excellent, that test also passes.

Now that we have warmed up, we add a third test to the test class:

As expected, this test also passes without any problem.

Unfortunately, that is not always true. A closer investigation reveals that all tests pass when they are executed individually. When executing all three tests (either from the IDE or from a build tool like Maven), the third test fails unexpectedly:

java.lang.AssertionError:
  Unexpected method call Delegate.doSomethingElse():
        at org.easymock.internal.MockInvocationHandler.invoke(MockInvocationHandler.java:44)
	at org.easymock.internal.ObjectMethodsFilter.invoke(ObjectMethodsFilter.java:85)
	at $Proxy8.doSomethingElse(Unknown Source)
        [...]

This is EasyMock telling us that the mock object is in the wrong state. An EasyMock mock object goes through a series of steps:

1. Initialization - Instantiate the mock object.
2. Record - Record the expectations of the mock object.
3. Replay - Call replay() on the mock object so that it can replay the recorded state.
4. Test - Execute the test assertions.
5. Verify - Call verify() on the mock object to certify that the recorded mock expectations are fulfilled.

In an ordinary unit test, the mock object is thrown away after the verify phase, a new mock instance is created before the next test that can be used to record the new behavior. However, since we are creating a Spring integration test, there is one more thing to consider.

Reused beans

In the testing chapter of the Spring manual it is stated that:

By default, once loaded, the configured ApplicationContext and all of its beans are reused for each test method. Thus the setup cost is incurred only once (per test fixture), and subsequent test execution is much faster.

Consequently, all Spring beans, including mocked beans, will preserve their state between different test methods. The mock object will be in the verify state when the last test begins and not in the record state when delegateMock.doSomethingElse() is called. To solve the problem, we need some mechanism to ensure that the mock object is in a known, well-defined state before each test.

Reset mocks

Both EasyMock and Mockito provide methods to reset() the state of the mocked objects. Normally, resetting mock objects in the middle of a test method is considered to be a potential code smell, because it is likely that you are testing too much and that the test could be refactored into two separate tests accordingly. Nonetheless, resetting the mock instance @Before each test allow us to keep the same mock instance through the entire lifetime of the test class and yet put it in the record state when before each test starts:

When executing all tests in the test class, we see that that the problem has been solved and all tests pass.
For reference, click the link for the full source code of the EasyMock test class (and the Mockito test class).

Comments

You may choose to reset the mocks @After each test, the result will be the same.

The reset() method is called between the different test methods (because of the nature of @After and @Before), so the code smell of calling reset within a test is avoided.

Tests written using Mockito mocks are generally more tolerant for this kind of problem. In contrast to EasyMock, Mockito has no replay() method and no notion of steps. Therefore, it is possible to add additional mock expectations and to replace existing expectations, both after the test has been executed as well as after the result has been verified. Hence, you should always reset your mock objects before each test so that you do not get any accidental mocking behavior.

As a side note, another solution would be to annotate the test class (or all of its test methods) with the @DirtiesContext annotation. This will effectively recreate the entire application context after each test has been executed, causing all Spring beans to be in their initial state and thus the mock beans to be in the record state. Although this will work, there will be a significant performance penalty compared to the suggested reset() and @Before / @After solution, because all beans in the application context will be recreated and re-wired before each test method.

Dependencies

• Spring 3.0.6
• Mockito 1.8.5
• jUnit 4.10
• EasyMock 3.1

References

• Spring Reference Manual - Test Context Management and Caching
• Spring Reference Manual - SpringJunit4ClassRunner
• Spring Reference Manual - ContextConfiguration
• EasyMock
• Mockito

Previous post: Spring Integration Tests, Part I, Creating Mock Objects

(In this post, Mockito has been used for creating mock objects, but the same problem applies to EasyMock as well. You can find the corresponding files if you follow the provided links.)

Test setup

In the following example the SomeClass needs a reference to an instance of the SomeDependency interface:

When writing the integration test, we use the static Mockito.mock(Class classToMock) method to create a mock instance. The straightforward approach would be to let Spring instantiate the mock object with a static factory method in the xml configuration, e.g.

(Corresponding EasyMock config.)

To verify that the beans are created and wired correctly, we write a simple test:

(Same test for EasyMock, just update the @ContextConfiguration with the EasyMock config file.)

To our surprise, the test fails:

[...]
java.lang.IllegalStateException: Failed to load ApplicationContext
Caused by: org.springframework.beans.factory.BeanCreationException:
    Error creating bean with name 'someClass': Injection of autowired dependencies failed;
nested exception is org.springframework.beans.factory.BeanCreationException:
    Could not autowire field: private com.jayway.example.SomeDependency com.jayway.example.SomeClass.someDependency;
nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException:
    No matching bean of type [com.jayway.example.SomeDependency] found for dependency:
    expected at least 1 bean which qualifies as autowire candidate for this dependency.
    Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
[...]

Explanation

In order to find a solution to the problem, we need to understand Spring's initialization process:

1. Load bean definitions.
   This step includes scanning the application context XML files, scanning packages defined by component-scan, and loading the bean definitions found into the bean factory. (BeanFactoryPostProcessors such as the PropertyPlaceHolderConfigurer may be called to update the bean definitions.)
2. Instantiate beans and store them in the application context.
   The bean factory creates the bean from the bean definitions. Bean dependencies get injected.
3. Bean post processing.
   All initializing methods (e.g. annotated with @PostConstruct, init-methods declared in the XML and the afterPropertiesSet() method will execute. Annotations such as @Required will be validated. Dynamic proxies in an AOP environment will be created and so on.

The problem that we see occurs due to a corner case in the first two steps. When the bean definition is created Spring scans through the application context XML file. It finds the declaration of the mock bean and a reflective call is made to find out the return type of the method defined by the factory-method declaration in an attempt to determine the bean's type. The return type of this method is declared as the formal type parameter <T> (remember that we use the Mockito.mock(Class<T> classToMock) as factory method). Consequently, the bean definition of someDependencyMock will be of java.lang.Object and not SomeDependency as one would think.

In the second step, the someClass bean is instantiated. Springs discovers the @Autowired annotation and tries to fetch an existing SomeDependency bean from the application context. None has yet been created, so the bean factory proceeds and looks for suitable bean definition in order to create an instance of the requested bean on the fly. This operation fails because the bean definition of someDependency is mapped to java.lang.Object. At this point, Spring bails out and throws the NoSuchBeanDefinitionException.

FactoryBean to the rescue

The problem is solved by implementing a custom FactoryBean that can be used whenever we need to mock an object:

(In the EasyMockFactoryBean, I have added an enum to define the type of mock object.)

Next, the Spring test config is updated to use the factory bean:

(Corresponding EasyMock config.)

Spring will still instantiate the beans in the order that they were declared in the mockito-test-config.xml, i.e. starting by creating an instance of SomeClass and then attempt to inject an instance of a SomeDependency into it before any has been created. The good news is that bean factory will scan through all declared FactoryBeans, calls its getObjectType() method, and if the returned type is correct, call its getObject() which should return an instance of the requested type.

Alternative solution

There is another more brittle solution to the problem. By simply changing the order of the tags in the initial application context, e.g. putting the bean declaration that defines the mock above the component scan, will cause the test to pass. The bean definition will still be wrong in the sense that the someDependencyMock will be defined as a java.lang.Object, but there is another important difference. In this case, the bean factory will invoke the Mockito.mock(Class<T> classToMock) method with SomeDependency as parameter causing the someDependencyMock bean to be created and subsequently stored in the application context. Later, the someClass bean is created and the dependency can obtained from the application context when requested by the bean factory and the wiring of the beans will succeed. However, making sure that the order of the bean declarations in the XML is always correct may neither be simple nor obvious, and it will get increasingly more difficult the more beans you add to the configuration.

Disclaimer

As stated in the in the first paragraph, I would only consider this solution for integration tests because of performance reasons. The application context itself adds some overhead, but the big performance hit is to create and wire all beans that are associated with it. When writing unit tests there is no need for this, so you could revert to constructor or setter dependency injection, Spring's ReflectionTestUtils.setField(), PowerMock's Whitebox.setInternalState(), Mockito's @InjectMocks or similar techniques.

Acknowledgements

Thanks to Wilhelm Kleu at stack overflow for suggesting the solution and to Mattias Hellborg Arthursson for valuable discussions.

Dependencies

• Spring 3.0.6
• Mockito 1.8.5
• jUnit 4.10
• EasyMock 3.1

References

• Spring Reference Manual - Static Factory Method
• Spring Reference Manual - FactoryBean
• Spring Reference Manual - Integration Testing
• Mockito
• EasyMock

Next post: Spring Integration Tests, Part II, Using Mock Objects

However, in some cases it is difficult to set up sufficiently precise expectations for the mocks

• because we don't know in advance with which arguments they get called or how often they get called
• or because the verification, we have to make in the test, involves relating the calls to one mock object to the calls to another mock object or relating calls to mock objects to data returned from the object being tested.

In such cases the solution may be to capture "traces" of the method calls to the mock objects, and then, after having exercised the object being tested, to make the needed verifications using the captured traces.

Example

Consider as an example a card game simulation with a Dealer object collaborating with multiple Player objects:

public interface Dealer {
    public void deal(Set<Card> deck);
    public void addPlayer(Player player);
}
 
public interface Player {
    public void addCard(Card card);
}

The rules for the Dealer are:

• Each player must get at least two, and at most six cards
• Four cards for each player must be dealt in total
• Cards cannot disappear, ie. after dealing, any card not given to a player must remain in the deck
• Cards dealt to the players must be removed from the deck
• The dealer may deal the cards randomly

Naive test

Now suppose we want to test the Dealer using EasyMock to mock the Players. How can we set up the expectations to test that the Dealer follows the rules? We can expect that the Dealer will call Player.addCard(), but we can't assume in advance exactly how often and with which cards. We may set up expectations like this:

@Test
public void testDealFor2Players()
{
    Player player1Mock = createMock(Player.class);
    player1Mock.addCard( (Card) anyObject() );
    expectLastCall().times(2, 6);
 
    Player player2Mock = createMock(Player.class);
    player2Mock.addCard( (Card) anyObject() );
    expectLastCall().times(2, 6);
 
    replay(player1Mock, player2Mock);
 
    Dealer dealer = new DealerImpl();
    dealer.addPlayer(player1Mock);
    dealer.addPlayer(player2Mock);
 
    Set<Card> deck = createDeck();
    dealer.deal(deck);
 
    verify(player1Mock, player2Mock);
}

With this naive test we only get to test the rule "Each player must get at least two, and at most six cards" - this is clearly not by any measure a good enough test of the Dealer.

In order to test the other rules, we have to somehow relate the actual calls to player1Mock.addCard() to the actual calls to player1Mock.addCard() and to the state of the deck after executing Dealer.deal().

Simple capture

EasyMock has a feature called Capture which allows us to capture the arguments with which mocks are called, such that we can verify the argument values after having exercised the object being tested. It works something like this:

Player player1Mock = createMock(Player.class);
Capture<Card> capturedCard = new Capture<Card>();
// Expect one call to addCard with any arg. Capture arg value for later
player1Mock.addCard( capture(capturedCard) ); 
 
// ...
 
dealer.deal(deck);
 
// Now we can verify what we want about the captured card, eg.
assertEquals( Card.Suit.SPADES, capturedCard.getValue().getSuit() );

This seems like a step in the right direction. However, using this capture feature is hard when we have to capture multiple values from multiple calls to the mock objects, and doesn't really seem to work at all when we don't know the number of calls in advance.

Capturing values from multiple calls

The solution to the problem lies in writing our own EasyMock argument matcher, similar to how EasyMock implements capture internally, but extended so it can accept multiple argument values for multiple methods calls:

• Create a class MultiCaptureMatcher implementing org.easymock.IArgumentMatcher. The class has a collection instance variable to collect the argument values. EasyMock delegates to an instance of this class whenever the corresponding mock object method gets called.
• Create a static method to be used when setting up expectations - this static method creates an instance of the MultiCaptureMatcher and tells EasyMock to use that matcher for the correponding expected method call.

Here is the code:

public class MultiCaptureMatcher<T> implements IArgumentMatcher {
 
    Collection<T> captureDestination;
 
    public MultiCaptureMatcher(Collection<T> captureDestination) {
        this.captureDestination = captureDestination;
    }
 
    @Override
    public void appendTo(StringBuffer buffer) {
        buffer.append("multiCapture(").append(captureDestination.toString()).append(")");
    }
 
    @Override
    public boolean matches(Object actual) {
        captureDestination.add((T) actual);
        return true;
    }
 
    public static <S> S multiCapture(Collection<S> destination) {
      reportMatcher(new MultiCaptureMatcher<S>(destination));
      return null;
  }
}

With this argument matcher we can now set up "expectations" like this:

Player player1Mock = createMock(Player.class);
Set<Card> player1Cards = new HashSet<Card>();
player1Mock.addCard( MultiCaptureMatcher.multiCapture(player1Cards) );
expectLastCall().anyTimes();

I put "expectations" in quotation marks because it is not really an expectation in any real sense, but rather tells EasyMock, that the addCard method may get called any number of times with any arguments - EasyMock shouldn't do any verification, but just capture the arguments for later manual verification.

Here the player1Cards set acts as a destination for captured values. Every time during the test that player1Mock.addCard() gets called, the argument gets added to the player1Cards set.

After the Dealer has been tested, we can then use the captured values in player1Cards to whatever verifications we may want.

Here is a complete example using MultiCaptureMatcher to test all of the Dealer rules:

@Test
public void testDealFor2Players()
{
    Player player1Mock = createMock(Player.class);
    Set<Card> player1Cards = new HashSet<Card>();
    player1Mock.addCard( MultiCaptureMatcher.multiCapture(player1Cards) );
    expectLastCall().anyTimes();
 
    Player player2Mock = createMock(Player.class);
    Set<Card> player2Cards = new HashSet<Card>();
    player2Mock.addCard( MultiCaptureMatcher.multiCapture(player2Cards) );
    expectLastCall().anyTimes();
 
    replay(player1Mock, player2Mock);
 
    Dealer dealer = new DealerImpl();
    dealer.addPlayer(player1Mock);
    dealer.addPlayer(player2Mock);
 
    Set<Card> deck = createDeck();
    // Store original deck for later comparison
    Set<Card> originalDeck = new HashSet<Card>(deck);
    dealer.deal(deck);
 
    verify(player1Mock, player2Mock);
 
    // Each player must have been dealt 2 to 6 cards
    assertTrue(player1Cards.size()>=2);
    assertTrue(player1Cards.size()<=6);
    assertTrue(player2Cards.size()>=2);
    assertTrue(player2Cards.size()<=6);
 
    // For two players 8 cards must have been dealt
    assertEquals(8, player1Cards.size()+player2Cards.size());
 
    // Check that no cards have disappeared, and that any card is either in
    // the deck or belongs to one but not both players
    assertEquals(originalDeck.size(), deck.size()+player1Cards.size()+player2Cards.size());
    assertEquals(originalDeck, SetUtils.union(deck, player1Cards, player2Cards));
}

Caveat

There is one catch to be aware of with this approach: When an argument object is captured, we capture a reference to the object - not a copy of the objects state. This means that if argument objects get modified after being captured, for example because the object being tested modifies the argument objects after calling the mocks, then the verification of captured objects will see the modified object state instead of the object state at the time of capture.

Framework independence

First of all the internal structure of the project has undergone major changes. PowerMock core is no longer coupled to EasyMock which means that PowerMock can now be used as a foundation for testability for other mock frameworks. Our intention is that in the future you pick a combination of a mock extension API of choice (such as EasyMock, Mockito or JMock) and a test framework of choice (such as JUnit or TestNG) to enable increased testability for your code. With the new release PowerMock has some basic support for Mockito using the Mockito extension api. This means that you can mock final and static methods, use annotations, enable partial mocking, mocking of signed classes even if they're package private and spying of final classes and methods. Expect more features in the upcoming releases! It's possible to create your own extension API's as well so you can benefit from the PowerMock features in virtually any mock framework.

Test listeners

Another noteworthy feature of the new release is the test listener support. This means that you can create listeners and get notifications of various events, such as before a test method is about to execute or the outcome of an executed test method etc. The purpose of these test listeners is to provide a framework-independent way to get and react to these notifications by implementing the org.powermock.core.spi.PowerMockTestListener and pass it to the PowerMockListener annotation. PowerMock has some built-in test listeners for you to use, for example it has an AnnotationEnabler that let's you use annotations to create your mock objects. For example consider the following piece of code:

@RunWith(PowerMockRunner.class)
@PowerMockListener(AnnotationEnabler.class)
 public class PersonServiceTest {

 	@Mock
  	private PersonDao personDaoMock;

  	private PersonService classUnderTest;

  	@Before
  	public void setUp() {
  		classUnderTest = new PersonService(personDaoMock);
  	}
   ...
  }

Using the @Mock annotation eliminates the need to setup and tear-down mocks manually which minimizes repetitive test code and makes the test more readable. The AnnotationEnabler works for both the EasyMock and Mockito API's. You can also supply the names of the methods that you wish to mock if you want to create a partial mock, for example:

@RunWith(PowerMockRunner.class)
@PowerMockListener(AnnotationEnabler.class)
 public class PersonServiceTest {

 	@Mock("getPerson")
  	private PersonDao personDaoMock;

  	private PersonService classUnderTest;

  	@Before
  	public void setUp() {
  		classUnderTest = new PersonService(personDaoMock);
  	}
   ...
  }

This piece of code will instruct PowerMock to create a partial mock of the PersonDao where only the "getPerson" method is mocked. Since EasyMock has support for nice and strict mocks you can use the @MockNice and @MockStrict annotations to get the benefits of this.

Mock policies

One really cool feature of the 1.1 release is something that we refer to as mock policies. A Mock Policy can be used to make it easier to unit test some code with PowerMock in isolation from a certain framework. A mock policy implementation can for example suppress some methods, suppress static initializers or intercept method calls and change their return value (for example to return a mock object) for a certain framework or set of classes or interfaces. A mock policy can for example be implemented to avoid writing repetitive setup code for your tests. Say that you're using a framework X that in order for you to test it requires that certain methods should always return a mock implementation. Perhaps some static initializers must be suppressed as well. Instead of copying this code between tests it would be a good idea to write a reusable mock policy.

PowerMock 1.1 provides three mock policies out of the box for mocking slf4j, java commons-logging and log4j. Let's pick slf4j as an example and let's say you have a class that looks like this:

public class Slf4jUser {
        private static final Logger log = LoggerFactory.getLogger(Slf4jUser.class);

        public final String getMessage() {
                log.debug("getMessage!");
                return "log4j user";
        }
}

To stub all calls to the logger you can use the Slf4j mock policy that takes care of doing all necessary setup for you. You use it like this:

@RunWith(PowerMockRunner.class)
@MockPolicy(Slf4jMockPolicy.class)
public class Slf4jUserTest {
     ...
}

Note that we don't have do any setup at all to mock slf4j, the Slf4jMockPolicy takes care of this.

Mock policies can also be chained or nested like this:

@RunWith(PowerMockRunner.class)
@MockPolicy({MockPolicyX.class, MockPolicyY.class})
public class MyTest {
    ...
}

You can of course create your own mock policies by implementing the org.powermock.core.spi.PowerMockPolicy interface. For more information on mock policies please have a look at the documentation.

Other

There are of course many other smaller features and improvements such as the ability to suppress individual fields and better support for resetting mocks in the EasyMock API. You can read about all the new features and bug fixes in the changelog. Please visit our web-page for more information and downloads.

We now think that the API is stable enough and we have also tested PowerMock in a couple of projects, but of course since we were presenting at the conference it is no coincidence that we released PowerMock 1.0 just days before.

The details of the changes can be found in the changelog but here are some of the highlights since PowerMock 0.8:

• The ability to set and get internal state based on an object type. This means that you get more refactor friendly code while still being able to set internal state.
• Implemented replayAll and verifyAll. These can be used to replay and verify all mocks created by PowerMock without having to explicitly specify each and everyone. Thanks to Ivar Grimstad for suggesting this!
• Added support for specifying parameter types for expectNew. Mocking constructor calls is a unique feature of PowerMock that people typically find very cool!

During the conference I talked to several people and got really good feedback. Everybody seemed to like the idea of being able to unit test legacy code and code that haven't been designed for testing. Kevlin Henney made a good job of defining and describing various types of unit testing during his talk at Øredev and we talked about the difference between essentially untestable code and accidentally untestable code. PowerMock solves the problem with code being untestable for technical reasons (part of the accidentally untestable). With all the great sessions and good discussions Øredev was yet again a great experience!

I have made the slides from the presentation available at powermock.org. Note that the demo we showed was actually from the trunk and not from the 1.0 release!

We have received some feedback and have also started some evaluation in one of our customer's project. Besides bug-fixes and documentation updates we are starting to stabilize the API and preparing for the 1.0 release. We are also presenting PowerMock at this years Øredev conference. You're more than welcome to try it out for yourself.

 
public class StupidSingleton {
	public static String doStatic(int i) {
		throw new UnsupportedOperationException("method not implemented yet...");
	}
 
	public static void sayHello() {
		System.out.println("Hello");
	}
 
	public static native void nativeMethod();
}
 

We would like to write something like this:

 
	public void testMocking() {
		mock(StupidSingleton.class);
		expect(StupidSingleton.doStatic(17)).andReturn("17");
		StupidSingleton.sayHello();
		StupidSingleton.nativeMethod();
		replay(StupidSingleton.class);
 
		assertEquals("17", StupidSingleton.doStatic(17));
		StupidSingleton.sayHello();
		StupidSingleton.nativeMethod();
 
		verify(StupidSingleton.class);
	}
 

After playing around with AspectJ I discovered that this is actually possible! Two things are needed:
* An aspect that allows us to catch calls to static methods!
* An implementation of the mock, replay, verify for classes

This turns out to be really easy! The following aspect catches ALL calls to ALL static methods. Might be a bit aggressive, but it works! If you want you can implement your own aspect with a more specialized pointcut.

 
@Aspect
public class StaticMockAll extends StaticMockAspect {
	@Pointcut("call(static * *(..))")
	public void anyStaticOperation() {}
 
	@Around("anyStaticOperation()")
	public Object aroundStaticMethods(ProceedingJoinPoint jp) throws Throwable {
		return super.aroundStaticMethods(jp);
	}
 
}
 

The mock, replay, verify methods turns out to be trivial:

 
public class StaticMock {
	static Map<Class, MockInvocationHandler> mocks = new HashMap<Class, MockInvocationHandler>();
 
	public static synchronized void mock(Class type) {
		MockInvocationHandler h = new MockInvocationHandler((MocksControl) EasyMock.createControl());
		mocks.put(type, h);
	}
 
	public static synchronized void replay(Class type) {
		mocks.get(type).getControl().replay();
	}
 
	public static synchronized void verify(Class type) {
		mocks.remove(type).getControl().verify();
	}
}
 

Finally, the base class for the aspect that does the actual work is also quite simple:

 
public class StaticMockAspect {
	public Object aroundStaticMethods(ProceedingJoinPoint jp) throws Throwable {
		Class type = jp.getSignature().getDeclaringType();
		InvocationHandler h = getInvocationHandler(type);
		if (h == null) {
			return jp.proceed();
		}
		MethodSignature methodSignature = ((MethodSignature)jp.getSignature());
		return h.invoke(type, methodSignature.getMethod(), jp.getArgs());
	}
 
	private MockInvocationHandler getInvocationHandler(Class type) {
		synchronized (StaticMock.class) {
			return StaticMock.mocks.get(type);
		}
	}
 
}
 

h3. Notes

Things to note and think about:
* Static methods and native methods can be mocked! By using the "call" pointcut we change all code that tries to call the native methods and not the native method itself.
* AspectJ is required to make this work. I have been using Eclipse AJDT
* It should be possible to replace the Aspect with "plain" byte code manipulation and "just" reload the class.