This category has served me well, and even got traction in the iOS developer community, so I never bothered to stop and think if there exist an even better solution.

Especially now that GDC exists, and doing an inline block to invoke on the main thread or a background queue is easier than ever before.

Until Today

Traveling home by bus as usual I got a flash of genius; Why not use proxy objects? Calling a method on the main thread should be no harder then this;

[[self.slider mainThreadProxy] setValue:0.5f animated:YES];

The old NSInvocation solutions have a few drawbacks. First of all there is no code completion for the arguments, secondly there is not even basic type checking, and lastly the implementation is quite ABI specific.

The NSObject class already has support for method forwarding, and wrapping method calls up into NSInvocation instances. Why not use seriously battle proven code that Apple provides.

The Skeleton

The implementation for -[NSObject mainThreadProxy] will be childishly simple:

-(id)mainThreadProxy;
{
    // Return self directly if already on main thread.
    if ([NSThread isMainThread]) {
        return self;
    } else {
        return [CWMainProxy proxyWithTarget:self];
    }
}

The CWMainProxy class is a simple private helper, that will perform all of the heavy lifting. The interface and life time code is very small:

@interface CWMainProxy : NSObject {
    id _target;
}
+(id)proxyWithTarget:(id)target;
@end

@implementation CWMainProxy

+(id)proxyWithTarget:(id)target;
{
    CWMainProxy* proxy = [[[self alloc] init] autorelease];
    proxy->_target = [target retain];
    return proxy;
}

-(void)dealloc;
{
    [_target release];
    [super dealloc];
}
@end

The Meaty Parts

Now comes the tricky parts; actually act like a proxy. The proxy is a simple NSObject subclass, first it needs to be able to fake to any caller that it can respond to everything that the proxies target responds to.

-(BOOL)respondsToSelector:(SEL)sel;
{
    return [super respondsToSelector:sel] ||
           [_target respondsToSelector:sel];
}

Next up if the object claims it responds to a selector but do not have an actual implementation, then we must provide a NSMethodSignature so the run-time can create a proper NSInvocation for us.

-(NSMethodSignature*)methodSignatureForSelector:(SEL)sel;
{
    if ([_target respondsToSelector:self) {
        return [_target methodSignatureForSelector:sel];
    } else {
        return [super methodSignatureForSelector:sel];
    }
}

And finally we need to actually forward the NSInvocation call to the main thread, as per the callers request:

-(void)forwardInvocation:(NSInvocation*)invocation;
{
    [invocation performSelectorOnMainThread:@selector(invokeWithTarget:)
                                 withObject:_target
                              waitUntilDone:YES];
}

Conclusions

Given some thought I regard this solution to be much more elegant than my old solution. And it even yields less and more readable code than using GCD on iOS 4 and later:

[[self.slider mainThreadProxy] setValue:0.5f animated:YES];
    // vs
dispatch_async(dispatch_get_main_queue(),
               ^{
                   [self.slider setValue:0.5f animated:YES];
               });

• The implementation is much smaller, and relies on the Foundation framework functionality, not my own Objective-C ABI hacks.
• It is less code to write to use the functionality.
• Xcode can do proper code completion and basic type checking.
• It is compatible all the way back to iPhone OS 2, or Mac OS X 10.0, if you should care.

The CWFoundation project on Github has a more complete implementation. With more support for optional blocking, delays, and more proxies not only for the main thread but also for background threads and operations queue.

You are seldom alone with great ideas, so a nod to Joachim Bengtsson who has written about an invocation grabber before.

The solution I have found is not yet another tool, or framework, but a simple pattern. I have not found an existing name for the pattern so I call it Sync-Async Pair. The idea is to hide the complexity of asynchronous calls and call-back behind a facade, and have a straightforward synchronous implementation. An implementation that is easy to write, test and extend.

Sync-Async Pair Pattern

As the name suggests the pattern consist of a method pair, one is synchronous, and the other is asynchronous. The asynchronous method is backed by the synchronous implementation, and is only responsible for erecting a simplified facade in front of the sometimes complex concurrent implementation.

The first priority is to expose a clean API, the second priority is an implementation that is simple, testable and maintainable. Turns out that beginning with a clean API design (how to use the code), steers you towards a clean implementation (how to write the code). Most developers do the mistake to first write the code, and then struggle to try to use their own code.

Sync-Asyn Pair begins by defining a public API that then steers towards a clean implementation. Let's assume we are implementing some kind of a book reader. We need two model objects; CWBook and CWChapter. Both books and chapters are fetched from the network, so we need concurrency in order to not block the UI thread.

For this example let's focus only on fetching the chapters associated with a book. All other operations would be implemented in a similar fashion. First we need a public API, two methods for the sync-async pair on CWBook, and a delegate protocol.

@protocol CWBookFetchChaptersDelegate <NSObject>
-(void)bookDidFetchChapters:(CWBook*)book;
-(void)book:(CWBook*)book failedFetchChaptersWithError:(NSError*)error;
@end
// ...
-(BOOL)fetchChaptersWithError:(NSError**)error;
-(void)fetchChaptersWithAsyncDelegate:(id<CWBookFetchChaptersDelegate>)delegate;

The synchronous method fetchChaptersWithError: is very straightforward to implement.

-(BOOL)fetchChaptersWithError:(NSError**)error;
{
    NSURL* url = [self URLForFetchingChapters];
    NSData* data = [NSData dataWithContentsOfURL:url
                                         options:0
                                           error:error];
    if (data) {
        return [self parseChaptersFromData:data
                                     error:error];
    }
    return NO;
}

How to construct the NSURL, or possibly NSURLRequest is not important. Nor is it important how you actually parse the resulting data sent from the server, that is totally left to our problem domain. Just make it synchronous and super easy to write automatic unit checks for!

Adding Concurrency

Now to the hard part, that is actually quite manageable; adding the concurrency. For this example I will use performSelectorInBackground:withObject:. Your specific implementation might need NSOperationQueue, or GCD if you have special needs such as queues or cancelable operations. The idea will remain the same.

The most basic, and common, implementation for the public async method simply fires off the concurrent operation:

-(void)fetchChaptersWithAsyncDelegate:(id<CWBookFetchChaptersDelegate>)delegate;
{
    [self performSelectorInBackground:@selector(fetchChaptersWithDelegate:)
                           withObject:delegate];
}

In most cases all it does is request a private method to be performed on a background thread at some point in time. This is also the perfect place to abort operations early, for example a no-op if a request such as fetching a thumbnail is already in progress.

The private method previously launched is where most of the work is done. This is where a NSAutoreleasePool is setup, the synchronous method is called, and the delegate is properly called on the proper thread. The basic implementation for fetching books would be:

-(void)fetchChaptersWithDelegate:(id<CWBookFetchChaptersDelegate>)delegate;
{
    NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
    NSError* error = nil;
    BOOL success = [self fetchChaptersWithError:&error];
    if (success) {
        [[NSInvocation invocationWithTarget:delegate
                                   selector:@selector(bookDidFetchChapters:)
                            retainArguments:YES, self] invokeOnMainThreadWaitUntilDone:NO];
    } else {
        [[NSInvocation invocationWithTarget:delegate
                                   selector:@selector(book:failedFetchChaptersWithError:)
                            retainArguments:YES, self, error] invokeOnMainThreadWaitUntilDone:NO];
    }
    [pool release];
}

For this example I am using my NSInvocation additions to perform the delegate callbacks to the main thread, but it could just as easily been done using GCD. The additions are described in detail and available for download in this blog post, or from in the CWFoundation repo on github.

As you can see the boiler plate code for managing concurrency is actually very minimal, much less than 10 statements in total. The boilerplate code is almost reduced to only calling other methods, leaving very very few opportunities for bugs to sneak in.

Core Data in the Mix

The Sync-Async Pair pattern also lend itself beautifully for writing multi-threaded Core Data. The hassle of juggling NSManagedObjectID and NSManagedObject instances from different contexts can be completely left up to the private implementation that calls the synchronous method.

The private method responsible for calling the synchronous method and handling callbacks could be implemented as such:

-(void)notifyDelegate:(id)delegate ofSuccess:(BOOL)success withError:(NSError*)error;
{
    self = [[NSManagedObjectContext threadLocalContext] objectWithID:[self objectID]];
    if (success) {
        [delegate bookDidFetchChapters:self];
    } else {
        [delegate book:self failedFetchChaptersWithError:error];
    }
}

-(void)fetchChaptersWithDelegate:(id<CWBookFetchChaptersDelegate>)delegate;
{
    NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
    self = [[NSManagedObjectContext threadLocalContext] objectWithID:[self objectID]];
    NSError* error = nil;
    BOOL success = [self fetchChaptersWithError:&error];
    [[NSInvocation invocationWithTarget:self
                               selector:@selector(notifyDelegate:ofSuccess:withError:)
                        retainArguments:YES, delegate, success, error] invokeOnMainThreadWaitUntilDone:NO];
    [pool release];
}

Not much extra code, and the ugliness of handling Core Data's managed contexts is neatly hidden for clients at a single point in the code, that is much easier to test, debug, and maintain than most conventional Core Data code.

Conclusions

The simple Sync-Async Pair pattern makes it easy to write robust concurrent code. It is even a pattern that by design yields an elegant public API. It is also very easy to test, extend and even change the internal implementation completely without affecting any clients.

NSOperation do not have any facilities for returning a result when done as the Future do, you are left to implement this on your own. Which I have done a few times, and as a rule of thumb, make the solution generic when needed for the third time.

Operations works perfectly for background network activities. One common network activity is search-as-you-type, in this case you need to also cancel the previous operation if it has not yet been completed, so lets built this behavior into the generic solution as well.

API Design

What we want is an subclass of NSOperation that provides a future result, lets call this class CWFutureOperation. It is abstract, so users are supposed to subclass and implement main method just as for NSOperation, only difference is that the main method implementation must call setResult: before exiting.

The result that is set in the main method implementation can then be fetched using the blocking method result, equivalent to get() from the Java Future. As an alternative we would also like to be handed the method upon completion via a target-action pair. If the target-action pair is only called for finished operations that are not cancelled, then we get the search-as-you-type behavior for free.

It is also nice to be able to wrap up any method returning an object in a future operation as a convenience, so lets add that as well to our class interface:

@interface CWFutureOperation : NSOperation {
@private
    id _result;
    BOOL _isResultSet;
    id _completionTarget;
    SEL _completionSelector;
}

-(id)result;
-(void)setCompletionTarget:(id)target selector:(SEL)selector;
-(void)setResult:(id)result;

+(CWFutureOperation*)operationWithTarget:(id)target
                                selector:(SEL)selector
                                  object:(id)object;

+(CWFutureOperation*)operationWithInvocation:(NSInvocation*)invocation;

@end

Implementation

The client of the code must be able to fetch the result and the CWFutureOperation subclass must be able to set the result. The result method should block until the operation is finished if needed, and must properly return a result that can survive the operation that might be in a release pool. The implementation for this is straight forward, only minor complexity added in order to allow the result to be set multiple times:

-(id)result;
{
    if (![self isFinished]) {
        [self waitUntilFinished];
    }
    if ([self isCancelled]) {
        return nil;
    }
    return [[_result retain] autorelease];
}

-(void)setResult:(id)result;
{
    _isResultSet = YES;
    if (_result != result) {
        [_result autorelease];
        _result = [result retain];
    }
}

The reason that we flag if the result has been set is because we want to throw an exception if the subclass implementation of the main method failes to set the result before exiting, we add this behavior by overriding the start method in CWFutureOperation.

The start method will also call the target-action pair upon completion if the operation has not been cancelled. The target selector is always called on the main thread, this is so that we safely can update any UI using the calculated result without adding even more threading code.

-(void)start;
{
    [super start];
    if (![self isCancelled]) {
        if (!_isResultSet) {
            [NSException raise:NSInternalInconsistencyException
                        format:@"%@ did not set result.", self];
        }
        [_completionTarget performSelectorOnMainThread:_completionSelector
                                            withObject:self
                                         waitUntilDone:NO];
    }
}

Conclusion

The total amount of code is not muFuturesch, but this generic class adds a pattern to follow making design decisions for new application easy and consistent.

Solving the age old search-as-you-type problem can be as simple as this (Many assumptions, but you get the idea):

-(NSArray*)performSearchWithString:(NSString*)str;
{
    NSMutableArray* searchResults = nil;
    // Do actual search over the interwebz.
    return searchResults;
}

-(void)searchOperationDidComplete:(CWFutureOperation*)op;
{
    self.currentSearchOperation = nil;
    self.currentSearchResult = [op result];
    [self.searchDisplayController.searchResultsTableView reloadData];
}

-(BOOL)searchDisplayController:(UISearchDisplayController*)controller
  shouldReloadTableForSearchString:(NSString*)searchString;
{
	CWFutureOperation* op = [CWFutureOperation operationWithTarget:self
                                                          selector:@selector(performSearchWithString:) object:searchString];
    [op setCompletionTarget:self
                   selector:@selector(searchOperationDidComplete:)];
    [[NSOperationQueue defaultQueue]
    		replaceOperation:self.currentSearchOperation
               withOperation:op];
    self.currentSearchOperation = op;
    return NO;
}

The full source code including unit tests, and some additional categories for making operation queue life easier, can be downloaded here.

Testing asynchronous systems is hard. Not only does it require handling threads, timeouts and concurrency issues, but the intent of the test code can be obscured by all these details. Awaility is a DSL that allows you to express expectations of a asynchronous system in a concise and easy to read manner.

Simple example

So let's assume that we send an "add user" message to our asynchronous system like this:

 
publish(new AddUserMessage("John Doe"));
 

In your test case Awaitility can help you to easily verify that the database has been updated. In its simplest form it may look something like this:

 
await().until(newUserIsAdded());
 

Awaitility will now block the test execution and wait until the new user is added to the database. newUserIsAdded is a method that you implement yourself in your test case. It specifies the condition that must be fulfilled in order for Awaitility to stop waiting.

 
private Callable<Boolean> newUserIsAdded() {
	return new Callable<Boolean>() {
		public Boolean call() throws Exception {
			return userRepository.size() == 1; // The condition that must be fulfilled
		}
	};
}
 

By default Awaitility will wait for 10 seconds and if the size of the user respository is not equal to 1 during this time it'll throw a TimeoutException failing the test. If you want a different timeout you can define it like this:

 
await().atMost(5, SECONDS).until(newUserWasAdded());
 

Better reuse

Awaitility also supports splitting up the condition into a supplying and a matching part for better reuse. The example above can also be written as:

 
await().until( userRepositorySize(), equalTo(1) );
 

The userRepositorySize method is now a Callable of type Integer:

 
private Callable<Integer> userRepositorySize() {
	return new Callable<Integer>() {
		public Integer call() throws Exception {
			return userRepository.size(); // The condition supplier part
		}
	};
}
 

equalTo is a standard Hamcrest matcher specifiying the matching part of the condition for Awaitility.

Now we could reuse the userRepositorySize in a different test. E.g. let's say we have a test that adds three users at the same time:

 
publish(new AddUserMessage("User 1"), new AddUserMessage("User 2"), new AddUserMessage("User 3"));
 

We now reuse the userRepositorySize supplier and simply update the Hamcrest matcher:

 
await().until( userRepositorySize(), equalTo(3) );
 

Reduce verboseness

Since Java is so verbose Awaitility also provides a way achieve the same result by building up the supplier part without defining a Callable:

 
await().untilCall( to(userRepository).size(), equalTo(3) );
 

to is method in Awaitility which you can use to define the suppling part inline in the await statement. Which option you like best depends on the use case and readability.

More features

Awaitility uses polling to check if a condition is fulfilled. You can easily specify a polling interval and poll delay (the delay before the first poll commits):

 
with().pollInterval(400, MILLISECONDS).and().pollDelay(2, SECONDS).await().forever().until(somethingHappens());
 

In some cases it's also useful to create named awaits for example if you need multiple awaits in one test. A naive example:

 
publish(new AddUserMessage("User 1"));
await("user 1").until(userRepositorySize(), is(equalTo(1)));
publish(new AddUserMessage("User 2"));
await("user 2").until(userRepositorySize(), is(equalTo(2)));
 

So if the first await statement fails the TimeoutException will indicate that it was the "user 1" statement that failed and so on.

Conclusion

We hope that you find Awaitility easy to use when you need to synchronize asynchronous behaviour in Java. However be aware that since it uses polling to verify that the condition is fulfilled it's not recommended to use it for precise performance testing. In these cases it could be better to use an AOP framework such as AspectJ and leverage on its compile-time weaving.

Please visit our homepage for more info and downloads.

[someTextField performSelectorOnMainThread:@selector(setText:)
                              	withObject:@"A new text"
                             waitUntilDone:YES];

But not everything is an object

Since Objective-C is a superset of C, there are still all the types from C available. Such as all the primitives, including enums, and more complex struct types. Cocoa Touch is pragmatic and will use the most proper type for the use-case, not forcing everything into a square OOP hole.

This makes it quite hard to call for example -[UIView setHidden:], that takes a single BOOL argument. Same for -[UIView setFrame:], that takes a single CGRect struct argument, that in turn consists of one CGPoint and one CGSize struct.

Every type imaginable in C can be bundled in an NSValue instance. So we could bundle up the BOOL primitive, or the CGRect struct in a NSValue object. Then pass that object to the main thread, where it is unbundled and passed to the desired method. Quite cumbersome as it requires us to bundle, and unbundle types manually, and implement at least one extra method.

There must be an easier way

It turns out that NSInvocation can also handle any imaginable C type, or else it would not be able to invoke any imaginable Objective-C method. Making a NSInvocation invoke its method on the main thread is easy enough. Just add a category to the NSInvocation class, with a new method like this:

-(void)invokeOnMainThreadWaitUntilDone:(BOOL)wait;
{
  [self performSelectorOnMainThread:@selector(invoke)
                         withObject:nil
                      waitUntilDone:wait];
}

So now all you need to do is to create a NSInvocation object, fill it in with the primitive or struct types, and invoke it on the main thread. But creating and setting up a NSInvocation object is also a bit on the boring side...

There must be an even easier way!

We could use variable argument lists from plain old C. Too bad that they are untyped?
No despair, we know the target object for our invocation, and we know what method to call. So we have what we need to fetch the NSMethodSignature object for the call, that contains all the type information we need to safely process the va_list.

Our target machine is a Mac running 32- or 64-bit Intel CPU, or an iPhone OS device with a 32 bit ARM CPU. Turns out that on both platforms va_list is simply a void* pointer, to the stack frame. Even better va_start() will always flush any argument passed into the register on the stack frame. So we can skip most of the boring argument handling, by treating the arguments like a byte buffer, only advancing and aligning the buffer according to the information in the NSMethodSignature object.

A convenience method for creating a NSInvocation object for a particular target, selector, and list of variable arguments, would turn out like this:

+(NSInvocation*)invocationWithTarget:(id)target
                            selector:(SEL)aSelector
                     retainArguments:(BOOL)retainArguments, ...;
{
  va_list ap;
  va_start(ap, retainArguments);
  char* args = (char*)ap;
  NSMethodSignature* signature = [target methodSignatureForSelector:aSelector];
  NSInvocation* invocation = [NSInvocation invocationWithMethodSignature:signature];
  if (retainArguments) {
    [invocation retainArguments];
  }
  [invocation setTarget:target];
  [invocation setSelector:aSelector];
  for (int index = 2; index < [signature numberOfArguments]; index++) {
    const char *type = [signature getArgumentTypeAtIndex:index];
    NSUInteger size, align;
    NSGetSizeAndAlignment(type, &size, &align);
    NSUInteger mod = (NSUInteger)args % align;
    if (mod != 0) {
      args += (align - mod);
    }
    [invocation setArgument:args atIndex:index];
    args += size;
  }
  va_end(ap);
  return invocation;
}

Conclusion

And now we can easily call any method on the main UI thread.

An example where a CGRect struct is used to update the UI components frame;

// Set new frame of a label.
[[NSInvocation invocationWithTarget:someLabel
                           selector:@selector(setFrame:)
                    retainArguments:NO, CGRectMake(40, 40, 200, 100)]
 invokeOnMainThreadWaitUntilDone:NO];

A slightly more complex example, where we send a primitive int, and also waits for and fetches the result from the main thread:

// Query a UITableView for the number of rows in section 2.
NSInvocation* i = [NSInvocation invocationWithTarget:tableView
                                            selector:@selector(numberOfRowsInSection:)
                                     retainArguments:NO, (NSUInteger)2];
// Block this thread until method has been invoked and result is available.
[i invokeOnMainThreadWaitUntilDone:YES];
NSInteger numberOfRows;
[i getReturnValue:&numberOfRows];

Download full source code here: NSInvocation+CWVariableArguments.zip

Update: Example 2 was not 64-bit compatible as David Remahl pointed out to me. Added type cast to fix the error.

Update 2: A new version with some more features and small bug-fixes to BOOL, float and uint16_t types, plus proper unit tests can be downloaded here: CWFoundationAsyncAdditions.zip

Any sane developer avoids premature optimization, so the task we later want to execute in a separate thread is almost always available in our current context. I am sure all Java-developers has seen something like this:

 new Thread() {
    public void run() {
        OuterClass.this.someMethod(arg);
    }
}.start();

And often an anonymous Runnable is involved as well. Executing a task in a separate thread with Cocoa and Cocoa Touch is ridiculously easy given the dynamic nature of Objective-C. This is done with Cocoa with this one-liner:

[self performSelectorInBackground:@selector(someMethod:) withObject:arg];

The performSelectorInBackground:withObject: is a companion method for performSelector:withObject:, and will create a new NSThread instance, and execute the specified method there. There are more companions such as performSelectorOnMainThread:withObject: that is used to execute a method on the main UI thread.

This easy access to executing in new threads often can result in code that spawns too many threads. In Cocoa on Mac OS X this is seldom a problem, not for Cocoa Touch applications under development when running in the iPhone simulator either. But on the more resource constrained iPhone and iPod Touch devices this will be a problem. Spawning a dozen background threads to download images will bring the iPhone OS to a staggering mess.

Update: I have removed CWSelectorOperation from this post, since using NSInvocationOperation is preferred.

Operation Queues

The solution is to use a NSOperationQueue, that as the name implies handles a queue of operations. The max number of concurrent operations can be set, as well as dependencies and priority. The downside is that NSOperationQueue can only handle instances of NSOperation.

NSOperation is an abstract class, where you as a developer override the main method to execute your task. In short NSOperation is programatically the equivalent of the Java Runnable interface. It is even worse as Objective-C do not support anonymous classes, so we will be required to actually implement our queued tasks as separate classes.

Not as neat as one would like, and definitely not even very Cocoa-like.

What is Missing

As a Cocoa developer I expect that executing a task on background queue, would be just as easy as executing a task on a background thread. The expected needed code is something like this:

[self performSelectorOnBackgroundQueue:@selector(someMethod:) withObject:arg];

To make this work we need:

1. A category on NSObject to add performSelectorOnBackgroundQueue:withObject:.
2. Implement a concrete subclass of NSOperation to support execution of a selector on any target.Use NSInvocationOperation to perform selector.
3. A category on NSOperationQueue to add a shared operation queue.

This is in the order of how a client developer would see it (most developers will only care for the NSObject category). For us this is the reverse order when implement this functionality.

A Shared Operation Queue

Cocoa and Cocoa Touch used many shared instances, for example a shared UIAccelerometer instance, and a shared NSURLCache instance. So this pattern of single point dependency injection is common in Cocoa and works very well. So let us follow suit by introducing an interface like this:

@interface NSOperationQueue (CWSharedQueue)
+(NSOperationQueue*)sharedOperationQueue;
+(void)setSharedOperationQueue:(NSOperationQueue*)operationQueue;
@end

The shared operation queue will be created lazily if it do not exist, and the client can easily swap it out at startup if needed. The implementation is very small, and a good example of how we should implement class variables with proper memory handling.

@implementation NSOperationQueue (CWSharedQueue)
 
static NSOperationQueue* cw_sharedOperationQueue = nil;
 
+(NSOperationQueue*)sharedOperationQueue;
{
  if (cw_sharedOperationQueue == nil) {
    cw_sharedOperationQueue = [[NSOperationQueue alloc] init];
    [cw_sharedOperationQueue setMaxConcurrentOperationCount:3];
  }
  return cw_sharedOperationQueue;
}
 
+(void)setSharedOperationQueue:(NSOperationQueue*)operationQueue;
{
  if (operationQueue != cw_sharedOperationQueue) {
    [cw_sharedOperationQueue release];
    cw_sharedOperationQueue = [operationQueue retain];
  }
}
 
@end

Enter CWSelectorOperation

Update: CWSelectorOperation is no longer needed, our implementation instead uses NSInvocationOperation.

Wrapping up NSObject

And at last we reach the category on NSObject, that is what we initially wanted, and what 95% of all use cases will be limited to.

@interface NSObject (CWSharedQueue)
 
-(NSInvocationOperation*)performSelectorInBackgroundQueue:(SEL)aSelector
    withObject:(id)arg;
-(NSInvocationOperation*)performSelectorInBackgroundQueue:(SEL)aSelector
    withObject:(id)arg dependencies:(NSArray*)dependencies
    priority:(NSOperationQueuePriority)priority;
 
@end

Most performSelector* methods do not have a return value, I have chosen to return the resulting NSInvocationOperation so that it can be used to setup dependencies for future queued operations. I have also added a second method performSelectorInBackgroundQueue:withObject:dependencies:priority: so that such dependencies can easily be setup, as well as prioritisation of operations.

The actual implementation is a simple wrapper around the NSInvocationOperation class, and category on NSOperationQueue that we implemented above:

@implementation NSObject (CWSharedQueue)
 
-(NSInvocationOperation*)performSelectorInBackgroundQueue:(SEL)aSelector
    withObject:(id)arg;
{
  NSInvocationOperation* operation = [[NSInvocationOperation alloc]
      initWithTarget:self selector:aSelector object:arg];
  [[NSOperationQueue sharedOperationQueue] addOperation:operation];
  return [operation autorelease];
}
 
-(NSInvocationOperation*)performSelectorInBackgroundQueue:(SEL)aSelector
    withObject:(id)arg dependencies:(NSArray*)dependencies
    priority:(NSOperationQueuePriority)priority;
{
  NSInvocationOperation* operation = [[NSInvocationOperation alloc]
      initWithTarget:self selector:aSelector object:arg];
  [operation setQueuePriority:priority];
  for (NSOperation* dependency in dependencies) {
    [operation addDependency:dependency];
  }
  [[NSOperationQueue sharedOperationQueue] addOperation:operation];
  return [operation autorelease];
}
 
@end

And that is it, if you want to avoid cut and paste of all this code to your own project just download the source code here. And happy concurrent hacking.

Download the older source code here, with a concrete subclass of NSOperation called CWSelectorOperation.

BDD in Java with easyb (John Ferguson Smart)

John started out by arguing that TDD is not about testing, it's about writing better code. That is, it's about making your code more maintainable, flexible, reliable and simple. TDD is forces the programmer to think about how to test the class before actually writing the class. BDD, on the other hand, helps to determine what to test and to write more focused code, by starting with the behavior. Instead of thinking that you're testing your class, you should think that you're validating your requirements!

easyb is a BDD testing framework for Java, written in Groovy (hence providing full access to all Java API's). It ensures that tests become clearer and easier to both write and read! It helps the developer to focus on the requirements only. A competitor to easyb is JBehave, which is an extension to JUnit. However, according to John, JBehave is more cumbersome than easyb.

easyb makes use of stories, much like the story cards in the Agile world. In a story, a narrative approach is used to describe a precise requirement. In easyb, a story could look like this:

 
scenario “Make initial deposit onto a new account”, {
	given “a new account”
	when “an initial deposit is made”
	then “the balance should be equal to the amount deposited”
}
 

This scenario is quite clear and readable, isn't it!? Furthermore, it's self-documenting - a very appreciated feature =) And the syntax is so trivial that even a non-technical stakeholder would have no problem understanding it. The developer can then reuse the given scenario (pseudo) code and implement the test case, like this:

 
import com.mycompany.bankonline.domain.Account
 
scenario “Make initial deposit onto a new account”, {
	given “a new account”, {
		account = new Account()
	}
	when “an initial deposit is made”, {
		initialAmount = 100
		account.makeDeposit(initialAmount)
	}
	then “the balance should be equal to the amount deposited”, {
		account.balance.shouldBe initialAmount
	}
}
 

The shouldBe call is an intuitive and readable way to verify that the requirement holds. Another way to verify the outcomes is to use the ensure syntax. Example:

 
ensure(account.balance > 0)
 

or

 
ensure(account) {
	has([balance:100])
}
 

easyb could be extended to use DBUnit (and further plugins for Grails and Excel are on the way). On the IDE support, the only good option is IntelliJ IDEA. It really comes down to the Groovy support, and Eclipse and NetBeans doesn't shine here. In those cases, you're better off running the tests in Maven or Ant, which both are well supported. In the upcoming version, easyb will have full support for Continuous Integration (CI), enabling automatic test execution in a CI-server such as Hudson.

Furthermore, easyb comes with a web application - Easiness (cmp. Fitnesse) - where stakeholders can create stories in plain text. The developers can then go ahead and implement the stories defined by the stakeholders. This is an important features, that further improves the cooperation between the customer and the provider. It really seems like a quite nice little framework! If you agree, have a look at it at http://easyb.org/.

From Concurrent to Parallel – Library-based parallelism in JDK 7 (Brian Goetz)

Brian is an expert in the concurrency area, but he's not a good presenter! He speaks faster than I can read, and his slides are so packed with info that he doesn't have time to say the half of it. That put aside, in this very interesting session Brian presented some really cool concurrency features to be included in JDK 7.

As you probably know, JDK 5 introduced a set of very useful classes for course-grained parallelism. JDK 7 will introduce a framework for fine-grained parallelism - the "fork-join" framework.

We have now reach a point in time when Moore's laws isn't valid anymore. The new reality is not faster CPUs but more CPUs. To adapt to these changes, programmers have to change the way they think about parallelism. The unit of work (UoW) that represents a task to be solved must be split into even smaller parts to be able to keep all hardware busy. Nowadays, it isn't enough to for ex. spawn a new thread for each service call in a server, the UoW must be split up even further.

Examples of finer-grained parallelism is searching, sorting and filtering a data set. Such a task is preferably solved by dividing it into sub-problems and combining the sub-results at the end. We could use the course-grained concurrency tools (for ex. Executor + Future) introduced in JDK 5 to solve this problem. However, with multiple CPUs available, these tools doesn't perform optimally. For ex., the shared work queue in the Executor becomes a bottle neck when the number of threads/CPUs increases. Furthermore, these tools provide no form of load balancing, i.e. if one thread finishes before the others, it is never reused.

The divide-and-conquer algorithm could also be applied to solve the problem. This great advantage of this approach is that it's independent of the number of CPUs used! Using plain threads for this approach is too expensive though, since thread creation costs way too much. However, the fork-join-operation in JDK 7 solves problems like these using the divide-and-conquer algorithm in a much more effective way! Internally, it uses an slim thread pool to overcome the cost of thread creation. Furthermore, no data-copying penalty exist since the data set (for ex. an array) is not divided/copied during the solution. Instead, the indexes within the data structure is used to define each data subset. Another advantage is that the code doesn't know how many CPUs it executes on, hence it's portable!

While the Executor (JDK 5) should be used for tasks consisting of both IO and computing, the fork-join-operation (JDK 7) should be used for highly compute-intensive tasks. Examples of such tasks are matrix operations, numerical integrations and game playing. The JDK 7 class to be used for these kind of problems is ParallelArray. It's a utility class that greatly simplifies the work of solving a task in parallel. You can think of it as an in-memory database for data sets. Typical usage is:

 
ParallelLongArray pa = ParallelLongArray.createUsingHandoff(array, forkJoinPool);
long max = pa.max();
 

The ParallelArray class provides operations for:

• Filtering (nestable)
• Mapping
• Replacement
• Aggregation (max, count, sum, ...)
• Application (one action performed for each selected element)

To compare, the ParallelArray class acts much like the Map-Produce concept introduced by Google but it works in a different domain (local system instead of in a distributed system).

The only downside really is that JDK 7 isn't scheduled until mid 2010, meaning we'll have to wait quite some time before we can actually use these new concurrency features. But from the looks of it ... it may be worth the wait!

The SpringSource DM Server (Joris Kuipers)

The reasons why SpringSource decided to develop a new application server were many. Today, modularity is important and developers spend much time on achieving it in their applications (layering, separation of concerns, etc.). However, when the application is deployed, all modularity is lost - the deployable unit is one huge, monolithic WAR file. That means that sharing code (let alone services) between applications become hard if not impossible. In order to perform the smallest update on the application, whole WAR file have to be redeployed. Furthermore, the servers themselves lack modularity; they come prepackaged with a lot of modules but there's no way to configure that your application only make use of a couple of them. Hence, they are all left in there, consuming unnecessary CPU cycles and memory.

The DM (Dynamic Modules) server was developed to tackle these issues of non-modularity. In order to achieve true modularity, is has been built on top of OSGi (namely the Equinox container). Since OSGi can be quite hard sometimes, the DM server conceal the complexity of OSGi in the same way as the Spring framework does with for ex. Remoting or JMX.

In order to be able to use a third-party library from within the DM server (and from any other OSGi containers), the standard JAR library needs to be extended with OSGi-specific meta data - the OSGi manifest. SpringSource has therefore created the Enterprise Bundle Repository, where they have started to upload and publish OSGi-ified Java EE libraries. If you're using Maven or Ivy, the bundle repository can easily be configured as a remote (or local) repository.

The DM server also introduces a couple of new OSGi import constructs, enabling ease of development. One example is the "Import-library" that can be useful if you want to make use of a framework (for ex. Spring) but don't want to waste your time having to specify every single Import-package (for example Spring-Remoting) within the framework (as you normally would have had to do). This doesn't affect performance improvement since (thanks to OSGi) none of the pre-installed bundles are loaded before they are actually used! Furthermore, with the DM server it's now possible to remove the library bloat of monolithic Java EE WARs. Instead of building the external libraries into your WAR, you can instead declare them as dependencies in the OSGi manifest headers. The server will then convert these into plain "Import-package" constructs during deployment. That way, the size of the built/deployed WAR can be reduced significantly!

In the DM server, one can of course deploy plain OSGi bundles but also WARs, EARs etc. Typical scenarios for deploying a plain OSGi bundle are stand-alone libraries, global services and small stand-alone applications. However, being able to deploy a bundle is not enough in most cases ... A normal application normally consist of multiple bundles, which becomes hard to un/deploy since there is no single deployable unit. Furthermore, with multiple bundles you don't get a common log file or a notion of an application scope. To tackle this issue, the DM server adds the notion of an application by introducing the PAR (Platform Archive) format. It's basically a JAR with a number of 'Application-*' manifest headers. You can think of it as an EAR in Java EE. PARs are versioned and the versions apply to all of their bundles. This mean that one can deploy a PAR twice with different versions but with the same bundles, without conflicts.

The DM server comes in several versions. Of course there's the community version, using a GPL and SpringSource license. Then there's the commercial license where full support from SpringSource is included. On the extension side, better Maven support will soon be released so that the dependencies can be specified in a single place (not like now when you have to duplicate them in the pom and in the manifest). There's a Eclipse-plugin available that enables re/deployments from within the IDE.

These are some of the features in the DM server. There are plenty or more cool ones, read all about them here. There, you'll also find a lot of sample applications that can get you going! With the DM server, SpringSource feel that they are filling the (up until now) empty space of server side OSGi and states that "The DM server is the healthy new way to run your apps!". I must say it looks very promising! Why don't you go ahead and try it out!? I know I will!

If you appreciated these posts, you will definitely enjoy next years conference! Pick up your calendar and reserve December 2009 for a great week at Devoxx!

Here is screenshot from the management console showing 81 active CPUs and 23 GB free memory:

I tried some Scala actors which worked very well. By running some CPU intensive calculations I concluded that it scaled linearly up to the number of CPUs. No surprise, but very cool when you are running 80 threads at full speed! Also cool to see that Scala runs smoothly which means that neither Scala nor Azul does anything strange. Just pure bytecode.

Somebody tried to configure the Azul JVM in Eclipse and was able to run JUnit tests on Azul. We were also able to run maven without any problems.

We tried to allocate lots of memory. No problem there either, although we had to specify -Xmx10G or something. By default you only get 1.6 GB....

A colleague started tomcat, produced lots of garbage and tried garbage collection at 8 GB of heap. No pause! However, there was some synchronization problem with Log4j since all 16 CPUs had to coordinate their work.

We encountered two problems:

• xstream apparently had a dependency to Unsafe which is not supported on Azul
• the program must be headless, so no Swing

A quick rewrite later and we had generated a 64 megapixel mandelbrot calculated in parallel on 40 CPUs. The only problem was saving the PNG file which was a single threaded operation...

Low level constructs

In the new API, there are representations of the classical concurrency constructs of
locks, semaphores, latches and barriers. The Lock interface, for example, behave very
much like the intrinsic lock (used in synchronized blocks). However, the biggest
advantage of Lock objects is their ability to back out if the lock isn’t available, either
immediately or within a specified timeout, allowing the implementer to take an al-
ternate action. Furthermore, the new Lock interface also allows locks to be acquired
and released in different code blocks. The drawback of the new Lock class is that
it’s more complex, above all it requires the programmer to remember to release the
lock. In a similar manner, there are useful implementations of semaphores, latches
and barriers.
The concept of atomicity is central in concurrency. An atomic operation is one
that cannot be interrupted by a concurrently running process. The Java increment
operator, i++, is not an atomic operation (internally it’s three operations) but a sim-
ple assign (i = 5;) is. The java.util.concurrent.atomic package defines classes that
support atomic operations on single variables. For example, in order to protect a
variable incrementation (i++) from thread interference without resorting to syn-
chronization, the variable can be kept in an AtomicInteger and the increment can be
performed using the incrementAndGet() method, thus ensuring atomicity.

New Collection types

The java.util.concurrent package includes a number of additions to the Java Collec-
tions Framework. The new concurrent collections are improvements over synchro-
nized collections in terms of throughput and efficiency. Synchronized collections
achieve their thread safety by protecting the collection’s state by a collection-wide
lock. When accessed by multiple threads, throughput suffers. Concurrent collec-
tions, on the other hand, are designed for concurrent access from multiple threads
and instead lock on the individual elements.
Two important new Collection types are BlockingQueue and ConcurrentMap.
BlockingQueue implements the (non-concurrent) Queue interface, which holds
elements prior to processing. The thread-safe BlockingQueue is primarily designed
for producer-consumer queues and allows clients to wait for an element to appear
(possibly within a specified timeout). Waiting occurs if you either try to insert an
element into a full queue or if you try to retrieve from an empty queue. Concur-
rentMap defines a couple of very useful atomic map operations:

• A key/value pair is removed/replaced only if the key is present in the map.
• A key/value pair is added only if the key is absent.

Hence, there’s no need to synchronize when accessing the ConcurrentMap.

High level constructs

The new Callable interface is responsible for encapsulating a task. It’s similar to the
Runnable interface; both are designed for classes whose instances are potentially
executed by another thread. However, Callable returns a result and may throw an
exception. The utility class Executors has methods for converting from other com-
mon task forms (such as Runnable) to a Callable. Furthermore, a Callable can be
executed by the ExecutorService class in the submit() method.
The Future interface is an important entity in the new Executor framework and
represents the result of an asynchronous computation. The result can be retrieved
using one of the get() methods when the computation has completed, blocking if
necessary until it’s ready. The interface provides a possibility to cancel the computa-
tion and it’s also possible to determine if the task completed normally or if it was
canceled. A Future is normally used in the following manner:

Future futureResult = executor.submit(
   new Callable() {
      public String call() {
         // perform heavy op and return result
      }
   }
);
doOtherThings();
try {
   useResult(futureResult.get());
} catch (ExecutionException ee) { cleanup(); }

Notice how the result is retrieved - get() might be blocking (depending on if the
computation is ready or not by the time of the invocation).
Executor is a simple interface for launching new tasks. It provides a way of decou-
pling task submission from the mechanics of how each task will be run, including
details of thread use, scheduling, etc. The single method in the interface, execute(),
has been designed to replace the way we normally use threads. That is, instead of
explicitly creating a new thread for each task to execute, all tasks are executed by a
single Executor. How the Executor actually executes the tasks internally is imple-
mentation-dependent.

The ExecutorService interface extends the Executor interface and adds fea-
tures for managing the life cycle of both individual tasks and the executor itself. It’s
submit() methods accept both Runnable and Callable objects and returns a
Future. An unused ExecutorService should be shutdown() to reclaim its resources.
The ExecutorService interface also provide methods for submitting large collections
of Callable objects.
It’s time for an example. An implementation of a simple web server might look
something like this:

public class WebServer {
   /**
    * Creates a server socket, accepts connections,
    * and handles each incoming request in a new thread.
    */
   public void run() throws IOException {
      ServerSocket socket = new ServerSocket(8080);
      while (true) {
         final Socket connection = socket.accept();
         Runnable task = new Runnable() {
            public void run() {
               handleRequest(connection);
            }
         };
         new Thread(task).start();
      }
   } 

   public static void main(String[] args)
         throws IOException {
      WebServer webServer = new WebServer();
      webServer.run();
   } 

   private void handleRequest(Socket connection) {
      // Impl. details not important here!
   }
}

In order to achieve high responsiveness, each new request is handled in a new
thread. Under light to moderate load with sufficient CPU resources available, this
implementation offers relatively good throughput too. However, it would be nice to
have a more flexible solution supporting a wide variety of task execution policies, i.e.
for ex. to be able to specify in what thread and in what order tasks will be executed,
and how many tasks may execute concurrently. Let’s introduce an Executor:

public class ThreadPerTaskExecutor implements Executor {
   public void execute(Runnable r) {
      new Thread(r).start();
   };
}

, which leaves us with the following run() method:

public void run() throws IOException {
   Executor exec = new ThreadPerTaskExecutor();
   ServerSocket socket = new ServerSocket(8080);
   while (true) {
      final Socket connection = socket.accept();
      Runnable task = new Runnable() {
         public void run() {
            handleRequest(connection);
         }
      };
      exec.execute(task);
   }
} 

The great advantage of this approach is of course that the execution policy could
very easily be substituted (even configured at deployment-time) without altering
the rest of the code base. However, one big problem still exists with this solution!
Under heavy load a lot of threads will be created, and thread creation and teardown
takes time and active threads consume a lot of memory. Furthermore, each platform
has an upper limit on how many threads that can be created, and when this limit is
hit an OutOfMemoryError is probably thrown. Hence, up to a certain point, more
threads can improve throughput but beyond that point, creating more threads just
slows down the application and may even lead to a crash. The way to stay out of
danger is to place some bound on how many active threads your application utilizes
concurrently.
Most of the executor implementations internally make use of thread pools,
which consists of worker threads. Worker threads are often used to execute mul-
tiple tasks and minimize the overhead caused by thread creation. One common
thread pool type is the fixed thread pool, which always has a specified number of
threads running. The Executors class contains factory methods for retrieving Execu-
torServices that in turn use different kind of thread pools internally. For example,
newCachedThreadPool() creates a thread pool that creates new threads as needed
but will try to reuse previously created threads when available.
To be on the safe side with regards to both throughput and responsiveness in the
web server example, an executor that uses a thread pool internally should be used:

public void run() throws IOException {
   Executor exec = Executors.newFixedThreadPool(100);
   ServerSocket socket = new ServerSocket(8080);
   while (true) {
      ... // same as above
   }
}

Conclusion

As you can imagine, Java 5 has given us a lot of great high level features in the
new concurrency package. However, they extend (not replace) the traditional low
level concurrency constructs, such as the synchronized keyword and the wait/notify
mechanism, who still have a vital role to play. Your concurrency toolbox has just
become bigger and more flexible.

References

1 The JavaDoc
2 Java 5 Collection Framework changes
3 “Java Concurrency in Practice” (Brian Goetz et. al.), ISBN 9780321349606
4 Concurrency Tutorial