In this post I want to show you a step by step guide on how you can start your Scala project with Spring Data for MongoDB.

You should have SBT 0.11 installed on your system since I am going to use it as the building tool through out this post.

1. Creating Project

Create the project like below:

$ mkdir springDataMongo
$ cd springDataMongo/
$ sbt
[info] Loading global plugins from /home/amir/.sbt/plugins
[info] Set current project to default-71a5a0 (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> set name := "SpringMongo"
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> set version := "1.0"
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> set scalaVersion := "2.9.1"
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> session save
[info] Reapplying settings...
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)

Open file build.sbt and add the dependencies:

Save the file and run the following in SBT console:

> reload
[info] Loading global plugins from /home/amir/.sbt/plugins
[info] Set current project to SpringMongo (in build file:/data/3-Projects/scala-projects/springDataMongo/)
> update
[info] Updating {file:/data/3-Projects/scala-projects/springDataMongo/}default-71a5a0...
[info] Done updating.

And generate the IDE related files. I am using IntelliJ:

> gen-idea

or if you use Eclipse:

> eclipse create-src

Now you have configured your project and you can start coding!

2. Configuring Spring

There are two ways to configure Spring Data for MongoDB: Annotations and XML. I explain annotations here. You have to extend AbstractMongoConfiguration class as follows to define your database settings:

As you see there are two methods to implement. mongo method should return an instance of Mongo class. If you are using a mongo instance in a network, you should then specify the address and possibly port (if not the default).

3. Data Model

Consider we have the following class for keeping accounts in a system. This class will be mapped to a mongo document by Spring:

4. Interacting with MongoDB

Now let's write a simple program to interact with MongoDB:

mongoTemplate is an implementation of MongoOperation interface. With MongoOperation you can do all CRUD operations. E.g. inserting/saving a document:

"accounts" is the name of collection. Or if you want to query:

Note that JavaConversions is required here since the result of find is a java.util.List[Account] and we need to convert it to Scala list.

Please refer to documentation of Spring Data for MongoDB for more info about what you can do more

There where lots of interesting talks on a wide variety of subjects, all the way from low level bit shifting in concurrent data structures all the way to how you build a real time sports betting system with Scala and Lift or integrate your Scala code with existing JEE applications. There is no doubt that there are lots of things happening around Scala right now, and the pace is fast.

One thing that really hit home though was the excellent keynote by David Pollak  that asked the question if Scala had really crossed the chasm, and reminded us that history is full of brilliant technology that has failed, since no single product can survive on technical merit alone.

But how can we ensure that Scala is successful? The obvious answers are of course tooling, support and continued innovation, but an equally important aspect is probably accessibility. People, both programmers, managers and other decision makers, need to feel at home with Scala. So we need to show them just how easy, familiar and powerful Scala is. This is where Phil Bagwells presentation about Kojo comes into the picture.

Kojo is a Scala based learning tool that was designed for teaching kids, but as Phil pointed out "We are all just 11 year olds mentally, who like shiny toys". That fact makes Kojo an excellent presentation tool. By not only showing people static slides with code and promises (yes I'm guilty of this as well), but letting them interact with the code in the presentation Kojo makes Scala directly and easily accessible.

Kojo is a simple download if you have Java installed, and Phil kindly let me post his story (that's what they're called in Kojo) on github.

So download Kojo, download the story, load it up, play with it, share it and introduce some people the wonders of Scala.

In this post I want to show you how you can write your own parallel collection in Scala. The example I use for this post is taken from Alex Prokopec's presentation at Scala eXchange 2011.

Consider we want to have a parallel String in our application. If you convert a String into a parallel collection you get a ParArray:

scala> "I Love Scala".par
res2: scala.collection.parallel.ParSeq[Char] = ParArray(I,  , L, o, v, e,  , S, c, a, l, a)

This is good enough to take advantage of your multi-core CPU architecture, but for some reason you want to write you own Parallel String collection!

Scala Parallel collection is implemented by a mechanism that is called split-combine. As the name implies a sequential collection can be splitted into sub-collections and each one can be stolen by a thread from a thread pool. Later the results of each sub-collection can be combined to make the final collection. You can find details about this implementation in [1].

In order to start with our Parallel String we need to extend scala.collection.parallel.immutable.ParSeq trait. There are 4 abstract methods that we need to implement:

seq, length and apply have obvious implementations. Method splitter returns a new instance of class ParallelStringSplitter which we are going to define next. Note that this instance is mixed-in into SignalContextPassingIterator which is responsible for passing the signal context along iterators. We define ParallelStringSplitter class inside ParallelString class:

Note that we need to extend Splitter trait. There are 6 abstract methods to implement. Method dup returns a new instance of ParallelStringSplitter with current values. Method split splits the iterator into a sequence of disjunct views and method psplit splits the splitter into a sequence of disjunct views.

Now we have the splitter. Let's test this parallel String in REPL:

scala> val pstr = new ParallelString("This is a long string" * 250000)
pstr: ParallelString = ParallelString(T, h, i, s,  , i, s,  , a,  , l, o, n, g,  ...

If you don't provide a combiner, then the default one is used. Sometimes you want to have control over the combination and you want to take the responsibility for it. Let's write our own combiner to show how you can do that. We need to mix in another trait into the definition of ParallelString: ParSeqLike

In order to specify our own combiner we need to override method newCombiner:

Now we need to define ParallelStringCombiner class by extending the Combiner trait:

There are 5 abstract methods that need to be implemented. As the names imply method combine combines the content of another builder to the current builder and produces a new builder. Method result produces a collection from added elements and += adds a single element to this builder.

To summarize, to build a parallel collection class A, one should:

• extend ParSeq trait
• implement all abstract methods from ParSeq trait
• define a splitter class that mixes in with Splitter and ParIterator traits and put it in class A
• extend ParSeqLike trait
• define a combiner class that extend Combiner trait and implement all abstract methods
• override newCombiner method in class A and return a new instance of defined combiner

For the complete code of this example please refer to Alex Prokopec's Github.

References

[1]: Aleksandar Prokopec, Tiark Rompf, Phil Bagwell, Martin Odersky, "A Generic Parallel Collection Framework", Euro-Par 2011

In this post I go through injectors and mostly extractors. You will see that how extractors can be employed for pattern matching.

Consider that we need to hold first names in an application. We can define a case class for Firstname:

we build a value from this case class in REPL:

scala> val fname = Firstname("Amir")
fname: Firstname = Firstname(Amir)

and now a pattern matching on this case class:

scala> fname match {
     |     case Firstname(f) => println(f)
     |     case _            => println("Nothing found")
     | }
Amir

Here fname is matched against its case class by constructor patterns mechanism and its field is extracted and printed. Very powerful and handy.

What if we need to do a pattern matching for a string? The problem is that strings are not case classes.

Scala provides us with a very interesting mechanism called Extractors. An extractor is an object that has a method called unapply as one of its members. Let's clarify this with an example: assume we want to have an extractor object for IP addresses. We define the extractor as the following:

Method unapply receives a string representing a possible IP address and returns an option of 4 strings. If the string is not a valid IP address, the method returns None. Method isValid is added for validating IP addresses. Let's try this in REPL:

scala> val ip = "127.0.0.1"
ip: java.lang.String = 127.0.0.1

scala> val nonIP = "128.-112.ABC."
nonIP: java.lang.String = 128.-112.ABC.

scala> IPAddress.unapply(ip)
res0: Option[(String, String, String, String)] = Some((127,0,0,1))

scala> IPAddress.unapply(nonIP)
res1: Option[(String, String, String, String)] = None

And if we use it in a pattern matching statement:

scala> ip match {
     |    case IPAddress(_, _, _, a) => println(a)
     |    case _                     => println("Invalid ip address")
     | }
1

scala> nonIP match {
     |    case IPAddress(_, _, _, a) => println(a)
     |    case _                     => println("Invalid ip address")
     | }
Invalid ip address

So ip was a valid string representation of an IP Address and it is matched against IPAddress while nonIP was not a valid one. In the example above we were only interested in the last byte of IP Address and we skipped the rest by _ wildcard. Of course you can extract all the four parts if you need them.

In IPAddress object we used 4 variables that is wrapped in a Some in the success case and returned. This can be generalized to N variables.

It is also possible that an extractor pattern does not bind to any variables. In this case the corresponding unapply method returns a boolean (true for success and false for failure). As an example we changed the unapply method to the following:

and we try it in REPL:

scala> "127.0.0.1" match {
     |    case IPAddress() => println("Valid")
     |    case _           => println("Invalid")
     | }
Valid

Remember that although there is no binding in this case but you have to put the parentheses in front of IPAddress.

So far we have seen fixed number of element values. But what if we have variable number of element values? For example if you have a string that can contain arbitrary number of IP addresses? In order to handle this case, Scala allows you to define a different extractor method called unapplySeq. To see how it can be used, assume we want to have pattern matching on a string containing arbitrary number of IP Addresses:

This time the method returns an option of a sequence of string. Now let's see what we can do with it:

scala> val ips = "192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4"
ips: java.lang.String = 192.168.0.1,192.168.0.2,192.168.0.3,192.168.0.4

scala> ips match {
     |    case IPAddresses(IPAddress(a, _, _, _), IPAddress(b, _, _, _), _*) => println(a + " " + b)
     |    case _                                                             => println("Invalid IP addresses")
     | }
192 192

In this example we used both IPAddress and IPAddresses objects in the pattern matching. There are 4 IP addresses in ips and we are looking for the first byte of the first two IPs in the string. How powerful and clean!

The unapply method is called extractor because it takes an element of the same set and extracts some of its parts. In our example IPAddress takes a string and validates and extracts all 4 bytes of the address. You can also define a method apply on an object that does vice versa which takes some arguments and yields an element of a given set. This method is called an injection. We can define injection for our IPAddress object like this:

Injections and extractions are often located in the same object because then you can use the object name for both constructions and pattern matching. However, it is also possible to have an extraction object without injection. The object itself is called extractor regardless of whether or not it has an apply method [1].

Remember if you include the injection method, it should be a dual to the extraction method, for example:

IPAddress.unapply( IPAddress.apply(a, b, c, d) )

should return:

Some(a,b,c,d)

Extractors vs. Case classes

There is a section in [1] that compares extractors and case classes which I summarize here:

• There is one shortcoming with case classes: they expose the concrete representation of data. This means that if a case succeeds in a matching, you know that the selector expression is an instance of that case class.
• Extractors do not expose the concrete representation of data.
• Case classes have less code and they are easier to set up. Scala compiler can optimize patterns over case classes much better than extractors.
• If a case class inherits from a sealed base class, Scala compiler checks for pattern matches for exhaustiveness and will complain in case there exists something while Scala compiler can not do the same thing for extractors.
• If you write code for a closed application, case classes are preferable because of their advantages in conciseness, speed and static checking.
• If you need to expose a type to unknown clients, extractors might be preferable.

References:

[1]: Martin Odersky, Lex Spoon, Bill Venners, "Programming in Scala", 2nd Edition

Lower bounds

Consider the class I defined in the first post:

Company is covariant in type T as you know by now. It's been a while and we realize it's the time to define a partnership relation for companies to connect them together and expand their co-operations. So we add the following method to Company class:

When we try to compile this code we get:

Error: covariant type T occurs in contravariant position in type T of value y

What happened here? To see what has happened it's better to take a look at Function1 definition in Scala:

This definition tells us that the input of a function is contravariant (negative) and what it returns is covariant (positive). So remember that no matter how many arguments a function has, all of them are in negative positions:

Now it should be more clear why we got the compile error. We were passing an argument of type T which is covariant (positive) and we know that arguments of a function has negative (contravariant) positions. This is a contradiction.

But how can we solve this? We can use lower type bounds to parameterize a method.

Lower type bounds: help you to declare a type to be a supertype of another type. T >: A indicates that type T or abstract type T refers to a supertype of type A.

Let's correct the defined method by using lower bounds:

By this new definition, partnerWith method accepts arguments that can be supertype of type T.

scala> val littleCompany: Company[SmallCompany] = new Company[SmallCompany](new SmallCompany)
littleCompany: Company[SmallCompany] = Company@149bc5a

scala> val bigCompany: Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany: Company[BigCompany] = Company@116ac04

scala> bigCompany.partnerWith(littleCompany)

So partnerWith method is now generalized and polymorphic.

Upper bounds

Assume that we want to restrict the partnership relation somehow in our example. We wanna say that a company can have partnership with another company that is a subtype of Company[BigCompany]. In order to apply this restriction in Scala, one can use upper bounds. So let's change the partnerWith method to reflect this:

Now partnerWith only accepts arguments that are a subtype of Company[BigCompany]. Let's prove this by introducing a new company:

This class does not extend BigCompany so Company[CrappyCompany] is not a subtype of Company[BigCompany] according to covariant type declaration. Now we try to build partnerships in REPL:

scala> val littleCompany: Company[SmallCompany] = new Company[SmallCompany](new SmallCompany)
littleCompany: Company[SmallCompany] = Company@1979eb4

scala> val bigCompany: Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany: Company[BigCompany] = Company@6d1901

scala> littleCompany.partnerWith(bigCompany)

scala> val crappyCompany: Company[CrappyCompany] = new Company[CrappyCompany](new CrappyCompany)
crappyCompany: Company[CrappyCompany] = Company@1d0d313

scala> littleCompany.partnerWith(crappyCompany)
<console>:11: error: inferred type arguments [Company[CrappyCompany]] do not conform to method 
partnerWith's type parameter bounds [U <: Company[BigCompany]]
       littleCompany.partnerWith(crappyCompany)
                     ^

So by upper type bound declaration we could restrict our partnership relation.

Remember that by definition a class is supertype and subtype of itself. Hence you can pass an instance of Company[BigCompany] to partnerWith:

scala> val bigCompany2 : Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany2: Company[BigCompany] = Company@b61e92

scala> bigCompany.partnerWith(bigCompany2)

Note that both lower and upper bound declarations have the colon as their second character so you don't mix the order ( <: >: )

Conclusion

In these three posts I showed how you can avail from Scala variance annotations and lower and upper bounds together to design your application in a type safe manner. Scala provides you with type-driven design where types of an interface guides its detailed design and implementation [1]. Examples provided in these posts were not the best examples but they could still show the purpose of their existence. I hope you have a better understanding of this cool feature in Scala.

References

[1]: Martin Odersky, Lex Spoon, Bill Venners, "Programming in Scala", 2nd Edition

Contravariant Subtyping

Do you remember the definition of covariant? Contravariant is the other way around. I will clarify this by the following example (example is adopted from [1]). Consider we have a trait:

This trait has a write method that is responsible for accepting an argument and writing it in the current network channel. Assume that we have two NetworkChannel of type AnyRef and String:

We just don't care about the write implementation now. There is another class that is responsible for controlling output channels:

The control method accepts an argument of type NetworkChannel[String]. Do you think we can pass NetworkChannel[AnyRef] to this method? Yes we can! NetworkChannel is declared to be contravariant in its type T.

Contrvariant subtyping means: If S is a subtype of T then NetworkChannel[T] is a subtype of NetworkChannel[S]

This seems to be a little bit strange in our example. We know that String is a subtype of AnyRef in Scala. How come NetworkChannel[AnyRef] can be a subtype of NetworkChannel[String] ?

Consider what you can do with each channel. NetworkChannel[String] accepts arguments of type String and write them in network channel. NetworkChannel[AnyRef] accepts arguments of type AnyRef and write them also in network channel. We can substitute NetworkChannel[String] with NetworkChannel[AnyRef] since whatever can be done by NetworkChannel[String] can be done also by NetworkChannel[AnyRef] (remember that NetworkChannel[AnyRef] can also write strings) but not vice versa. We can not use NetworkChannel[String] instead of NetworkChannel[AnyRef] since it only accepts strings.

Remember that contravariant orders types from more genetic to more specific.

How should I know if two types are covariants or contravariants? Read the following section.

Liskov Substitution Principle (LSP)

LSP is a principle in object-oriented programming. It states:

It is safe to assume that S is a subtype of T if you can substitute a value of type S wherever a value of type T is required without violating the desirable properties of the program [2].

So let's say we have S as a subtype of T. If Class of type S is a subtype of Class of type T then there is a covariant subtyping between them. Otherwise if Class of type T is a subtype of Class of type S then there is contravariant subtyping between them. if there is no subtyping relation between Class of type S and T, then there is invariant subtyping relation.

I hope you have understood Scala Types variances. In the next post I will discuss Lower bounds and upper bounds.

References

[1]: Martin Odersky, Lex Spoon, Bill Venners, "Programming in Scala", 2nd Edition
[2]: Barbara Liskov, Jeannette Wing, "A behavioral notion of subtyping", ACM Transactions on Programming Languages and Systems (TOPLAS), Volume 16, Issue 6 (November 1994), pp. 1811 – 1841

In this series of posts I will try my best to clear out type variance annotations in Scala; a powerful tool to handle type orderings from specific to generic and vice verca.

Covariant Subtyping

We start by covariant subtyping. If you don't know what it is, please be patient. First consider the following code in Scala:

We have a Company class that has type parameter T. This class accepts a value of type T for its constructor. We have two companies defined here: BigCompany and SmallCompany. As it is obvious from the code SmallCompany is a subtype of BigCompany. Finally we have a class Investor that accepts only companies of type BigCompany so it invests only on big ones!
Now let's do some constructions in REPL:

scala> val littleCompany: Company[SmallCompany] = new Company[SmallCompany](new SmallCompany)
littleCompany: Company[SmallCompany] = Company@1e70822

scala> val bigCompany: Company[BigCompany] = new Company[BigCompany](new BigCompany)
bigCompany: Company[BigCompany] = Company@9bb740

scala> val bigInvestor:Investor = new Investor(bigCompany)
bigInvestor: Investor = Investor@1004566

So far so good. After a while it is decided that we need investors that can take care of small companies. We don't want to define a new brand of investor thus we use the existing one. So we try to construct:

scala> val smallInvestor:Investor = new Investor(littleCompany)

But we get:

<console>:8: error: type mismatch;
 found   : Company[SmallCompany]
 required: Company[BigCompany]
Note: SmallCompany <: BigCompany, but class Company is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val smallInvestor:Investor = new Investor(littleCompany)
                                                 ^

Bang! What happened? Compile error!

Let's see what happened exactly here. We know by definition that SmallCompany is subtype of BigCompany. But what is the relation between Company[SmallCompany] and Company[BigCompany] ? It is not defined by default and we need somehow to tell compiler about this. In order to solve this we change class Company to the following:

The only thing that is changed is the type parameter that has a + sign in front. This means that subtyping is covariant (flexible) in paramter T.

Covariant subtyping means: if S is a subtype of T then Company[S] is subtype of Company[T].

Remember that covariant orders types from more specific to more generic.

Let's construct a small investor again with new Company definition:

scala> val smallInvestor:Investor = new Investor(littleCompany)
smallInvestor: Investor = Investor@9fad9c

Great! We could solve this by Scala variance subtyping. By a single + sign that is called variance annotation you could tell Scala compiler that you want Company[SmallCompany] be a subtype of Company[BigCompany].

In the next post, we will investigate contravariant subtyping.

Parallel Collections were introduced in Scala 2.9 release which are built on the same abstractions and provide the same interfaces as existing collection implementation. It follows the model proposed by Doug Lea in his Fork/Join framework [1]. They are really easy to use and this is one of the advantages you can see for them. If you are familiar with parallel programming, it is quite difficult to convert a sequential program into a parallel one. This difficulty comes from the fact that each has its own paradigm, algorithm and data structure.

Scala enables you to invoke a method called par on a collection to get a parallel collection. This parallel collection then employs all the available cores on the target system. You can easily try this out in REPL:

scala> List(1,2,3,4,5).par.filter { _ % 2 == 0 }
res1: scala.collection.parallel.immutable.ParSeq[Int] = ParVector(2, 4)

This is a simple filtering on a list. The only new thing I added is 'par' after list definition.

Now let's have a comparison between sequential and parallel versions and see how parallel collection can help us. Consider that we have a Data class with some data as follows:

and a companion object to construct Data instances:

Now we define computation trait to perform some operations on the aforementioned data class:

This trait has two implementations:

In each implementation, method compute applies a map with function f and then calculate the maximum element in the list. The only difference between these two implementations is the use of 'par' method. Now let's run and time each implementation:

Running the code with SBT on a machine with 4 CPU cores:

> run 3
[info] Running com.jayway.parallel.ParallelExperiment 3
Sequential 44 (ms)
Parallel 14 (ms)

> run 100
[info] Running com.jayway.parallel.ParallelExperiment 100
Sequential 1020 (ms)
Parallel 268 (ms)

As expected, we gained speed up in parallel version. Results for a couple of run with different list sizes is depicted in the following fgure:

Now let's change function f to the following:

  def f(data:Data):Int =  {
    data.a + data.b
  }

I removed the Thread.sleep so the method returns immediately. Now let's have a look at results for different runs again:

Interestingly, this time the result is not good at all! It's even vice versa and sequential implementation has lower running time for the first 5 runs. This is because using parallel collection has some overhead for distributing (fork) and gathering (join) the data between cores. Thus one can conclude having heavy computations (which is the case in most applications), parallel collections can be of great performance improvement. But there might be scenarios (like the second scenario we discussed) where sequential collection is faster

For a better understanding of Scala Parallel Collection internals, please watch Aleksander Prokopec's presentation from Scala Days 2010. If you are interested to know more then you can read [2].

References:

[1]: Doug Lea, A Java Fork/Join Framework, 2000.
[2]: Aleksandar Prokopec, Tiark Rompf, Phil Bagwell, Martin Odersky, "A Generic Parallel Collection Framework", Euro-Par 2011

To start with here's a simplified version of the trait that I wanted to intercept:

trait MyTrait {
    var someStuff = "someStuff"
    def someMethod() : String = {
        "do something"
    }
}

class SomeClass extends MyTrait

As I wanted to intercept the constructor and some method calls I defined my aspect like this:

public aspect MyAspect {

	public pointcut newInstance() : execution(MyTrait+.new(..));

	public pointcut someInvocation() : execution(String MyTrait+.someMethod(..));

	after() : newInstance() {
	    System.out.println("after MyTrait constructor invocation");
	}

	after() : someInvocation() {
	    System.out.println("after someMethod invocation");
	}
}

Runnning the code I got quite puzzled that the method invocation pointcut worked fine, but not the constructor pointcut. After consulting the AspectJ reference and convincing myself that the pointcut definition was ok, I had a closer look at the byte code. As we don't have traits in Java the byte code for my trait becomes equivalent to the following pseudo-javacode:

public interface MyTrait extends scala.ScalaObject{
    public String someMethod();
}

public abstract class MyTrait$class extends java.lang.Object{
    public static String someMethod(MyTrait myTrait) {
      // invoke someMethod
    }
    public static void $init$(MyTrait myTrait) {
      // some init stuff
    }
}

So in bytecode I've got an interface and a class with some static method calls. The stuff that goes in the constructor of my trait is done in the $init$ method of the MyTrait$class class. And that's the reason why my pointcut won't work. So actually I have to intercept the $init$ method of the MyTrait$class to get what i want:

public pointcut newInstance() : execution(void MyTrait$class.$init$(..));

We start with creating a new eclipse plugin project and create our wizard class. The NewElementWizard we want to extend resides in the org.eclipse.jdt.ui bundle. So we add both org.eclipse.jdt.ui and org.eclipse.jdt.core to our plugin manifest:

Then we create a new class "NewActivityWizard":

package newwizard;

import org.eclipse.jdt.internal.ui.wizards.NewElementWizard;

public class NewActivityWizard extends NewElementWizard {

}

First thing we get is a warning
"Discouraged access: The type NewElementWizard is not accessible due to restriction on required library /Applications/eclipse/plugins/org.eclipse.jdt.ui_3.5.1.r351_v20090821-0800.jar".

The NewElementWizard belongs to an internal package and is only exported (using the 'xfriends' directive) to some chosen bundles. So we have the choice to have a look at the code of NewElementWizard and implement our wizard by extending org.eclipse.jface.wizard.Wizard instead, or we could make our plugin a fragment. Fragments are part of the OSGi R4 release and according to the specification a fragment is “a bundle that is attached to a host bundle”, adding content to the target bundle. For now we choose the later alternative, just because it requires less work.

Next thing we have to do is to override some methods of the superclass and of course we want to add our own wizard page. Our wizardpage will extend the org.eclipse.jdt.ui.wizards.NewTypeWizardPage. The following code is pretty much the same as in the wizards used internally in JDT as for example org.eclipse.jdt.internal.ui.wizards.NewClassCreationWizard.

public class NewActivityWizard extends NewElementWizard {

	private NewActivityWizardPage newActivityPage = null;

	public NewActivityWizard() {
        setWindowTitle("My new Android Activity");
        newActivityPage = new NewActivityWizardPage();
    }

	@Override
	public void addPages() {
        addPage(newActivityPage);
    }

    @Override
	 public void init(IWorkbench workbench, IStructuredSelection selection) {
    	newActivityPage.init(selection);
    }

    @Override
	public boolean canFinish() {
       return super.canFinish() &&
                   getContainer().getCurrentPage() == newActivityPage;
    }

	@Override
	protected void finishPage(IProgressMonitor monitor)	throws InterruptedException, CoreException {
		newActivityPage.createType(monitor);
	}

	@Override
	public IJavaElement getCreatedElement() {
		return newActivityPage.getCreatedType();
	}
}

Next we have to define the extension in the fragment.xml:

The most interesting part of the wizard page is the code generation part. We want to generate a onCreate method that takes a android.os.Bundle as argument:

private void generateOnCreate(IType type, ImportsManager imports) throws CoreException, JavaModelException {
	StringBuilder buf = new StringBuilder();
	final String delim = "\n";
	buf.append("@Override");
	buf.append(lineDelim);
	buf.append("public void onCreate(");
	buf.append(imports.addImport("android.os.Bundle"));
	buf.append(" savedInstanceState) {");
	buf.append(lineDelim);
	buf.append("super.onCreate(savedInstanceState);");
	buf.append(lineDelim);
	final String content =
            CodeGeneration.getMethodBodyContent(type.getCompilationUnit(),
               type.getTypeQualifiedName('.'), "onCreate", false, "", delim);
	if (content != null && content.length() != 0)
		buf.append(content);
	buf.append(lineDelim);
	buf.append("}"); //$NON-NLS-1$
	type.createMethod(buf.toString(), null, false, null);
}

This is just the basic version of the new wizard, we also want to add intent actions and categories and update the Android manifest file. The complete source is available at http://github.com/kobmic/adt-extensions.

So how do we update the android manifest? As a Java programmer dom4j and XStream are some of my favourite XML frameworks - but I'm quite enthusiastic about Scala and Scala provides XML support baked into the language. The XML generation code for a new Android activtiy with a given list of intent actions and categories in Scala looks like this:

def createXML(activityName: String, intentActions: List[String], intentCategories: List[String]) : Node = {
  val xml =
    
         {for(action <- intentActions)
             yield {}}
	 {for(category <- intentCategories)
             yield {}}
      
    
   xml
}

Putting it all together

I chose to do the XML manipulation (written in Scala) and the Wizard (written in Java) in different OSGi bundles. If you don't have the eclipse Scala IDE installed you even need the Scala library 2.7.7 as OSGi bundle. You can download the wizard with or without Scala dependencies from http://github.com/kobmic/adt-extensions/downloads.

The complete wizard looks like this:

At time of writing there's a new version of MOTODEV studio available, now even as a set of plugins to eclipse:
http://developer.motorola.com/docstools/library/Installing_MOTODEV_Studio_for_Android/#installingStudioPlugins

If you want to follow the example you need:
gradle 0.8
scala 2.7.7
apache felix framework 2.0.1 (or your favourite OSGi container)

Instead of a HelloWorld I'll take Neill's BundleActivator example that prints out a list of currently installed bundles when started.

package com.jayway.osgi.hello

import org.osgi.framework._

class Activator extends BundleActivator {  

    def start( context: BundleContext ) {
          var bundleNames = context.getBundles()
                   .map   (b => b.getSymbolicName())
                   .filter(b => b != context.getBundle())
          println("Installed bundles: " + bundleNames.toString())
     }    

     def stop( context: BundleContext )  {
     }
 }

Gradle comes with both Scala and OSGi plugins that make it quite easy to build the bundle. The Scala plugin assumes the same standard project layout as used in Maven, i.e java production code will reside in the 'src/main/java', scala production code in the 'src/main/scala' folder. So I put the Activator class in 'src/main/scala' and add the following build.gradle file in my project folder:

group = 'com.jayway'
version = '1.0.0'

usePlugin 'scala'
usePlugin 'osgi'

repositories {
  mavenCentral()  // http://repo1.maven.org/maven2/
  mavenRepo(urls: 'http://www.ibiblio.org/maven2/')
  mavenRepo(urls: 'http://scala-tools.org/repo-releases/')
}

dependencies {
    // Libraries needed to run the scala tools
    scalaTools 'org.scala-lang:scala-compiler:2.7.7'
    scalaTools 'org.scala-lang:scala-library:2.7.7'

    compile 'org.scala-lang:scala-library:2.7.7'
    // osgi framework
    compile 'org.apache.felix:org.apache.felix.framework:2.0.1'
}

configure(jar.osgi) {
    // generate the bundle manifest
    version = '1.0.0'
    name = 'scala osgi bundle built with gradle'
    symbolicName = 'com.jayway.osgi.hello'
    instruction 'Require-Bundle', 'scala.library'
    instruction 'Bundle-Activator', 'com.jayway.osgi.hello.Activator'
    instruction 'Import-Package', 'org.osgi.framework'
    instruction 'Export-Package', 'com.jayway.osgi.hello'
    instruction 'Bundle-Vendor', 'JAYWAY AB'
}

The Gradle OSGi plugin makes use of the BND tool to generate the bundle manifest. I added the scala library OSGi bundle as required and imported the package org.osgi.framework. Now I'm good to go and call gradle from the commandline:

C:\ws-scala-test\com.jayway.osgi.hello>gradle build

Running on apache felix 2.0.1

Now start the osgi container and install the bundles starting with the scala library as OSGi bundle:

C:\felix-framework-2.0.1>java -jar bin/felix.jar

Welcome to Felix
================

->install http://www.scala-lang.org/downloads/distrib/files/scala.library_2.7.7.final.jar
Bundle ID: 24

-> install file:C:\ws-scala-test\com.jayway.osgi.hello\build\libs\com.jayway.osgi.hello-1.0.0.jar
Bundle ID: 25

-> ps
START LEVEL 1
   ID   State         Level  Name
[   0] [Active     ] [    0] System Bundle (2.0.1)
[   1] [Active     ] [    1] Apache Felix Bundle Repository (1.4.2)
[   2] [Active     ] [    1] Apache Felix Shell Service (1.4.1)
[   3] [Active     ] [    1] Apache Felix Shell TUI (1.4.1)
[  24] [Resolved   ] [    1] Scala Library (2.7.7.final)
[  25] [Installed  ] [    1] scala osgi bundle built with gradle (1.0.0)

-> start 25
Installed bundles: Array(org.apache.felix.framework, org.apache.felix.bundlerepository, org.apache.felix.shell, org.apa
he.felix.shell.tui, scala.library, com.jayway.osgi.hello)

Running on eclipse equinox 3.5.1

If you want to use equinox instead of felix you could change the dependencies like this:

compile 'org.eclipse:osgi:3.5.1-R35x_v20090827'

I had some problems to find a maven repository containing the current equinox osgi bundles (3.5.1). If you are running maven you can publish the bundles from your eclipse installation to your local maven repository:

mvn eclipse:to-maven -DeclipseDir="[eclipse-installation-dir]"

Finally add your local repository to your build.gradle:

mavenRepo(urls: 'file://[path-to-user-home]/.m2/repository/')

// this is easy since everything are Strings
scala> List("one", "two", "three")
res0: List[java.lang.String] = List(one, two, three)

scala> res0.foreach(x => println(x.length()))
3
3
5

// with a mixed collection it gets harder
scala> List("one", 2, "three", 4.0)
res2: List[Any] = List(one, 2, three, 4.0)

scala> res2.foreach(x => println(x.length()))
<console>:6: error: value length is not a member of Any

// it is not enough to just filter out the objects that are Strings
scala> res2.filter(_.isInstanceOf[String])
res4: List[Any] = List(one, three)

// as we still have a List of Any
scala> res4.foreach(x => println(x.length()))
<console>:7: error: value length is not a member of Any

// both filtering and mapping is needed
scala> res2.filter(_.isInstanceOf[String]).map(_.asInstanceOf[String])
res6: List[String] = List(one, three)

// finally it works
scala> res6.foreach(x => println(x.length()))
3
5

// another option is to use pattern matching
scala> res2.foreach(_ match { case x:String => println(x.length); case _ => })
3
5

I wanted a more elegant solution with less boiler plate so after playing around with implicit conversions and reified types I managed to come up with rather nice solution:

// define a class that implements a restrictTo method
scala> class IterableWithRestrictTo[A](from:Iterable[A]) {
     | def restrictTo[B](implicit m: scala.reflect.Manifest[B]) = {
     | from.filter(m.erasure.isInstance(_)).map(_.asInstanceOf[B])
     | }
     | }
defined class IterableWithRestrictTo

// define implicit conversion
scala> implicit def makeIterableWithRestrictTo[A](from : Iterable[A]) = {
     | new IterableWithRestrictTo(from)
     | }
makeIterableWithRestrictTo: [A](from: Iterable[A])IterableWithRestrictTo[A]

scala> List("one", 2, "three", 4.0)
res0: List[Any] = List(one, 2, three, 4.0)

// restrict to objects of type String
scala> res0.restrictTo[String]
res1: Iterable[String] = List(one, three)

// works since the collection have correct type
scala> res1.foreach(x => println(x.length()))
3
5

Using the name restrictTo clearly shows that not only is there a type conversion, but also a restriction to a subset of the elements. Notice that the manifest is needed to get the reified type, since the type B is actually erased at compile time. Read more about reified types in Scala.

Let me know you have another solution!

When the ratio between a and b is equal to the ratio between a+b and a, then you have the Golden Ratio. The value of the Golden Ratio is:

Why is it so special? Because if you know about it, it can be seen in many different places; in geometrical figures, in architecture, in paintings, and in nature. I'll show you. Draw a pentagon:

Draw straight lines between all vertices of the pentagon. This will result in a pentagram (five-pointed star). Now mark the lengths a and b as this picture shows:

Guess what the ratio is between the lengths a and b? The Golden Ratio. Now, mark two other lengths a' and b':

You guessed it. The ratio between a' and b' is the Golden Ratio.

It's actually pretty easy to approximate Phi (φ), as the Golden Ratio normally is called. Take any two numbers and add them, then divide the sum with the larger number. For example, 11 and 74:

11.0+74.0=85.0
85.0/74.0=1.1486486486486487

Now we add the larger of the numbers (74) to the sum (85), and then we again divide the new sum (159) with the the previous sum (85):

74.0+85.0=159.0
159.0/85.0=1.8705882352941177

We're getting there. One more time:

85.0+159.0=244.0
244.0/159.0=1.5345911949685536

No, this is too tedious. Let's write a program that does it. For no reason whatsoever, I'll pick Scala. What is Scala? I'm on a Mac, so I'll just do:

% port info scala
scala @2.7.5 (lang, java)

Description:          Scala is a modern multi-paradigm programming language designed to express common programming patterns
                      in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and
                      functional languages. It runs inside a Java Virtual Machine and is fully interoperable with Java.
Homepage:             http://www.scala-lang.org/

This is how I install Scala:

% sudo port install scala

I'll want some editor support, so I install the TextMate bundle that accompanies Scala:

% cd /Library/Application\ Support/TextMate/Bundles/
% unzip /opt/local/share/scala/misc/scala-tool-support/textmate/Bundles/Scala.tmbundle.zip

I'll tell TextMate to reload its bundles by selecting the menu "Bundles | Bundle Editor | Reload Bundles". Now I'm all set.

Consider this Scala program (phi1.scala):

var x = 11.
var y = 74.

for (i <- 1 to 5) {
    var sum = x + y
    var q = sum / y
    println(q)
    x = y
    y = sum
}

I'll run it like this:

% scala phi1.scala
1.1486486486486487
1.8705882352941177
1.5345911949685536
1.651639344262295
1.6054590570719602

It's not pretty, but it does the job. Let's spice it up with some input arguments and better printout (phi2.scala):

var x = 11.
var y = 74.
var n = 13

if (args.length > 3) {
    println("Usage: scala phi2.scala [n [x [y]]]")
    println("  n: number of times to perform calculation (default: " + n + ")")
    println("  x: start value of 'x' (default: " + x + ")")
    println("  y: start value of 'y' (default: " + y + ")")
    System.exit(1)
}
if (args.length >= 1) n = Integer.parseInt(args(0))
if (args.length >= 2) x = Integer.parseInt(args(1))
if (args.length == 3) y = Integer.parseInt(args(2))

for (i <- 1 to n) {
    var sum = x + y
    print(i + ": " + x + "+" + y + "=" + sum)
    var q = sum / y
    println(", " + sum + "/" + y + "=" + q)
    x = y
    y = sum
}

I'll run it with four arguments to test it:

% scala phi2.scala 1 2 3 4
Usage: scala phi2.scala [n [x [y]]]
  n: number of times to perform calculation (default: 13)
  x: start value of 'x' (default: 11.0)
  y: start value of 'y' (default: 74.0)

I won't bore you with more tests, but instead do what we intended to do from the start: have a program run the iterations for us. I'll run it with no arguments, using the defaults:

% scala phi2.scala
1: 11.0+74.0=85.0, 85.0/74.0=1.1486486486486487
2: 74.0+85.0=159.0, 159.0/85.0=1.8705882352941177
3: 85.0+159.0=244.0, 244.0/159.0=1.5345911949685536
4: 159.0+244.0=403.0, 403.0/244.0=1.651639344262295
5: 244.0+403.0=647.0, 647.0/403.0=1.6054590570719602
6: 403.0+647.0=1050.0, 1050.0/647.0=1.6228748068006182
7: 647.0+1050.0=1697.0, 1697.0/1050.0=1.6161904761904762
8: 1050.0+1697.0=2747.0, 2747.0/1697.0=1.618738951090159
9: 1697.0+2747.0=4444.0, 4444.0/2747.0=1.6177648343647615
10: 2747.0+4444.0=7191.0, 7191.0/4444.0=1.618136813681368
11: 4444.0+7191.0=11635.0, 11635.0/7191.0=1.6179947156167431
12: 7191.0+11635.0=18826.0, 18826.0/11635.0=1.6180489901160293
13: 11635.0+18826.0=30461.0, 30461.0/18826.0=1.6180282587910337

Actually, I will run one more test, but this time I'll choose the starting values more carefully. One of many wonderful properties of the Fibonacci series is that the ratio between the numbers is very close to the Golden Ratio. This means it will converge much quicker. Check this out, four correct decimals after just four iterations:

% scala phi2.scala 4 21 34
1: 21.0+34.0=55.0, 55.0/34.0=1.6176470588235294
2: 34.0+55.0=89.0, 89.0/55.0=1.6181818181818182
3: 55.0+89.0=144.0, 144.0/89.0=1.6179775280898876
4: 89.0+144.0=233.0, 233.0/144.0=1.6180555555555556

Here is the final version, with some stuff that makes the file executable (phi.scala):

#!/bin/sh
exec scala $0 $@
!#
var x = 11.
var y = 74.
var n = 13

if (args.length > 3) {
    println("Usage: scala phi.scala [n [x [y]]]")
    println("  n: number of times to perform calculation (default: " + n + ")")
    println("  x: start value of 'x' (default: " + x + ")")
    println("  y: start value of 'y' (default: " + y + ")")
    System.exit(1)
}
if (args.length >= 1) n = Integer.parseInt(args(0))
if (args.length >= 2) x = Integer.parseInt(args(1))
if (args.length == 3) y = Integer.parseInt(args(2))

for (i <- 1 to n) {
    var sum = x + y
    print(i + ": " + x + "+" + y + "=" + sum)
    var q = sum / y
    println(", " + sum + "/" + y + "=" + q)
    x = y
    y = sum
}

I'll make it executable and run it:

% chmod +x phi.scala
% ./phi.scala 10 13 21
1: 13.0+21.0=34.0, 34.0/21.0=1.619047619047619
2: 21.0+34.0=55.0, 55.0/34.0=1.6176470588235294
3: 34.0+55.0=89.0, 89.0/55.0=1.6181818181818182
4: 55.0+89.0=144.0, 144.0/89.0=1.6179775280898876
5: 89.0+144.0=233.0, 233.0/144.0=1.6180555555555556
6: 144.0+233.0=377.0, 377.0/233.0=1.6180257510729614
7: 233.0+377.0=610.0, 610.0/377.0=1.6180371352785146
8: 377.0+610.0=987.0, 987.0/610.0=1.618032786885246
9: 610.0+987.0=1597.0, 1597.0/987.0=1.618034447821682
10: 987.0+1597.0=2584.0, 2584.0/1597.0=1.6180338134001253

Check out Golden Ratio on Wikipedia for more details.

The eclipse plugin has pretty basic support for code completion, syntax highlihting, and some wizards to create projects, scala classes, objects and traits.
We started out with one java project and one scala project. Testing with the latest release, it was a bit tricky to resolve our imported java classes at build and runtiime and vice versa:

To use a Scala class in your Java project:

• add the scala project to the build path of your java project (project properties -> Java Build Path -> Projects -> Add...)
• add the scala nature to your java project (rightclick your java project -> Scala -> add Scala nature)

To use a Java class in your Scala project:

• import of java.* works fine (for example using java.util.Date in scala, like in the Scala Tutorial for Java programmers)
• add the java project to the build path of your scala project

With the current eclipse plugin it is not possible to use scala objects in your java classes. This seems to be a known issue.

It's even possible to create scala and java classes in the same project if your project has both scala and java nature. But the mixed project support in the plugin is still quite buggy: adding a java class to your scala project sometimes make eclipse think that your scala class is a java class. Closing and reopening your project is a way to get around this.

Overall working with the scala eclipse plugin is quite frustating, as it is still very buggy. Actually the first impression was that interoperability does not work at all in the plugin.

The IntelliJ Scala plugin 0.2.22340 had much better Scala support with all the above and then refactoring support (rename, move, introduce variable) and open declaration navigation. IntelliJ also supported Java and Scala interoperability even within the same module.

Here is screenshot from the management console showing 81 active CPUs and 23 GB free memory:

I tried some Scala actors which worked very well. By running some CPU intensive calculations I concluded that it scaled linearly up to the number of CPUs. No surprise, but very cool when you are running 80 threads at full speed! Also cool to see that Scala runs smoothly which means that neither Scala nor Azul does anything strange. Just pure bytecode.

Somebody tried to configure the Azul JVM in Eclipse and was able to run JUnit tests on Azul. We were also able to run maven without any problems.

We tried to allocate lots of memory. No problem there either, although we had to specify -Xmx10G or something. By default you only get 1.6 GB....

A colleague started tomcat, produced lots of garbage and tried garbage collection at 8 GB of heap. No pause! However, there was some synchronization problem with Log4j since all 16 CPUs had to coordinate their work.

We encountered two problems:

• xstream apparently had a dependency to Unsafe which is not supported on Azul
• the program must be headless, so no Swing

A quick rewrite later and we had generated a 64 megapixel mandelbrot calculated in parallel on 40 CPUs. The only problem was saving the PNG file which was a single threaded operation...