In my previous post, I mentioned how AppHarbor provides an online tool for manually testing transformations. Also, the Visual Studio extension SlowCheetah provides support for manually testing and transformation and diffing them against the base configuration from inside Visual Studio. These tools are useful and may be fine for tinkering with getting a transform just right, but being software professionals, we find it highly desirable to reliably automate our transformation testing.

In this article we shall look at one way to exercise our Web.config transformations from our own code, so that we can TDD our transformations and regression test them as part of our continuous integration cycle, like everything else.

Starting Point

The actual Web.config transformation process is normally invoked by MSBuild, as described in detail by Elton Stoneman in this post. (One approach to testing the transformations could, thus, be to fork off MSBuild processes on custom build targets, but we'd like to take a less cumbersome route.)

Being an MSBuild task means that the transformation is implemented as an ITask realization. The concrete implementation is called TransformXml and resides in the Microsoft.Web.Publishing.Tasks.dll assembly. Note that this assembly is not pre-installed with Windows/.NET but rather installed along with Visual Studio 2010, typically to C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web). Given that its location is not fixed, you'd probably want to add it to your dependencies folder (caveat emptor: check license terms before passing it around).

From Elton's post, we can also see 3 arguments being passed to the TransformXml task:

• Source (the XML file to transform)
• Transform (the transformation XML file itself)
• Destination (the target of the transformed XML file)

Armed with that background knowledge, let's move on and try to get the transformation to run outside of the MSBuild environment in which it normally executes. For our experiment, we'll be transforming the following source (app.config):

   <configuration>
     <appSettings>
	   <add key="testKey" value="testValue"/>
     </appSettings>
   </configuration>

The transformation we'll be testing (app.Debug.config) should change the "testValue" of the "testKey" to something transformation specific ("debugValue" in this case):

   <configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
     <appSettings>
       <add key="testKey"
            value="debugValue"
            xdt:Locator="Match(key)" xdt:Transform="SetAttributes" />
     </appSettings>
   </configuration>

Test Approach

Our initial stab at automatically testing the transformation will be to invoke the XmlTransform through its outermost interface; i.e. the ITask interface. First, we need to add a reference to Microsoft.Web.Publishing.Tasks.dll as mentioned above. Additionally, we need to reference Microsoft.Build.Framework and Microsoft.Build.Utilities.v4.0 (these should be available from your usual Project, Add Reference, .NET tab).

Opening up the XmlTransform class in Visual Studio's Object Browser shows us a single method, Execute(), plus a handful of properties, most of which we recognize from Stoneman's previously mentioned post. A little experimentation reveals the following details:

• SourceRootPath
• The base path to which the XML file specified in Source is relative. If not specified, this will  be the current directory (which, when running from a test runner, would typically be your build target directory).
• Source
• The name of the XML source file to be transformed. If this is not an absolute path, it will be relative to the SourceRootPath.
• TransformRootPath
• The base path to which the XML file specified in Transform is relative. If not specified, this will be the same as SourceRootPath.
• Transform
• The name of the XML transformation to be applied to the source. If this is not an absolute path, it will be relative to the TransformRootPath.
• Destination
• The name of the resulting transformed XML file. If this is not an absolute path, it will be relative to the current directory (typically your build target directory, in a test context).

So all we need to do is something like the following, right?

   var transformer = new TransformXml
   {
      SourceRootPath = @"..\..\..\Jayway.WebTransformTesting\",
      Source = @"app.config",
      Transform = @"app.Debug.config",
      Destination = @"app.transformed.config"
   };
   var transformationSucceeded = transformer.Execute();

Well, as the System.InvalidOperationException with the message "Task attempted to log before it was initialized" thrown in our face subtly tells us, this is not quite the case...

Investigating the task implementation with Reflector shows the problem to be due to the fact that a property, BuildEngine, of the Task base class has not been set. It is used as a dependency injection mechanism, giving the task access to some environment services (see IBuildEngine on MSDN) normally provided by MSBuild.

Well, the whole purpose of this experiment was to run outside of MSBuild, so we'll roll our own "build engine" instead:

   public class BuildEngineForTest : IBuildEngine
   {
      public void LogErrorEvent(BuildErrorEventArgs e)
      {
         Console.WriteLine(LogFormat, "ERROR", e.Message);
      }

      public void LogWarningEvent(BuildWarningEventArgs e)
      {
         Console.WriteLine(LogFormat, "WARNING", e.Message);
      }

      public void LogMessageEvent(BuildMessageEventArgs e)
      {
         Console.WriteLine(LogFormat, e.Importance, e.Message);
      }

      public void LogCustomEvent(CustomBuildEventArgs e)
      {
         Console.WriteLine(LogFormat, "CUSTOM", e.Message);
      }

      public bool BuildProjectFile(
         string projectFileName, string[] targetNames,
         IDictionary globalProperties, IDictionary targetOutputs)
      {
         return true;
      }

      public bool ContinueOnError
      {
         get { return true; }
      }

      public int LineNumberOfTaskNode
      {
         get { return 0; }
      }

      public int ColumnNumberOfTaskNode
      {
         get { return 0; }
      }

      public string ProjectFileOfTaskNode
      {
         get { return string.Empty; }
      }

      private const string LogFormat = "{0} : {1}";
   }

This was the piece we were missing to be able to run the transformation outside MSBuild. We are, thus, able to write our coveted test:

...
   [Test]
   public void ApplyingDebugTransformationToConfigurationSetsExpectedKey()
   {
      const string targetFile = @"app.transformed.config";

      var transformer = new TransformXml
      {
         BuildEngine = new BuildEngineForTest(),
         SourceRootPath = @"..\..\..\Jayway.WebTransformTesting\",
         Source = @"app.config",
         Transform = @"app.Debug.config",
         Destination = targetFile
      };

      // Left as exercise for the reader... ;]
      AssertKeyInTransformedFile(targetFile);
   }
...

Our transformation runs successfully, yielding the following console output from our "build engine":

Normal : Transforming Source File: ..\..\..\Jayway.WebTransformTesting\app.config
Normal : Applying Transform File: ..\..\..\Jayway.WebTransformTesting\app.Debug.config
Low : Executing SetAttributes (transform line 6, 35)
Low : on /configuration/appSettings/add[@key='testKey']
Low : Applying to 'add' element (source line 4, 6)
Low : Set 'key' attribute
Low : Set 'value' attribute
Low : Set 2 attributes
Low : Done executing SetAttributes
Normal : Output File: app.transformed.config
Normal : Transformation succeeded

Yay? Yay.

Conclusion

We have shown how Web.config Transformations can be invoked programmatically, outside the MSBuild process by providing a simple custom IBuildEngine realization; a technique that might prove useful for other scenarios as well.

We have applied that knowledge to enable automated testing of our transformations.

In my next post in this series, we'll continue our investigation, aiming to get rid of the whole, nasty detour around the file system.

I was adding a transformation to adapt our NLog configuration for the various environments, when I ran into a problem. I'd added the following transformation, intended to set the log level to Warning on the given deloyment:

<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd">
  <rules>
    <logger name="*" minlevel="Warn"
      xdt:Transform="SetAttributes(minlevel)" xdt:Locator="Match(name)"
    />
  </rules>
</nlog>

However, the original fragment kept being passed through without modification. I tried various modes of matching (incidentally, the online Web Config Transformation Tester is a good tool for such tinkering with transformations) to no avail, until I decided to try and remove the NLog namespace from the transformation:

<nlog>
  <rules>
    <logger name="*" minlevel="Warn"
      xdt:Transform="SetAttributes(minlevel)" xdt:Locator="Match(name)"
    />
  </rules>
</nlog>

And lo and behold; it worked.

That could have been the end of it, but I decided to dig a bit deeper, since we have the NLog schema in our solution and properly referencing it allows us to have IntelliSense within the web.config.

Eventually, as an experiment, I tried associating a namespace prefix with the NLog namespace at the top level, and then explicitly qualify the NLog elements with that prefix, rather than relying on specifying a default namespace on the NLog element:

<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform"
  xmlns:nlog="http://www.nlog-project.org/schemas/NLog.xsd"
>

  <nlog:nlog>
    <nlog:rules>
      <nlog:logger name="*" minlevel="Warn"
        xdt:Transform="SetAttributes(minlevel)" xdt:Locator="Match(name)"
      />
    </nlog:rules>
  </nlog:nlog>

</configuration>

Success!

Conclusion

Explicitly prefixing elements from foreign XML namespaces in config transformations must be described as a workaround, since semantically the first and the third NLog fragments are equivalent.

In spite of this flaw in the web.config transformations implementation, this workaround allows you to get the benefits of IntelliSense and better validation when you have the schema for NLog or any other foreign fragment available.

The built-in support for rendering XML from RESTful web services in Spring is built on top of the Marshaller abstraction originally from Spring Web Services (now moved to Spring core). This is a pretty nice abstraction if you want to use one of the existing Object-XML marshalling frameworks (JAXB, Castor, XMLBeans, etc.), but if you want to move out of that box you're pretty much stranded.

I really do want to move out of the box because I think the control and readability greatly improves if you do your XML marshalling in Groovy. It's quite common that the same domain object will need to be marshalled differently depending on the scenario (an object will e.g. quite probably need to be marshalled differently in a detail view than in a list). Also, it is not uncommon that more complicated application logic will be required when rendering the XML; a common example of this is URI generation in 'linked' REST applications (HATEOAS). Hence, Groovy is the perfect choice: Very little code is needed, you are able to include some logic if necessary, and the code written exactly reflects the XML that will be produced.

The Building Blocks

I started out by defining a simple interface for a completely generic Marshaller:

 
public interface SimpleMarshaller {
  public void marshal(Writer writer, Object o);
}
 

I now needed a View implementation to render Model objects using the SimpleMarshaller interface (much of this code is actually borrowed from the Spring MarshallingView - Apache license rocks!):

 
public class SimpleMarshallingView extends AbstractView {
  public final static String DEFAULT_MODEL_KEY = "objectToMarshal";
  private static final boolean AUTOMATIC_FLUSH = true;
  private static final String DEFAULT_CHARSET = "utf8";
  private final SimpleMarshaller marshaller;
 
  public SimpleMarshallingView(SimpleMarshaller marshaller) {
    this.marshaller = marshaller;
  }
 
  @Override
  protected final void renderMergedOutputModel(Map<String, Object> model, HttpServletRequest request,
      HttpServletResponse response) throws Exception {
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    PrintWriter writer = new PrintWriter(new OutputStreamWriter(outputStream, DEFAULT_CHARSET), AUTOMATIC_FLUSH);
    marshaller.marshal(writer, objectToRender(model));
    response.setContentType(getContentType());
    response.setContentLength(outputStream.size());
    FileCopyUtils.copy(outputStream.toByteArray(), response.getOutputStream());
  }
 
  private Object objectToRender(Map<String, Object> model) {
    Object o = model.get(DEFAULT_MODEL_KEY);
    if (o == null) {
      throw new IllegalArgumentException("Model does not contain the expected key '" + DEFAULT_MODEL_KEY + "'");
    }
    return o;
  }
}
 

Next I created an abstract SimpleMarshaller implementation for marshalling using a Groovy MarkupBuilder:

 
public abstract class AbstractMarkupBuilderSimpleMarshaller implements SimpleMarshaller {
  public final void marshal(Writer writer, Object o) {
    MarkupBuilder builder = new MarkupBuilder(writer);
    marshalToBuilder(builder, o);
  }
  protected abstract void marshalToBuilder(MarkupBuilder builder, Object o);
}
 

Now all that is left is to create a new Groovy View implementation for specific XML response you want, e.g.:

 
class UserXmlMarshallingView extends SimpleMarshallingView {
  @Autowired
  public UserXmlMarshallingView(UriConstructionService uriConstructionService){
    super(new UserXmlMarshaller(uriConstructionService))
    setContentType("application/xml")
  }
 
  static class UserXmlMarshaller extends AbstractMarkupBuilderSimpleMarshaller {
    private UriConstructionService uriConstructionService;
 
    public UserXmlMarshaller(UriConstructionService uriConstructionService) {
      this.uriConstructionService = uriConstructionService;
    }
 
    @Override
    protected void marshalToBuilder(MarkupBuilder builder, Object user) {
      builder.'user'('name': user.name, 'email': user.email) {
        user.orders.each {
          'order-ref'('dateCreated': it.dateCreated.format('yyyy-MM-dd hh:mm:ss'), 'uri': uriConstructionService.orderUri(it))
        }
      }
    }
  }
}
 

Note above that we included some processing logic as well - we get a formatted date and we also make use of a service for producing URIs for referencing sub-entities.

Each individual View implementations now needs to be registered in the DispatcherServlet's ApplicationContext:

 
<bean id="user" class="com.jayway.demo.xml.UserXmlMarshallingView"/>
...
<bean class="org.springframework.web.servlet.view.BeanNameViewResolver"/>
 

Since we have defined a BeanNameViewResolver we can now refer to these Views using their bean names when pointing them out from a Controller:

 
@Controller
public class UserController {
  @Autowired
  private Repo repo;
 
  @RequestMapping(value = "/users/{id}")
  public ModelAndView user(@PathVariable int id) {
    User user = repo.userById(id);
    return new ModelAndView("user", SimpleMarshallingView.DEFAULT_MODEL_KEY, user);
  }
}
 

Infrastructure

Obviously, for the above to work we need organize our code appropriately and make sure that our build system properly builds the groovy files. This turned out to be surprisingly easy (using maven). All groovy code should be placed under src/main/groovy. Then maven needs to be configured to build the groovy files using the gmaven plugin. Also, make sure you include the groovy libs in your dependency list:

 
<dependency>
  <groupId>org.codehaus.groovy</groupId>
  <artifactId>groovy-all</artifactId>
  <version>1.7.1</version>
</dependency>
 

Conclusion

With the above setup it's easy to quickly implement (and test) new marshalling views for the different resources in a REST application. Even though this was used in a rather small proof-of-concept project my general feeling is that the approach will work pretty well. My next step with this will be to add some simplifications for working with HTTP status codes (the approach to HTTP response codes with the @ResponseCode annotation in Spring MVC feels insufficient for many cases).

Finally it should be noted that the approach with a completely generic marshaller makes it easy to expand this to working with other representations as well, e.g. JSON (although the JSON support in Groovy is still a little bit shaky).

 
for (Iterator companies =
      database.getCompanies().iterator();
      companies.hasNext();) { 
 
   Company company = (Company)companies.next(); 
 
   if (company.getLocation().equals("CA")) { 
 
      for (Iterator departments =
            company.getDepartments().iterator();
            departments.hasNext(); ) { 
 
         Department department = (Department)departments.next(); 
 
         if (department.getEmployees().size() > 10) {
            System.out.println(department);
         }
      }
   }
}
 

Using JXPath you can simplify this:

 
// only needed once
JXPathContext context = JXPathContext.newContext(database); 
 
Iterator departments = context.iterate(
      "/companies[location='CA']" +
      "/departments[count(employees) > 10]");
while (departments.hasNext()) {
   System.out.println(departments.next());
}
 

Two points should be noted from this example:

• The code using JXPath is considerably shorter. The query is concisely expressed in one statement.
• You can take that expression out of the code and parameterize it, or put it in a configuration file.

Perfect for Unit Testing

Unit testing often requires checking object hierarchies for particular values. You can use JXPath to express the components to be tested. For example, you can write a helper method in a JUnit test:

 
/**
 * tests using a JXPath to express the required
 * component to test against. Note that nulls
 * aren't catered for!
 * @param path the path to find the object to
 *        test against
 * @param testable the base object to test
 * @param required the required result
 */
private void assertFromPath(String path,
                            Object testable,
                            Object required) { 
 
   JXPathContext context = JXPathContext.newContext(testable);
   assertTrue(required.equals(context.getValue(path)));
}
 

and then you can use this in tests:

 
public void testBankAccount() { 
 
   // you get a bank account object from some operation...
   Account account = .... 
 
   // then test against it
   assertFromPath("/accountHolder/name", account, "John Smith");
   assertFromPath("/accountHolder/opened", account, DATE_OPENED);
   assertFromPath("/accountHolder" + "/transactions[1]/amount",
         account, INITIAL_PAYMENT);
}
 

Read more in this java.net article.