Integration testing with maven 2.0

One thing to keep in mind is that unit tests are not integration tests.

The characteristics of unit tests are :
- They test your code in isolation
- They must be fast because they have to be run a lot of times

The problem with unit testing is that even if they cover a lot of your code, you can still have errors on integration when you put all the pieces together. That's why you must create integration tests.

The characteristics of integration tests are :
- They tests all your pieces of code together.
- They are pretty slow because they can be fired in a context like a spring context. They rely on real database or web services.

Integration tests are slow so they must not be run in the same phase as unit tests. Unit testing must be run in the maven test phase while integration tests must be run in the maven integration-test phase. The problem is that maven doesn't help you with that.

The trick is to customize the maven surefire plugin. You have to write your integration tests in a specific package and exclude them from the execution of the maven test phase. Then, you have to bind the execution of your integration tests to the maven integration-test phase.

Here's how you do that :

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <excludes>
      <exclude>**/integration/*Test.java</exclude>
    </excludes>
  </configuration>
  <executions>
    <execution>
      <id>integration-tests</id>
      <phase>integration-test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <skip>false</skip>
        <excludes>
          <exclude>none</exclude>
        </excludes>
        <includes>
          <include>**/integration/*Test.java</include>
        </includes>
      </configuration>
    </execution>
  </executions>
</plugin>

A common thing to do is to start a web server like Jetty in the pre-integration-test phase using the maven-jetty plugin.

<plugin>
  <groupId>org.mortbay.jetty</groupId>
  <artifactId>maven-jetty-plugin</artifactId>
  <version>6.1.10</version>
  <configuration>
    <contextPath>/yourContextPath</contextPath>
  </configuration>
  <executions>
    <execution>
      <id>start-jetty</id>
      <phase>pre-integration-test</phase>
      <goals>
        <goal>run-exploded</goal>
      </goals>
      <configuration>
        <scanIntervalSeconds>0</scanIntervalSeconds>
        <daemon>true</daemon>
      </configuration>
    </execution>
  </executions>
</plugin>

You can even populate your database in the pre-integration-test phase with the dbunit-maven plugin if you need to add specific test case in your database.

Maybe maven 3.0 will improve the integration of integration testing in the maven process. A good solution would be to have something similar to the test phase like a src/it directory where you will put all your integration tests.

Updated on July 23, 2009 :

Another solution is to use the Maven Failsafe Plugin. The Failsafe Plugin is a fork of the Surefire plugin designed to run integration tests.

By default, the Surefire plugin executes **/Test*.java, **/*Test.java, and **/*TestCase.java test classes. The Failsafe plugin will look for **/IT*.java, **/*IT.java, and **/*ITCase.java

Take a look at this article on Sonatype Blog.