junit - Running all tests from a @Category using Maven

Question

I want to run only a subset of my unit tests, the ones defined by a specific @Category.

So I read several SO questions, such as this one (which is exactly what I am looking for), and also this one.

The solution of my problem seems to be provided by the ClasspathSuite project. So I started to write the NewTest and OldTest interfaces that will define my test categories. Then, I created the AllTests suite:

@RunWith(ClasspathSuite.class)
public class AllTests { }

After that, I created a AllNewTests suite:

@RunWith(Categories.class)
@IncludeCategory(NewTest.class)
@SuiteClasses( { AllTests.class })
public class AllNewTests { }

Finally, I create two JUnit classes, one per category:

@Category(NewTest.class)
public class SomeNewTests {
    // some tests...
}

@Category(OldTest.class)
public class SomeOldTests {
    // some tests...
}

---

Now, if I run AllTests, Eclipse launches all the tests of my project, while Maven fails as no test are found:

mvn test -Dtest=AllTests

...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running my.company.AllTests
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.093 sec
There are no tests to run.

If I run AllNewTests (which is the correct thing to do, right?), in Eclipse everything is fine (i.e. it only run the tests annoted with @Category(NewTest.class)) but Maven returns an error:

mvn test -Dtest=AllNewTests

...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running my.company.AllNewTests
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.125 sec <<< FAILURE!

Results :

Tests in error:
  initializationError(my.company.AllNewTests)

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

The exception thrown is the following:

org.junit.runner.manipulation.NoTestsRemainException
    at org.junit.runners.ParentRunner.filter(ParentRunner.java:256)
    at org.junit.experimental.categories.Categories.<init>(Categories.java:142)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:35)
    at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:24)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
    at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:29)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:57)
    at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:24)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:33)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:146)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:97)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.maven.surefire.booter.ProviderFactory$ClassLoaderProxy.invoke(ProviderFactory.java:103)
    at $Proxy0.invoke(Unknown Source)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:145)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:70)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:69)

---

My question is what I did wrong?

---

Technical details: Java 6, Maven 3.0.2, JUnit 4.8.1, Surefire plugin 2.7.1, cpsuite-1.2.5

Answer 1

As an update: as of Surefire plugin v2.11, JUnit 4.8+ style categories are now supported.

The release notes for Surefire v2.11 mention the new feature. The surefire:test goal can is configured using groups.