scala - How to run subprojects tests (including setup methods) sequentially when testing

Question

EDIT4 Most of the text below the horizontal rule doesn't have anything to do with the real problem. At the beginning I thought forking is the problem, but that is not true.

I'm trying to run all the tests of an aggregates subprojects. Before all the tests in a subproject a setup method should run, and after the tests in a subproject a cleanup method should run.

When running the test task for the aggregate project, I expect the following sequence

• setup method of project A is called
• tests for project A are executed
• cleanup method for project A is called
• the same for project B

But the sequence is:

• setup methods of project A and B are called
• tests of project A are executed
• tests of project B are executed
• cleanup methods of project A and B are called

A build script having this behaviour can be found here.

How can I fix this problem to get my expected sequence?

---

I'm using forking to run the tests in my subprojects. For every subproject a mongo db is started before the tests and stopped after the tests.

The tests within one project run sequentially; this works well if I run the tests for a single project.

But if I run the task test for project root (the aggregate containing the sub projects), I want the forked jvms to be started sequentially, i.e.

1. the jvm for project A is forked and its tests are executed
2. the jvm for project B is forked and its tests are executed
3. ...

But it looks like the jvms are started in parallel; this is not what I want.

I tried the following (which should be set to 1 already, according to the documentation):

  concurrentRestrictions in Test := Seq(
    Tags.limit(Tags.ForkedTestGroup, 1)
  )

But it didn't work. Directly after starting the test task, the following is printed (before any test log is printed) from my setup method:

startupDb, thread name = pool-4-thread-5 
startupDb, thread name = pool-4-thread-7 
startupDb, thread name = pool-4-thread-2 
startupDb, thread name = pool-4-thread-6 
startupDb, thread name = pool-4-thread-8
startupDb, thread name = pool-4-thread-3 
startupDb, thread name = pool-4-thread-9

These are my test related settings:

parallelExecution in Test := false,
testOptions in Test += Tests.Setup( () => MongoTest.startupDb() ),
testOptions in Test += Tests.Cleanup( () => MongoTest.shutdownDb() ),
fork in Test := true,
concurrentRestrictions in Test := Seq(
  Tags.limit(Tags.ForkedTestGroup, 1)
)

It's important for me to use forking, but in the way as described above.

I'm using sbt 0.13.0 on Windows 7.

EDIT I created a gist with a sample build.

EDIT2 In the documentation it says:

Control the number of forked JVMs allowed to run at the same time by setting the limit on Tags.ForkedTestGroup tag, which is 1 by default

So this should work theoretically. Is this a bug? If not, how can I manage to do that?

EDIT3 It looks like forking is not the problem. The problem is that the setup methods are called immediately for all subprojects.

Answer 1

Hmm. Trivial answer convert to comment. It sounds like you are asking the same question that got solved in the other question (see comments). The answer seems to be

parallelExecution in ThisBuild := false

and I found another blog post that suggested

parallelExecution in Global := false

to be the answer. The "Global" answer also suggested that would turn off parallel execution for things like compilation but that's probably less important than running tests.