javascript - node.js how to get better error messages for async tests using mocha

Question

A typical test in my node.js mocha suite looks like the following:

 it("; client should do something", function(done) {
    var doneFn = function(args) {
      // run a bunch of asserts on args
      client.events.removeListener(client.events.someEvent, userMuteFn);
      done();
    }

    client.events.on(someEvent, doneFn);

    client.triggerEvent();
});

The problem here is that if client.triggerEvent() doesn't do something right, or if the server breaks and never invokes someEvent, then done() will never get invoked. This leaves an ambiguous error for someone who hasn't worked with the testsuite before like:

Error: timeout of 10500ms exceeded. Ensure the done() callback is being called in this test.

My question is, is there a way to rewrite these tests, whether it's with mocha or in addition to another lib, that makes async work like this easier to follow. I'd love to be able to output something such as:

the callback doneFn() was never invoked after clients.event.on(...) was invoked

or perhaps something similar.

I'm not sure if using something such as promises would help this. More meaningful error messages for async/callback type code would be a great thing. If it means moving from callback/async to another workflow, I'm OK with that as well.

What are some solutions?

Answer 1

When you get a timeout error rather than a more precise error, the first thing to do is to check that your code does not swallow exceptions, and that it does not swallow promise rejections. Mocha is designed to detect these in your tests. Unless you do something unusual like running test code in your own VM or manipulate domains, Mocha will detect such failures, but if your code swallows them, there's nothing Mocha can do.

This being said, Mocha won't be able to tell you that done was not called because your implementation has a logic bug that causes it to fail to call the callback.

Here is what could be done with sinon to perform a post mortem after test failures. Let me stress that this is a proof of concept. If I wanted to use this on an ongoing basis, I'd develop a proper library.

var sinon = require("sinon");
var assert = require("assert");

// MyEmitter is just some code to test, created for the purpose of
// illustration.
var MyEmitter = require("./MyEmitter");
var emitter = new MyEmitter();

var postMortem;
beforeEach(function () {
  postMortem = {
    calledOnce: []
  };
});

afterEach(function () {
  // We perform the post mortem only if the test failed to run properly.
  if (this.currentTest.state === "failed") {
    postMortem.calledOnce.forEach(function (fn) {
      if (!fn.calledOnce) {
        // I'm not raising an exception here because it would cause further
        // tests to be skipped by Mocha. Raising an exception in a hook is
        // interpreted as "the test suite is broken" rather than "a test
        // failed".
        console.log("was not called once");
      }
    });
  }
});

it("client should do something", function(done) {
  var doneFn = function(args) {
    // If you change this to false Mocha will give you a useful error.  This is
    // *not* due to the use of sinon. It is wholly due to the fact that
    // `MyEmitter` does not swallow exceptions.
    assert(true);
    done();
  };

  // We create and register our spy so that we can conduct a post mortem if the
  // test fails.

  var spy = sinon.spy(doneFn);
  postMortem.calledOnce.push(spy);
  emitter.on("foo", spy);

  emitter.triggerEvent("foo");
});

Here is the code of MyEmitter.js:

var EventEmitter = require("events");

function MyEmitter() {
    EventEmitter.call(this);
}

MyEmitter.prototype = Object.create(EventEmitter.prototype);
MyEmitter.prototype.constructor = MyEmitter;

MyEmitter.prototype.triggerEvent = function (event) {
    setTimeout(this.emit.bind(this, event), 1000);
};

module.exports = MyEmitter;