Safer Testing with Akka EventFilter

Akka provides a useful feature for integration testing called the EventFilter. This enables assertions to be made on log messages. I love this feature as it gives me confidence that the correct log messages are being generated and if my system starts struggling in production, I will have the necessary information to quickly understand the problem.

Recently, however, I came across an issue which took away much of the confidence I had in these tests.

I reduced the problem to an example with the following Actor:

This Actor is very simple; after receiving a Crash message it kicks off an asynchronous process using the Scheduler which results in logging the error “Crash”. Our integration test is also fairly simple:

The test contains two test cases:

  • crash twice: This sends two Crash messages to our Actor, but makes no assertions
  • crash once: This sends once Crash messages to our Actor, and asserts that “Crash” appears in the error logs

This test should pass, right? Well let’s look at the output from running it:

[info] FailingTest:
[info] MyActor
[info] - should crash twice
[info] - should crash once *** FAILED ***
[info]   java.lang.AssertionError: Received 2 messages too many on ErrorFilter(class akka.event.Logging$Error$NoCause$,None,Left(Crash),true)

All three log messages are picked in the second test case. This is terrible and could mean my test suite to be full of false positives. I will ask the Akka team for their input on this, however for the meantime it is fortunately very easy to work around. If you are using ScalaTest, simply mix-in the OneInstancePerTest trait and now each test case will get its own ActorSystem and is completely isolated from other test cases.

The above test now passes as expected! OneInstancePerTest will make your tests a bit slower as they will now create a new ActorSystem for each test case but when it comes to testing, correctness is surely better than speed?

The complete example project is up on github

Print Friendly

2 thoughts on “Safer Testing with Akka EventFilter

  1. The logging subsystem is of course a shared resource within one actor system and every actor can log to it as it likes. To make your test deterministic you would have to wait for the log messages of the first case to be processed before going on to the second case (e.g. by using the event filter there as well, with occurrences=2). The other possibility is to filter also on the log source:

    EventFilter.error(message = “Crash”, source = actor.path.toString, occurrences = 1) { … }

    • Thanks Roland,

      The thing I don’t like with the first option is you have to go digging around in other tests cases to find the cause. This annoying when all I want to do focus on the test I am writing and have it isolated from other tests.

      I do like the second option though, I will try it out!

Leave a Reply