JUnit 5 – Conditions

We recently learned about JUnit’s new extension model and how it allows us to inject customized behavior into the test engine. I left you with the promise to look at conditions, which can be used to seöectively deactivate tests. Let’s do that now!

Conditions allow us to define flexible criteria when tests should or shouldn’t be executed. Their official name is Conditional Test Execution.

Overview

Other posts in this series about JUnit 5:

(If you'd rather hear me talk about JUnit 5, check out the recent vJUG session or my presentation at JavaZone.)

This series is based on the pre-release version Milestone 3 and will get updated when a new milestone or the general availability release gets published.

Most of what you will read here and more can be found in the emerging JUnit 5 user guide (that link went to the Milestone 3 version - you can find the most current version here). The code samples I show here can be found on GitHub.

Extension Points For Conditions

Remember what we said about extension points? No? In short: There’s a bunch of them and each relates to a specific interface. Implementations of these interfaces can be handed to JUnit (with the @ExtendWith annotation) and it will call them at the appropriate time.

Two conditional extension points exist, ContainerExecutionCondition and TestExecutionCondition, and they can be used to deactivate either all tests in a container (likely a class) or individual tests (likely a test method).

And that’s already pretty much it. Any condition should implement one or both of these interfaces and do the required checks in its evaluate implementation(s).

@Disabled

The easiest condition is one that is not even evaluated: We simply always disable the test if our hand-crafted annotation is present.

So let’s create @Disabled:

And the matching extension:

Easy as pie, right? And correct, too, because it is almost the same as the real @Disabled implementation. There are only two small differences:

  • The official annotation does not need to carry its own extension with it because it is registered by default.
  • It can be given a reason, which is logged when the disabled test is skipped.

Small caveat (of course there’s one, what did you think?): AnnotationUtils is internal API but it is likely that its functionality will be officially available soon.

Now let’s try something less trivial.

@DisabledOnOs

Maybe we only want to run some tests if we are on the right operating system.

Simple Solution

Again, we start with the annotation:

This time it takes a value, or rather a bunch if values, namely the operating systems on which the test should not run. OS is just an enum with a value for each operating system. And it has a handy static OS determine() method, which, you guessed it, determines the operating system the code is running on.

With that, let’s turn to OsCondition. It has to check whether the annotation is present but also whether the current OS is one of those given to the annotation.

We can use it as follows:

Nice.

Less Ceremony

But we can do better! Thanks to JUnit’s customizable annotations we can make this condition even smoother:

To implement @TestExceptOnOs, it would be great to just do this:

When executing a test and scanning for @DisabledOnOs in OsCondition::evaluateIfAnnotated, we would find it meta-annotated on @TestExceptOnOs and our logic would Just Work™. But I couldn’t find a way to make the OS values given to @TestExceptOnOs accessible to @DisabledOnOs. :( (Can you?)

The next best option is to simply use the same extension for the new annotation:

Then we pimp OsCondition::evaluateIfAnnotated to include the new case…

… and we’re done, check out the full source. Now we can indeed use it as we hoped we could.

Polishing

Creating the inverted annotations (disabling if not on one of the specified operating systems) is just more of the same but with them, improved names, and static imports we could end up here:

Not bad, eh?

@DisabledIfTestFails

Let’s try one more thing – and this time we’ll make it really interesting! Assume there are a bunch of (integration?) tests and if one of them fails with a specific exception, other tests are bound to fail as well. So to save time, we’d like to disable them. To that end, we want to create a class level annotation @DisabledIfTestFailedWith that takes one or more exception types and disables all test methods once one of them was thrown.

So what do we need here? Right off the bat it’s clear that we have to somehow collect the exceptions thrown during test execution. This has to be bound to the lifetime of the test class so we don’t disable tests because some exception flew in a totally different test class. And then we need a condition implementation that checks whether a specific exception was thrown and disables the test if so.

Collect Exceptions

Looking over the list of extension points we find “Exception Handling”. The corresponding interface looks promising:

So we’ll implement handleException to store and then rethrow the exception.

You may remember what I wrote about extensions and state:

The engine makes no guarantees when it instantiates extensions and how long it keeps instances around. Hence, extensions have to be stateless. Any state they need to maintain has to be written to and loaded from the store that the extension context makes available.

Ok, so we use the store – effectively a keyed collection of things we want to remember. We can access it via the extension context but which one do we use? There is one context per test method ( TestExtensionContext) and another one for the whole test class ( ContainerExtensionContext). Remember that we want to store all exceptions thrown during the execution of all tests in a class but not more, i.e. not the ones thrown by other test classes. Turns out that the ContainerExtensionContext and its store are exactly what we need.

So here we go getting the container context and using it to store a set of thrown exceptions:

Now adding an exception is simple:

This is actually an interesting extension of its own, maybe it could be used for analytics as well, so we’ll put into its very own CollectExceptionExtension. Anyways, we will want to have a look at the thrown exceptions so we need a public method for that:

With this, given an extension context, any other extension can check which exceptions have been thrown so far.

Disable

The rest is much like before so let’s be quick about it:

Note that we only allow this annotation on classes, which makes sense because it needs to apply to every test method in the class to make sense. We apply the CollectExceptionExtension we just built and only need to create DisabledIfTestFailedCondition now.

The condition only disables individual test methods, so we only implement TestExecutionCondition. Because we want this extension to apply as soon as it is present on the test class, there is no reason to check whether individual test methods are annotated. Instead we simply access the user provided exception classes and disable if such an exception was already thrown:

Putting It Together

And this is how we use those annotations to disable tests if an exception of a specific type was thrown before:

Summary

Wow, that was a lot of code! But by now we really know how to implement conditions in JUnit 5:

  • create the desired annotation and @ExtendWith your condition implementation
  • implement ContainerExecutionCondition, TestExecutionCondition, or both
  • check whether the new annotation is even present
  • perform the actual checks and return the result

We have also seen that this can be combined with other extension points, how the store can be used to persist information, and that custom annotations can make using an extension much more elegant.

For more fun with flags extension points, check the next post in this series when we’ll be discussing parameter injection.

Share & Follow

You liked this post? Then share it with your friends and followers!
twittergoogle_plusredditlinkedin
And if you like what I'm writing about, why don't you follow me?
twittergoogle_plusrss

Other Posts