Parameterized tests allow developers to efficiently test their code with a range of input values. In the field of JUnit testing, experienced users have long struggled with the complexity of implementing these tests. But with the release of JUnit 5.7, a new era of test parameterization is entering, offering developers first-class support and improved capabilities. Let’s dive into the exciting possibilities that JUnit 5.7 brings to the table for parameterized testing!
Parameterization samples from the JUnit 5.7 documents
Let’s look at some examples from the documents:
@ParameterizedTest
@ValueSource(strings = "racecar", "radar", "able was I ere I saw elba" )
void palindromes(String candidate)
assertTrue(StringUtils.isPalindrome(candidate));
@ParameterizedTest
@CsvSource(
"apple, 1",
"banana, 2",
"'lemon, lime', 0xF1",
"strawberry, 700_000"
)
void testWithCsvSource(String fruit, int rank)
assertNotNull(fruit);
assertNotEquals(0, rank);
@ParameterizedTest
@MethodSource("stringIntAndListProvider")
void testWithMultiArgMethodSource(String str, int num, List<String> list)
assertEquals(5, str.length());
assertTrue(num >=1 && num <=2);
assertEquals(2, list.size());
static Stream<Arguments> stringIntAndListProvider()
return Stream.of(
arguments("apple", 1, Arrays.asList("a", "b")),
arguments("lemon", 2, Arrays.asList("x", "y"))
);
The @ParameterizedTest the annotation must be accompanied by one of several source notes given that describe where to take the parameters from. The parameter source is often referred to as the “data provider”.
I won’t go into their detailed description here: the JUnit user guide does it better than I could, but let me share a few observations:
- The
@ValueSourceit is limited to providing only one parameter value. In other words, a test method cannot have more than one argument, and the types that can be used are also limited. - Passing multiple arguments is somewhat fixed
@CsvSource, parsing each string into a record which is then passed as an argument field by field. This can easily become difficult to read with long strings and/or lots of arguments. The types that can be used are also limited – more on that later. - All sources that declare actual values in comments are limited to values that are compile-time constants (a limitation of Java comments, not JUnit).
@MethodSourceand@ArgumentsSourceprovides a stream/collection of (untyped) n-tuples which are then passed as method arguments. Various actual types are supported to represent an array of n-tuples, but none of them are guaranteed to match the method’s argument list. This type of source requires additional methods or classes, but does not provide restrictions on where and how to get the test data.
As you can see, the available native types range from simple (easy to use but limited functionality) to extremely flexible that require more code to work.
- Side note — This is generally a sign of good design: little code is needed for essential functionality, and adding extra complexity is justified when used to enable a more demanding use case.
What does not seem to fit this hypothetical continuum from simple to flexible does @EnumSource. Look at this non-trivial example of four sets of parameters with 2 values each.
- Note – While
@EnumSourcepasses an enum value as one parameter of the test method, conceptually, the test is parameterized by the enum’s fields, which is not a limit to the number of parameters.
enum Direction
UP(0, '^'),
RIGHT(90, '>'),
DOWN(180, 'v'),
LEFT(270, '<');
private final int degrees;
private final char ch;
Direction(int degrees, char ch)
this.degrees = degrees;
this.ch = ch;
@ParameterizedTest
@EnumSource
void direction(Direction dir)
assertEquals(0, dir.degrees % 90);
assertFalse(Character.isWhitespace(dir.ch));
int orientation = player.getOrientation();
player.turn(dir);
assertEquals((orientation + dir.degrees) % 360, player.getOrientation());
Just imagine: a hard-coded list of values severely limits its flexibility (without external or generated data), while the amount of extra code required to declare enum makes this a fairly comprehensive alternative to, say, @CsvSource.
But that’s just the first impression. We’ll see how elegant this can be when the true power of Java enums is harnessed.
- Sidenote: This article is not about checking enums that are part of your production code. They, of course, had to be declared regardless of how you choose to check them. Instead, it focuses on when and how express your test data in the form of an enum.
When to use it
There are situations where enums perform better than the alternatives:
Multiple parameters per test
When all you need is a single parameter, you probably don’t want to overcomplicate things @ValueSource. But as soon as you need multiples — say, inputs and expected results — you have to resort @CsvSource, @MethodSource/@ArgumentsSource or @EnumSource.
In some way, enum allows you to “smuggle” any number of data fields.
So when you need to add more test method parameters in the future, simply add more fields to your existing enums, leaving the test method signatures intact. This becomes invaluable when you reuse your data provider in multiple tests.
Other sources should be employed ArgumentsAccessorwith or ArgumentsAggregators for the flexibility that enums have out of the box.
Type Security
For Java developers this should be huge.
Parameters read from CSV (files or literals), @MethodSource or @ArgumentsSourcethey do not guarantee during compilation that the number of parameters and their types will match the signature.
Obviously, JUnit will complain at runtime, but forget about code help from your IDE.
As before, this adds up when you reuse the same parameters for multiple tests. Using the type-safe approach would be a big win when expanding the parameter set in the future.
Custom types
This is mainly an advantage over text sources, such as those that read data from CSV — text-encoded values must be converted to Java types.
If you have a custom class to instantiate from a CSV record, you can do this using ArgumentsAggregator. However, your data declaration is still not type-safe — any mismatch between the method signature and the declared data will appear at runtime when “aggregating” the arguments. Not to mention that declaring an aggregator class adds more support code required for your parameterization to work. And we always favored @CsvSource over @EnumSource to avoid additional code.
Documented
Unlike other methods, the enum source has Java symbols for parameter sets (enum instances) and any parameters they contain (enum fields). They provide a simple place where you can attach documentation in its more natural form — JavaDoc.
It’s not that documentation can’t be placed elsewhere, but it will – by definition – be located away from what it documents and therefore harder to find and more easily out of date.
But there is more!
Now: enums. Are they. Teaching.
It seems that many younger developers have yet to realize how powerful Java enums truly are.
In other programming languages, they are really just glorified constants. But in Java, they are convenient little implementations of the Flyweight design pattern with (largely) the benefits of full classes.
Why is that a good thing?
Test behavior related to fixture
As with any other class, methods can be added to enums.
This becomes handy if enum test parameters are reused between tests — the same data, just tested slightly differently. To work with parameters efficiently without significant copy and paste, some helper code should also be shared between these tests.
It’s not something a helper class and a few static methods won’t “solve”.
- Sidenote: Notice that such a design suffers from Feature Envy. Test methods – or worse, helper class methods – would have to extract data from enum objects in order to perform operations on that data.
Although this is the (only) way in procedural programming, in the object-oriented world we can do better.
By declaring “helper” methods in the enum declaration itself, we would move the code to where the data is. Or, to put it in OOP jargon, helper methods would become the “behavior” of test devices implemented as enums. This would not only make the code more idiomatic (calling sensible methods on instances instead of static methods that pass data), but also make it easier to reuse enum parameters in test cases.
Heritage
Enums can implement interfaces with (default) methods. When used judiciously, this can be used to share behavior between several data providers — several enums.
An example that easily comes to mind is a separate list for positive and negative tests. If they represent a similar type of test, chances are they have common behavior.
Talk is cheap
Let’s illustrate this with a test suite of a hypothetical source code file converter, not unlike one that performs a Python 2 to 3 conversion.
To have real confidence in what such a comprehensive tool is doing, we would end up with an extensive set of input files that manifest various aspects of the language and corresponding files to compare the conversion results to. Additionally, it is necessary to check what warnings/errors are served to the user for problematic entries.
This is a natural fit for parameterized tests due to the large number of patterns to check, but it doesn’t really fit any of the simple JUnit parameter sources, since the data is somewhat complex.
See below:
enum Conversion
CLEAN("imports-correct.2.py", "imports-correct.3.py", Set.of()),
WARNINGS("problematic.2.py", "problematic.3.py", Set.of(
"Using module 'xyz' that is deprecated"
)),
SYNTAX_ERROR("syntax-error.py", new RuntimeException("Syntax error on line 17"));
// Many, many others ...
@Nonnull
final String inFile;
@CheckForNull
final String expectedOutput;
@CheckForNull
final Exception expectedException;
@Nonnull
final Set<String> expectedWarnings;
Conversion(@Nonnull String inFile, @Nonnull String expectedOutput, @NotNull Set<String> expectedWarnings)
this(inFile, expectedOutput, null, expectedWarnings);
Conversion(@Nonnull String inFile, @Nonnull Exception expectedException)
this(inFile, null, expectedException, Set.of());
Conversion(@Nonnull String inFile, String expectedOutput, Exception expectedException, @Nonnull Set<String> expectedWarnings)
this.inFile = inFile;
this.expectedOutput = expectedOutput;
this.expectedException = expectedException;
this.expectedWarnings = expectedWarnings;
public File getV2File() ...
public File getV3File() ...
@ParameterizedTest
@EnumSource
void upgrade(Conversion con)
try
File actual = convert(con.getV2File());
if (con.expectedException != null)
fail("No exception thrown when one was expected", con.expectedException);
assertEquals(con.expectedWarnings, getLoggedWarnings());
new FileAssert(actual).isEqualTo(con.getV3File());
catch (Exception ex)
assertTypeAndMessageEquals(con.expectedException, ex);
Using enums does not limit us in how complex the data can be. As you can see, we can define several convenient constructors in enums, so declaring new parameter sets is nice and clean. This prevents the use of long argument lists that often end up filled with many “empty” values (nulls, empty strings, or collections) that make one wonder what argument #7 — you know, one of the nulls — actually represents.
Notice how enums allow the use of complex types (Set, RuntimeException) without limitations or magical transformations. Sending such data is also completely safe for the guy.
Now, I know what you’re thinking. This is passionately wordy. Well, somewhat. Realistically, you will have many more sample data to check, so the amount of boilerplate will be less significant in comparison.
Also see how related tests can be written using the same enums and their helper methods:
@ParameterizedTest
@EnumSource
// Upgrading files already upgraded always passes, makes no changes, issues no warnings.
void upgradeFromV3toV3AlwaysPasses(Conversion con) throws Exception
File actual = convert(con.getV3File());
assertEquals(Set.of(), getLoggedWarnings());
new FileAssert(actual).isEqualTo(con.getV3File());
@ParameterizedTest
@EnumSource
// Downgrading files created by upgrade procedure is expected to always pass without warnings.
void downgrade(Conversion con) throws Exception
File actual = convert(con.getV3File());
assertEquals(Set.of(), getLoggedWarnings());
new FileAssert(actual).isEqualTo(con.getV2File());
After all, a little more talk
Conceptually, @EnumSourceencourages you to create a complex, machine-readable description of individual test scenarios, blurring the line between data providers and test instruments.
Another great thing about having each data set expressed as a Java symbol (an enum element) is that they can be used individually; completely outside the data provider/parameterized tests. Because they have a reasonable name and are self-contained (in terms of data and behavior), they contribute to nice and readable tests.
@Test
void warnWhenNoEventsReported() throws Exception
FixtureXmls.Invalid events = FixtureXmls.Invalid.NO_EVENTS_REPORTED;
// read() is a helper method that is shared by all FixtureXmls
try (InputStream is = events.read())
EventList el = consume(is);
assertEquals(Set.of(...), el.getWarnings());
Now, @EnumSource is not it will be one of your most used sources of arguments, and that’s a good thing because overusing it wouldn’t help. But in the right circumstances, it’s good to know how to take advantage of all they have to offer.