validation-file-assertions

File-based assertions for Java.

Idea shown by example

Let’s consider the following test:

import de.cronn.assertions.validationfile.junit5.JUnit5ValidationFileAssertions;

class MyTest implements JUnit5ValidationFileAssertions {
    @Test
    void myTestMethod() {
        assertWithFile("actual value");
    }
}

When we run this test for the first time, it will create the following files:

• data/test/output/MyTest_myTestMethod.txt (this one always contains the actual value from the last run; this file should be ignored in your SCM) with

  actual value
  

• data/test/validation/MyTest_myTestMethod.txt (this one is prefilled automatically once during the first test run; on subsequent runs it will not be modified automatically) containing

  === new file "data/test/validation/MyTest_myTestMethod.txt" ===
  actual value
  

The test will fail as long as these two files are different. It’s the developer’s job to manually review the content of the validation file and remove the "new file" marker to make the test pass. The reviewed validation file is then committed to the SCM repository together with the test itself. Starting from this point, the validation file is the expected value of the test.

Any change to "actual value" will cause the test to fail and in this case the developer has to compare the content of the output file with the validation file and if satisfied copy output to validation - this makes the test green.

We recommend using a good diffing tool, such as the built-in differ of IntelliJ or Meld, that allows you to diff the two directories data/test/output vs data/test/validation. We also provide an IntelliJ plugin that diffs the two directories with one shortcut.

Getting started

1. Add the dependency

   Gradle (Kotlin DSL)

   testImplementation("de.cronn:validation-file-assertions:{version}")

   Maven

   <dependency>
       <groupId>de.cronn</groupId>
       <artifactId>validation-file-assertions</artifactId>
       <version>{version}</version>
       <scope>test</scope>
   </dependency>

2. Exclude the output files from your SCM, e.g. in .gitignore

   data/test/output/
   data/test/tmp/
   

3. Let your test class implement the JUnit5ValidationFileAssertions interface if you are using JUnit5, otherwise use ValidationFileAssertions

4. Pick a suitable assertWithFile method and enjoy your first validation file assertion.

Gradle integration

Declare the validation files as inputs and the output files as outputs of the test task. Otherwise, Gradle considers the task up-to-date after a validation file was changed. PathSensitivity.RELATIVE keeps the build cache reusable across different checkout locations.

tasks.test {
    inputs.files("data/test/validation").withPathSensitivity(PathSensitivity.RELATIVE)
    outputs.dir("data/test/output")
    outputs.dir("data/test/tmp")
}

tasks.clean {
    delete("data/test/output", "data/test/tmp")
}

We also recommend excluding data/test/output and data/test/tmp in IntelliJ (e.g. via idea.module.excludeDirs of the Gradle idea plugin), so that their content is not indexed and doesn’t show up in search results.

Custom validation files directory

It is possible to customize the path where validation files are stored, in order to do that:

• Implement de.cronn.assertions.validationfile.config.Configuration and override the method getDataDirectory() with path to desired location.

• Register the implemented configuration via Java Service Provider interface (namely: put fully qualified configuration class name in resources/META-INF/services/de.cronn.assertions.validationfile.config.Configuration)

Normalization

Validation files only work with deterministic output (see also our stance on determinism). Where possible, make the output itself deterministic, for example by using ordered collections and a fixed clock, locale and time zone. Values that are random by nature, such as generated UUIDs, can be replaced by stable placeholders using a ValidationNormalizer:

assertWithJsonFile(json, new UuidNormalizer());

{"id": "UUID_1", "parentId": "UUID_1", "otherId": "UUID_2"}

Equal UUIDs get the same placeholder, so references between them remain visible. See the normalization and replacements packages for further normalizers, such as IdNormalizer, SimpleRegexReplacement and JsonReplacerBuilder.

Soft Assertions

File based validation can be combined with AssertJ’s soft assertions.

Example

import org.assertj.core.api.SoftAssertions;
import org.assertj.core.api.junit.jupiter.InjectSoftAssertions;
import org.assertj.core.api.junit.jupiter.SoftAssertionsExtension;

import de.cronn.assertions.validationfile.junit5.JUnit5ValidationFileAssertions;

@ExtendWith(SoftAssertionsExtension.class)
class MyTest implements JUnit5ValidationFileAssertions {

    @InjectSoftAssertions
    private SoftAssertions softly;

    @Override
    public FailedAssertionHandler failedAssertionHandler() {
        return callable -> softly.check(callable::call);
    }

    @Test
    void myTestMethod() {
        assertWithFileWithSuffix("actual value 1", "file1");
        assertWithFileWithSuffix("actual value 2", "file2");
    }
}

Markdown tables

Tabular data is easiest to review as a table. MarkdownTable renders rows as an aligned Markdown table:

MarkdownTable table = new MarkdownTable(List.of("Name", "Price"));
table.addRow("Coffee", 2.5);
table.addRow("Croissant", 1.9);
assertWithFile(table.toString(), FileExtensions.MD);

| Name      | Price |
|-----------|-------|
| Coffee    | 2.5   |
| Croissant | 1.9   |

See also

• IntelliJ plugin for validation file comparison
• File snapshots for Playwright and Vitest