OpenJFX unit tests

Understanding OpenJFX Unit tests.

OpenJFX contains many unit tests that can be run using 'gradle test' as part of a build. These are JUnit based tests.

Locations

There are two primary locations to find the unit tests in the repository:

• modules/_module_name_/src/tests
• tests/system/src/test/

In these locations, we have

• java/test/* - for all java source files for Unit tests
• java/* - if not in the test package, must be a shim class
• resources - for all related resources

In a non-modular build, there is nothing really separating the unit tests and the shims but convention.

The Shims

A shim is a java class that provides a backdoor into the core classes for testing purposes.They are special in a number of ways:

• they use the same classloader as the core classes
• they provide test entry points that circumvent restrictions on API visibility for unit testing. 
• they usually must live in the same package as the API they are exposing.
• they cannot use classes on the Application class loader - like the JUnit API (no import org.junit.* for example)
• with modules - they must live in a package referenced in the module-info for the class.

It is best to keep these classes as simple as possible, retaining as much of the test logic in the Unit test classes as possible.

In the OpenJFX repository, the shims are any test class that does not reside in the test package.

With modules, the shim classes must live in a package already referenced in the module-info for the class, even if the package is not public. The addExports option can be used to loosen module restrictions, but not expose a package that is not already listed.

The Unit Tests

The unit tests run with the application class loader, and not the extension class loader as the core classes are. With modules, they usually are run in the "unnamed" module by default (OpenJFX does not create a "test" module containing the tests).

All unit tests must be:

• JUnit tests
• part of the test package hierarchy (i.e. package test.*).
• Certain tests are optionally selected by gradle using run time options.

Optional Tests

• -PFULL_TEST=true - long running tests, robot tests, or ones that flash windows making the host difficult to use
• -PUSE_ROBOT=true - enable Robot based tests
• -PHEADLESS_TEST=true - enable headless tests on some platforms
• -PAWT_TEST=true - enable AWT related tests
• -PUNSTABLE_TEST=true - enable tests tagged as unstable

All unit tests must:

• use the documented public API
• one of the shim classes
• in a few cases may be able to directly use internal API (like com.sun.*) classes with the right module addExport

(mention @ignore ? unstable test pattern ?)

When creating a new unit test, think about some of the special cases listed above and talk to the team about any special needs.

Fun in a modular world

With the addition of modules in JDK9, there are some additional details that must be understood when dealing with the unit tests.

Module-info and the packages

It helps to review the module-info.java for the module you are working on to understand the visibility of the module.

The unit tests run in the "unnamed" module, and so by default can only see the public packages like this one:

      exports javafx.beans;

many packages are not exported as public like this one:

    exports com.sun.javafx to

        javafx.controls,

        javafx.graphics;

To access a class in this package, we will need to use an addExport option to override the default package protections for the test. Also note that package specifications do not include any sub packages - each must be explicitly mentioned.

Each of the build modules in OpenJFX has an addExports file that is imported using @argfile into the test invocation. The addExports file contains entries that export packages that are not public so that the unit tests in the unnamed module can see them. Here is an excerpt from one of the files:

"-XaddExports:\

    javafx.base/com.sun.javafx.collections=ALL-UNNAMED,\

    javafx.graphics/com.sun.javafx.application=ALL-UNNAMED,\

    javafx.graphics/com.sun.javafx.font=ALL-UNNAMED,\

Building an Xpatch

As part of the modular test build, we need to create a directory containing the freshly built core classes with the shim classes added in. This combined mix of classes can be used with the Xpatch command to override the modules in the JDK.

In the OpenJFX build, this results in -Xpatch:build/testing/modules

NOTE: Xpatch does not override module-info.

Finding our shared libraries

By default, java will look for the module shared libraries in the JDK, which may mean problems if was are trying to use the libraries we just built. To access those libraries, we need to use -Djava.library.path=build/sdk/lib/_the_right_arch_  to find them

And example manual command line:

The following is an example command line that can run a junit test from within a built Linux OpenJFX modular tree: