Testing

Skara offers two ways for contributors to test their changes before they are integrated:

• The /test pull request command
• Automatic testing via GitHub Actions

This page will describe how do testing with the help of GitHub Actions.

GitHub Actions

GitHub Actions is a way to describe workflows that should be triggered when certain events occur, such as the push of a commit to personal fork. For contributors that have a personal fork of the jdk repository then builds and tests will be executed when a commit is pushed to the contributor's personal fork.

Builds

GNU/Linux

x86-64

• Release
• Debug ("fastdebug")
• Release without precompiled headers

macOS

x86-64

• Release
• Debug ("fastdebug")

Windows

x86-64

• Release
• Debug ("fastdebug")

Tests

tier1

• GNU/Linux x86-64
• macOS x86-64
• Windows x86-64

Triggering builds and tests

The builds and tests will be automatically run when a contributor pushes a commit to their personal fork. Contributors can also manually re-run the actions for a certain commit, see the GitHub documentation for details.

Analyzing results

If a contributor has run testing with the help of GitHub Actions then the Skara bots will automatically display a summary of the build and test results in the pull request body. Anyone can see the result of the builds and tests, but only the contributor can analyze the logs. Please see the GitHub Actions documentation for how to view the logs from a build and/or test run.

Conditionally enable/disable builds and tests

Contributors who wishes to push commits to their personal fork without triggering automatic builds an tests being run can do so by opting out from automatic testing. A contributor who wishes to disable automatic builds and tests for all changes can disable GitHub Actions for their personal fork. See the GitHub documentation for how to disable GitHub Actions completely for a repository.

Contributors can choose to only enable automatic builds and tests for commits pushed to branches with certain names. To enable this the contributors creates a GitHub Secret with the name JDK_TEST_ENABLE. The value of JDK_TEST_ENABLE is a regular expression denoting the branch names for which commits should be tested. For example, setting the secret JDK_TEST_ENABLE to test-.* makes automatic builds and tests only being run for commits pushed to branches whose names matches the regular expression test-.* . The default value for JDK_TEST_ENABLE is .*.

Contributors can choose to disable automatic builds and tests for commits pushed to branches with certain names. To enable this the contributors creates a GitHub Secret with the name JDK_TEST_DISABLE. The value of JDK_TEST_DISABLE is a regular expression denoting the branch names for which commits should not be tested. For example, setting the secret JDK_TEST_DISABLE to wip-.* makes automatic builds and tests not being run for commits pushed to branches whose names matches the regular expression wip-.* . The default for JDK_TEST_DISABLE is wip(-|/).*|master.

Running locally

The run the same tests that are being run using GitHub Actions locally, run the following command:

$ make test-tier1

Note that the GitHub Actions runs the above tests both with release and debug builds. For more information about how to run tests locally, see doc/testing.md.

Definitions

The definitions for the GitHub Actions for the jdk repository are available in the .github/workflows directory.