Backlog / deficit 

There are a lot of tests and we have decent functional coverage but even so there's a lot of accumulated issues that we should address.
In rough priority order the main categories for the tests themselves are :

• Fix tests that are currently failing and  / or are problem listed
• Update tests that use the Applet API to remove that dependency
• Migrate more tests to OpenJDK from closed test suites.
• Update manual tests to use a common framework for reporting the results
• Make more manual tests into automated tests

Writing stable automated client tests

Reliable automated GUI testing is hard work since rendering may be

1. different from platform to platform
2.  Affected by O/S desktop changes
3.  Affected by platform specifics such as display resolution, color profiles, keyboard shortcuts
4.  Affected by timing issues

 etc.

We expect automated tests to be resilient against these and pass reliably 99.99% of the time
Tests which are considered to be "important" should pass with even greater reliability.

Many of our automated tests that are headful make use of the java.awt.Robot API to

- deliver mouse and key input events
- grab portions of the screen for comparison against expected rendering.
   These have additional considerations.

The following are some guidelines that should be followed in writing client tests and
to ensure stability and reliability of those tests. This will be added to and refined over time.

• Tests should be automated. If it does not need to be manual - automate it
  
•   Headless tests are preferred - they run faster and for our CI testing on much available hardware and they may be better candidates to run in github actions (some day).
   For extra credit make a UI "optional" - via a command line option to help someone evaluating a failure
  
•  Headless tests can grab screen pixels from a BufferedImage if testing rendering and should be reliable
  
• Automated tests that need to test screen rendering can use java.awt.Robot to grab screen pixels
     But beware of (1) rounded corners of application windows (2) Unexpected overlapping  (3) Shadow effects near the edges of windows.
  • Try to ensure you sample safely inside a window in an area where all colours should be as expected
    
•  Beware of tests that may be unstable due to some other test leaving a modifier key pressed.
     Or CAPS LOCK activated. And make sure YOUR test always cleans up after itself so that no matter of error, or failure, or what it leaves the keyboard state as it found it.
  
•  Make sure to hide() and dispose() all windows
  
•  Use Robot.setAutoDelay() so you don't have to insert delays after every key press
  
•  Use additional delays where the auto delay may not be enough but don't over do it !
  
•  Use SwingUtilities.invokeAndWait() etc to make sure the UI is created before your test runs
• In general follow threading rules
• Use Toolkit/Robot sync to flush all requests
•  use Robot.waitForIdle() to ensure all events are processed  Use volatile or synchronized or Atomic classes or whatever is appropriate to make sure threads see changes, or see consistent state
•  Make sure the test can safely run in the default time allowed by jtreg (180 seconds). Otherwise increase the time out
  
•  If the test fails log as much info as you can as to what failed "test failed" is terribly unhelpful if a test failure can't be reproduced. Use printStackTrace(), print the state of variables, save  as PNG a BufferedImage that isn't rendered as expected etc .. make sure everything points  back to the source of the failure, and you aren't losing that and just summarising the result.
• The fewer windows the better.