• How do release numbers of releases on the binary download page relate to the repository or the trunk of the repository?

  A: During development of a release, the trunk is used to create the binary development (dev) images. Developers add to the trunk during their development process, so it is a changing body of code. When a release is completed, that final body of code is copied into the branches/code/ directory in the SVN repository and marked with the appropriate version number. That code corresponds to the milestone release you will see on the download page.

  If you ever have trouble finding something, please post a question in the JT harness forum. One of the developers will explain the current status.

• In the tutorial, the metadata to describe a test is located in the test source .java files. Is this required? What if my test suite does not contain .java files?

  A: This is just one way that a test suite can present the metadata. Another popular method is to use HTML files for this purpose. Different TestFinders are included in the project for this purpose. You can also write a custom TestFinder that extracts this information in a way that works best for your test suite. See the JavaTest Architect's Guide and Javadoc generated documentation for information about this.

• The jtharness.zip file does not have a top-level directory to encapsulate the contents being unpacked. This makes a mess in my current directory if I forget to unpack it in a separate directory. Is this intentional?

  A: Yes, it is created this way because the product is usually packaged within another deliverable (for example, a test suite product). Because the harness is often imported from the Zip bundle. we do not create a top-level directory because it creates an unnecessary directory. We realize this is not optimal for a person previewing the JT harness and will consider changing the Zip layout for future releases.

• Is there a way to pause or resume a test run?

  A: This functionality is not provided by JT harness directly, but it can be provided by a test framework. For example, the ME Framework allows users to pause a test for an unlimited time.

• Is the ordering of items on the command line significant?

  A: Yes. See the JavaTest User's Guide - Command-Line User Interface available from java.sun.com or the JT harness online help for more information.

• We have a test suite and run it as part of the nightly builds. How can I avoid having to re-answer the configuration interview questions each day?

  A: You can save the interview to a .jti file and specify that file when you run the tests in batch mode. You can change the answers as required on the batch command line.

• My entire organization uses basically the same configuration settings for running the test suite. Do we all need to answer the entire configuration interview and keep track of our own interview files?

  A: No. An "administrator" can set up a central repository of interview templates that answer all the questions that the organization has in common. Users only have to answer questions that vary from individual to individual. The templates can be created using bookmarks so that only the questions that require change are displayed.

• Are interviews forward and backwards compatible?

  A: If the configuration interview changes and a set of answers is loaded, question-answer pairs that were saved from the old interview are still valid in the new interview. However, depending on the changes, the new interview might be incomplete.

• I run tests using a status filter, and the tests seem to disappear from the display when their result status changes to "pass". Also, the progress bar indicates that the tests passed, failed, and so on, but they do not appear in statistics that way. What's going on?

  A: Most likely the tests moved into the "filtered out" category because they no longer fit the selection criteria defined in the current configuration, which is what the tree and its statistics show by default. You can change your view filter to change this effect. Try using the "All Tests" or "Last Test Run" filters. This is a side-effect of maintaining a GUI state that is always consistent. It is important to remember that the tree and its statistics represent the contents of the entire work directory. The progress bar and progress monitor dialog represent only the last or current test run. Understanding that is vital to understanding the numbers in the GUI display.

• The progress monitor dialog box (or any dialog box) insists on staying on top of the main JT harness window. Is there anything I can do about that?

  A: Unfortunately, the JT harness does not have complete control over window ordering. This is a function that your operating system window manager controls. The implementation of this functionality might be changed in a future release to allow the main window to cover the dialog box.

• When I exit the JT harness GUI are my settings saved?

  A: All GUI settings are saved to your desktop unless you set the preference that prevents them from being saved. If you restart the JT harness, your settings are reinstated. However, if you start the JT harness on the command line with explicit configuration flags or with the -newDesktop flag, the settings are not reinstated. A few options are saved in the preferences subsystem, that survive beyond the saved "desktop".

  Note: Beginning in version 4.3 of the harness, settings are no longer saved by default. You must specify that you want them saved using the Preferences dialog box. If you use version 4.3 or later on a work directory created by an earlier version, the default settings used by the earlier version are retained.

• Why doesn't the log viewer show any output?

  A: The harness itself generates minimal output to the logging subsystem. Most of the output to the logging subsystem is generated by the test suite itself. Some test suites create more output than others. Developers should attempt to use the built-in logging system whenever possible, instead of sending output to System.err and System.out.

• I'm testing my app in an ME emulator. Can the harness set up my emulation environment for me?

  A: Yes, starting with version 4.2 of the harness, an ME emulator can be started using the service management facility. Emulators (as well as other test services) can be specified as services to be started by the harness before a test run. The built-in service management facility can identify this at runtime and then request that service to start. The request to start the service is done through the use of Ant, which allows the architect or tester to configure the operations required to activate a particular service.

  For more information about using service management see the JavaTest Harness 4.2 Architect's Guide. Note that earlier versions of this guide do not include this information.

• Can I exchange configurations (.jti files) across different locales?

  A: Yes. In harness versions prior to 4.2 it should normally work, but unintended behavior might occur in cases where questions store numeric information. The interview stores incorrect answers as strings and saves them for later use. If the user later returns to that question in the Configuration Editor, the invalid answer appears for editing. For example, if a user enters 1,4 for a floating point value in the "en,US" (US English) locale it is invalid. However, if that JTI is transferred to the "fr,FR" (France French) locale, the number becomes valid because that locale uses the comma as the floating point separator.

  Beginning with version 4.2, invalid values are no longer stored. In addition, the original and current locales are taken into account when reading floating point and integer answers from the JTI file. Invalid answers now stay invalid or blank, and valid answers remain valid.

  If the JTI file is written out again, it is done so using the current locale. There is currently no way to save the results in a locale different from the one you are using. As a workaround, you can use your operating system settings and run the harness in a different locale.

• Where does the harness store information on my computer? How can I delete it or use it to troubleshoot?

  A: The harness writes information to only one location that the user does not explicitly specify — the .javatest folder. The .javatest folder is created in the directory specified by the user.home JVM system property. This folder primarily contains your preference settings and stored desktop information. If you want to restore your system to the state it was in before you ever ran JT harness, you can delete this folder. It is not usually necessary to manipulate this folder. All other data is written to locations specified by the user in the harness GUI.

  The work directory contains multiple log files that can sometimes be useful in sorting out problems. Feel free to contact the developers via the forum on java.net if you want help interpreting the output in those files.

• My test suite needs some separate processes for the tests to run, can the harness start/stop them for me?

  A: Yes, there are two general ways to do this. In the past, this was usually accomplished by a wrapper script that started and stopped the harness. This is a reasonable solution, but does not allow for more integrated behavior such as restarting services during the test run for example, capturing of log output within the harness and starting services on-demand.

  The harness provides two ways to start and stop services. First is by monitoring the observer messages from the harness that indicate the start and completion of a test run. See the API documentation for the TestSuite and Harness.Observer interfaces for information. determined Beginning with version 4.2.1 of the harness, an integrated service management subsystem exists that allows the starting and stopping of services on a per-test run or as-needed basis. These services can be programatically specified, through a XML based configuration file or by a combination of those. See Chapter 11 in the JavaTest Architect's Guide for more information.

• With services, how are the binaries located on the machine under test?

  A: This is configured by the test suite. Consult the test suite documentation for details. These paths are usually set by answering a question in the configuration interview or by changing a value in an XML file.

• Are any port (number) management capabilities provided?

  A: This is not available yet, but would be useful for dynamically finding an available port on which to run a service — for example, if you want to run two test runs simultaneously and both tests need an HTTP and HTTPS server. The test suite needs to find four available ports to start the services, and having this managed by the harness might be useful.

  If you would like to help implement a post/resource management system, contact the developers in the JT harness forum on java.net.

Reporting Test Results