VirtualKeyboard User Experience Documentation

Author: Jindrich Dinga

1. Introduction

This document is the user experience specification for the JavaFX Virtual Keyboard control. 

2. Overview

Virtual Keyboard, which may also be known as a software keyboard, is a control that allows users to type text on devices that lack of hardware keyboard, i.e. tablets, TVs, smart phones. It operates the same way as any hardware keyboard except that numbers and symbols might be located one tap away due to space constraints.

Basic assumptions and requirements for 2.2 release are as follows:

1. Virtual Keyboard must be shown automatically when a text input field is in focus. If there is a native virtual keyboard available on the device, this keyboard must be used instead of our own if possible.
2. Virtual Keyboard must support US English locale only.
3. Virtual Keyboard must work with touch devices (non-touch devices will be supported in a future release).
4. Virtual Keyboard must support context hints.
5. Virtual Keyboard must react when devices are rotated.
6. Minimum screens size is VGA (640x480) and up.

The following picture shows overview of Virtual Keyboard UI Control for JavaFX Embedded.

Figure 1 Overview of Alphanumeric Virtual Keyboard

Since the virtual keyboard should rotate when devices are flipped over, we should make sure that the virtual keyboard also nicely fits in VGA screens in the portrait mode (480x640) and up.

Figure 2 - Alphanumeric Virtual Keyboard in Portrait mode

3. Interaction Design

3.1 Basic Functionality

Virtual keyboard always appears automatically when users need to enter text into TextField, TextArea, PasswordField, and possibly into other controls in case their edit state is enabled.

When required, the keyboard slides out of the bottom area pushing the parent stage up in order to keep the control which the keyboard is associated with visible on the screen. When the keyboard is dismissed, the parent stage should return to its original position. Please note that if control that is associated with the keyboard is visible when the keyboard is displayed, there is no need to push the parent stage up. To hide the keyboard, users must tap outside the control or "hide the keyboard" button.

When the virtual keyboard appears, users can type a character by tapping the appropriate key. After that, character that is associated with the key is sent to the control.

Figure 3 Example of a tap feedback on touch device

In case users tap and hold a key for longer time a pop-up dialog appears showing the primary character, alternative character, and characters with diacritics (if available) that are associated with that key. Users can select the character they need by tapping the character in the pop-up dialog. Please note that the first character in the pop-up dialog is focused.

Figure 4 Dialog that appears above the letter after long press

In order to type capitalized letters, users need to tap the Shift/CapsLock key. After that, the virtual keyboard shows capital letters. Functionality of Shift/CapsLock key depends on interaction with the key.

Let's assume that the virtual keyboard shows small letters. Pressing the Shift key once causes the virtual keyboard to show capital letters. Once a character is typed, the virtual keyboard returns to the previous state to show small letters. If users do not tap any character, but the Shift key instead the virtual keyboard also returns to the previous state to show small letters. In order to enable the CapsLock functionality, users need to double tap the Shift key. After that, the virtual keyboard shows capital letters only. To switch CapsLock functionality OFF and return to the virtual keyboard's original state, users must tap the Shift key.

Figure 5 Diagram of Shift/CapsLock key functionality

Sometimes, users may want to type more than one alternative character into the input field. Even though users can do that by holding the appropriate key for longer time, there is faster way how to do that. Users can simply switch keyboards by tapping the "Switch the Keyboard" key and type primary characters by tapping appropriate keys. To access less frequently used characters, users need to press the appropriate key for longer time.

Figure 6 Mock-up of "secondary" alphanumeric keyboard showing numbers and symbols

Please note that label of the "Switch the Keyboard" key is !#123 for keyboard that shows alpha keys. Otherwise, the label reads as ABC. In addition to that, keys that do not provide users with any functionality or the functionality is not available are grayed out.

In some cases, showing regular virtual keyboard may be a little bit cumbersome. For example, if an input field requires users to enter numbers only it is recommended for application developers to show numeric virtual keyboard instead.

Figure 7 Example of a numeric virtual keyboard

Apart from providing app developers with a numeric virtual keyboard, there are another keyboards available to them. These keyboards can be used when users are required to type a URL or e-mail address, because they provide them with @ and .com keys. To access less common domains, such as .org, .net, .edu., and .tv,, users can press .com key for longer time.

Figure 8 Example of "primary and secondary" E-mail virtual keyboard

Figure 9 Example of "primary and secondary" URL virtual keyboard

If users want to edit a letter in the middle of a word, they can tap the place where they want to place the caret. If the caret is not in the intended place, users can adjust it by dragging the handle (see Figure 10). In order to select whole word, users can double tap it. After the selection is displayed, users can adjust the selection by dragging the handles (see Figure 11). Please note, that behavior described in this paragraph is not something that the virtual keyboard should be responsible for. It should be handled by the input text field instead.

Figure 10 Example of the caret with a handle

Figure 11 Example of a selection with handles

When users want to delete a character, they can do that by tapping the Delete key. After that, character that is on the left side of the caret is removed from the control the virtual keyboard is associated with. Keeping the Delete key pressed for longer time deletes word after word.

Please note that tapping the Enter key should perform action that is associated with focused control. E.g. tapping the Enter key in TextArea moves the caret on the new line.

3.2 Additional Functionality

Hide the Keyboard

It would be good if the Virtual Keyboard has the "Hide the Keyboard" key that would allow users to dismiss the keyboard when it is not needed. The key would be situated on the bottom left. For more information, see the pictures.

Capitalization of the first letter in each sentence

It would be nice if the Shift key is selected every time a text field gets focus and at the beginning of each sentence so users do not have to select it by themselves in order to start sentences with capital letter.

Assumption: Each sentence starts after a "dot".

Custom button

It would be nice if application developers are allowed to create their own virtual keyboard in which they can substitute our keys with their own keys. This can be especially useful when default keys, such as Return, do not make sense in given context. For example, when the virtual keyboard is associated with a search field the Return key could be substituted with a Search key. Apart from that, it should also be possible for application developers to change color of their own keys so they do not use the default color scheme as other keys.

Figure 12 Example of a custom button

Previous and Next buttons

In some cases, i.e. when users are required to fill out a form, application developers should be allowed to provide users with buttons that would allow them to easily navigate between controls within the form. These two buttons should be placed right above the virtual keyboard and their behavior should be as follows:

• Previous button is disabled when focus is on the first control and
• Next button is disabled when focus is on the last control.
• Otherwise these buttons are always enabled.

Please note that application developers should also be allowed to add their custom buttons to the bar as needed, i.e. a button that would indicate that users are done.

Figure 13 Example of Previous and Next buttons along with a custom button

4. Visual Design

5. Navigation

On touch devices, users use fingers or stylus to navigate within an app and keyboard. The following table describes navigation within the virtual keyboard on a touch device.