Overview

Monocle is a proposed rewrite of Lens, the Glass implementation for the implementation of the Glass windowing component of JavaFX for embedded systems. Monocle provides windowing functionality and access to native graphics for simple embedded systems that do not have an underlying window manager. Unlike Lens, in which most of the work is done in C, Monocle will be almost all Java code. Where required, interactions with C code will be through simple Java-C bindings where the logic is in Java and only the low-level system or library call is in C. The goals of Monocle are:

• Functional and quality parity with Lens on all platforms: Embedded Linux/ARM, Android and headless
• Simplify the process of porting to a new platform. Ideally this could be done in some cases without rebuilding JavaFX.
• Improve maintainabiity and debuggability of the embedded ports. Where possible, simplify the code.
• Pluggability of Linux input device handlers. Ideally it would be possible to add support for a new input device without rebuilding JavaFX.

Like Lens, Monocle assumes it has full control of the screen and does not have to cooperate with other graphical applications. It does not rely on an underlying window system.

Generic Components

Glass Integration

The classes named Monocle* integrate with class. So MonocleApplication, MonocleWindow and MonocleView extends the Glass classes Application, Window and View.

All code running at this level is on the application thread.

Window Management

Monocle windows do not exist at a native level. Window state is held in MonocleWindow, while MonocleWindowManager maintains the Z-ordering of the window stack and assigns ID numbers to MonocleWindows.

All window management code runs on the application thread.

Input Device Capabilities

An InputDevice represents a single device that can generate input events. An InputDevice can report on its input capabilities. For example, it can declare itself as a multitouch screen or as a 5-way keypad. InputDevices are registered with an InputDeviceRegistry; MonocleApplication listens on changes to the InputDeviceRegistry to get notification on what classes of devices are attached.

Implementations of InputDeviceRegistry are also responsible for making sure input devices are recognized and their events delivered.

All generic input device code runs on the application thread. Platform-specific InputDeviceRegistry implementations can contain code that runs on other threads. 

Input

Three input handler classes process input of different kinds: MouseInput, TouchInput and KeyInput. Each of these maintains its own state in an input state class: MouseState, TouchState or KeyState. When the one of the input handler is notified of a change to the input state it generates events accordingly based on the current state of the window stack and input focus. Low-level input classes do not communicate directly with the window stack.

Each input handler contains a single input state object. When the input handler is notified of an input state change, it receives a state object containing the new state. The input handler does not store this external state object, but copies its contents into its own records. This helps us to minimize object creation  during event processing.

All generic input code runs on the application thread.

Platform

The platform-specific components are: NativePlatform, NativeScreen, NativeCursor and InputDeviceRegistry.

NativePlatform is instantiated by NativePlatformFactory. NativePlatformFactory looks at the system property monocle.platform to get an ordered list of factory classes to attempt to use. NativePlatformFactory then instantiates these factory classes, querying each in turn whether it can support the current platform it is running on. When a matching NativePlatformFactory is found, its corresponding NativeFactory will be created.

NativePlatform provides a single-threaded java.util.concurrent.ExecutorService. This ExecutorService is the application thread.

NativeScreen is instantiated by the NativePlatform. NativeScreen reports on the physical characteristics of the screen. It is possible that this class will be used by Prism as well; in this case the class will have to be thread-safe.

NativeCursor is instantiated by the NativePlatform. NativeCursor is responsible for updating the visible cursor state, using a platform-specific hardware cursor where possible. NullCursor is an empty implementation of this that does not display a cursor.

Generic Linux Port

Native Interfaces

Most of the interaction between the Linux port of Monocle and the OS level is using the pseudo-filesystem under /sys. This is accessed using utility methods in the class SysFS. This is sufficient to read screen and input device characteristics and to request notification on what input devices are attached.

The Udev class is an interface to the Linux udev monitor to get notification when devices are attached and removed from the system. This requires some C code, since Java does not have an API for connecting to Unix domain sockets. 

it might be necessary to add another native interface to use ioctl calls to read absolute axis range data for touch screens. This information does not seem to be available in sysfs.

Implementations

• FBDevScreen reads screen data from the framebuffer device /dev/fb0.
• No cursor implementation is provided for the generic port, since there is no usable standard for hardware cursors on Linux.
• LinuxInputDeviceRegistry receives hot plug notifications of input device addition and removal and sets up input event processors that work with the generic Monocle input handlers.

Input

When an input device is detected by Udev, a LinuxInputDevice instance is created for it. The LinuxInputDevice reads events from the Linux input node into a ByteBuffer. As soon as any data is ready in the ByteBuffer to be processed, a Runnable (actually a singleton EventProcessor) is submitted to the application thread to process this data. "Data ready" means that a complete event has been received, including the terminating EV_SYN SYN_REPORT. If the EventProcessor is already pending execution then it is not resubmitted; the expectation is that every time the EventProcessor runs it will process all pending events.

Each LinuxInputDevice has an implementation of LinuxInputProcessor attached to it. This LinuxInputProcessor is called by the EventProcessor on the application thread. The LinuxInputProcessor iterates over pending input events and notifies input handlers of input state changes.

Note that raw linux input events are not objects. Where possible we avoid object creating during input processing. This will be particularly important in touch and mouse processing where events can be received very quickly.

OMAP Port

The OMAP port is a subclass of the generic Linux port that also provides a hardware cursor on OMAP platforms.

X11 Port

The X11 port is a subclass of the generic Linux port that uses EGL/X11 for rendering instead of EGL/Framebuffer. It also takes its input from X11 events instead of directly from Linux input devices.

OMAPX11 Port

The OMAPX11 port is a subclass of the OMAP port that uses EGL/X11 for rendering instead of EGL/Framebuffer.  Like the OMAP port, it uses Linux input devices for input events and a hardware OMAP cursor.

Status

What's working:

• Rendering and input on X11
• Rendering on Freescale i.MX6
• Mouse input
• Cursor on OMAP3 (partially)
• Most of HelloSanity on BeagleBoard xM

What's not working:

system.

Description of Monocle components

This is hard to read in the web interface, so you'll want to download the PDF if you are interested in Monocle's inner workings.

How to build Monocle

If you build OpenJFX for embedded Linux/ARM platforms from the 8u-dev repository then you get a functioning Monocle as well. You can also build it for Linux/x86 using the x86egl compile target (build with -PCOMPILE_TARGETS=x86egl on a system with GLESv2 and EGL development libraries installed). This builds the embedded stack for the Linux desktop; it is not the same as the GTK implementation of Glass. You cannot currently build any of the graphical implementations of Monocle for desktop windowing systems.

How to run Monocle

Monocle is now the default implementation of Glass on embedded platforms. 

You can select one of the specific back-ends of Monocle with the system property monocle.platform. Some of the back-ends work only with hardware rendering (the es2 pipeline); some work only with software rendering (the sw pipeline).

If you are running the desktop build of JavaFX or OpenJFX then your only monocle option is Headless. Desktop JavaFX does not support the javafx.platform system property, but you can select Monocle with:
-Dglass.platform=Monocle -Dmonocle.platform=Headless -Dprism.order=sw

On MacOS and Windows, removing -Dprism.order=sw can be critical to prevent crashes.

Status

What's working:

• Accelerated rendering on Freescale i.MX6, BeagleBoard xM and Raspberry Pi
• Mouse, key and single-point/multi-point touch input with Linux device nodes
• Synthesis of mouse events from touch events
• Double-buffered software rendering to memory-mapped /dev/fb0, with a software cursor
• Hardware cursors on OMAP3, i.MX6 and Raspberry Pi
• Touch coordinate transformations for screen calibration 
• Pluggable pipeline for touch event cleanup
  
• Nested event loops
• Robot input and capture
• HelloSanity is working on Freescale i.MX6, BeagleBoard xM and Raspberry Pi
• Headless implementation running on embedded and desktop platforms, passing unit and system tests (base/graphics tests, system tests and Linux input tests)
• VNC server mode with remote display and mouse input
• Mouse input and accelerated rendering on X11
• Drag and Drop
• Full screen and minimized
• Transparency in OMAP3 cursor
• Multiple windows

What's not done yet:

• Touch, Key input
• Freescale i.MX6 Cursor
• Dispman port
• Nested event loops
• Window grabbing
• Integration into 8u-dev
• Robot
• Unification with Prism's platform recognition

• Android port
• DirectFB port

See also the JIRA query for open issues on Monocle

How to port Monocle

See Porting JavaFX to additional embedded Linux devices