Page History

The Wakefield committers may hold off-line discussions from time-to-time. Since these meetings are just for the committers and invited wayland experts, summary minutes will be recorded here as time permits

 Online Zoom Meeting 9am PDT 17th Aug 2023

Attendees:   Niels de Graef,  Victor D'yakov,  Phil Race,  Olivier Fourdan, Maxim Kartashev, Kevin Rushforth, Jonas Adahl, Alexander Zvegintsev

Alexander :

As suggested, I implemented the approach when the session is not closed right away, but after a few seconds of inactivity. It gave a significant boost for certain scenarios, e.g. taking pixel by pixel in 50x50 area took 375s before the fix and only 28s after the fix.

Other than that working on various HiDPI scenarios for taking screen data in X11 compatibility mode.

Maxim:

We have made some progress since we last met, Vulkan rendering pipeling work is ongoing, we are actually able to render a colored rectangle on the screen, still the bulk of the work was setting up the environment to be able to start making progress, it is almost done.

We also implemented input support for testing through our Weston plugin, so the robot can generate keyboard events, mouse clicks, including absolute positioning, etc. We are getting ready to start running tests in our environment, but we are not there yet because the rest of the toolkit is not ready to start testing.

I finished working on tooltips, menus, now all the stuff works more or less, because the positioning left for the Wayland server entirely with only hint that we can give. It works pretty well for us, but things may change in the production, with all variety of desktop configurations.

I am finishing the clipboard support and plan to get to work on DnD soon.

I think, that primary focus for our work in medium term is going to be Vulkan and getting the IDE window up on the screen and interactable to show our users the progress being made.

Phil:

The hints you refer to positioning are a bit vague, did it go where you wanted it to go, you don't know it went there, but it actually did go there?

Maxim:

It is probably the most complicated part of the protocol I have dealt with so far. I can post a link to the protocol interface, called the Positioner, but it will not make things clearer.

In short, you throw away all the calculations you did in the Java code, except for the coordinates relative to the top left corner of the parent window, and then you should suggest to Wayland what to do if that particular position is not suitable, for example, if it is between screens, on the edge of the screen, off the screen. I think this logic in Wayland is quite similar to the Java code we have.

https://wayland.app/protocols/xdg-shell#xdg_wm_base:request:create_positioner

https://wayland.app/protocols/xdg-shell#xdg_positioner

The Wayland API does not allow you to use the results of calculations made by common Java code, because the result of this code is in an absolute position on the screen.

Phil:

So this is a mismatch between what Wayland expects and the way our code has been written historically?

There's nothing fundamentally wrong with either side, it's just that they come from two different worlds and it's going to be hard to make them meet.

Maxim:

Yes,  basically the work that we do in Java right now  for positioning and popups, all that work is done on the Wayland server side, and you can only nudge it in the right direction that you think is preferable for you by giving hints.

Phil:

There is something about you posted on a Wakefield mailing list about libei. Could you please elaborate what are doing there?

Maxim:

The idea for our testing so far has been to rely on a plugin for Weston to use the Weston instance and run tests there. The only ramining part that was not implemented was the input emulation.

One of the approaches to achieve that is to use more or less standard way to use the lebei. I asked if Weston supported it so we could just implement it through libei. But it will not be supported, so we decided to use another approach.

Niels:

people might be interested in Peter Hutterer’s GUADEC talk about libei: https://www.youtube.com/live/dVVyokoQl2k?t=15312 

He is the maintainer of libinput and the author of libei. It is a good talk about input capturing and  what libei is.

Gnome 45 will be released with libei support in a less than a month.

Phil:

X11 compatibility mode mostly seems to be pretty stable, the only issue we have is related to the Meta.disable_unredirect_for_display(global.display) workaround. It seems to work.

I did some pretty exaustive testing, we can run thousands of test with screencaptures, whatever else, the token does seems to remain valid, unless we run this test that does the fullscreen capture and everything goes south.

Is there a way to do this Meta.disable_unredirect_for_display(global.display) in some kind of startup file?

Jonas:

You can try to use an extension(basically a js) which loads with a system
https://extensions.gnome.org/extension/1873/disable-unredirect-fullscreen-windows/

In Gnome 45 and xdg-desktop-portal 1.18 you also get the same token behavior(reuse token and restore session) as in Screencast for Remote Desktop, so you can do a remote control.

https://github.com/flatpak/libportal/pull/114

Olivier:

For X11 compatibility mode, XWayland has direct support for libei, so XTest calls will call libei internally. This allows to control mouse and keyboard outside the X11 server, which is not possible today.

For example, clicking on the title of a window to move it to the front should work.

However, it does not use tokens, but a dialog for user confirmation.

Alexander:

Is this permission a one-time thing that lasts forever?

Olivier:

No, the XWayland will keep the connection for some time, if there is no activity from the client for a certain amount of time the connection will be closed and a user will be asked again.

It does not use the token. From the Wayland point of view you never know if some legit X11 application or a rogue client is trying to hijack your pointer.

Alexander:

What if I need to keep this permission between reboots?

Olivier:

In this case, you would need to use Remote Desktop Portal with a token and use libei directly, not XWayland.

Phil:

For native Wayland, if we use libei directly - we are good, but for XWayland it gets better only when user is there and can click on a session.

It is not even a login session , it is XTest - libei behind the scenes session.

Maxim:

I forgot to mention one more thing, the splash screen support is also finalized.

I managed to get it to the screen center, the idea is to show a screen sized transparent window and position against it and hope that the Wayland will place the window exactly matching the screen size.

I am not sure how well it will work in a multi-monitor configuration.

Jonas:

There may be some glitches, such as animations showing windows appearing, it will not work well with tiling setups.

It does not look like a future-proof solution.

The proper solution for the splash screen is to work on the splash screen role window.

https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/140

Alexander:

We could use a X11's splash screen implementation for a while, it does not depend on XToolkit, and does not have such issues with positioning.

Jonas:

It will be problematic in many compositors that use Wayland to do coordinate systems in a compositor. In Gnome terms, this is called fractional scaling mode, and it may cause the splash screen to be blurry.

Online Zoom Meeting 9am PDT 22nd June 2023

...

he topic of the day was a discussion around Alexey's email to the mailing list ..libei

https://mail.openjdk.java.net/pipermail/wakefield-dev/2021-December/000025.html

proposing what we might do for the "full" wayland native port and the various points it raised

Note: Alexey will add a fleshed out version on the project wiki.

The proposal suggests we'd code to the low level wayland APIs rather than using a high level toolkit (ie GTK4)

As previously observed we should explore both. Probably we could get bootstrapped a lot faster with GTK4
and it would give flexibility for backends so we'd probably not need to create an OGL and Vulkan backend ..
if Vulkan was an option on that distro then we'd expect the underlying Cairo (?) based renderer to be
using it or have it as an option.

Likely there are pros and cons to both options. GTK4 might make assumptions or so things in a way
that isn't compatible with some other functionality. A low-level port might mean a lot of re-inventing
the wheel and having to use functionality that is distor-specific but would have been hidden / abstracted
by the platform GTK 4.

So due diligence on both options, and not expecting completely smooth sailing for either.
Ease of implementation and maintenance and portability are strong arguments, but so are flexibility and
internal consistency in behaviour and behavioural compatibility  with the existing X11 implementation.
So if doing it "right" is more work, we'll have to absorb that, and we want a stable base so we aren't
in a perpetual maintenance game.

Mario observed that we need to not get ahead of ourselves because we still have the list
of technical challenges that need to be resolved that apply to X compat mode and to a native port too.

We'll work on those but whilst they work through the eco-system over 2 years (?) we could be
doing something useful ..

One element of the proposal not directly related to wayland is we could merge EDT and Toolkit thread
since we no longer need these separate for applets. Possibly, but we kick off a new thread for modal
dialogs too .. could this be special cased ? Might depend on whether you can limit what happens
in such a case but I'm not sure you can.

We do have these documented on the wiki but it needs another pass to have a consitent format
- what the problem is
- what JDK can do
- what we need from the wayland ecosystem
- how much of a blocker it is
- can we "tweak" the spec - at the cost of making backports harder

We should look at this soon so we can get started on spec tweaks we think may be unavoidable but acceptable ..

Eg What's acceptable for screenshots ? We can't deprecate but can we make something optional ?

Alex Z: Is the SplashScreen API supportable ?

Apparently splash screens display in top lefe (the window positioning problem)

A splash screen (output only) surface role could probably be proposed upstream.
it'd just need a fallback implementation for compositors that doesn't support it, e.g. a dialog

Maybe similar to this PiP request : https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/132
For more info on roles: https://wayland-book.com/surfaces/roles.html
Add input to this existing issue : https://gitlab.freedesktop.org/wayland/wayland-protocols/-/issues/67

Accessibility ?

Is there a library ? Like ATK ?
There are existing DBUS based A11Y APIs ..
For the accessibility protocol Jonas is talking about: https://www.freedesktop.org/wiki/Accessibility/AT-SPI2/
For a quick overview of what Emmanuele did for accessibility in GTK4: https://blog.gtk.org/2020/10/21/accessibility-in-gtk-4/

...