Page History

The Wakefield committers may hold off-line discussions from time-to-time. Since these meetings are just for the committers and invited wayland experts, summary minutes will be recorded here as time permits

Online Zoom Meeting 9am PDT January 31, 2025

Attendees: Olivier Fourdan, Alexey Ushakov, Maxim Kartashev, Harshitha Onkar, Phil Race, Kevin Rushforth, Victor D'yakov, Alexander Zvegintsev

Alexander:

Fixed a problem with focusable popups being closed prematurely, it was not handled correctly before.

I'm updating the pipewire header files to a more modern version.

Alexey:

We implemented pixel grabber functionality for Vulcan hardware accelerated surfaces.

We're creating a sort of shortcut for the robot functionality to grab pixels directly from it. 

I implemented it to do single window testing for rendering.

Within this work I implemented surface to software blit that is necessary to get the pixels, now running some tests from AWT part for regression tests, investigating the remaining failures.

Maxim:

As you probably all know, there are no absolute coordinates in Wayland. So for getLocationOnScreen I just returned zeros.

It worked for a simple Swing application, and it worked perfectly.

But it turns out that there are actually cases when you do have some glimpse into the Wayland coordinate system when you look at the relative location of the monitors that you have attached to a system.

And in XToolkit, it is visible, it matters, but the applications do depend on this.

So if you have two displays that are not perfectly aligned, for example, if the other monitor, not that your main window is on, is above.

So your monitor has a positive offset in the absolute coordinate system.

And this created a small problem, which I fixed by making all windows returning their primary monitors offsets as their absolute locations.

So you might have some non-zero offset, if you have more than one monitor, 

For example, the monitor on the right would be the normal size, the monitor on the left would be the horizontal offset size, and so on.

So having this done, I started going around and finding and fixing all sorts of offset issues, mostly with pop-ups.

And it took quite a while because I don't normally use a setup where the monitors are not aligned. But people do, and they report problems.

And I started using this setup so that I could catch problems before others did.

And the other day, for example, I discovered that the robot implementation actually also depends on this feature, that the specification expects the robot to have coordinates on the screen that didn't matter before, but now they do.

One is the test that would just get the location of a JFrame, which is wrong because its specification says that this location is relative to the parent and not to the screen.

So to properly feed the robot the correct coordinates, you need to get the location on the screen.

But even if you do that in our Wayland toolkit implementation, you may still get the wrong result because of screen offsets in a multi-monitor configuration.

So the problem is simple, but it took a long time to figure out which places to fix and how, and I'm sure we're not done yet, as not all holes have been plugged.

As more people start using our pure Wayland toolkit, more bugs are reported, unobvious configurations are discovered, etc.

For example, someone recently tested the clipboard and discovered that it would not accept more than 64K of data.

And it turned out that at least on my Ubuntu with GNOME 24.04, the pipe opened for clipboard content transfer is in non-blocking mode.

And the old IO in Java doesn't handle this very well. So it just exits as if successful.

And fortunately after writing the buffer capacity, which is 64K, it continues as if nothing happened.

Online Zoom Meeting 9am PDT December 5  2024

Attendees: Jonas Ådahl, Olivier Fourdan, Alexey Ushakov, Harshitha Onkar, Phil Race, Kevin Rushforth, Victor D'yakov, Alexander Zvegintsev

Alexander:

Fixed a couple of JavaFX/Swing interop issues related to our screencast usage for receiving screen data:

1. hang when FX is running, it internally runs the gtk main loop, which wasn't accounted for on the JDK side.
So now it handles 3 cases:
    * When there is no GTK main loop running
    * If there is a GTK main loop running, but we are not requesting pixels on its thread
    * If there is a GTK main loop running and we request pixels on its thread

2. crash,
Internally the ScreenCast session keeps open for 2s (both JDK and JFX, and their implementations are almost identical).
When we perform a cleanup to close the session, we internally called pw_deinit before the fix.
It becomes a problem if these sessions overlap in time, so that the second session cleanup crashes when it tries to call pipewire functions without initializing the pipewire system by pw_init

Working on JDK side fix of the emulated resolution change does not come issue.
The system side fix has landed on the Ubuntu 22.04 only partially, as it doesn't receive configure notification for change back to the native resolution(other works), but it 24.04 it works with native application as it has both fixes..
It still needs changes on the JDK side.

And there are a few more issues to fix.
The ones that were already discussed before like HiDPI support. It was decided that it's hard to implement for x11 compatibility mode. didn't dig into it, so I'll take a look at it later.
There is another issue with drag and drop when we do a drag and drop within a single window when the CTRL key is pressed. I haven't managed to make a native renderer for this yet.

Alexey:

Regarding high DPI support, I would suggest to discuss it with Nikita Gubarkov via email.
Because he spent a lot of time trying to support high DPI in Xwayland and got some basic limitations.
So he even implemented some logic and we tried to make it work in production.
But we found some really hard corner cases and were not able to fix them.

Alexander:

I seem to recall that it had something to do with multiscreen.
So probably we can implement this support at least for a single screen.

Jonas:

For Gnome 47 or Fedora 41, I've implemented support for what we call native scaling for XWayland(for mutter).
It's more or less similar to how one of the KDE HiDPI X11 modes works where we tell XWayland to draw bigger.
And then we tell X11 clients to draw bigger than our logical monitor layout in the compositor.
And then we take each window from the X server and scale it down to the same scale that we told XWayland to scale itself up to.
This works well for all X11 applications.
We have applications that use what we could call traditional X11 high DPI, where the Xserver has a global scale and windows, all different windows in the application try to follow that scale.
But when I was running this with Intellij, it was using X11, but if you had access to a Wayland connection, it dynamically adjusted its scale to whatever monitor you were using.
It was according to the WL output and that caused problems because whatever it was on a low DPI monitor, there was no way for the compositor to know that it wasn't large anymore, and it got very, very tiny.
Is there a way to detect this kind or disable this from a window manager or Wayland compositor direction so that this doesn't happen if you run Intellij?

Alexey:

I suppose that we're going to revert this logic in the future versions of EDA so it shouldn't be a case anymore.
But it's interesting. Do you have some links to read about this feature of XWayland? So maybe we can find out the way how we can use it.

Jonas:

https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/3567#note_2159205

Alexey:

Yes, we have been collecting feedback from our users with the Vulkan rendering preview and we have found some issues with different environments.
Some users are trying to run Vulkan with hardware accelerated WSL2 and we actually looked into it and it didn't work, but we found out why.
That kind of acceleration is not fully supported by the drivers that they a have implemented.
And in particular, it was an unimportant feature, the XOR mode.
So probably if we make the XOR mode optional, we'll be able to run on WSL with hardware acceleration as well.
There were also some issues with the NVIDIA hardware.
We are currently working on all of those issues that we got from our users.
And we're also currently implementing the feature that helps us test Vulkan.
In the Wayland software mode, we have implemented some of the PixelGrabber functionality for a window.
We go directly to the surface of the window and grab the color from there.
So we are able to test at least one window application without the help of the operating system.
And right now we've almost implemented the approach to grab pixels from Vulkan windows.
And I think we'll have an update next week.
And also Maxim fixed some corner cases in Wayland software mode.
Connected with some dialogues and context menus with the multiple monitors.

Online Zoom Meeting 9am PDT November 8  2024

Attendees: Olivier Fourdan,  Alexey Ushakov, Phil Race, Kevin Rushforth, Maxim Kartashev, Jonas Ådahl, Victor D'yakov, Alexander Zvegintsev

Alexander:

ImageDecoratedDnD.java  now fails on Gnome 47 (Ubuntu 24.10), DnD is aborted when trying to run it with CTRL pressed.

It looks like a non-java issue, as it doesn't reproduce at all when mouse and keyboard interactions are done via XTest(java.awt.Robot)

The issue is reproducible when an actual user performs mouse and keyboard actions with physical devices (which didn't happen before), sometimes it passes though(maybe 1 out of 20 attempts).

Now I am trying to make a native reproducer for it. 

Alexey:

Yes, it actually took longer than we expected.

In Vulkan we have validation layers and despite the fact that it works visually we had a certain amount of validation errors in the pipeline because of some mis-synchronization and things like that and we actually fixed all the validation issues.

They're actually enabled in the fast debug build and you also need to have validation layers installed in the system to make them work. 

We are even sending out some preliminary builds to our users and have gotten some good feedback from them recently.

The Vulkan support provides the list of the hardware devices(if you enable it with the True with capital)

We've got a list of hardware devices that our users are using, some people are using m1 macbooks with Asahi Linux installed, and Vulkan has been working on hardware accelerated drivers for that Linux.

So currently we are putting all the changes we have made in our JBR into the separate branch that Maxim recently updated, which is a jdk21.0.1-wayland.

Obviously it's not as performant now, but we're working on the primitives.

And the ultimate goal is to use an accelerated image for rendering after all the primitives are done.

In the future we will work on the renderer.

Actually, one of our engineers is working on enabling Vulkan on X11. So it looks like it's a fairly simple change, but it's quite tricky. So maybe in this case we will have Vulkan in both pipelines.

Maxim:

One issue wasn't just a bug fix.

It sort of pointed out to me a potential incompatibility between Java and Wayland: Wayland requires proof of user involvement (like serial numbers from user events) for actions like clipboard access.

The protocol, however, doesn’t specify which events are accepted, nor does it provide feedback, leaving developers to experiment and guess.

After extensive testing with various Wayland implementations, I found a solution that works for now.

This is extremely fragile and any other Wayland implementation that does things differently will kill the Java assumption and things will start to break.

We also got rid of the generated code in our development branch, it is now generated on the fly during the build.

You only need to install one extra package, wayland-protocols, which is not hard at all, except for a gGnome-only protocol, which should be brought by hand, as I don't know where to get it on any distribution. I made it optional as it is not essential, it just makes the dialogs work better and more gnome-like.

Jonas:

We recently upstreamed the xdg_dialog protocol, and it has set_modal.

And regarding the clipboard and  the ambiguity of what kind of serial input events are respected and all that, I suggest opening an issue in the Wayland repository issue tracker to see if it can be clarified. At least to know which parts are expected and which are implementation specific.

Online Zoom Meeting 9am PDT October 10  2024

Attendees: Olivier Fourdan,  Alexey Ushakov, Phil Race, Kevin Rushforth, Maxim Kartashev, Victor D'yakov, Alexander Zvegintsev

Alexey:

We have some achievements from the Vulkan implementation.

We finally implemented blit support, for blits, which is necessary to actually present content on the screen.

So currently we actually have a software rendering mode presented via Vulkan.

It's even possible to run our IDEs, for example Idea, in this mode.

We have like in a metal texture pool and able to render used textures as a back buffer in Vulkan and present them on the screen.

We also implemented some primitives in pure rendering mode, but they do not work now because currently we have volatile image as a buffered image.

Now we still have rendering in software mode presented via Vulkan.

We've now implemented high DPI support for this mode, so it's possible to use this hybrid rendering mode with different scale factors.

We're going to be polishing these changes and maybe early next week we'll get them into the Wakefield repository.

From a performance point of view it doesn't make sense, because this hybrid mode is slower than even the pure software rendering mode in wayland.

It's a pretty good step from an implementation point of view, because now we can test our texture pool and drag application windows from screen to screen with a different DPI and see how it reacts and rebuild the texture buffers and stuff like that.

We continue to implement primitives to finally use a real hardware accelerated volatile image as a backbuffer.

Phil:

What platforms are you testing on? Is everything equally stable on the Vulkan pipeline?

Alexey:

Currently it only works for Linux and Wayland. We have discussed other platforms, it is quite easy to extend this support to Windows and X11, it is an isolated part of the pipeline.

Phil:

What kind of hardware are you testing on? Raw, VirtualBox or something like that as well?

Alexey:

We tried it mostly on AMD hardware. So we tried desktop cards and mobile cards.

Unfortunately NVIDIA is very unstable for Wayland.

By the way, Vulkan has its own software rendering pipeline inside, you can actually run Vulkan pipeline in software rendering mode.

We've implemented the ability to switch between the different cards and the software rendering pipeline via the VM property.

You can get the list of devices that you have in the system, you can run on that particular device in the next run and run on that particular device.

Online Zoom Meeting 9am PDT September 12  2024

Attendees: Niels de Graef, Olivier Fourdan,  Alexey Ushakov, Kevin Rushforth, Victor D'yakov, Phil Race, Alexander Zvegintsev

Alexander:

The fix for the DnD issue is verified, everything works fine(including compatibility testing).

Relax The java.awt.Robot specification is now shipped with earlier releases such as 17u, 11u, 8u, so the X11 compatibility mode related fixes can now be backported.

The fix for taking screenshots is also delivered for Java FX, however it introduced two more issues:

JDK-8335468 calling java.awt.Robot#getPixelColor hangs when the JavaFX is running. I have the fix, but it leads to the second issue.
JDK-8335469 Crash when OpenJDK and JavaFX screencast sessions overlap, e.g. getPixelColor calls from FX and JDK robots within 2s (keep sessions open interval to avoid overhead of creating/closing sessions for frequent screenshots). Also have a fix, but it is hard to find an object to sync on in JDK and JavaFX without bringing new extra dependencies between them.

Alexey:

We successfully launched software rendering wayland as a preview in 2024.2 release and we generally get quite good feedback from the users.

They switched to our mode and use it on the wayland, but there are still some issues in different parts input methods for some tricky input frameworks doesn't work. But in general it looks like it's the right way to do it.

Currently we are fixing some bugs and implementing missing parts in input and continue to work on Vulkan rendering.

So we finally got it right and are working on implementing certain rendering primitives and internal stuff. So hopefully we'll get new mode of Vulkan produced in the next release of our products.

Online Zoom Meeting 9am PDT June 20  2024

Attendees: Olivier Fourdan, Jonas Ådahl,  Maxim Kartashev,   Kevin Rushforth, Victor D'yakov, Phil Race

Vacation season : several people out today, and because of same we will not meet in July

Phil : DnD bug https://gitlab.gnome.org/GNOME/mutter/-/issues/3511 is a TCK compliance issue for JDK
Jonas :  DnD bug should be fixed in Gnome 46.3 (6-29-2024)
We may need to follow up with distros to get it backported.

Phil : is libei ready for experimentation
Olivier: yes

Discussion about c/h files generated from wayland protocol files - we should
generate these at build time, not check them in. JDK build will tell a developer
of missing dependencies and how to fix during configure. To build on older OSes
such as are used by CI build systems will instead require a devkit.
These are already extensively used by the JDK build.


Maxim:
Keeping Wakefield repo up to date with WIP.

Figured out what seems a reliable way to count FPS on wayland.

Have someone to work on Input Method support - common request
Jonas : Is this V3 - also there may be a meeting this summer in an occasional
series to discuss requirements for input methods. Maxim will send contact info
so the person working on it can participate if/when it happens.

Online Zoom Meeting 9am PST May 23  2024

Attendees: Niels de Graef, Jonas Adahl,  Alexey Ushakov, Kevin Rushforth, Victor D'yakov, Phil Race, Alexander Zvegintsev

Alexander:

I was mostly focused on stabilizing tests. Many of them were failing, some for the non-obvious reasons.

For example, there is a difference. It looks like there is a difference in implementation between XWayland server and regular X11 server.

We have a couple of tests where we are not releasing the mouse keystroke or the keyboard keystroke.

This looks like the regular X11 server would release these keys when the client disconnects.

But for some reason, XWayland does not. For example, we can get a failing test like a few tests after those tests which failed to release this key.

But just releasing this key is good enough to fix that.

There was some test that counted mouse enter/exit events and making some action to frame like iconify, deiconify, maximize and so on.

For some reason after maximizing both for a frame, mouse coordinates reported by XWayland go crazy and we are missing these events.

Is it our issue or not? But it seems like a bug somewhere outside.

Victor:

And just to explain to everybody on this call, by stabilizing tests, we are talking about open JDK in the mainline, which right now is JDK 23.

So we will fork in two weeks.

So it means at that moment mainline will be 24 and we don't have any plans to work on 23 at that moment unless it's really critical issue.

So it means we will continue to work only on mainline and 23 will be for stabilization.

Phil:

So the tests that Alexander fixes that are unstable are the ones that are only unstable when we run them on Wayland.

And these are, it's all about just tweaks to the tests, not about tweaks to, not about changes to the product code.

Alexey:

We're currently stabilizing the Pure Wayland branch, Pure Wayland toolkit in our fork, JetBrains runtime repository, fixing some issues to be able to release it in preview mode, not by default for the users.

There are some active community members that would like to implement something in Pure Wayland toolkit.

Maybe you've seen the recent pull request to the wakefield repository with server-side declaration.

We're also working on hardware acceleration. 

Phil:

Alexander is going to have to go off and be distracted from the pure Wakefield project focused on AWT in order to help Java FX catch up.

Because FX is going to have the same problem that we have had with the JDK.

We're not going to be able to run FX on upcoming distros unless we actually fix this.

And that will be right after the fork. I already mentioned that in two weeks.

But I, I think 95% of what it shouldn't be anything like as hard to do as what Alexander had to do for AWT.

I think it should be mostly lift the code, move it over here, drop, integrate, hook things up.

So it shouldn't be too bad. I think all the hard learning has been done.

Alexey:

We discussed the high DPI support for XWayland with Nikita.

And based on our experience, we decided not to port it to OpenJDK, because it works in some cases, but in general that approach doesn't work well.

And we have a lot of regressions and we fix them.

And we decided to fix those problems and just move forward.

Phil:

Were the issues with fractional scaling or with integer scaling as well?

Alexey:

If you have monitors with different scales, and if they are placed in a strange position in virtual space, e.g. on top of each other, it may be difficult to calculate the position of the windows, because some information is missing on the X11 side.

So it works for monitors that are placed one side of each other.

But if you put them on top of each other, it causes some problems with, for example, menus or things like that.

Nikita provided the initial solution to the open JDK alias with this fix.

We can actually provide the complete solution just to look at it and see if we can reuse it in the XWayland area. We don't have a lot of resources to work on pure Wayland.

Phil:

If you're basically saying the high DPI stuff isn't working.

Is it because of issues in, is it just a matter of working through the issues on our side?

Or is there something, is there a problem on the compositor side or is there something missing in terms of an API?

Alexey:

Yes, exactly. There's something missing in the API. We cannot get all the information to calculate location in some configuration, some display, multiple display configuration, unless we provide some additional API for XWayland sandbox.

It's less tricky in a pure Wayland mode. So we decided to just switch it.

Alexander:

Speaking of RHEL10, there will be removed Xorg session. Will it be possible to add it back somehow?

Or will it be removed completely?

Niels:

So there's a there's X11, there's the X11 session and there's Xorg.

And so X11 itself is a protocol and will be there in the form of X Wayland.

The Xorg session, that will be gone by in the default trial.

It could be that that's going to be so I think there will be people who will have an Apple package.

So Apple is the repository that people can install, but it's not supported by Red Hat.

So people will probably be able to install it, but it won't be supported.

Phil:

OK, so in other words, you can't just go to the Red Hat package repository, pick something, install it, and get Xorg back.

Niels:

No, indeed.

I recently looked into some of the Wayland protocols that are still under discussion.

So one of the things that piqued my interest was the splash screen protocol.

And one of the things I was wondering is if it makes sense to get some feedback from different people and see what they expect from a splash screen,

because that's one of the common ones. It's one of the things that we know that people are asking for.

For example, for global positioning, they want to have a splash screen that is centered somewhere in the middle of the screen, so it presents the user nicely.

So one of the things we could do is have a separate splash window, and a special protocol for splash windows.

Phil:

But the OpenJDK splash screen protocol and its API has couple of different there's some interesting there's an interesting aspect to it.
You can basically say I'm starting Java and you provide a command line option dash splash colon.
And then my pretty picture dot JPEG or GIF or whatever, or PNG.
And that will get put up by slap bang in the middle of the default screen. (Usually)
And it's done by some native code that gets up and running before the VM is even running.
So you can't there's not even anything running Java code at that point.

In the back in it just forks that off sets runs a thread to do that and then starts them.
That can theoretically be done in a very short amount of time.
And then you still have to boot up this massive thing called the VM and get it to start loading classes and stuff.
Now, once it's got up and loaded classes and all the rest of it, there's a kind of callback mechanism.
And we have splash screen APIs in the AWT.
And there can be a transition for that window to where that window is to actually transition into something where we can then start rendering into it using AWTs.
It doesn't have to be the same window. But the point is, is that for the splash screen to work for us, it's not just it can't just be a fire and forget thing.
We actually need to know where that window is and then be able to seamlessly transition to having an AWT, a normal l top level window.

They're both top level windows, but the first top level window isn't being run by AWT.

It's just a pure native block of that thing on the screen.

No event processing or anything. It's not an AWT window. It's just a native would be a native toolkit or X11 window into which we just call X put image.

Or something. And then but it transitions into this full on Java AWT window in which swing can draw or anything.

And we can put additional information as the application is then starting up.

Another question, what toolkit are you actually going to use, XToolkit or WLToolkit, before you've even run the tool, started to read,  run the code that parses the options that decides that.

We really need to know that window, the position of that splash screen, so that we can ask another window to go to the exact same position and size, not just kind of a habit fire and forget. Otherwise, this, the transition doesn't work.

Alexey:

We actually implemented the splash screen in our WL toolkit.

And there were some tricky architectural decisions to decide which window to show before anything existed, the X11 window or the Wayland window.

And yeah, I don't remember the exact logic, but it was a tricky question

Jonas:

The question was, does the splash screen image thingy share the same display connection as the AWT one, but if AWT isn't even started yet, does it need to, can it inherit the connection, or is that even possible if you don't know what kind of connection it's going to be until like after.

I guess from you there is that since it's going to be X11 or Wayland, when the splash screen shows up, it's not possible to know if it's even possible to share the connection because it's going to be X11 or Wayland.

Phil:

You ask a very good question. I would be inclined to guess. There must be a separate X connection, but I'm reasonably sure that with a bit of work that we could actually use the same connection.

It might be that we don't share the same connection, but I'm pretty sure we could.

Jonas:

 Is the splash screen always the same size as the window itself? Is that what the purpose is that it shows something?

Phil:

That's the purpose. Yes, that's exactly the purpose.

There's the splash screen that gets loaded initially, where there's, you can't do any rendering into it.

You just basically give it a GIF or a PNG and it's displayed via very lightweight native code.

So the splash screen is fast. The splash screen, the window on the screen that has the PNG in it is like supposed to be instant, like 10th of a second.

And then you got another little bit of time while the VM boots everything up. 

It could be drawing a progress bar as it basically ticks off loading all the plugins and all sorts of other things that some big application might do, making connections, reconnecting to the server.

And then, then generally that window then, most, in most applications, that window is not going to become your application window. It is just , the information we're getting going, we're getting going.

And then boom, it disappears because now your main application window is up. And that's the usual way people use it. 

JonasL

It makes me wonder what exactly is missing from having a splash screen protocol that gets you the PNG on the screen, more or less instantaneously.

Is it that you have to morph it to the next one somehow? Like you need to create an association between the splash and the next window? Or what exactly?

Phil:

I have to go off and double check our API, but to see how the transition is done. The same connection, but I don't think it's the same actual window is the point.

It's the same server connection, but it's not the same window. We just replace the window with the new one.

Alexey:

As far as I can remember, in Wayland Toolkit, we actually use the same window.

Because it's impossible to create the similar window with the same, the similar position.

Jonas:

With the splash protocol that I suspected would have to be a different window that you would.

And if there needs to be some kind of association of one knowing about the next, then that's something to take into account in how you design the protocol.

Niels:

Another question is about making a splash screen output only, which means that basically it would not be allowed to get any input.

I don't know what kind of problem that might be for you, or if that would be acceptable to people.

Phil:

The window is definitely "output only", so I think that part fits into the proposed protocol. 

Alexander

And one aspect we didn't mention, probably not an issue, is that the splash screen window should support shaped windows.

Phil:

You probably right. because any window can be a shaped window(e.g. a window that looks to you like a circle rather than a rectangle).

And generally it's done by applying a clip, essentially, between the implementation and the display server.

And there's generally some kind of transparency involved to help make it work. And for the initial image that you're displaying as a PNG, if you're displaying a JPEG, which doesn't support transparency, you can't.

But PNG and GIF could actually have transparent pixels and some clever, some very clever developer could basically craft their nice PNG, which is 500 by 500 or something.

They know exactly what part of the window is transparent and what part isn't.

And then they have to apply the same, apply a clip to create the AWT window when it comes up that matches that exactly.

Otherwise you won't have a seamless transition.

Alexander:

It should also be input transparent, not just visually transparent.

Jonas:

Something to keep in mind about the input region, maybe not relevant for splash screens, but if you have a shaped window that only wants to receive input on the shape, the representation is trivial. It's just a texture or a buffer with an alpha channel, which is supported without any problems.

The problem is that the input region is a region. This means a set of rectangles.

So if you have smooth lines, like bezier curves, that's going to be a lot of regions, a lot, a lot of rectangles. So it's very inefficient.

Phil:

The only saving grace is shaped windows are, not many people use complex shapes for, in practice for, it's it's the usual sort of thing where some nice curved, some nice curved corners or something, right.

But it's, which means that maybe the problem is not quite as bad. You don't see many real user interface applications that have a some kind of a complicated thing that looks like a Mobius strip as your, as your window or something now that's not exactly. I would say it's almost nonexistent.

Online Zoom Meeting 9am PST Apr 25  2024

Attendees: Niels de Graef, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Olivier Fourdan, Phil Race, Maxim Kartashev, Alexander Zvegintsev

Phil:

We made changes in the specification of the Java 21 to accomodate Wayland.

At that time wasn't expected to be backported, but in order to be able to run 8,11,17 which are LTS and will be supported for a significant period of time,

that will overlaps with the lifetime of RedHat Linux 10 without the X11 session support.

Maintenance Releases of Java 8, 11 and 17 specifications, will have these specification changes to allow the implementation update on those.

Alexey:

Do you plan to port all the code that we are going to write to that releases? Pure Wayland Toolkit?

Phil:

Not a chance. This is purely by XWayland, this is a spec change, we not even backporting the Robot changes we did in 21.

At some point RedHat, Oracle, Azul or somebody else might decide to put the Robot changes into their versions of 8, 11 or 17 as well.

After the specification changes in MR, you don't have to backport Robot changes, you are compliant at this point, but it just doesn't work very well.

Alexander:

I've been busy with beforementioned spec backports for the MR, discovered and fixed missing doPriviliged calls on our TokenStorage implementation,

Fix tests that try to interact outside the XWayland server. I also tried to test the XTEST - libei support(should be available on Gnome 45) on beta of Ubuntu 24.04(Gnome 46), but it doesn't work, no confirmation dialog appeared.

I tried the xdotool and java.awt.Robot(XTEST internally). Is this supported only on Fedora?

Olivier:

just a couple of comments on xdotool, it heavily using XTEST for everything, I personally fixed upstream, it might now work.

I believe mutter had support of libei before XWayland, this is still optional, meaning that it depends whether XWayland itself have been build with libei support or not.

So it depends how XWayland was build on Ubuntu, but if they build the XWayland with libeie, than the XTEST would go through the libei as you expected.

Alexander:

Another question, what about the HiDPI support patch for the XWayland? It's probably good time to make it into JDK 23

Alexey:

We actually switched to the pure Wayland implementation now, but I ask Nikita to continue work on this.

I continue my work on Vulkan implementation, hopefully we will have something next week. It is similar to the Metal work, but this API is more complicated.

I have something working, but I need to put it together to at least have something interesting to show.

Niels:

Last meeting you mentioned that software rendering practically complete. It is interesting to know how people evaluate this.

Alexey:

Software rendering works quite good, but we got everighing possible from it. It is slower than hardware rendering. From our perspective is quite good for the preview.

In some cases we have low performance.

Maxim can provide more details.

Maxim:

We have received quite a few reports from users who tried this Wayland Toolkit and run our IDEs on that, discover some bugs.

Actually nobody complains about performance, I don't think that expectations are high, people know that it is not hardware accelerated.

Main complaint was from, I accidently discovered that window resizing was way slower on Wayland Toolkit comparing to XToolkit.

I made a few changes and now we got ~30 fps in 4k when it was ~10 fps before, it's quite smooth, even with software rendering.

We made a bunch of fixes here and ther since last meeting, e.g. one of the guys added support for more than 3 mouse buttons, which was not present before.

Olivier:

There is  a proposal for a new category of Wayland protocols at compatibility. That could be useful for Wakefield project as well. Right now it is more about creating the category.

The reason why I wanted to mention it, somebody mentioned Wakefield as part of the discussion, specifically about something Wakefield does, about a bug with the maximum size of a window.

Is that something that we know about?

Maxim:

If understand correctly, I did a small change relatively recently with regards to limiting the maximum window size, because one of the tests started crashing JVM. It creates a window of size MAX INT.

Now it limits the size twice the size of combined monitors.

Online Zoom Meeting 9am PST Feb 29  2024

Attendees: Jonas Adahl, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Olivier Fourdan, Phil Race, Maxim Kartashev, Alexander Zvegintsev

Alexander:

Not much to report, continuing my aarch64 testing on Wayland, so far so good.

...

Online Zoom Meeting 9am PST Feb 29  2024

Attendees: Niels de Graef, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Olivier Fourdan, Phil Race, Maxim Kartashev

Phil & Alexey : FOSDEM Free Java room session on Wakefield went well.
Session can be viewed here : https://fosdem.org/2024/schedule/event/fosdem-2024-2154-openjdk-project-wakefield-the-wayland-desktop-for-jdk-on-linux/


Alexey : Not much to report, been occupied with other  tasks

Maxim : Ramped up testing so ran all AWT + Swing groups.
With WLToolkit - 70 % of Swing tests passed and about 25% of AWT tests passed
Trying to run in a weston instance via plugin which supports setting location, pixel grabbing,
input events. First obstacle was the stability of weston window manager which crashed in 10 mins.
Have just implemented a  special mode in jtreg which would run each test in a new weston instance

Niels: Perhaps this helper daemon (written by Olivier ?)  could help
 https://gitlab.gnome.org/ofourdan/gnome-ponytail-daemon


Phil : to Neils: Is there any update adding restore token support in RHEL 9.x
Neils : yes, https://issues.redhat.com/browse/RHEL-4526 - shows it is targeted to be fixed in 9.4

Maxim :
Their users when learning that wayland doesn't support setLocation for window positioning are not happy.

Phil: whilst we have screencast for pixel grabbing, libei upcoming for events,
positioning is not something we know the answer to yet. Yes, spec. allows for it,
but it is a practical problem.

Niels: there are proposals around protocols for specific cases
Eg splashscreen protocol but not much upstream interest so far in implementing it
The session management protocol is also proposed
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/18
and there is at least some work going on there :
https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1444
Picture in picture has its own protocol used by firefox

Suggest go to upstream discussion groups and try to  have productive discussions with
But a core API for setting global positioning is a hard pass for the compositor,

Maxim : session mgmt might help Intellij

Maxim: What are Oracle's plans for Wayland
Phil : Finishing up Xwayland support in JDK 22, there's also a need to add support
for XWayland to JavaFX's robot class. Sometime after that we will be able to turn attention
to pure wayland but OpenJDK projects don't have specific deadlines. When they are ready
we can have a JEP to propose to integrate it, but it has to be 100% TCK compliant for that.

Online Zoom Meeting 9am PST Jan 4th  2024

Attendees: Niels de Graef, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Alexander Zvegintsev

Alexander:

In December I did some testing regarding the "automated tests become manual on OL 9 because of confirmation for each screencapture".

How many user interactions (select display, click share screen button) are required to do this?

The answer is ~11k, so we are definitely not going to do this and will wait for the restore_token backport to 9.4.

Regarding not getting a displayChanged event after a setDisplayMode call:

On Ubuntu 22.04, we get a ConfigureNotify for the root window when we switch to a non-native resolution, but we don't get it when we switch back to the native resolution.

On Ubuntu 23.04 and 23.10 we get both notifications.

...

Online Zoom Meeting 9am PST Dec 7th  2023

Attendees: Niels de Graef,  Victor D'yakov,   Alexey Ushakov, Philip Race,  Olivier Fourdan, Maxim Kartashev, Alexander Zvegintsev

Alexander:

I don't have much to share regarding Wakefield, just that the fix for not showing focusable popups has been pushed.

...

Recently Nikita asked about usage of C++ standary library in on Wakefield mainling list, I understand is not quite good for us. What about just C++? Hotspot use it.

Philip:

I do not like C++ at all, hotspots themselves had consequnces it terms of they basically had to abandom supporting some OSes because of C++,

and the Solaris organization said that they deliberately never allowed any part of core solaris to be written in C++. It just takes away a problem if we just stick to C.

...

The question is whether we should use C heavily or just move to java where it is possible?

Philip:

Yes, that sort of the architechture we've had in every other implementation of everything, it all lives in java then you just call native.

If we were to look at a long term future, we could use FFM. The idea there is that you write everything in java, and the native library just accessed throught FFM.

So writing it oriented around the native implementation, makes it a little bit harder to move to the FFM.

So I am not saying that you should adopt FFM for this, but you might want to think of it.

FFM is final as of JDK 22, I already pushed a small usage of it, but I have to jump through hoops. The biggest issue that we have that it needs some time to warm up.

...

The largest chunk of work that I did was to try to merry code in the Idea to the new reality of Wayland, some quirks that I asked on the mailing list,

when some window loses focus and it cannot know which one is going to get it.

The approach of waiting just  a little bit for the other window to aquire focus works quite well(so far in a very limited context), but I am still trying things out.

Philip:

We saw that the Red Hat Linux 10 is planning to drop the X.org and X11 session, do you know when Fedora is going to do so?

...

Since Fedora is a community project, it is up to the community to decide. Fedora can take many more packages, if they want they can have a Cinnamon or other sessions, but we will not have X.org in RHEL10.

Philip:

I have a question regarding the OL/RHEL 9, Red Hat empasized stability, but the version of Gnome doesn't seem to support the restore token functionality that we make important use of it the screen capture code.

It was added in later version of Gnome, which did not make into 9, and it seems to be dooming us never be able to test Red Hat Linux 9 in X11 compatibility mode. If is definitely the case that they won't be ever upgrade Gnome in 9?

...

Online Zoom Meeting 9am PST Nov 9th  2023

Attendees: Niels de Graef,  Victor D'yakov,   Alexey Ushakov,  Kevin Rushforth, Philip Race,  Olivier Fourdan, Maxim Kartashev, Alexander Zvegintsev

Alexander:

We have a regression for my fix for a popup dismiss on focus lost. It was reported by an external developer.

My old fix does not take into account that popup may be focusable(e.g. has TextField). I have a solution and will publish it soon. And we'need to backport it to JDK 21.

...

Most of the progress was not in the wakefield framework, we started experimenting with running our IDEs using the pure Wayland toolkit, so far so good. I had to make quite a few modifications to the IDE code that relied on certain XToolkit related classes and made a lot of assumptions out of that. There are, of course, there are quite a few hiccups here and there.

For example, some of the popups just flash instead of showing themselves at a certain position on the screen, or pasting does not always work for some reason, it starts choking every 5 times. There are weird corner cases and I'll have to go through mostly the IDE code, rather than the toolkit code to iron them out before we go public with this.

It is still usable, performance wise feels about the same as XToolkit. Resource wise, it seems to consume slightly more memory, but I haven't spent too much time looking into that.

My main concern for me at this point is a window placement, and I was wondering if guys from RedHat might have some advice on that. I discovered that gnome has a kind of extension interface for Wayland, where you can set a modality for your window. It has a very nice effect on modal dialogs, it darkens the background, it does not allow to it does not allow the focus to be transferred to the parent window, it shows the dialog centered on top of the window that it is its parent, it is all very well, except that many of our windows are not really dialogs, although they act like ones. For example, the find window, it is not modal, it does not prevent you from focusing a window underneath it, and the problem for me, and I suspect for many users, is going to be that it always opens towards to the top left corner of the screen no matter where you opened it last time.

So I was wondering if there is any way to at least open the window in the same position it was opened before?

Is there anything else I can do, like give the window manager a hint that I want it centered relative to my parent window?

Philip:

I played with the stuff you pushed into the wakefield project repo, it was clearly software rendering. And it is pretty good, reasonably fast, so it doesn't feel like it slow at all.

But there are some other things that are noticable, there was not gtk integration, or gnome integration, so the font were all black and white, because it wasn't picking up any of the desktop settings. The positioning thing was very obvious. The exceptions and messages continually being streamed to the console, but the overall UI was fine.

Regards to the windows positioning, clearly we have a modal and non-modal dialog, and it would be nice if non modal dialog could get the same positioning and treatment as modal dialogs.

...

The xdg-session-management protocol is under discussion and it has not yet been merged. Basically, it allows a client to request that client windows be restored to the same positions as before.

Philip:

We talked about the session manager in the past quite a while ago. I think we need to understand how we actually distinguish Java applications. It doesn't help with the first time, and even if this is absolute positioning, this window that I can recognize comes up with this position, but you really want it relative to your main window, and you window can may be moved around. It is not the something that happens only at startup, e.g. how to recognize a window shown before is the same window shown after, maybe the session manager takes care of all this in some clever way, but I am not sure about it.

...

Online Zoom Meeting 9am PDT 12th Oct 2023

Attendees: Niels de Graef,  Victor D'yakov,  Phil Race,  Olivier Fourdan, Maxim Kartashev, Alexander Zvegintsev

Alexander:

The fix for taking screenshots on a displays with different scales is integrated.

It looks like we will support X11 compatibility mode in the next JDK release(at least for Ubuntu 22.04 for now). Probably we'll also include the latest Oracle Linux and RHEL.

Phil:

Let's try to test the latest Fedora as well, I know it is not on Oracle supported Linux list.

Alexander:

I recall that a fix for HiDPI support in X11 compatibility mode was posted to the Wakefield mailing list for the Wakefield sandbox, I think it is a good time to integrate it to mainline(JDK 22).

Maxim:

I think it was dependent on some other minor thing that I did, but I also don't remember the fate of it. I'll check on that, if the dependency is in the mainline, I'll ping the guy responsible for the fix.

 

Actual rendering with Vulkan started to happen, there is some progress in this area, but not very visual.

I did some minor fixes here and there in the Wayland prototype, I implemented the clipboard support, I started working on DnD, but it turned out that is a lot larger that clipboard support.

We are trying to wrap things up to come up with a more or less working solution to show the JetBrains IDE, actually at this point, the goal is almost on the side IDE,

they do some funny stuff with popups, some of them show up in weird places. It is practically the main major thing that prevents IDE from being usable from pure Wayland toolkit.

Tried to run this thing on KDE, but faced some KDE issues, like XServer is crashing every several minutes, unable to logout without killing something, etc.

We also have some difficulties to setup out testing virtual systems in the cloud to start with Wayland session, no matter of the config, investigation is ongoing.

The amount of tests is not significant, I managed to choose 400 hand picked tests, that still run in pure Wayland session, we have to use som jtreg 7.3 to pass all required env variable.

Those 400 are picked that can be used unmodified, so don't use Robot, don't rely on positioning of a window exactly, etc, but they still make sense.

I have a question related to splashscreen, how do we implement the decision which toolkit to use, especially when the splashscreen to be shown?

Alexander:

We still have the positioning issue for Wayland windows(we can't contol their position), we can use the X11 splashscreen since it doesn't have one(for now). Even if the toolkit will be Wayland later.

Maxim:

The positioning problem is sort of solvable with a large transparent window to the size of the screen, and then draw you splashscreen inside of it, but the problem of having an X11 splashscreen

and then proceeding later with the Wayland toolkit that you have to have the ability to draw over the splashscreen once toolkit get loaded.

You have to have a handle of a window that you showed during the startup, and then that handle has to be reused somehow when you have your AWT,

when you can paint and everything else, so it has to stay within one toolkit.

Niels:

Can we just tear down a splashscreen window and then immediately start a new one?

Maxim:

It is not technically possible, the shared code, the infrastructure that works on all the other platforms is set up such that at a startup you fire up a window

with the ability to save its opaque handle to the window for the code that comes later and be able to draw on that window, other than a static image that we took from a file(jpg, png gif).

Phil:

If we tear it down, at least there'll be some kind of flicker.

For now, if you want just to test this out,  if this in production, I would stick for X11 for a while, and I only switch to the Wayland implementation where there is a high level of confidence.

But you obviously want to test out the Wayland implementation, so I would just use an environment variable.

Maxim:

That is the only thing that touches shared code. Do you have some recomendations what's not to do in shared code?

Because this platform is so specific, so it has two toolkits.

Phil:

For now just do what you have to do in the shared code, and we refine it later.

Niels:

Just wondering in regards to CI problem that you have, so one thing you could do is just run mutter or even gnome shell as headless.

That you can use to immediatelly run it as Wayland compositor.

gnome-shell --headless --wayland --virtual-monitor 1920x1080

Online Zoom Meeting 9am PDT 17th Aug 2023

Attendees:   Niels de Graef,  Victor D'yakov,  Phil Race,  Olivier Fourdan, Maxim Kartashev, Kevin Rushforth, Jonas Adahl, Alexander Zvegintsev

Alexander:

As suggested, I implemented the approach when the session is not closed right away, but after a few seconds of inactivity. It gave a significant boost for certain scenarios, e.g. taking pixel by pixel in 50x50 area took 375s before the fix and only 28s after the fix.

Other than that working on various HiDPI scenarios for taking screen data in X11 compatibility mode.

...

Attendees:  Victor D'yakov,  Alexey Ushakov, Phil Race,  Kevin Rushforth, Maxim Kartashev, Olivier Fourdan, Alexander Zvegintsev

Update from Alexander :

Screencast support for AWT Robot was integrated into JDK 21

...

Attendees:  Victor D'yakov,  Alexey Ushakov, Phil Race,  Kevin Rushforth, Maxim Kartashev, Olivier Fourdan, Jonas Adahl, Alexander Zvegintsev

Alexander:

we have pushed several changes related to the X11 compatibility :

JDK-8307779 - Relax the java.awt.Robot specification

JDK-8280993 - [XWayland] Popup is not closed on click outside of area controlled by XWayland

CSR JDK-8307456 approved for the java.awt.Robot taking screenshots issue wth Screencast. This fix is also targeted to JDK 21.

Drag and Drop does not work in java -> wayland app direction: while this DND fix has been released on the system side(1, 2), changes are still required on the JDK side.

Our code it quite picky for the drop target, and requires it to be a toplevel with WM_STATE property, whereas the Wayland server uses dummy windows without this property set.

I already have a fix for this on the JDK side that I want to put in 21, but first I want to make sure I didn't break anything.  In any case, it will be active when the Wayland session is active.

...

There will be a separate PR for changes: code changes and spec changes to make it easier to review(dependent on each other). I'll post a head-up message to wakefield mailing list.

Niels:

About libEI, progress is still being made, and we are getting closer to something stable and ready to merge . One of the latest protcol changes , to not depend on protobuf anymore.

We are still planning to make it Gnome 45 and Fedora 39.

...

For libEI we are expecting last few API tweaks, obviously we want to do all that before we declare the API stable.

Niels:

How is the pure Wayland stuff going on? I understand that it is not going into JDK21

...