Page History

The Wakefield committers may hold off-line discussions from time-to-time. Since these meetings are just for the committers and invited wayland experts, summary minutes will be recorded here as time permits

Online Zoom Meeting 9am PDT December 5  2024

Attendees: Jonas Ådahl, Olivier Fourdan, Alexey Ushakov, Harshitha Onkar, Phil Race, Kevin Rushforth, Victor D'yakov, Alexander Zvegintsev

Alexander:

Fixed a couple of JavaFX/Swing interop issues related to our screencast usage for receiving screen data:

1. hang when FX is running, it internally runs the gtk main loop, which wasn't accounted for on the JDK side.
So now it handles 3 cases:
    * When there is no GTK main loop running
    * If there is a GTK main loop running, but we are not requesting pixels on its thread
    * If there is a GTK main loop running and we request pixels on its thread

2. crash,
Internally the ScreenCast session keeps open for 2s (both JDK and JFX, and their implementations are almost identical).
When we perform a cleanup to close the session, we internally called pw_deinit before the fix.
It becomes a problem if these sessions overlap in time, so that the second session cleanup crashes when it tries to call pipewire functions without initializing the pipewire system by pw_init

Working on JDK side fix of the emulated resolution change does not come issue.
The system side fix has landed on the Ubuntu 22.04 only partially, as it doesn't receive configure notification for change back to the native resolution(other works), but it 24.04 it works with native application as it has both fixes..
It still needs changes on the JDK side.

And there are a few more issues to fix.
The ones that were already discussed before like HiDPI support. It was decided that it's hard to implement for x11 compatibility mode. didn't dig into it, so I'll take a look at it later.
There is another issue with drag and drop when we do a drag and drop within a single window when the CTRL key is pressed. I haven't managed to make a native renderer for this yet.

Alexey:

Regarding high DPI support, I would suggest to discuss it with Nikita Gubarkov via email.
Because he spent a lot of time trying to support high DPI in Xwayland and got some basic limitations.
So he even implemented some logic and we tried to make it work in production.
But we found some really hard corner cases and were not able to fix them.

Alexander:

I seem to recall that it had something to do with multiscreen.
So probably we can implement this support at least for a single screen.

Jonas:

For Gnome 47 or Fedora 41, I've implemented support for what we call native scaling for XWayland(for mutter).
It's more or less similar to how one of the KDE HiDPI X11 modes works where we tell XWayland to draw bigger.
And then we tell X11 clients to draw bigger than our logical monitor layout in the compositor.
And then we take each window from the X server and scale it down to the same scale that we told XWayland to scale itself up to.
This works well for all X11 applications.
We have applications that use what we could call traditional X11 high DPI, where the Xserver has a global scale and windows, all different windows in the application try to follow that scale.
But when I was running this with Intellij, it was using X11, but if you had access to a Wayland connection, it dynamically adjusted its scale to whatever monitor you were using.
It was according to the WL output and that caused problems because whatever it was on a low DPI monitor, there was no way for the compositor to know that it wasn't large anymore, and it got very, very tiny.
Is there a way to detect this kind or disable this from a window manager or Wayland compositor direction so that this doesn't happen if you run Intellij?

Alexey:

I suppose that we're going to revert this logic in the future versions of EDA so it shouldn't be a case anymore.
But it's interesting. Do you have some links to read about this feature of XWayland? So maybe we can find out the way how we can use it.

Jonas:

https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/3567#note_2159205

Alexey:

Yes, we have been collecting feedback from our users with the Vulkan rendering preview and we have found some issues with different environments.
Some users are trying to run Vulkan with hardware accelerated WSL2 and we actually looked into it and it didn't work, but we found out why.
That kind of acceleration is not fully supported by the drivers that they a have implemented.
And in particular, it was an unimportant feature, the XOR mode.
So probably if we make the XOR mode optional, we'll be able to run on WSL with hardware acceleration as well.
There were also some issues with the NVIDIA hardware.
We are currently working on all of those issues that we got from our users.
And we're also currently implementing the feature that helps us test Vulkan.
In the Wayland software mode, we have implemented some of the PixelGrabber functionality for a window.
We go directly to the surface of the window and grab the color from there.
So we are able to test at least one window application without the help of the operating system.
And right now we've almost implemented the approach to grab pixels from Vulkan windows.
And I think we'll have an update next week.
And also Maxim fixed some corner cases in Wayland software mode.
Connected with some dialogues and context menus with the multiple monitors.

Online Zoom Meeting 9am PDT November 8  2024

Attendees: Olivier Fourdan,  Alexey Ushakov, Phil Race, Kevin Rushforth, Maxim Kartashev, Jonas Ådahl, Victor D'yakov, Alexander Zvegintsev

Alexander:

ImageDecoratedDnD.java  now fails on Gnome 47 (Ubuntu 24.10), DnD is aborted when trying to run it with CTRL pressed.

It looks like a non-java issue, as it doesn't reproduce at all when mouse and keyboard interactions are done via XTest(java.awt.Robot)

The issue is reproducible when an actual user performs mouse and keyboard actions with physical devices (which didn't happen before), sometimes it passes though(maybe 1 out of 20 attempts).

Now I am trying to make a native reproducer for it. 

Alexey:

Yes, it actually took longer than we expected.

In Vulkan we have validation layers and despite the fact that it works visually we had a certain amount of validation errors in the pipeline because of some mis-synchronization and things like that and we actually fixed all the validation issues.

They're actually enabled in the fast debug build and you also need to have validation layers installed in the system to make them work. 

We are even sending out some preliminary builds to our users and have gotten some good feedback from them recently.

The Vulkan support provides the list of the hardware devices(if you enable it with the True with capital)

We've got a list of hardware devices that our users are using, some people are using m1 macbooks with Asahi Linux installed, and Vulkan has been working on hardware accelerated drivers for that Linux.

So currently we are putting all the changes we have made in our JBR into the separate branch that Maxim recently updated, which is a jdk21.0.1-wayland.

Obviously it's not as performant now, but we're working on the primitives.

And the ultimate goal is to use an accelerated image for rendering after all the primitives are done.

In the future we will work on the renderer.

Actually, one of our engineers is working on enabling Vulkan on X11. So it looks like it's a fairly simple change, but it's quite tricky. So maybe in this case we will have Vulkan in both pipelines.

Maxim:

One issue wasn't just a bug fix.

It sort of pointed out to me a potential incompatibility between Java and Wayland: Wayland requires proof of user involvement (like serial numbers from user events) for actions like clipboard access.

The protocol, however, doesn’t specify which events are accepted, nor does it provide feedback, leaving developers to experiment and guess.

After extensive testing with various Wayland implementations, I found a solution that works for now.

This is extremely fragile and any other Wayland implementation that does things differently will kill the Java assumption and things will start to break.

We also got rid of the generated code in our development branch, it is now generated on the fly during the build.

You only need to install one extra package, wayland-protocols, which is not hard at all, except for a gGnome-only protocol, which should be brought by hand, as I don't know where to get it on any distribution. I made it optional as it is not essential, it just makes the dialogs work better and more gnome-like.

Jonas:

We recently upstreamed the xdg_dialog protocol, and it has set_modal.

And regarding the clipboard and  the ambiguity of what kind of serial input events are respected and all that, I suggest opening an issue in the Wayland repository issue tracker to see if it can be clarified. At least to know which parts are expected and which are implementation specific.

Online Zoom Meeting 9am PDT October 10  2024

Attendees: Olivier Fourdan,  Alexey Ushakov, Phil Race, Kevin Rushforth, Maxim Kartashev, Victor D'yakov, Alexander Zvegintsev

Alexey:

We have some achievements from the Vulkan implementation.

We finally implemented blit support, for blits, which is necessary to actually present content on the screen.

So currently we actually have a software rendering mode presented via Vulkan.

It's even possible to run our IDEs, for example Idea, in this mode.

We have like in a metal texture pool and able to render used textures as a back buffer in Vulkan and present them on the screen.

We also implemented some primitives in pure rendering mode, but they do not work now because currently we have volatile image as a buffered image.

Now we still have rendering in software mode presented via Vulkan.

We've now implemented high DPI support for this mode, so it's possible to use this hybrid rendering mode with different scale factors.

We're going to be polishing these changes and maybe early next week we'll get them into the Wakefield repository.

From a performance point of view it doesn't make sense, because this hybrid mode is slower than even the pure software rendering mode in wayland.

It's a pretty good step from an implementation point of view, because now we can test our texture pool and drag application windows from screen to screen with a different DPI and see how it reacts and rebuild the texture buffers and stuff like that.

We continue to implement primitives to finally use a real hardware accelerated volatile image as a backbuffer.

Phil:

What platforms are you testing on? Is everything equally stable on the Vulkan pipeline?

Alexey:

Currently it only works for Linux and Wayland. We have discussed other platforms, it is quite easy to extend this support to Windows and X11, it is an isolated part of the pipeline.

Phil:

What kind of hardware are you testing on? Raw, VirtualBox or something like that as well?

Alexey:

We tried it mostly on AMD hardware. So we tried desktop cards and mobile cards.

Unfortunately NVIDIA is very unstable for Wayland.

By the way, Vulkan has its own software rendering pipeline inside, you can actually run Vulkan pipeline in software rendering mode.

We've implemented the ability to switch between the different cards and the software rendering pipeline via the VM property.

You can get the list of devices that you have in the system, you can run on that particular device in the next run and run on that particular device.

Online Zoom Meeting 9am PDT September 12  2024

Attendees: Niels de Graef, Olivier Fourdan,  Alexey Ushakov, Kevin Rushforth, Victor D'yakov, Phil Race, Alexander Zvegintsev

Alexander:

The fix for the DnD issue is verified, everything works fine(including compatibility testing).

Relax The java.awt.Robot specification is now shipped with earlier releases such as 17u, 11u, 8u, so the X11 compatibility mode related fixes can now be backported.

The fix for taking screenshots is also delivered for Java FX, however it introduced two more issues:

JDK-8335468 calling java.awt.Robot#getPixelColor hangs when the JavaFX is running. I have the fix, but it leads to the second issue.
JDK-8335469 Crash when OpenJDK and JavaFX screencast sessions overlap, e.g. getPixelColor calls from FX and JDK robots within 2s (keep sessions open interval to avoid overhead of creating/closing sessions for frequent screenshots). Also have a fix, but it is hard to find an object to sync on in JDK and JavaFX without bringing new extra dependencies between them.

Alexey:

We successfully launched software rendering wayland as a preview in 2024.2 release and we generally get quite good feedback from the users.

They switched to our mode and use it on the wayland, but there are still some issues in different parts input methods for some tricky input frameworks doesn't work. But in general it looks like it's the right way to do it.

Currently we are fixing some bugs and implementing missing parts in input and continue to work on Vulkan rendering.

So we finally got it right and are working on implementing certain rendering primitives and internal stuff. So hopefully we'll get new mode of Vulkan produced in the next release of our products.

Online Zoom Meeting 9am PDT June 20  2024

Attendees: Olivier Fourdan, Jonas Ådahl,  Maxim Kartashev,   Kevin Rushforth, Victor D'yakov, Phil Race

Vacation season : several people out today, and because of same we will not meet in July

Phil : DnD bug https://gitlab.gnome.org/GNOME/mutter/-/issues/3511 is a TCK compliance issue for JDK
Jonas :  DnD bug should be fixed in Gnome 46.3 (6-29-2024)
We may need to follow up with distros to get it backported.

Phil : is libei ready for experimentation
Olivier: yes

Discussion about c/h files generated from wayland protocol files - we should
generate these at build time, not check them in. JDK build will tell a developer
of missing dependencies and how to fix during configure. To build on older OSes
such as are used by CI build systems will instead require a devkit.
These are already extensively used by the JDK build.


Maxim:
Keeping Wakefield repo up to date with WIP.

Figured out what seems a reliable way to count FPS on wayland.

Have someone to work on Input Method support - common request
Jonas : Is this V3 - also there may be a meeting this summer in an occasional
series to discuss requirements for input methods. Maxim will send contact info
so the person working on it can participate if/when it happens.

Online Zoom Meeting 9am PST May 23  2024

Attendees: Niels de Graef, Jonas Adahl,  Alexey Ushakov, Kevin Rushforth, Victor D'yakov, Phil Race, Alexander Zvegintsev

Alexander:

I was mostly focused on stabilizing tests. Many of them were failing, some for the non-obvious reasons.

For example, there is a difference. It looks like there is a difference in implementation between XWayland server and regular X11 server.

We have a couple of tests where we are not releasing the mouse keystroke or the keyboard keystroke.

This looks like the regular X11 server would release these keys when the client disconnects.

But for some reason, XWayland does not. For example, we can get a failing test like a few tests after those tests which failed to release this key.

But just releasing this key is good enough to fix that.

There was some test that counted mouse enter/exit events and making some action to frame like iconify, deiconify, maximize and so on.

For some reason after maximizing both for a frame, mouse coordinates reported by XWayland go crazy and we are missing these events.

Is it our issue or not? But it seems like a bug somewhere outside.

Victor:

And just to explain to everybody on this call, by stabilizing tests, we are talking about open JDK in the mainline, which right now is JDK 23.

So we will fork in two weeks.

So it means at that moment mainline will be 24 and we don't have any plans to work on 23 at that moment unless it's really critical issue.

So it means we will continue to work only on mainline and 23 will be for stabilization.

Phil:

So the tests that Alexander fixes that are unstable are the ones that are only unstable when we run them on Wayland.

And these are, it's all about just tweaks to the tests, not about tweaks to, not about changes to the product code.

Alexey:

We're currently stabilizing the Pure Wayland branch, Pure Wayland toolkit in our fork, JetBrains runtime repository, fixing some issues to be able to release it in preview mode, not by default for the users.

There are some active community members that would like to implement something in Pure Wayland toolkit.

Maybe you've seen the recent pull request to the wakefield repository with server-side declaration.

We're also working on hardware acceleration. 

Phil:

Alexander is going to have to go off and be distracted from the pure Wakefield project focused on AWT in order to help Java FX catch up.

Because FX is going to have the same problem that we have had with the JDK.

We're not going to be able to run FX on upcoming distros unless we actually fix this.

And that will be right after the fork. I already mentioned that in two weeks.

But I, I think 95% of what it shouldn't be anything like as hard to do as what Alexander had to do for AWT.

I think it should be mostly lift the code, move it over here, drop, integrate, hook things up.

So it shouldn't be too bad. I think all the hard learning has been done.

Alexey:

We discussed the high DPI support for XWayland with Nikita.

And based on our experience, we decided not to port it to OpenJDK, because it works in some cases, but in general that approach doesn't work well.

And we have a lot of regressions and we fix them.

And we decided to fix those problems and just move forward.

Phil:

Were the issues with fractional scaling or with integer scaling as well?

Alexey:

If you have monitors with different scales, and if they are placed in a strange position in virtual space, e.g. on top of each other, it may be difficult to calculate the position of the windows, because some information is missing on the X11 side.

So it works for monitors that are placed one side of each other.

But if you put them on top of each other, it causes some problems with, for example, menus or things like that.

Nikita provided the initial solution to the open JDK alias with this fix.

We can actually provide the complete solution just to look at it and see if we can reuse it in the XWayland area. We don't have a lot of resources to work on pure Wayland.

Phil:

If you're basically saying the high DPI stuff isn't working.

Is it because of issues in, is it just a matter of working through the issues on our side?

Or is there something, is there a problem on the compositor side or is there something missing in terms of an API?

Alexey:

Yes, exactly. There's something missing in the API. We cannot get all the information to calculate location in some configuration, some display, multiple display configuration, unless we provide some additional API for XWayland sandbox.

It's less tricky in a pure Wayland mode. So we decided to just switch it.

Alexander:

Speaking of RHEL10, there will be removed Xorg session. Will it be possible to add it back somehow?

Or will it be removed completely?

Niels:

So there's a there's X11, there's the X11 session and there's Xorg.

And so X11 itself is a protocol and will be there in the form of X Wayland.

The Xorg session, that will be gone by in the default trial.

It could be that that's going to be so I think there will be people who will have an Apple package.

So Apple is the repository that people can install, but it's not supported by Red Hat.

So people will probably be able to install it, but it won't be supported.

Phil:

OK, so in other words, you can't just go to the Red Hat package repository, pick something, install it, and get Xorg back.

Niels:

No, indeed.

I recently looked into some of the Wayland protocols that are still under discussion.

So one of the things that piqued my interest was the splash screen protocol.

And one of the things I was wondering is if it makes sense to get some feedback from different people and see what they expect from a splash screen,

because that's one of the common ones. It's one of the things that we know that people are asking for.

For example, for global positioning, they want to have a splash screen that is centered somewhere in the middle of the screen, so it presents the user nicely.

So one of the things we could do is have a separate splash window, and a special protocol for splash windows.

Phil:

But the OpenJDK splash screen protocol and its API has couple of different there's some interesting there's an interesting aspect to it.
You can basically say I'm starting Java and you provide a command line option dash splash colon.
And then my pretty picture dot JPEG or GIF or whatever, or PNG.
And that will get put up by slap bang in the middle of the default screen. (Usually)
And it's done by some native code that gets up and running before the VM is even running.
So you can't there's not even anything running Java code at that point.

In the back in it just forks that off sets runs a thread to do that and then starts them.
That can theoretically be done in a very short amount of time.
And then you still have to boot up this massive thing called the VM and get it to start loading classes and stuff.
Now, once it's got up and loaded classes and all the rest of it, there's a kind of callback mechanism.
And we have splash screen APIs in the AWT.
And there can be a transition for that window to where that window is to actually transition into something where we can then start rendering into it using AWTs.
It doesn't have to be the same window. But the point is, is that for the splash screen to work for us, it's not just it can't just be a fire and forget thing.
We actually need to know where that window is and then be able to seamlessly transition to having an AWT, a normal l top level window.

They're both top level windows, but the first top level window isn't being run by AWT.

It's just a pure native block of that thing on the screen.

No event processing or anything. It's not an AWT window. It's just a native would be a native toolkit or X11 window into which we just call X put image.

Or something. And then but it transitions into this full on Java AWT window in which swing can draw or anything.

And we can put additional information as the application is then starting up.

Another question, what toolkit are you actually going to use, XToolkit or WLToolkit, before you've even run the tool, started to read,  run the code that parses the options that decides that.

We really need to know that window, the position of that splash screen, so that we can ask another window to go to the exact same position and size, not just kind of a habit fire and forget. Otherwise, this, the transition doesn't work.

Alexey:

We actually implemented the splash screen in our WL toolkit.

And there were some tricky architectural decisions to decide which window to show before anything existed, the X11 window or the Wayland window.

And yeah, I don't remember the exact logic, but it was a tricky question

Jonas:

The question was, does the splash screen image thingy share the same display connection as the AWT one, but if AWT isn't even started yet, does it need to, can it inherit the connection, or is that even possible if you don't know what kind of connection it's going to be until like after.

I guess from you there is that since it's going to be X11 or Wayland, when the splash screen shows up, it's not possible to know if it's even possible to share the connection because it's going to be X11 or Wayland.

Phil:

You ask a very good question. I would be inclined to guess. There must be a separate X connection, but I'm reasonably sure that with a bit of work that we could actually use the same connection.

It might be that we don't share the same connection, but I'm pretty sure we could.

Jonas:

 Is the splash screen always the same size as the window itself? Is that what the purpose is that it shows something?

Phil:

That's the purpose. Yes, that's exactly the purpose.

There's the splash screen that gets loaded initially, where there's, you can't do any rendering into it.

You just basically give it a GIF or a PNG and it's displayed via very lightweight native code.

So the splash screen is fast. The splash screen, the window on the screen that has the PNG in it is like supposed to be instant, like 10th of a second.

And then you got another little bit of time while the VM boots everything up. 

It could be drawing a progress bar as it basically ticks off loading all the plugins and all sorts of other things that some big application might do, making connections, reconnecting to the server.

And then, then generally that window then, most, in most applications, that window is not going to become your application window. It is just , the information we're getting going, we're getting going.

And then boom, it disappears because now your main application window is up. And that's the usual way people use it. 

JonasL

It makes me wonder what exactly is missing from having a splash screen protocol that gets you the PNG on the screen, more or less instantaneously.

Is it that you have to morph it to the next one somehow? Like you need to create an association between the splash and the next window? Or what exactly?

Phil:

I have to go off and double check our API, but to see how the transition is done. The same connection, but I don't think it's the same actual window is the point.

It's the same server connection, but it's not the same window. We just replace the window with the new one.

Alexey:

As far as I can remember, in Wayland Toolkit, we actually use the same window.

Because it's impossible to create the similar window with the same, the similar position.

Jonas:

With the splash protocol that I suspected would have to be a different window that you would.

And if there needs to be some kind of association of one knowing about the next, then that's something to take into account in how you design the protocol.

Niels:

Another question is about making a splash screen output only, which means that basically it would not be allowed to get any input.

I don't know what kind of problem that might be for you, or if that would be acceptable to people.

Phil:

The window is definitely "output only", so I think that part fits into the proposed protocol. 

Alexander

And one aspect we didn't mention, probably not an issue, is that the splash screen window should support shaped windows.

Phil:

You probably right. because any window can be a shaped window(e.g. a window that looks to you like a circle rather than a rectangle).

And generally it's done by applying a clip, essentially, between the implementation and the display server.

And there's generally some kind of transparency involved to help make it work. And for the initial image that you're displaying as a PNG, if you're displaying a JPEG, which doesn't support transparency, you can't.

But PNG and GIF could actually have transparent pixels and some clever, some very clever developer could basically craft their nice PNG, which is 500 by 500 or something.

They know exactly what part of the window is transparent and what part isn't.

And then they have to apply the same, apply a clip to create the AWT window when it comes up that matches that exactly.

Otherwise you won't have a seamless transition.

Alexander:

It should also be input transparent, not just visually transparent.

Jonas:

Something to keep in mind about the input region, maybe not relevant for splash screens, but if you have a shaped window that only wants to receive input on the shape, the representation is trivial. It's just a texture or a buffer with an alpha channel, which is supported without any problems.

The problem is that the input region is a region. This means a set of rectangles.

So if you have smooth lines, like bezier curves, that's going to be a lot of regions, a lot, a lot of rectangles. So it's very inefficient.

Phil:

The only saving grace is shaped windows are, not many people use complex shapes for, in practice for, it's it's the usual sort of thing where some nice curved, some nice curved corners or something, right.

But it's, which means that maybe the problem is not quite as bad. You don't see many real user interface applications that have a some kind of a complicated thing that looks like a Mobius strip as your, as your window or something now that's not exactly. I would say it's almost nonexistent.

Online Zoom Meeting 9am PST Apr 25  2024

Attendees: Niels de Graef, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Olivier Fourdan, Phil Race, Maxim Kartashev, Alexander Zvegintsev

Phil:

We made changes in the specification of the Java 21 to accomodate Wayland.

At that time wasn't expected to be backported, but in order to be able to run 8,11,17 which are LTS and will be supported for a significant period of time,

that will overlaps with the lifetime of RedHat Linux 10 without the X11 session support.

Maintenance Releases of Java 8, 11 and 17 specifications, will have these specification changes to allow the implementation update on those.

Alexey:

Do you plan to port all the code that we are going to write to that releases? Pure Wayland Toolkit?

Phil:

Not a chance. This is purely by XWayland, this is a spec change, we not even backporting the Robot changes we did in 21.

At some point RedHat, Oracle, Azul or somebody else might decide to put the Robot changes into their versions of 8, 11 or 17 as well.

After the specification changes in MR, you don't have to backport Robot changes, you are compliant at this point, but it just doesn't work very well.

Alexander:

I've been busy with beforementioned spec backports for the MR, discovered and fixed missing doPriviliged calls on our TokenStorage implementation,

Fix tests that try to interact outside the XWayland server. I also tried to test the XTEST - libei support(should be available on Gnome 45) on beta of Ubuntu 24.04(Gnome 46), but it doesn't work, no confirmation dialog appeared.

I tried the xdotool and java.awt.Robot(XTEST internally). Is this supported only on Fedora?

Olivier:

just a couple of comments on xdotool, it heavily using XTEST for everything, I personally fixed upstream, it might now work.

I believe mutter had support of libei before XWayland, this is still optional, meaning that it depends whether XWayland itself have been build with libei support or not.

So it depends how XWayland was build on Ubuntu, but if they build the XWayland with libeie, than the XTEST would go through the libei as you expected.

Alexander:

Another question, what about the HiDPI support patch for the XWayland? It's probably good time to make it into JDK 23

Alexey:

We actually switched to the pure Wayland implementation now, but I ask Nikita to continue work on this.

I continue my work on Vulkan implementation, hopefully we will have something next week. It is similar to the Metal work, but this API is more complicated.

I have something working, but I need to put it together to at least have something interesting to show.

Niels:

Last meeting you mentioned that software rendering practically complete. It is interesting to know how people evaluate this.

Alexey:

Software rendering works quite good, but we got everighing possible from it. It is slower than hardware rendering. From our perspective is quite good for the preview.

In some cases we have low performance.

Maxim can provide more details.

Maxim:

We have received quite a few reports from users who tried this Wayland Toolkit and run our IDEs on that, discover some bugs.

Actually nobody complains about performance, I don't think that expectations are high, people know that it is not hardware accelerated.

Main complaint was from, I accidently discovered that window resizing was way slower on Wayland Toolkit comparing to XToolkit.

I made a few changes and now we got ~30 fps in 4k when it was ~10 fps before, it's quite smooth, even with software rendering.

We made a bunch of fixes here and ther since last meeting, e.g. one of the guys added support for more than 3 mouse buttons, which was not present before.

Olivier:

There is  a proposal for a new category of Wayland protocols at compatibility. That could be useful for Wakefield project as well. Right now it is more about creating the category.

The reason why I wanted to mention it, somebody mentioned Wakefield as part of the discussion, specifically about something Wakefield does, about a bug with the maximum size of a window.

Is that something that we know about?

Maxim:

If understand correctly, I did a small change relatively recently with regards to limiting the maximum window size, because one of the tests started crashing JVM. It creates a window of size MAX INT.

Now it limits the size twice the size of combined monitors.

Online Zoom Meeting 9am PST Feb 29  2024

Attendees: Jonas Adahl, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Olivier Fourdan, Phil Race, Maxim Kartashev, Alexander Zvegintsev

Alexander:

Not much to report, continuing my aarch64 testing on Wayland, so far so good.

Maxim:

Very minor update from my side, just mostly bugfixes and small changes, people continue to use our unfinished toolkit on various Linux distros and report lots of bugs and wishes.

For example, many seem to be bothered by the mismatched title bar thing and stuff like that.

Alexey:

We have solved all performance issues with our runtime based on JDK21, so we will release JBR21 runtime based on it inside our product with 24.2 release.

It'll be bundled with Wayland support, ofc is not by default, we may have more testing within our users.

There is some unofficial testing of pure Wayland toolkit within our community. I am personally working on CLion and Idea running on top of Wayland, of course we have some issues, but it generally works.

Even in software mode it has quite good performance.

I also continue to implement hardware accelerated rendering using Vulkan, just got some basic rendering working.

Phil:

Have you guys tested the arbitrary shaped windows support?

Maxim:

Yes, the ruler example works, it is actually semi-transparent by the way.

Online Zoom Meeting 9am PST Feb 29  2024

Attendees: Niels de Graef, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Olivier Fourdan, Phil Race, Maxim Kartashev

Phil & Alexey : FOSDEM Free Java room session on Wakefield went well.
Session can be viewed here : https://fosdem.org/2024/schedule/event/fosdem-2024-2154-openjdk-project-wakefield-the-wayland-desktop-for-jdk-on-linux/


Alexey : Not much to report, been occupied with other  tasks

Maxim : Ramped up testing so ran all AWT + Swing groups.
With WLToolkit - 70 % of Swing tests passed and about 25% of AWT tests passed
Trying to run in a weston instance via plugin which supports setting location, pixel grabbing,
input events. First obstacle was the stability of weston window manager which crashed in 10 mins.
Have just implemented a  special mode in jtreg which would run each test in a new weston instance

Niels: Perhaps this helper daemon (written by Olivier ?)  could help
 https://gitlab.gnome.org/ofourdan/gnome-ponytail-daemon


Phil : to Neils: Is there any update adding restore token support in RHEL 9.x
Neils : yes, https://issues.redhat.com/browse/RHEL-4526 - shows it is targeted to be fixed in 9.4

Maxim :
Their users when learning that wayland doesn't support setLocation for window positioning are not happy.

Phil: whilst we have screencast for pixel grabbing, libei upcoming for events,
positioning is not something we know the answer to yet. Yes, spec. allows for it,
but it is a practical problem.

Niels: there are proposals around protocols for specific cases
Eg splashscreen protocol but not much upstream interest so far in implementing it
The session management protocol is also proposed
https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/18
and there is at least some work going on there :
https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1444
Picture in picture has its own protocol used by firefox

Suggest go to upstream discussion groups and try to  have productive discussions with
But a core API for setting global positioning is a hard pass for the compositor,

Maxim : session mgmt might help Intellij

Maxim: What are Oracle's plans for Wayland
Phil : Finishing up Xwayland support in JDK 22, there's also a need to add support
for XWayland to JavaFX's robot class. Sometime after that we will be able to turn attention
to pure wayland but OpenJDK projects don't have specific deadlines. When they are ready
we can have a JEP to propose to integrate it, but it has to be 100% TCK compliant for that.

Online Zoom Meeting 9am PST Jan 4th  2024

Attendees: Niels de Graef, Kevin Rushforth, Victor D'yakov,  Alexey Ushakov, Alexander Zvegintsev

Alexander:

In December I did some testing regarding the "automated tests become manual on OL 9 because of confirmation for each screencapture".

How many user interactions (select display, click share screen button) are required to do this?

The answer is ~11k, so we are definitely not going to do this and will wait for the restore_token backport to 9.4.

Regarding not getting a displayChanged event after a setDisplayMode call:

On Ubuntu 22.04, we get a ConfigureNotify for the root window when we switch to a non-native resolution, but we don't get it when we switch back to the native resolution.

On Ubuntu 23.04 and 23.10 we get both notifications.

Alexey:

We are continuing our internal testing of Intellij Idea on JBR 21 + pure wayland prototype, so far so good. It is only software rendering for now, but for the IDE purposes it works pretty well.

Also, after our discussion with Phil,, we are rewriting our hardware accelerated code for Vulkan in C, it is less fun than it was before, but we are moving forward with this approach.

Niels:

Alexey, when you mentioned the software rendering, did you get things working?

Alexey:

Yes, it works fine. We use Wayland surfaces, and our java2d C loops to render geometry and stuff like that, Marlin for anti-aliasing rendering. It is able to support all the features that we have in Idea.

The funny thing is that it works better than X11 code.

We want to launch internal and external preview with non-default Wayland toolkit, provide proper VM options for some people who want to test it.

We already have some people from our tracker who have built our branches and launched Idea.

Of course there are some missing features, for example, we don't have some special shaders for better text rendering, but grayscale antialiasing works quite well.

I suppose that most of the work was done by the Wayland server, we present the stuff and in the actual Idea we don't need to pass rendering with hardware acceleration.

Probably with some crazy animations it would suffer.

Niels

I didn't expect such positive surprises at the beginning of the year. If it is necessary, I can ask people from Fedora community to test it. But I do not know how far we are from that.

Alexey:

It is not in the main OpenJDK repository, we do some regular syncs with the wakefield repository. And yes, it would be interesting, but it will probably require some manual patching of the distribution you provide.

Niels:

That's where RPM Fusion could come in, so if people want to test it, here's a random package you can install to see if it works for you, and if it doesn't you can go there and file a bug.

Alexey:

Sounds good, we can provide a current branch or prepare a special one in the wakefield for this purpose.

Online Zoom Meeting 9am PST Dec 7th  2023

Attendees: Niels de Graef,  Victor D'yakov,   Alexey Ushakov, Philip Race,  Olivier Fourdan, Maxim Kartashev, Alexander Zvegintsev

Alexander:

I don't have much to share regarding Wakefield, just that the fix for not showing focusable popups has been pushed.

Alexey:

Recently Nikita asked about usage of C++ standary library in on Wakefield mainling list, I understand is not quite good for us. What about just C++? Hotspot use it.

Philip:

I do not like C++ at all, hotspots themselves had consequnces it terms of they basically had to abandom supporting some OSes because of C++,

and the Solaris organization said that they deliberately never allowed any part of core solaris to be written in C++. It just takes away a problem if we just stick to C.

Alexey:

The question is whether we should use C heavily or just move to java where it is possible?

Philip:

Yes, that sort of the architechture we've had in every other implementation of everything, it all lives in java then you just call native.

If we were to look at a long term future, we could use FFM. The idea there is that you write everything in java, and the native library just accessed throught FFM.

So writing it oriented around the native implementation, makes it a little bit harder to move to the FFM.

So I am not saying that you should adopt FFM for this, but you might want to think of it.

FFM is final as of JDK 22, I already pushed a small usage of it, but I have to jump through hoops. The biggest issue that we have that it needs some time to warm up.

Maxim:

The largest chunk of work that I did was to try to merry code in the Idea to the new reality of Wayland, some quirks that I asked on the mailing list,

when some window loses focus and it cannot know which one is going to get it.

The approach of waiting just  a little bit for the other window to aquire focus works quite well(so far in a very limited context), but I am still trying things out.

Philip:

We saw that the Red Hat Linux 10 is planning to drop the X.org and X11 session, do you know when Fedora is going to do so?

Niels:

Since Fedora is a community project, it is up to the community to decide. Fedora can take many more packages, if they want they can have a Cinnamon or other sessions, but we will not have X.org in RHEL10.

Philip:

I have a question regarding the OL/RHEL 9, Red Hat empasized stability, but the version of Gnome doesn't seem to support the restore token functionality that we make important use of it the screen capture code.

It was added in later version of Gnome, which did not make into 9, and it seems to be dooming us never be able to test Red Hat Linux 9 in X11 compatibility mode. If is definitely the case that they won't be ever upgrade Gnome in 9?

Niels:

We usually keep packages the same, but we do backports when required. Specifically for the restore token I thought we solve this in 9.3(released few weeks ago). But if it isn't there you can always file a bug for that, and we can internally investigate it.

My advice would be to check the latest available.

Online Zoom Meeting 9am PST Nov 9th  2023

Attendees: Niels de Graef,  Victor D'yakov,   Alexey Ushakov,  Kevin Rushforth, Philip Race,  Olivier Fourdan, Maxim Kartashev, Alexander Zvegintsev

Alexander:

We have a regression for my fix for a popup dismiss on focus lost. It was reported by an external developer.

My old fix does not take into account that popup may be focusable(e.g. has TextField). I have a solution and will publish it soon. And we'need to backport it to JDK 21.

Alexey:

We are making a good progress so I will let Maxim give more details about on that, I am more focused on rendering, there is not much progress there.

Maxim:

Most of the progress was not in the wakefield framework, we started experimenting with running our IDEs using the pure Wayland toolkit, so far so good. I had to make quite a few modifications to the IDE code that relied on certain XToolkit related classes and made a lot of assumptions out of that. There are, of course, there are quite a few hiccups here and there.

For example, some of the popups just flash instead of showing themselves at a certain position on the screen, or pasting does not always work for some reason, it starts choking every 5 times. There are weird corner cases and I'll have to go through mostly the IDE code, rather than the toolkit code to iron them out before we go public with this.

It is still usable, performance wise feels about the same as XToolkit. Resource wise, it seems to consume slightly more memory, but I haven't spent too much time looking into that.

My main concern for me at this point is a window placement, and I was wondering if guys from RedHat might have some advice on that. I discovered that gnome has a kind of extension interface for Wayland, where you can set a modality for your window. It has a very nice effect on modal dialogs, it darkens the background, it does not allow to it does not allow the focus to be transferred to the parent window, it shows the dialog centered on top of the window that it is its parent, it is all very well, except that many of our windows are not really dialogs, although they act like ones. For example, the find window, it is not modal, it does not prevent you from focusing a window underneath it, and the problem for me, and I suspect for many users, is going to be that it always opens towards to the top left corner of the screen no matter where you opened it last time.

So I was wondering if there is any way to at least open the window in the same position it was opened before?

Is there anything else I can do, like give the window manager a hint that I want it centered relative to my parent window?

Philip:

I played with the stuff you pushed into the wakefield project repo, it was clearly software rendering. And it is pretty good, reasonably fast, so it doesn't feel like it slow at all.

But there are some other things that are noticable, there was not gtk integration, or gnome integration, so the font were all black and white, because it wasn't picking up any of the desktop settings. The positioning thing was very obvious. The exceptions and messages continually being streamed to the console, but the overall UI was fine.

Regards to the windows positioning, clearly we have a modal and non-modal dialog, and it would be nice if non modal dialog could get the same positioning and treatment as modal dialogs.

Niels:

As far as I know, Wayland does not provide any kind of positioning, as you already know. Definitely for modal windows, transient for kind of relationship.

It is not GTK that decides to put one window on top of the other, it is actually the Gnome compositor, and I think, for example, a tiling compositor may decide not to have modal windows, since it may not have a concept of it.

At this point there is no way to specify that one window should be on top of the another.

Maxim:

I understand that it is in general it is an unsolvable problem, as you said for a different kinds of compositors, for some of them the whole concept just does not make sense, but I think I saw some talk about maybe letting the window position to where it was last dissapered to at least give the user that consistency.

Niels:

The xdg-session-management protocol is under discussion and it has not yet been merged. Basically, it allows a client to request that client windows be restored to the same positions as before.

Philip:

We talked about the session manager in the past quite a while ago. I think we need to understand how we actually distinguish Java applications. It doesn't help with the first time, and even if this is absolute positioning, this window that I can recognize comes up with this position, but you really want it relative to your main window, and you window can may be moved around. It is not the something that happens only at startup, e.g. how to recognize a window shown before is the same window shown after, maybe the session manager takes care of all this in some clever way, but I am not sure about it.

Maxim:

First of all the problem of restoring a session and the problem of restoring a window position within the same session are slightly different. 

For the latter, if you just want a window to appear where it was last time, and you haven't closed your session, I would be satisfied if a window manager would restore a window that has the same parent-child relationship with another window, and the same size as the last time this happened, but I'm dreaming.

Niels:

It makes sense for some compositors, not sure if we do that. I think the Mutter already does some restoration things already, like when you unplug it and plug your external monitor back in, it also tries to do smart things.

At this point, the modal dialog is probably the best choice. The Gnome side has something centered in the middle, but it may not work across compositors.

Olivier:

I wanted to mention that the feature that centers modal dialogs in the center of its parent window is actually a notion, that can be disabled by user, so we can't rely on it, because some people just like me have disabled it. And it is not a standard, other compositors (e.g. KDE) may not be doing that.

Maxim:

Speaking of KDE, it worked by default, it tends to center a child window on top of the parent anyway. And it even centered our splash screen, which is essentially a borderless window. I understand that we should not rely on this all the time, but it is as good as it gets, if it works most of the time, or almost always is good enough for me.

Speaking of popups being different, there was a curious problem with popups that I ran into, while making the IDE work under Wayland. One of the most popular dialogs in the IDE is the search everywhere which comes up with double shift tap. It is actually implemented as a popup, and while I was implemented the ability to move it on the screen, with popups you cannot ask the compositor to do that for you, but you do have the ability to do it programatically.  There is a curious problem with the interactive movement, when you move it together with your mouse, the window moves and the relative position(to the window that just moved) of the mouse also changes, it does not follow your mouse, it just jumps very erratically. You have to be very slow and careful to grab it.

Looks like there is no solution for that, other than to remake it with a proper dialog.

Alexander:

At the last meeting we raised a question about getting the Nikita's HiDPI fix into the mainline, is there an update on that? We have less than a month to get it into JDK 22.

Maxim:

If I understand correctly, just yesterday a problem was discovered with this fix, or rather an opportunity to improve it.

An XFCE user discovered a problem with window positioning, and it is quite persistent.

Online Zoom Meeting 9am PDT 12th Oct 2023

Attendees: Niels de Graef,  Victor D'yakov,  Phil Race,  Olivier Fourdan, Maxim Kartashev, Alexander Zvegintsev

...

I think it was dependent on some other minor thing that I did, but I also don't remember the fate of it. I'll check on that, if the dependency is in the mainline, I'll ping the guy responsible for the fix.

 

Actual rendering with Vulkan started to happen, there is some progress in this area, but not very visual.

I did some minor fixes here and there in the Wayland prototype, I implemented the clipboard support, I started working on DnD, but it turned out that is a lot larger that clipboard support.

We are trying to wrap things up to come up with a more or less working solution to show the JetBrains IDE, actually at this point, the goal is almost on the side IDE,

they do some funny stuff with popups, some of them show up in weird places. It is practically the main major thing that prevents IDE from being usable from pure Wayland toolkit.

Tried to run this thing on KDE, but faced some KDE issues, like XServer is crashing every several minutes, unable to logout without killing something, etc.

We also have some difficulties to setup out testing virtual systems in the cloud to start with Wayland session, no matter of the config, investigation is ongoing.

The amount of tests is not significant, I managed to choose 400 hand picked tests, that still run in pure Wayland session, we have to use som jtreg 7.3 to pass all required env variable.

Those 400 are picked that can be used unmodified, so don't use Robot, don't rely on positioning of a window exactly, etc, but they still make sense.

I have a question related to splashscreen, how do we implement the decision which toolkit to use, especially when the splashscreen to be shown?

...