Experiments with Incorporating Graal changes into Sumatra JDK Build

During our development phase, we build a Sumatra JDK using the two-step process outlined in Sumatra JDK build instructions

1. Build the Sumatra-modified JDK without graal functionality
   1.  there are a small number of classes (less than 10) from java.util.stream that get modified to enable the gpu offload.  If offload is detected it then calls into graal.
      
      
2. With JAVA_HOME set to the Sumatra-modified JDK, build graal "server" compiler. This uses its own special "mx build" command and produces a new JDK image with the following changes:
   1. from the graal java sources, a graal.jar in the jre/lib directory
   2. from the graal hotspot sources, a new hotspot libjvm.so in the jre/lib/amd64/server directory

We wanted to estimate the footprint of including the graal functionality directly into the Sumatra JDK build.  That is, to add enough classes to sumatra-dev/jdk and sumatra-dev/hotspot so that we could skip step 2 above but still build a functional sumatra JDK able to offload to hsail.  So the necessary classes from graal.jar would be built into rt.jar

Ideally this would include only that part of graal java and graal hotspot that would be necessary for an HSAIL backend.   A webrev from before vs. after then would let us see what the hsail/graal additional footprint would be. For a first cut, we wouldn't spend too much time getting down to the absolute minimum footprint.

What was Done

So far we have some success.

• I modified mx/projects so that what I thought were the unnecessary project sets would not be included in graal.jar (so truffle, sparc, ptx.  Note that we do need the graal amd64 backend because of the way hsail deoptimization is done).
  
  
• Then I took the graal java sources for the classes that were still in graal.jar and added them to the sumatra-dev/jdk.
  
  
•  I also took the hotspot src tree from graal (which is a modification of hs25-b63, graal has stayed at that hotspot revision lately) and used that as an overlay of the vanilla hs25-b63 tag in sumatra-dev/hotspot.
  • There were a couple of graal-generated files graalRuntime.inline.hpp and HotspotVMConfig.inline.hpp which I also manually added to sumatra-dev/hotspot
  • Some minor modifications to the hotspot/Makefile to make the graal flavor be the default flavor.

With the above we are able to build the release build and execute HSAIL offloads there. (there is some problem with building the fastdebug build involving libjsig.so).

Brief Summary of Footprint

All changes are in either sumatra-dev/jdk, or in sumatra-dev/hotspot

sumatra-dev/jdk

The changes are the addition of the src files that made up the abbreviated graal.jar.  As mentioned earlier this includes the graal AMD64 backend because of the way we do deoptimization thru trampoline code.and the way the hsail backend defers to the host backend for things that it doesn't override.  It is possible that some of the files in core graal are not really used by the HSAIL backend of the AMD64 bakckend trampoline code but this fine tuning has not been done yet.

• abbreviated graal sources, 171K lines of code,  1083 java files
• full graal.jar                            209K lines of code,  1319 java files
• rest of JDK                                                                8200 java files

hg diff-stat of sumatra-dev/jdk with added graal files

sumatra-dev/hotspot

The changes are the changes that the graal project for their own hotspot fork.  At the time of this experiment, graal was still forked off of hs25-b63 so I used that as a starting point for this diff.

hg diff-stat of sumatra-dev/hotspot vanilla hs25-b63