[Proposal] jlink plugin which replaces dynamic shared libraries

[Severin Gehwolf]

Hi,

Apologies if this isn't the right list. Please let me know where jlink
related issues are being discussed.

The Problem:
-----------------------------------------------
This issue originally surfaced in our packaged OpenJDK 11 builds for
Fedora. If a user created a custom image from the packaged JDK, the
resulting custom image would be huge in size because of non-stripped
debug symbols in the packaged binaries (~500MB vs. ~50MB).

Native libraries and executables in the packaged JDK image would be
properly stripped of debug symbols, but copies of those files in the
jmod files would not be. That has to do with a) How we build for the
Fedora distribution: --with-native-debug-symbols=internal configure
switch. b) When the stripping happens. In the Fedora case, b) happens
external to the OpenJDK build by the RPM build system *after* the
OpenJDK build actually completes. This means copies of native libraries
and executables with full debug symbols would already be "zipped up" in
jmod files when the distribution build's stripping process starts.
Hence, the distribution build system doesn't "see" DSO/EXE copies in
jmod files in the JDK images directory. They are archived in jmod
files. Because of this we end up with properly stripped libraries and
executables in the default, extracted JDK image, but DSOs/EXEs in the
'jmods' directory would still have full debug symbols internal to the
binaries. Yet, when jlink is being run, jmod files in directory 'jmods'
are being used for creating custom modular JDK images.

The original downstream issue is:
https://bugzilla.redhat.com/show_bug.cgi?id=1652177


Proposed Solution:
-----------------------------------------------
The solution we came up with is a new jlink plugin which - at runtime -
replaces matching DSOs from jmod files with versions from the JDK image
(or a JDK image of the same version pointed to by a property). As a
result, properly stripped DSOs are being used for the custom JDK image
and the size issue goes away. We are aware that this is a rather
distribution specific use-case, but it might be useful for other
downstream consumers (Debian?) too. That's why I'm proposing this here.
Would there be interest in having this jlink plugin upstream? If so,
I'd create a bug and will propose a formal RFR.

webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/jlink-native-replace-plugin/webrev/

Thoughts?

Thanks,
Severin

[Alan Bateman]

Post by Severin Gehwolf
-----------------------------------------------
The solution we came up with is a new jlink plugin which - at runtime -
replaces matching DSOs from jmod files with versions from the JDK image
(or a JDK image of the same version pointed to by a property). As a
result, properly stripped DSOs are being used for the custom JDK image
and the size issue goes away. We are aware that this is a rather
distribution specific use-case, but it might be useful for other
downstream consumers (Debian?) too. That's why I'm proposing this here.
Would there be interest in having this jlink plugin upstream? If so,
I'd create a bug and will propose a formal RFR.
webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/jlink-native-replace-plugin/webrev/
Thoughts?

If I read this correctly then it makes assumption that jlink is run on a
JDK build that matches exactly the build of the packaged modules on the
module path specified to the tool - is that right? If so then it will
work for the scenario you described but it won't work in general as the
builds may not match.  Also it could never work when cross targeting,
e.g. using jlink on linux-x64 to create a run-time image for an embedded
linux-arm.

You mail makes me wonder if it would be feasible to have a plugin that
does the stripping, maybe you've looked into that already?

-Alan

[Severin Gehwolf]

Hi Alan,

Post by Alan Bateman

Post by Severin Gehwolf
-----------------------------------------------
The solution we came up with is a new jlink plugin which - at runtime -
replaces matching DSOs from jmod files with versions from the JDK image
(or a JDK image of the same version pointed to by a property). As a
result, properly stripped DSOs are being used for the custom JDK image
and the size issue goes away. We are aware that this is a rather
distribution specific use-case, but it might be useful for other
downstream consumers (Debian?) too. That's why I'm proposing this here.
Would there be interest in having this jlink plugin upstream? If so,
I'd create a bug and will propose a formal RFR.
webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/jlink-native-replace-plugin/webrev/
Thoughts?

If I read this correctly then it makes assumption that jlink is run on a
JDK build that matches exactly the build of the packaged modules on the
module path specified to the tool - is that right?

The inherent assumption is that jlink is run on a JDK build that has a
module set at least equal to the modules specified to the tool. Other
than that, yes.

Post by Alan Bateman
If so then it will
work for the scenario you described but it won't work in general as the
builds may not match. Also it could never work when cross targeting,
e.g. using jlink on linux-x64 to create a run-time image for an embedded
linux-arm.

Yes, understood.

Post by Alan Bateman
You mail makes me wonder if it would be feasible to have a plugin that
does the stripping, maybe you've looked into that already?

We've considered it, but I haven't looked into it.

Thanks,
Severin

[Alan Bateman]

Post by Severin Gehwolf
The inherent assumption is that jlink is run on a JDK build that has a
module set at least equal to the modules specified to the tool. Other
than that, yes.

The approach would need integrity as it would be too easy to end up with
a mismatch between the native libraries and the other resources in the
module.

Post by Severin Gehwolf

Post by Alan Bateman
You mail makes me wonder if it would be feasible to have a plugin that
does the stripping, maybe you've looked into that already?

We've considered it, but I haven't looked into it.

I think it's worth exploring, even if it means requiring the `strip`
tool on the system.

-Alan.

[Severin Gehwolf]

Post by Alan Bateman

Post by Severin Gehwolf

Post by Alan Bateman
You mail makes me wonder if it would be feasible to have a plugin that
does the stripping, maybe you've looked into that already?

We've considered it, but I haven't looked into it.

I think it's worth exploring, even if it means requiring the `strip`
tool on the system.

If that's an option I'll look into it:
https://bugs.openjdk.java.net/browse/JDK-8214796

Thanks,
Severin