Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

[Martijn Verburg]

Hi All,

Stephen's comments are in his blog post: http://blog.joda.org/2018/03/j
pms-negative-benefits.html

He does raise some interesting points. Perhaps it's worth spending some
time before 11 is out to see if there is a combination of Maven and JPMS
changes / clarifications that can address some of these pain points.

If Stephen is missing something, then that's an area we could document
(and/or publicise better if already documented. That is something I and
others can help to address through the usual JUG Leaders / Champions / Info
Q etc channels.

@Stephen are you able to share the source code? It may be that there is
another
path that could be explored to mitigate some of the challenges you faced.

@Robert - I've been thinking about the difficulty of getting enough
engineering time for Maven committers to add support for major features
like JPMS. I'll start a separate thread with you on that.

Cheers,
Martijn

[David Lloyd]

On Fri, Mar 23, 2018 at 5:19 AM, Martijn Verburg

Post by Martijn Verburg
Hi All,
Stephen's comments are in his blog post: http://blog.joda.org/2018/03/j
pms-negative-benefits.html
He does raise some interesting points. Perhaps it's worth spending some
time before 11 is out to see if there is a combination of Maven and JPMS
changes / clarifications that can address some of these pain points.
If Stephen is missing something, then that's an area we could document
(and/or publicise better if already documented. That is something I and
others can help to address through the usual JUG Leaders / Champions / Info
Q etc channels.
@Stephen are you able to share the source code? It may be that there is
another
path that could be explored to mitigate some of the challenges you faced.
@Robert - I've been thinking about the difficulty of getting enough
engineering time for Maven committers to add support for major features
like JPMS. I'll start a separate thread with you on that.

Case in point: I've had
https://github.com/apache/maven-compiler-plugin/pull/1 sitting out
there since January. This commit is a key to what is at least a
somewhat workable strategy for MR JARs and multi-JVM build and testing
for us. Robert has said that the change is acceptable, but indicated
that there may be a resource problem in terms of getting the plugin to
a releasable state.

I (and others I work with) are more than willing to contribute what
extra time we have to fixing or improving things in Maven and its
plugins, and even in the JDK itself, but the other side of that is
that the fixes do need to be accepted in a timely manner. With Java
11 just around the corner, the time is now.

[Stephen Colebourne]

This post might be inappropriate. Click to display it.

[Remi Forax]

----- Mail original -----

Envoyé: Vendredi 23 Mars 2018 13:51:06
Objet: Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

http://blog.joda.org/2018/03/jpms-negative-benefits.html

Firstly, I want to emphasise that my goal is effectively that of a
retrospective, to examine what hasn't worked so well and to improve
things from where we are. The jigsaw team did a great job in getting
the feature out at all - I want to make sure it is used rather than
ignored or used as a blocker to progressing the JDK.

@Stephen are you able to share the source code?

https://github.com/ThreeTen/threeten-extra
https://github.com/JodaOrg/joda-parent
https://github.com/JodaOrg/joda-convert
https://github.com/JodaOrg/joda-beans
While some of the problems are tool-based, the most fundamental issues
are about JPMS itself.
At the heart of the problem is the split between class-path and
module-path. Since this split has happened now, and can't "unhappen"
what is needed is a way to make it easier to manage (fix the problems
the split has created).
As it stands, a library developer cannot control whether they run on
the class-path or module-path. This increases the bug-surface of the
library, and requires testing in two different environments (which is
not widely known). Without being able to insist that a library is on
the module-path it is also clear that the benefits of strong
encapsulation and reliable configuration don't apply to library
consumers.
My belief is that a way needs to be found for a library author to
insist that their library is run as a named module. There are probably
a number of ways this could be achieved - I'm interested in whether
the change makes things better, not what the specific change is.
One approach is to say that modular jar files are always treated as
- a modular jar on the class-path is treated as being named, not part
of the unnamed module It is therefore encapsulated, but depends on the
unnamed module (class-path), so does not have full reliable
configuration. There is the potential for some incompatibility with
this change where code that uses the modular jar now can't access the
encapsulated packages, but this is a Good thing (as the library author
specifically coded for that encapsulation). Any incompatibilities are
smaller than JPMS has already caused, and could be managed with the
existing command line flags. I would hope this does not require a JVM
spec change to achieve.
- a module author could mark a module as "strict" in module-info.java
so that it is not permitted to be on the class-path. This does not
force the whole application to move to the module-path - only strict
modules and their dependencies would need to move. At runtime, if a
strict module is found on the class-path an error occurs. I suspect
this requires a JVM spec change, so may be harder.
There may well be more approaches. My gut feeling is that the first
approach is the better one, being relatively simple and implementable,
but of course I may be wrong.
Ultimately, the requirement is that library authors who go to the
effort of adding module-info.java should see some benefits in doing
so, where today they only have increased cost through running as both
an unnamed and named module.
thanks
Stephen

You can check dynamically if a class is in the unamed module (classpath) or not and emit an error message.

This is exactly what the VM does when the class version is wrong.

Rémi

[David Lloyd]

On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne

Post by Stephen Colebourne
One approach is to say that modular jar files are always treated as
- a modular jar on the class-path is treated as being named, not part
of the unnamed module It is therefore encapsulated, but depends on the
unnamed module (class-path), so does not have full reliable
configuration. There is the potential for some incompatibility with
this change where code that uses the modular jar now can't access the
encapsulated packages, but this is a Good thing (as the library author
specifically coded for that encapsulation). Any incompatibilities are
smaller than JPMS has already caused, and could be managed with the
existing command line flags. I would hope this does not require a JVM
spec change to achieve.

This would cause problems for containers which do not use JPMS yet
(which is to say: containers); such frameworks may behave poorly
(having an expectation that they were loaded as JPMS modules) or fail
to load (if there is some kind of hypothetical enforcement at a JVM
level).

Post by Stephen Colebourne
- a module author could mark a module as "strict" in module-info.java
so that it is not permitted to be on the class-path. This does not
force the whole application to move to the module-path - only strict
modules and their dependencies would need to move. At runtime, if a
strict module is found on the class-path an error occurs. I suspect
this requires a JVM spec change, so may be harder.

I don't think this would resolve the container problem, though it
would allow containers to refuse to load such JARs. I'm not sure
that's a net improvement for users though.

Post by Stephen Colebourne
Ultimately, the requirement is that library authors who go to the
effort of adding module-info.java should see some benefits in doing
so, where today they only have increased cost through running as both
an unnamed and named module.

I agree, we've effectively doubled the testing effort for frameworks
due to JPMS, and tripled in the case where frameworks must include MR
JAR supplements for 8 vs 9+. In a word, this sucks.

[Stephen Colebourne]

Post by David Lloyd
On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne

Post by Stephen Colebourne
One approach is to say that modular jar files are always treated as
- a modular jar on the class-path is treated as being named, not part
of the unnamed module It is therefore encapsulated, but depends on the
unnamed module (class-path), so does not have full reliable
configuration. There is the potential for some incompatibility with
this change where code that uses the modular jar now can't access the
encapsulated packages, but this is a Good thing (as the library author
specifically coded for that encapsulation). Any incompatibilities are
smaller than JPMS has already caused, and could be managed with the
existing command line flags. I would hope this does not require a JVM
spec change to achieve.

This would cause problems for containers which do not use JPMS yet
(which is to say: containers); such frameworks may behave poorly
(having an expectation that they were loaded as JPMS modules) or fail
to load (if there is some kind of hypothetical enforcement at a JVM
level).

The classes in the modular jar file would still be in the same
classloader (because same classloader module loading is the standard
strategy). Other than encapsulating internal packages, the container
really shouldn't see any changes should it? At some point containers
will support JPMS anyway.

Stephen

[David Lloyd]

On Fri, Mar 23, 2018 at 8:49 AM, Stephen Colebourne

Post by Stephen Colebourne

Post by David Lloyd
On Fri, Mar 23, 2018 at 7:51 AM, Stephen Colebourne

Post by Stephen Colebourne
One approach is to say that modular jar files are always treated as
- a modular jar on the class-path is treated as being named, not part
of the unnamed module It is therefore encapsulated, but depends on the
unnamed module (class-path), so does not have full reliable
configuration. There is the potential for some incompatibility with
this change where code that uses the modular jar now can't access the
encapsulated packages, but this is a Good thing (as the library author
specifically coded for that encapsulation). Any incompatibilities are
smaller than JPMS has already caused, and could be managed with the
existing command line flags. I would hope this does not require a JVM
spec change to achieve.

This would cause problems for containers which do not use JPMS yet
(which is to say: containers); such frameworks may behave poorly
(having an expectation that they were loaded as JPMS modules) or fail
to load (if there is some kind of hypothetical enforcement at a JVM
level).

The classes in the modular jar file would still be in the same
classloader (because same classloader module loading is the standard
strategy). Other than encapsulating internal packages, the container
really shouldn't see any changes should it? At some point containers
will support JPMS anyway.

The container won't see any changes, but the framework will if the
container is not creating a JPMS module for it.

At some point, containers _may_ support JPMS. However, this seems to
be turning into quite a technical challenge in practice. And until
there are some implementations supporting JPMS, there will be no specs
for supporting JPMS (at least, unless the world has gone mad, which
perhaps it has).

[Jason Greene]

Post by Stephen Colebourne
Without being able to insist that a library is on
the module-path it is also clear that the benefits of strong
encapsulation and reliable configuration don't apply to library
consumers.

Why would a framework author ever want to cut off 99% of their users that are not using JPMS, most of which are completely happy with the arguably more flexible classpath? I think the only way you solve this problem is if you resurrect (and extend) some of the original goals to support common unification and bridging across runtime models. Then the industry as a whole could be pushing adoption.

-Jason