Discussion:
Proposal (revised): Allow illegal access to internal APIs by default in JDK 9
m***@oracle.com
2017-06-05 18:45:27 UTC
Permalink
(Thanks for all the feedback on the initial proposal [1]. Here's a
revised version, which incorporates some of the suggestions received and
includes a bit more advice. An implementation is already available for
testing in the Jigsaw EA builds [2]. Further comments welcome!)

Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention to the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given at run time in JDK 8.

To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal-access operations to
internal APIs from code on the class path by default in JDK 9, and to
disallow them in a future release. This will enable smoother application
migration in the near term, yet still enable and motivate the maintainers
of libraries and frameworks that use JDK-internal APIs to fix their code
to use proper exported APIs.

New command-line option: `--illegal-access`
-------------------------------------------

The recently-introduced `--permit-illegal-access` option [3] will be
replaced by a more-general option, `--illegal-access`. This option takes
a single keyword parameter to specify a mode of operation, as follows:

`--illegal-access=permit`

This mode opens each package in each module in the run-time image to
code in all unnamed modules, i.e., code on the class path, if that
package existed in JDK 8. This enables both static access, i.e., by
compiled bytecode, and deep reflective access, via the platform's
various reflection APIs.

The first reflective-access operation to any such package causes a
warning to be issued, but no warnings are issued after that point.
This single warning describes how to enable further warnings.

This mode will be the default for JDK 9. It will be removed in a
future release.

`--illegal-access=warn`

This mode is identical to `permit` except that a warning message is
issued for each illegal reflective-access operation. This is roughly
equivalent to the current `--permit-illegal-access` option.

`--illegal-access=debug`

This mode is identical to `warn` except both a warning message and a
stack trace are issued for each illegal reflective-access operation.
This is roughly equivalent to combining `--permit-illegal-access`
with `-Dsun.reflect.debugModuleAccessChecks`.

`--illegal-access=deny`

This mode disables all illegal-access operations except for those
enabled by other command-line options, e.g., `--add-opens`.

This mode will become the default in a future release.

When `deny` becomes the default mode then `permit` will likely remain
supported for at least one release, so that developers can continue to
migrate their code. The `permit`, `warn`, and `debug` modes will, over
time, be removed, as will the `--illegal-access` option itself. (For
launch-script compatibility the unsupported modes will most likely just
be ignored, after issuing a warning to that effect.)

How to prepare for the future
-----------------------------

The default mode, `--illegal-access=permit`, is intended to make you
aware when you have code on the class path that reflectively accesses
some JDK-internal API at least once. To learn about all such accesses
you can use the `warn` or `debug` modes. For each library or framework
on the class path that requires illegal access you have two options:

- If the component's maintainers have already released a new,
fixed version that no longer uses JDK-internal APIs then you
can consider upgrading to that version.

- If the component still needs to be fixed then we encourage you
to contact its maintainers and ask them to replace their use
of JDK-internal APIs with proper exported APIs [4].

If you must continue to use a component that requires illegal access then
you can eliminate the warning messages by using one or more `--add-opens`
options to open just those internal packages to which access is required.

To verify that your application is ready for the future, run it with
`--illegal-access=deny` along with any necessary `--add-opens` options.
Any remaining illegal-access errors will most likely be due to static
references from compiled code to JDK-internal APIs. You can identify
those by running the `jdeps` tool with the `--jdk-internals` option.
(JDK 9 does not issue warnings for illegal static-access operations
because that would require deep JVM changes and degrade performance.)

Warning messages
----------------

The warning message issued when an illegal reflective-access operation is
detected has the following form:

WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM

where:

- $PERPETRATOR is the fully-qualified name of the type containing
the code that invoked the reflective operation in question plus
the code source (i.e., JAR-file path), if available, and

- $VICTIM is a string that describes the member being accessed,
including the fully-qualified name of the enclosing type

In JDK 9's default mode, `--illegal-access=permit`, at most one of these
warning messages will be issued, accompanied by additional instructive
text. Here is an example, from running Jython on the current Jigsaw EA
build [2]:

$ java -jar jython-standalone-2.7.0.jar
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$

If `--illegal-access=warn` is used then only warnings are displayed, with
no instructive text. The run-time system makes a best-effort attempt to
suppress duplicate warnings for the same $PERPETRATOR and $VICTIM. Here
is an example, again running Jython:

$ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$

Notes
-----

- There is no `--illegal-access` mode that suppresses all warnings.
This is intentional: It ensures that developers know that all
illegal-access operations will be denied by default in a future
release, at which time code that generates warnings today will fail.
Warnings can be suppressed completely via one or more `--add-opens`
options.

- The first proposal [1] opened every package in every explicit module,
rather than just the packages in modules in the run-time image, to
every unnamed module. Peter Levart pointed out [5] that this could
tempt developers to use internal APIs that are new in JDK 9 (e.g.,
`jdk.internal.misc.Unsafe`) and thus make the eventual transition
from JDK 9 no less painful than that from JDK 8. This proposal thus
only opens internal packages that existed in JDK 8.

- This proposal will require adjustments to JEP 260, "Encapsulate Most
Internal APIs" [6]. APIs that are internal to the JDK will still be
strongly encapsulated from the standpoint of code in modules, whether
those modules are automatic or explicit, but they will not appear to
be encapsulated at run time from the standpoint of code on the class
path.

- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
[2] http://jdk.java.net/jigsaw/
[3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[4] This will usually but not always be possible, since there are still a
few critical internal APIs without exported replacements [6].
[5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
[6] http://openjdk.java.net/jeps/260
David Blevins
2017-06-05 23:41:49 UTC
Permalink
I think this is the most pragmatic and reasoned middle ground one could possibly imagine.

I agree with the finely-tuned choices, specifically:

- Not going completely silent. Some asked for the ability to completely shut off the warnings. This goes a little too far to one extreme. There has to be some cost or discomfort or there won’t be the right movement. IMHO, it would have been too far back the other way to go silent. People will see these messages, they will post them to StackOverflow and get an education on how to go forward. We need that.

- Not choosing `--illegal-access=deny` by default for this version. Some cited this as a setback. There is an argument to be made that it would have been the bigger setback to go too strict too soon. The danger would be when developers who don’t know this conversation and the details behind it learn “use this flag to make problems go away”. At which point it becomes cargo cult and they’ll be afraid to stop using it because they never understood it completely in the first place. Worse, we’re now buried under 3 years of lazy StackOverflow answers effectively telling people how to go backwards.

So although it feels less strict, one set of choices creates a conversation on how to move forward and one creates a conversation on how to go backward.

Of course time will tell, but I believe this proposal gets us what we want the soonest.


-David
Post by m***@oracle.com
(Thanks for all the feedback on the initial proposal [1]. Here's a
revised version, which incorporates some of the suggestions received and
includes a bit more advice. An implementation is already available for
testing in the Jigsaw EA builds [2]. Further comments welcome!)
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention to the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given at run time in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal-access operations to
internal APIs from code on the class path by default in JDK 9, and to
disallow them in a future release. This will enable smoother application
migration in the near term, yet still enable and motivate the maintainers
of libraries and frameworks that use JDK-internal APIs to fix their code
to use proper exported APIs.
New command-line option: `--illegal-access`
-------------------------------------------
The recently-introduced `--permit-illegal-access` option [3] will be
replaced by a more-general option, `--illegal-access`. This option takes
`--illegal-access=permit`
This mode opens each package in each module in the run-time image to
code in all unnamed modules, i.e., code on the class path, if that
package existed in JDK 8. This enables both static access, i.e., by
compiled bytecode, and deep reflective access, via the platform's
various reflection APIs.
The first reflective-access operation to any such package causes a
warning to be issued, but no warnings are issued after that point.
This single warning describes how to enable further warnings.
This mode will be the default for JDK 9. It will be removed in a
future release.
`--illegal-access=warn`
This mode is identical to `permit` except that a warning message is
issued for each illegal reflective-access operation. This is roughly
equivalent to the current `--permit-illegal-access` option.
`--illegal-access=debug`
This mode is identical to `warn` except both a warning message and a
stack trace are issued for each illegal reflective-access operation.
This is roughly equivalent to combining `--permit-illegal-access`
with `-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This mode disables all illegal-access operations except for those
enabled by other command-line options, e.g., `--add-opens`.
This mode will become the default in a future release.
When `deny` becomes the default mode then `permit` will likely remain
supported for at least one release, so that developers can continue to
migrate their code. The `permit`, `warn`, and `debug` modes will, over
time, be removed, as will the `--illegal-access` option itself. (For
launch-script compatibility the unsupported modes will most likely just
be ignored, after issuing a warning to that effect.)
How to prepare for the future
-----------------------------
The default mode, `--illegal-access=permit`, is intended to make you
aware when you have code on the class path that reflectively accesses
some JDK-internal API at least once. To learn about all such accesses
you can use the `warn` or `debug` modes. For each library or framework
- If the component's maintainers have already released a new,
fixed version that no longer uses JDK-internal APIs then you
can consider upgrading to that version.
- If the component still needs to be fixed then we encourage you
to contact its maintainers and ask them to replace their use
of JDK-internal APIs with proper exported APIs [4].
If you must continue to use a component that requires illegal access then
you can eliminate the warning messages by using one or more `--add-opens`
options to open just those internal packages to which access is required.
To verify that your application is ready for the future, run it with
`--illegal-access=deny` along with any necessary `--add-opens` options.
Any remaining illegal-access errors will most likely be due to static
references from compiled code to JDK-internal APIs. You can identify
those by running the `jdeps` tool with the `--jdk-internals` option.
(JDK 9 does not issue warnings for illegal static-access operations
because that would require deep JVM changes and degrade performance.)
Warning messages
----------------
The warning message issued when an illegal reflective-access operation is
WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM
- $PERPETRATOR is the fully-qualified name of the type containing
the code that invoked the reflective operation in question plus
the code source (i.e., JAR-file path), if available, and
- $VICTIM is a string that describes the member being accessed,
including the fully-qualified name of the enclosing type
In JDK 9's default mode, `--illegal-access=permit`, at most one of these
warning messages will be issued, accompanied by additional instructive
text. Here is an example, from running Jython on the current Jigsaw EA
$ java -jar jython-standalone-2.7.0.jar
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$
If `--illegal-access=warn` is used then only warnings are displayed, with
no instructive text. The run-time system makes a best-effort attempt to
suppress duplicate warnings for the same $PERPETRATOR and $VICTIM. Here
$ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$
Notes
-----
- There is no `--illegal-access` mode that suppresses all warnings.
This is intentional: It ensures that developers know that all
illegal-access operations will be denied by default in a future
release, at which time code that generates warnings today will fail.
Warnings can be suppressed completely via one or more `--add-opens`
options.
- The first proposal [1] opened every package in every explicit module,
rather than just the packages in modules in the run-time image, to
every unnamed module. Peter Levart pointed out [5] that this could
tempt developers to use internal APIs that are new in JDK 9 (e.g.,
`jdk.internal.misc.Unsafe`) and thus make the eventual transition
from JDK 9 no less painful than that from JDK 8. This proposal thus
only opens internal packages that existed in JDK 8.
- This proposal will require adjustments to JEP 260, "Encapsulate Most
Internal APIs" [6]. APIs that are internal to the JDK will still be
strongly encapsulated from the standpoint of code in modules, whether
those modules are automatic or explicit, but they will not appear to
be encapsulated at run time from the standpoint of code on the class
path.
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
[2] http://jdk.java.net/jigsaw/
[3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[4] This will usually but not always be possible, since there are still a
few critical internal APIs without exported replacements [6].
[5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
[6] http://openjdk.java.net/jeps/260
Pepper Lebeck-Jobe
2017-06-06 00:12:54 UTC
Permalink
Hi, I'm a new poster to this list. I'm actively working on getting Gradle
to work on JDK 9, and I also think this proposal strikes a very good
balance between pragmatism (don't just break people today) and getting
developers motivated to clean up their code for the future.

Thank you,
Pepper Lebeck-Jobe
Post by David Blevins
I think this is the most pragmatic and reasoned middle ground one could possibly imagine.
- Not going completely silent. Some asked for the ability to completely
shut off the warnings. This goes a little too far to one extreme. There
has to be some cost or discomfort or there won’t be the right movement.
IMHO, it would have been too far back the other way to go silent. People
will see these messages, they will post them to StackOverflow and get an
education on how to go forward. We need that.
- Not choosing `--illegal-access=deny` by default for this version.
Some cited this as a setback. There is an argument to be made that it
would have been the bigger setback to go too strict too soon. The danger
would be when developers who don’t know this conversation and the details
behind it learn “use this flag to make problems go away”. At which point
it becomes cargo cult and they’ll be afraid to stop using it because they
never understood it completely in the first place. Worse, we’re now buried
under 3 years of lazy StackOverflow answers effectively telling people how
to go backwards.
So although it feels less strict, one set of choices creates a
conversation on how to move forward and one creates a conversation on how
to go backward.
Of course time will tell, but I believe this proposal gets us what we want the soonest.
-David
Post by m***@oracle.com
(Thanks for all the feedback on the initial proposal [1]. Here's a
revised version, which incorporates some of the suggestions received and
includes a bit more advice. An implementation is already available for
testing in the Jigsaw EA builds [2]. Further comments welcome!)
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention to the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given at run time in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal-access operations to
internal APIs from code on the class path by default in JDK 9, and to
disallow them in a future release. This will enable smoother application
migration in the near term, yet still enable and motivate the maintainers
of libraries and frameworks that use JDK-internal APIs to fix their code
to use proper exported APIs.
New command-line option: `--illegal-access`
-------------------------------------------
The recently-introduced `--permit-illegal-access` option [3] will be
replaced by a more-general option, `--illegal-access`. This option takes
`--illegal-access=permit`
This mode opens each package in each module in the run-time image to
code in all unnamed modules, i.e., code on the class path, if that
package existed in JDK 8. This enables both static access, i.e., by
compiled bytecode, and deep reflective access, via the platform's
various reflection APIs.
The first reflective-access operation to any such package causes a
warning to be issued, but no warnings are issued after that point.
This single warning describes how to enable further warnings.
This mode will be the default for JDK 9. It will be removed in a
future release.
`--illegal-access=warn`
This mode is identical to `permit` except that a warning message is
issued for each illegal reflective-access operation. This is roughly
equivalent to the current `--permit-illegal-access` option.
`--illegal-access=debug`
This mode is identical to `warn` except both a warning message and a
stack trace are issued for each illegal reflective-access operation.
This is roughly equivalent to combining `--permit-illegal-access`
with `-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This mode disables all illegal-access operations except for those
enabled by other command-line options, e.g., `--add-opens`.
This mode will become the default in a future release.
When `deny` becomes the default mode then `permit` will likely remain
supported for at least one release, so that developers can continue to
migrate their code. The `permit`, `warn`, and `debug` modes will, over
time, be removed, as will the `--illegal-access` option itself. (For
launch-script compatibility the unsupported modes will most likely just
be ignored, after issuing a warning to that effect.)
How to prepare for the future
-----------------------------
The default mode, `--illegal-access=permit`, is intended to make you
aware when you have code on the class path that reflectively accesses
some JDK-internal API at least once. To learn about all such accesses
you can use the `warn` or `debug` modes. For each library or framework
- If the component's maintainers have already released a new,
fixed version that no longer uses JDK-internal APIs then you
can consider upgrading to that version.
- If the component still needs to be fixed then we encourage you
to contact its maintainers and ask them to replace their use
of JDK-internal APIs with proper exported APIs [4].
If you must continue to use a component that requires illegal access then
you can eliminate the warning messages by using one or more `--add-opens`
options to open just those internal packages to which access is required.
To verify that your application is ready for the future, run it with
`--illegal-access=deny` along with any necessary `--add-opens` options.
Any remaining illegal-access errors will most likely be due to static
references from compiled code to JDK-internal APIs. You can identify
those by running the `jdeps` tool with the `--jdk-internals` option.
(JDK 9 does not issue warnings for illegal static-access operations
because that would require deep JVM changes and degrade performance.)
Warning messages
----------------
The warning message issued when an illegal reflective-access operation is
WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM
- $PERPETRATOR is the fully-qualified name of the type containing
the code that invoked the reflective operation in question plus
the code source (i.e., JAR-file path), if available, and
- $VICTIM is a string that describes the member being accessed,
including the fully-qualified name of the enclosing type
In JDK 9's default mode, `--illegal-access=permit`, at most one of these
warning messages will be issued, accompanied by additional instructive
text. Here is an example, from running Jython on the current Jigsaw EA
$ java -jar jython-standalone-2.7.0.jar
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
(file:/tmp/jython-standalone-2.7.0.jar) to method
sun.nio.ch.SelChImpl.getFD()
Post by m***@oracle.com
WARNING: Please consider reporting this to the maintainers of
jnr.posix.JavaLibCHelper
Post by m***@oracle.com
WARNING: Use --illegal-access=warn to enable warnings of further
illegal reflective access operations
Post by m***@oracle.com
WARNING: All illegal access operations will be denied in a future
release
Post by m***@oracle.com
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$
If `--illegal-access=warn` is used then only warnings are displayed, with
no instructive text. The run-time system makes a best-effort attempt to
suppress duplicate warnings for the same $PERPETRATOR and $VICTIM. Here
$ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
(file:/tmp/jython-standalone-2.7.0.jar) to method
sun.nio.ch.SelChImpl.getFD()
Post by m***@oracle.com
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
(file:/tmp/jython-standalone-2.7.0.jar) to field
sun.nio.ch.FileChannelImpl.fd
Post by m***@oracle.com
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper
(file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
Post by m***@oracle.com
WARNING: Illegal reflective access by org.python.core.PySystemState
(file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
Post by m***@oracle.com
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$
Notes
-----
- There is no `--illegal-access` mode that suppresses all warnings.
This is intentional: It ensures that developers know that all
illegal-access operations will be denied by default in a future
release, at which time code that generates warnings today will fail.
Warnings can be suppressed completely via one or more `--add-opens`
options.
- The first proposal [1] opened every package in every explicit module,
rather than just the packages in modules in the run-time image, to
every unnamed module. Peter Levart pointed out [5] that this could
tempt developers to use internal APIs that are new in JDK 9 (e.g.,
`jdk.internal.misc.Unsafe`) and thus make the eventual transition
from JDK 9 no less painful than that from JDK 8. This proposal thus
only opens internal packages that existed in JDK 8.
- This proposal will require adjustments to JEP 260, "Encapsulate Most
Internal APIs" [6]. APIs that are internal to the JDK will still be
strongly encapsulated from the standpoint of code in modules, whether
those modules are automatic or explicit, but they will not appear to
be encapsulated at run time from the standpoint of code on the class
path.
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
[1]
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
Post by m***@oracle.com
[2] http://jdk.java.net/jigsaw/
[3]
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
Post by m***@oracle.com
[4] This will usually but not always be possible, since there are still a
few critical internal APIs without exported replacements [6].
[5]
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
Post by m***@oracle.com
[6] http://openjdk.java.net/jeps/260
Stephen Felts
2017-06-06 03:34:06 UTC
Permalink
The new format is missing information that was useful in the earlier -Dsun.reflect.debugModuleAccessChecks=true. It printed something like
'java.base does not "opens java.text" to unnamed module ...' so it was really obvious what option was needed.


-----Original Message-----
From: jigsaw-dev [mailto:jigsaw-dev-***@openjdk.java.net] On Behalf Of ***@oracle.com
Sent: Monday, June 5, 2017 2:45 PM
To: jigsaw-***@openjdk.java.net
Subject: Proposal (revised): Allow illegal access to internal APIs by default in JDK 9

(Thanks for all the feedback on the initial proposal [1]. Here's a revised version, which incorporates some of the suggestions received and includes a bit more advice. An implementation is already available for testing in the Jigsaw EA builds [2]. Further comments welcome!)

Over time, as we've gotten closer and closer to the JDK 9 GA date, more and more developers have begun paying attention to the actual changes in this release. The strong encapsulation of JDK-internal APIs has, in particular, triggered many worried expressions of concern that code that works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of this change was given at run time in JDK 8.

To help the entire ecosystem migrate to the modular Java platform at a more relaxed pace I hereby propose to allow illegal-access operations to internal APIs from code on the class path by default in JDK 9, and to disallow them in a future release. This will enable smoother application migration in the near term, yet still enable and motivate the maintainers of libraries and frameworks that use JDK-internal APIs to fix their code to use proper exported APIs.

New command-line option: `--illegal-access`
-------------------------------------------

The recently-introduced `--permit-illegal-access` option [3] will be replaced by a more-general option, `--illegal-access`. This option takes a single keyword parameter to specify a mode of operation, as follows:

`--illegal-access=permit`

This mode opens each package in each module in the run-time image to
code in all unnamed modules, i.e., code on the class path, if that
package existed in JDK 8. This enables both static access, i.e., by
compiled bytecode, and deep reflective access, via the platform's
various reflection APIs.

The first reflective-access operation to any such package causes a
warning to be issued, but no warnings are issued after that point.
This single warning describes how to enable further warnings.

This mode will be the default for JDK 9. It will be removed in a
future release.

`--illegal-access=warn`

This mode is identical to `permit` except that a warning message is
issued for each illegal reflective-access operation. This is roughly
equivalent to the current `--permit-illegal-access` option.

`--illegal-access=debug`

This mode is identical to `warn` except both a warning message and a
stack trace are issued for each illegal reflective-access operation.
This is roughly equivalent to combining `--permit-illegal-access`
with `-Dsun.reflect.debugModuleAccessChecks`.

`--illegal-access=deny`

This mode disables all illegal-access operations except for those
enabled by other command-line options, e.g., `--add-opens`.

This mode will become the default in a future release.

When `deny` becomes the default mode then `permit` will likely remain supported for at least one release, so that developers can continue to migrate their code. The `permit`, `warn`, and `debug` modes will, over time, be removed, as will the `--illegal-access` option itself. (For launch-script compatibility the unsupported modes will most likely just be ignored, after issuing a warning to that effect.)

How to prepare for the future
-----------------------------

The default mode, `--illegal-access=permit`, is intended to make you aware when you have code on the class path that reflectively accesses some JDK-internal API at least once. To learn about all such accesses you can use the `warn` or `debug` modes. For each library or framework on the class path that requires illegal access you have two options:

- If the component's maintainers have already released a new,
fixed version that no longer uses JDK-internal APIs then you
can consider upgrading to that version.

- If the component still needs to be fixed then we encourage you
to contact its maintainers and ask them to replace their use
of JDK-internal APIs with proper exported APIs [4].

If you must continue to use a component that requires illegal access then you can eliminate the warning messages by using one or more `--add-opens` options to open just those internal packages to which access is required.

To verify that your application is ready for the future, run it with `--illegal-access=deny` along with any necessary `--add-opens` options.
Any remaining illegal-access errors will most likely be due to static references from compiled code to JDK-internal APIs. You can identify those by running the `jdeps` tool with the `--jdk-internals` option.
(JDK 9 does not issue warnings for illegal static-access operations because that would require deep JVM changes and degrade performance.)

Warning messages
----------------

The warning message issued when an illegal reflective-access operation is detected has the following form:

WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM

where:

- $PERPETRATOR is the fully-qualified name of the type containing
the code that invoked the reflective operation in question plus
the code source (i.e., JAR-file path), if available, and

- $VICTIM is a string that describes the member being accessed,
including the fully-qualified name of the enclosing type

In JDK 9's default mode, `--illegal-access=permit`, at most one of these warning messages will be issued, accompanied by additional instructive text. Here is an example, from running Jython on the current Jigsaw EA build [2]:

$ java -jar jython-standalone-2.7.0.jar
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$

If `--illegal-access=warn` is used then only warnings are displayed, with no instructive text. The run-time system makes a best-effort attempt to suppress duplicate warnings for the same $PERPETRATOR and $VICTIM. Here is an example, again running Jython:

$ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$

Notes
-----

- There is no `--illegal-access` mode that suppresses all warnings.
This is intentional: It ensures that developers know that all
illegal-access operations will be denied by default in a future
release, at which time code that generates warnings today will fail.
Warnings can be suppressed completely via one or more `--add-opens`
options.

- The first proposal [1] opened every package in every explicit module,
rather than just the packages in modules in the run-time image, to
every unnamed module. Peter Levart pointed out [5] that this could
tempt developers to use internal APIs that are new in JDK 9 (e.g.,
`jdk.internal.misc.Unsafe`) and thus make the eventual transition
from JDK 9 no less painful than that from JDK 8. This proposal thus
only opens internal packages that existed in JDK 8.

- This proposal will require adjustments to JEP 260, "Encapsulate Most
Internal APIs" [6]. APIs that are internal to the JDK will still be
strongly encapsulated from the standpoint of code in modules, whether
those modules are automatic or explicit, but they will not appear to
be encapsulated at run time from the standpoint of code on the class
path.

- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
[2] http://jdk.java.net/jigsaw/
[3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[4] This will usually but not always be possible, since there are still a
few critical internal APIs without exported replacements [6].
[5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
[6] http://openjdk.java.net/jeps/260
Alan Bateman
2017-06-06 06:09:03 UTC
Permalink
Post by Stephen Felts
The new format is missing information that was useful in the earlier -Dsun.reflect.debugModuleAccessChecks=true. It printed something like
'java.base does not "opens java.text" to unnamed module ...' so it was really obvious what option was needed.
`-Dsun.reflect.debugModuleAccessChecks=true` can still be used to hunt
code that swallows IllegalAccessException or
InaccessibleObjectException, particularly useful when running with
`--illegal-access=deny`.

-Alan
Peter Levart
2017-06-07 06:02:34 UTC
Permalink
Hi,

I think the fine-tuning is right to the point now. Allows most of what
was allowed in JDK8, but no more than that.

Regards, Peter
Post by m***@oracle.com
(Thanks for all the feedback on the initial proposal [1]. Here's a
revised version, which incorporates some of the suggestions received and
includes a bit more advice. An implementation is already available for
testing in the Jigsaw EA builds [2]. Further comments welcome!)
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention to the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given at run time in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal-access operations to
internal APIs from code on the class path by default in JDK 9, and to
disallow them in a future release. This will enable smoother application
migration in the near term, yet still enable and motivate the maintainers
of libraries and frameworks that use JDK-internal APIs to fix their code
to use proper exported APIs.
New command-line option: `--illegal-access`
-------------------------------------------
The recently-introduced `--permit-illegal-access` option [3] will be
replaced by a more-general option, `--illegal-access`. This option takes
`--illegal-access=permit`
This mode opens each package in each module in the run-time image to
code in all unnamed modules, i.e., code on the class path, if that
package existed in JDK 8. This enables both static access, i.e., by
compiled bytecode, and deep reflective access, via the platform's
various reflection APIs.
The first reflective-access operation to any such package causes a
warning to be issued, but no warnings are issued after that point.
This single warning describes how to enable further warnings.
This mode will be the default for JDK 9. It will be removed in a
future release.
`--illegal-access=warn`
This mode is identical to `permit` except that a warning message is
issued for each illegal reflective-access operation. This is roughly
equivalent to the current `--permit-illegal-access` option.
`--illegal-access=debug`
This mode is identical to `warn` except both a warning message and a
stack trace are issued for each illegal reflective-access operation.
This is roughly equivalent to combining `--permit-illegal-access`
with `-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This mode disables all illegal-access operations except for those
enabled by other command-line options, e.g., `--add-opens`.
This mode will become the default in a future release.
When `deny` becomes the default mode then `permit` will likely remain
supported for at least one release, so that developers can continue to
migrate their code. The `permit`, `warn`, and `debug` modes will, over
time, be removed, as will the `--illegal-access` option itself. (For
launch-script compatibility the unsupported modes will most likely just
be ignored, after issuing a warning to that effect.)
How to prepare for the future
-----------------------------
The default mode, `--illegal-access=permit`, is intended to make you
aware when you have code on the class path that reflectively accesses
some JDK-internal API at least once. To learn about all such accesses
you can use the `warn` or `debug` modes. For each library or framework
- If the component's maintainers have already released a new,
fixed version that no longer uses JDK-internal APIs then you
can consider upgrading to that version.
- If the component still needs to be fixed then we encourage you
to contact its maintainers and ask them to replace their use
of JDK-internal APIs with proper exported APIs [4].
If you must continue to use a component that requires illegal access then
you can eliminate the warning messages by using one or more `--add-opens`
options to open just those internal packages to which access is required.
To verify that your application is ready for the future, run it with
`--illegal-access=deny` along with any necessary `--add-opens` options.
Any remaining illegal-access errors will most likely be due to static
references from compiled code to JDK-internal APIs. You can identify
those by running the `jdeps` tool with the `--jdk-internals` option.
(JDK 9 does not issue warnings for illegal static-access operations
because that would require deep JVM changes and degrade performance.)
Warning messages
----------------
The warning message issued when an illegal reflective-access operation is
WARNING: Illegal reflective access by $PERPETRATOR to $VICTIM
- $PERPETRATOR is the fully-qualified name of the type containing
the code that invoked the reflective operation in question plus
the code source (i.e., JAR-file path), if available, and
- $VICTIM is a string that describes the member being accessed,
including the fully-qualified name of the enclosing type
In JDK 9's default mode, `--illegal-access=permit`, at most one of these
warning messages will be issued, accompanied by additional instructive
text. Here is an example, from running Jython on the current Jigsaw EA
$ java -jar jython-standalone-2.7.0.jar
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Please consider reporting this to the maintainers of jnr.posix.JavaLibCHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$
If `--illegal-access=warn` is used then only warnings are displayed, with
no instructive text. The run-time system makes a best-effort attempt to
suppress duplicate warnings for the same $PERPETRATOR and $VICTIM. Here
$ java --illegal-access=warn -jar jython-standalone-2.7.0.jar
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to method sun.nio.ch.SelChImpl.getFD()
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field sun.nio.ch.FileChannelImpl.fd
WARNING: Illegal reflective access by jnr.posix.JavaLibCHelper (file:/tmp/jython-standalone-2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/tmp/jython-standalone-2.7.0.jar) to method java.io.Console.encoding()
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9-internal
Type "help", "copyright", "credits" or "license" for more information.
^D
$
Notes
-----
- There is no `--illegal-access` mode that suppresses all warnings.
This is intentional: It ensures that developers know that all
illegal-access operations will be denied by default in a future
release, at which time code that generates warnings today will fail.
Warnings can be suppressed completely via one or more `--add-opens`
options.
- The first proposal [1] opened every package in every explicit module,
rather than just the packages in modules in the run-time image, to
every unnamed module. Peter Levart pointed out [5] that this could
tempt developers to use internal APIs that are new in JDK 9 (e.g.,
`jdk.internal.misc.Unsafe`) and thus make the eventual transition
from JDK 9 no less painful than that from JDK 8. This proposal thus
only opens internal packages that existed in JDK 8.
- This proposal will require adjustments to JEP 260, "Encapsulate Most
Internal APIs" [6]. APIs that are internal to the JDK will still be
strongly encapsulated from the standpoint of code in modules, whether
those modules are automatic or explicit, but they will not appear to
be encapsulated at run time from the standpoint of code on the class
path.
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012673.html
[2] http://jdk.java.net/jigsaw/
[3] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[4] This will usually but not always be possible, since there are still a
few critical internal APIs without exported replacements [6].
[5] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012708.html
[6] http://openjdk.java.net/jeps/260
Loading...