Proposal: Allow illegal reflective access by default in JDK 9

[m***@oracle.com]

Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.

To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.

In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.

In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
will take a single keyword parameter, as follows:

`--illegal-access=permit`

This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.

The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.

`--illegal-access=warn`

This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.

`--illegal-access=debug`

This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.

`--illegal-access=deny`

This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.

Notes:

- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.

- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.

- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.

- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)

- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.

Comments?

- Mark


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[2] http://openjdk.java.net/jeps/260

[Andrew Haley]

Post by m***@oracle.com
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.

Mmm. There are many scripts which parse the output of java,and many
would break. Might '--illegal-access=permit,quiet' be worth
considering?

Andrew.

[Cédric Champeau]

This looks like a very reasonable proposal, +1

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
March/011763.html
[2] http://openjdk.java.net/jeps/260

[Uwe Schindler]

Hi Mark,

To me this proposal is a Desaster. I'd not do this. Buggy software may use the big kill switch.

Sorry Red Hat guys: that's what you triggered with your "no". Bravo! I am impressed!

Sorry Gradle, the worst design in software about environment variables made the whole world again as unsafe as before. We will again see ongoing security updates in Java just fix fix holes that are opened by default. When I have read the mails yesterday, I thought: do you really want to build your software with such a broken tool and it's ecosystem? Can you not just tell the plug-in authors to fix their shit and fix your API to work correct?

Amazon S3 software dilettantes: Fix your EC2 security software to not undermine the Java security system! I can bring many more: Don't do that in security relevant tools or build systems many people rely on!

Today is the worst day in Java history.

Uwe

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the
`--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1]
http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[2] http://openjdk.java.net/jeps/260

[Mario Torre]

Post by Uwe Schindler
Hi Mark,
To me this proposal is a Desaster. I'd not do this. Buggy software may use the big kill switch.
Sorry Red Hat guys: that's what you triggered with your "no". Bravo! I am impressed!
Sorry Gradle, the worst design in software about environment variables made the whole world again as unsafe as before. We will again see ongoing security updates in Java just fix fix holes that are opened by default. When I have read the mails yesterday, I thought: do you really want to build your software with such a broken tool and it's ecosystem? Can you not just tell the plug-in authors to fix their shit and fix your API to work correct?
Amazon S3 software dilettantes: Fix your EC2 security software to not undermine the Java security system! I can bring many more: Don't do that in security relevant tools or build systems many people rely on!
Today is the worst day in Java history.

Hello Uwe,

I appreciate your enthusiasm, but please let's try to keep this
discussion focused.

Cheers,
Mario
--
pgp key: http://subkeys.pgp.net/ PGP Key ID: 80F240CF
Fingerprint: BA39 9666 94EC 8B73 27FA FC7C 4086 63E3 80F2 40CF

Java Champion - Blog: http://neugens.wordpress.com - Twitter: @neugens
Proud GNU Classpath developer: http://www.classpath.org/
OpenJDK: http://openjdk.java.net/projects/caciocavallo/

Please, support open standards:
http://endsoftpatents.org/

[Michael Nascimento]

Sounds like a well thought compromise, Mark. Real world applications need
this and it's a very welcome move.

Regards,
Michael

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-
March/011763.html
[2] http://openjdk.java.net/jeps/260

[Markus Keller]

The command-line argument and the default mode sound good.

`--illegal-access=quiet` could be added for use by clients that cannot
cope with additional warning messages.

The second ingredient for broad compatibility with Java 8 would be an
automatic --add-modules=ALL-SYSTEM.
This could be implemented as an automatic compatibility mode in Java 9
that would only be active as long as there's no --modulepath on the
command line.

An alternative to automatic compatibility would be another sub-option for
--illegal-access, so that affected end users only have one command-line
argument for all accessibility concerns, e.g.:

`--illegal-access=permit-all`

Equivalent to `--illegal-access=permit --add-modules=ALL-SYSTEM`


FYI: In the Eclipse launcher for the Oxygen (4.7) release, we've added
forward-compatibility handling for --permit-illegal-access. Our deadline
to revert/adjust that is May 30 ( https://bugs.eclipse.org/516911 ).

Regards,
Markus

Date: 2017-05-18 16:49
Subject: Proposal: Allow illegal reflective access by default in JDK 9
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the

`--permit-illegal-access`

option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still

different,

`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1]

http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html

[2] http://openjdk.java.net/jeps/260

[Nicolai Parlog]

Hi!

I like the flag per se and think a "quiet" argument would be helpful. I
also understand that there is pressure to improve the spec and that many
observers will see this change as progress[1].

But I think making the lenient option the default is a bad decision,
needlessly prolonging a proper clean-up of the ecosystem without any
practical benefit!

Java's stewards have been warning against using internal APIs for 20
years. Jigsaw announced to make them inaccessible for nigh two years
now. Java 8 will be supported until at least July 2018 and even after
that all that is needed to get around this specific Java 9 compatibility
problem is to add `--permit-illegal-access`.

All that is not enough, though? Even adding that single flag is too
hard? If spending an hour or two reading up on JPMS and then adding that
flag is too much to ask, then how can one expect the same people to
clean up their code?

With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!

so long ... Nicolai


[1](where in fact it only creates the appearance thereof)
[2]
https://krautblog-ulrich.blogspot.de/2012/07/word-of-month-armutszeugnis.html

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[2] http://openjdk.java.net/jeps/260

--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
http://codefx.org
a blog about software development
https://www.sitepoint.com/java
high-quality Java/JVM content
http://do-foss.de
Free and Open Source Software for the City of Dortmund

Twitter:
https://twitter.com/nipafx

[Christoph Engelbert]

Hey,

I also think the new styled parameter is great and I would appreciate a fully
“quiet” mode for our customers. I don’t mandate it to be the default but I think
to make it the default for (and only for) Java 9 sounds meaningful.

The reason is simple: It is not just us (a few people, a few vendors) to set
this parameter but it will be tons of customers or users, running the actual
production workloads, to add it to their command line. I guess most of us
know the burden of making a change to the command line at some bigger
companies, such as banks.

On the other side, as a dev I want it to print as much warnings as possible,
when I’m working on fixing the actual “misbehaving” code fragment.

In general I’m ok with making users add it to their command line but would
like to see something like quiet as the default for Java 9.

PS: the `—illegal-access=permit` as default works for me though :)

Thanks for the revised proposal, good stuff!

Cheers,
Chris

Post by Nicolai Parlog
Hi!
I like the flag per se and think a "quiet" argument would be helpful. I
also understand that there is pressure to improve the spec and that many
observers will see this change as progress[1].
But I think making the lenient option the default is a bad decision,
needlessly prolonging a proper clean-up of the ecosystem without any
practical benefit!
Java's stewards have been warning against using internal APIs for 20
years. Jigsaw announced to make them inaccessible for nigh two years
now. Java 8 will be supported until at least July 2018 and even after
that all that is needed to get around this specific Java 9 compatibility
problem is to add `--permit-illegal-access`.
All that is not enough, though? Even adding that single flag is too
hard? If spending an hour or two reading up on JPMS and then adding that
flag is too much to ask, then how can one expect the same people to
clean up their code?
With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!
so long ... Nicolai
[1](where in fact it only creates the appearance thereof)
[2]
https://krautblog-ulrich.blogspot.de/2012/07/word-of-month-armutszeugnis.html

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[2] http://openjdk.java.net/jeps/260

--
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
http://codefx.org
a blog about software development
https://www.sitepoint.com/java
high-quality Java/JVM content
http://do-foss.de
Free and Open Source Software for the City of Dortmund
https://twitter.com/nipafx

[Tagir Valeev]

Regarding proposed "quiet" option: there's existing
non-standard -XX:-PrintWarnings JVM option. I wonder whether it will work
to suppress the illegal access warning as well. If yes, then no additional
"quite" mode is necessary.

With best regards,
Tagir Valeev.

Post by Christoph Engelbert
Hey,
I also think the new styled parameter is great and I would appreciate a fully
“quiet” mode for our customers. I don’t mandate it to be the default but I think
to make it the default for (and only for) Java 9 sounds meaningful.
The reason is simple: It is not just us (a few people, a few vendors) to set
this parameter but it will be tons of customers or users, running the actual
production workloads, to add it to their command line. I guess most of us
know the burden of making a change to the command line at some bigger
companies, such as banks.
On the other side, as a dev I want it to print as much warnings as possible,
when I’m working on fixing the actual “misbehaving” code fragment.
In general I’m ok with making users add it to their command line but would
like to see something like quiet as the default for Java 9.
PS: the `—illegal-access=permit` as default works for me though :)
Thanks for the revised proposal, good stuff!
Cheers,
Chris

Post by Nicolai Parlog
Hi!
I like the flag per se and think a "quiet" argument would be helpful. I
also understand that there is pressure to improve the spec and that many
observers will see this change as progress[1].
But I think making the lenient option the default is a bad decision,
needlessly prolonging a proper clean-up of the ecosystem without any
practical benefit!
Java's stewards have been warning against using internal APIs for 20
years. Jigsaw announced to make them inaccessible for nigh two years
now. Java 8 will be supported until at least July 2018 and even after
that all that is needed to get around this specific Java 9 compatibility
problem is to add `--permit-illegal-access`.
All that is not enough, though? Even adding that single flag is too
hard? If spending an hour or two reading up on JPMS and then adding that
flag is too much to ask, then how can one expect the same people to
clean up their code?
With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!
so long ... Nicolai
[1](where in fact it only creates the appearance thereof)
[2]
https://krautblog-ulrich.blogspot.de/2012/07/word-of-

month-armutszeugnis.html

Post by Nicolai Parlog

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the

`--permit-illegal-access`

Post by Nicolai Parlog

Post by m***@oracle.com
option [1] will become the default behavior of the JDK 9 run-time

system,

Post by Nicolai Parlog

Post by m***@oracle.com
though without as many warnings. The current behavior of JDK 9, in

which

Post by Nicolai Parlog

Post by m***@oracle.com
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-

March/011763.html

Post by Nicolai Parlog

Post by m***@oracle.com
[2] http://openjdk.java.net/jeps/260

--
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
http://codefx.org
a blog about software development
https://www.sitepoint.com/java
high-quality Java/JVM content
http://do-foss.de
Free and Open Source Software for the City of Dortmund
https://twitter.com/nipafx

[Christoph Engelbert]

There was a bug report which is closed, but that was, obviously, before the changed proposal. I think it would be subject to reconsider: https://bugs.openjdk.java.net/browse/JDK-8177916 <https://bugs.openjdk.java.net/browse/JDK-8177916>

Chris

Regarding proposed "quiet" option: there's existing non-standard -XX:-PrintWarnings JVM option. I wonder whether it will work to suppress the illegal access warning as well. If yes, then no additional "quite" mode is necessary.
With best regards,
Tagir Valeev.
Hey,
I also think the new styled parameter is great and I would appreciate a fully
“quiet” mode for our customers. I don’t mandate it to be the default but I think
to make it the default for (and only for) Java 9 sounds meaningful.
The reason is simple: It is not just us (a few people, a few vendors) to set
this parameter but it will be tons of customers or users, running the actual
production workloads, to add it to their command line. I guess most of us
know the burden of making a change to the command line at some bigger
companies, such as banks.
On the other side, as a dev I want it to print as much warnings as possible,
when I’m working on fixing the actual “misbehaving” code fragment.
In general I’m ok with making users add it to their command line but would
like to see something like quiet as the default for Java 9.
PS: the `—illegal-access=permit` as default works for me though :)
Thanks for the revised proposal, good stuff!
Cheers,
Chris

Post by Nicolai Parlog
Hi!
I like the flag per se and think a "quiet" argument would be helpful. I
also understand that there is pressure to improve the spec and that many
observers will see this change as progress[1].
But I think making the lenient option the default is a bad decision,
needlessly prolonging a proper clean-up of the ecosystem without any
practical benefit!
Java's stewards have been warning against using internal APIs for 20
years. Jigsaw announced to make them inaccessible for nigh two years
now. Java 8 will be supported until at least July 2018 and even after
that all that is needed to get around this specific Java 9 compatibility
problem is to add `--permit-illegal-access`.
All that is not enough, though? Even adding that single flag is too
hard? If spending an hour or two reading up on JPMS and then adding that
flag is too much to ask, then how can one expect the same people to
clean up their code?
With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!
so long ... Nicolai
[1](where in fact it only creates the appearance thereof)
[2]
https://krautblog-ulrich.blogspot.de/2012/07/word-of-month-armutszeugnis.html <https://krautblog-ulrich.blogspot.de/2012/07/word-of-month-armutszeugnis.html>

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html <http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html>
[2] http://openjdk.java.net/jeps/260 <http://openjdk.java.net/jeps/260>

--
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509 <http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509>
http://codefx.org <http://codefx.org/>
a blog about software development
https://www.sitepoint.com/java <https://www.sitepoint.com/java>
high-quality Java/JVM content
http://do-foss.de <http://do-foss.de/>
Free and Open Source Software for the City of Dortmund
https://twitter.com/nipafx <https://twitter.com/nipafx>

[Peter Levart]

Post by Nicolai Parlog
With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!

+1

I think that --illegal-access=permit-silently or equivalent is a
reasonable request (in addition to other modes), but please make the
--illegal-access=deny the default!

There was a lot of work put into moving various internal classes into
jdk.internal.* packages (except for some widely used ones), so enabling
illegal access from classpath by default allows careless developers to
start using those new internal APIs (such as jdk.internal.misc.Unsafe)
because they can!!! And we are back to square one when JDK 9 -> JDK 10
transition is to be made. Not only will we be forced to search for
public alternatives to existing private (JDK 8-) APIs, but in addition
to all jdk.internal.* APIs that were newly introduced in JDK 9 too...

Regards, Peter

[Andrew Haley]

Post by Peter Levart

Post by Nicolai Parlog
With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!

+1
I think that --illegal-access=permit-silently or equivalent is a
reasonable request (in addition to other modes), but please make the
--illegal-access=deny the default!

I don't think that's practical because, as Mark Reinhold put it, "the
strong encapsulation of JDK-internal APIs has, in particular,
triggered many worried expressions of concern that code that works on
JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of
this change was given in JDK 8."

Andrew.

[Stephen Felts]

Here's something to think about with respect to noise by whatever choice is made for the default for --illegal-access.

 

I've run a lot of code using the current --permit-illegal-access.

 

One of the problems with this option is that there can be errors detected for code that is working correctly and/or has already been fixed for JDK9.

Since I have been working on identifying and entering bugs for JDK9 problems, I have also been collecting a list of warnings for code that is valid so I can ignore them.

Here are a few public ones.

org.python.core.PyJavaClass - Jython

com.sun.xml.bind.v2.runtime.reflect.opt.Injector – standalone jaxb

com.sun.xml.ws.model.Injector – standalone jaxws

net.sf.cglib.core.ReflectUtils$ - cglib

 

Summarizing the use cases:

1. The code calls setAccessible and correctly catches and ignores the exception for those that fail.  It would be a big change to make this call setAccessible lazily.

2. The code continues to use the JDK8 approach and if that fails it falls back to an approach that works on JDK9 and/or other non-Hotspot platforms.  In some cases, this code has been around for a decade or more (e.g., to handle platforms without Unsafe).

 

In work flows that I have seen, a message would always be printed for every process, even though there is no real problem.  That’s not acceptable in a production environment.

 

Let's assume that there is at least one illegal access per process that is run.

The current proposal will print one message per process.  That can be a lot of noise.  Examples:

- Running tests with fork for every test (obviously not a production environment).

- If you are doing internal process generation that used to be invisible to your customers, it's now visible.

 

 

[Stephen Felts]

By the way, there is no available API so that one can write

 

If an object is not accessible and setAccessible on the object will succeed

  call setAccessible

 

That means that the code needs to call setAccessible and handle the exception, which triggers a warning.

 

 

 

[Alan Bateman]

Post by Stephen Felts
By the way, there is no available API so that one can write
If an object is not accessible and setAccessible on the object will succeed
call setAccessible
That means that the code needs to call setAccessible and handle the exception, which triggers a warning.

AccessibleObject canAccess is new in 9 to test if a reflected object is
accessible. Another new API is trySetAccessible to avoid using
exceptions for control flow.

To your example then if setAccessible succeeds in suppressing access to
a JDK internal member then you'll get a warning if it's the first
illegal access. If setAccessible fails then you'll
InaccessibleObjectException will be thrown as it does not, there will be
no warning.

-Alan.

[Stephen Felts]

Right. I'm asking for a new API so code can be written to avoid the warning.

-----Original Message-----
From: Alan Bateman
Sent: Friday, May 19, 2017 10:11 AM
To: Stephen Felts; jigsaw-***@openjdk.java.net
Subject: Re: Proposal: Allow illegal reflective access by default in JDK 9

Post by Stephen Felts
By the way, there is no available API so that one can write
If an object is not accessible and setAccessible on the object will succeed
call setAccessible
That means that the code needs to call setAccessible and handle the exception, which triggers a warning.

AccessibleObject canAccess is new in 9 to test if a reflected object is accessible. Another new API is trySetAccessible to avoid using exceptions for control flow.

To your example then if setAccessible succeeds in suppressing access to a JDK internal member then you'll get a warning if it's the first illegal access. If setAccessible fails then you'll InaccessibleObjectException will be thrown as it does not, there will be no warning.

-Alan.

[Jochen Theodorou]

On 19.05.2017 15:38, Stephen Felts wrote:
[...]

Post by Stephen Felts
Here are a few public ones.
org.python.core.PyJavaClass - Jython
com.sun.xml.bind.v2.runtime.reflect.opt.Injector – standalone jaxb
com.sun.xml.ws.model.Injector – standalone jaxws
net.sf.cglib.core.ReflectUtils$ - cglib
1. The code calls setAccessible and correctly catches and ignores the exception for those that fail. It would be a big change to make this call setAccessible lazily.
2. The code continues to use the JDK8 approach and if that fails it falls back to an approach that works on JDK9 and/or other non-Hotspot platforms. In some cases, this code has been around for a decade or more (e.g., to handle platforms without Unsafe).

with the same summary you can include Groovy here.

bye Jochen

[Nicolai Parlog]

Hi,

I don't want to pick on Andrew in particular but his paragraph is a

Post by Andrew Haley
I don't think that's practical

Interesting, waiting for a real, practical, technical reason not to do that.

Post by Andrew Haley
many worried expressions

Err, what? So something is impractical because people are worried? I'm
sure they're worried because they think it's impractical.

Post by Andrew Haley
code that works on JDK 8 today will not work on JDK 9 tomorrow,
yet no advance warning of this change was given in JDK 8

So twenty years of claiming internal APIs are internal and subject to
change[1] was not enough? Two years of this change being announced was
no enough? One more year of Java 8 is not enough? A simple flag to make
it work is not enough?

so long ... Nicolai


[1]
http://web.archive.org/web/19980215011039/http://java.sun.com/products/jdk/faq/faq-sun-packages.html

Post by Andrew Haley

Post by Peter Levart

Post by Nicolai Parlog
With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!

+1
I think that --illegal-access=permit-silently or equivalent is a
reasonable request (in addition to other modes), but please make the
--illegal-access=deny the default!

I don't think that's practical because, as Mark Reinhold put it, "the
strong encapsulation of JDK-internal APIs has, in particular,
triggered many worried expressions of concern that code that works on
JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of
this change was given in JDK 8."
Andrew.

--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
http://codefx.org
a blog about software development
https://www.sitepoint.com/java
high-quality Java/JVM content
http://do-foss.de
Free and Open Source Software for the City of Dortmund

Twitter:
https://twitter.com/nipafx

[Russell Gold]

There are two separate issues, here:

1) Access to internal APIs. Yes, we have been warned of these for two decades; it shouldn’t actually be a shock that they have been banned. It shouldn’t, but in some cases, people didn’t really believe that they would go away. Still, it seems reasonable to say that anything marked with a deprecation warning for some time can vanish without further notice. That is the point of them, after all.

2) Reflective access to internals. This one isn’t so easy. Obviously, when you are using reflection to hack into somebody else’s code, you should expect that it could change without warning; however, this has always seemed to work, and people have come to rely on it. In many cases, it has allowed Java developers to do things that were not possible in other languages. Because it is runtime functionality, there were no equivalents of the deprecation warnings. This new proposal creates them, giving notice that we need to find alternatives.

As I see it, the ideal default is compilation failures against internal APIs by default, but access with warnings of to-be-removed reflective access to internals. Command-line switches for runtime is challenging, since many Java programs are run from user-written shell scripts (the executable jar case is handled by manifest entries). Command-line switches for compilation shouldn’t be that much of a burden.

Some of us have been working to remove all of our internal accesses, but twenty-years of technical debt isn’t easily cleaned up in a short time - especially when some it relies on third-party code. This proposal allows us to move forward. It makes it abundantly clear that things are changing: deprecated and internal APIs are really going away, and reflective access is going to be limited. At the same time, it allows the vast amount of legacy code out there to migrate to the new world sooner rather than later.

Post by Nicolai Parlog
Hi,
I don't want to pick on Andrew in particular but his paragraph is a

Post by Andrew Haley
I don't think that's practical

Interesting, waiting for a real, practical, technical reason not to do that.

Post by Andrew Haley
many worried expressions

Err, what? So something is impractical because people are worried? I'm
sure they're worried because they think it's impractical.

Post by Andrew Haley
code that works on JDK 8 today will not work on JDK 9 tomorrow,
yet no advance warning of this change was given in JDK 8

So twenty years of claiming internal APIs are internal and subject to
change[1] was not enough? Two years of this change being announced was
no enough? One more year of Java 8 is not enough? A simple flag to make
it work is not enough?
so long ... Nicolai
[1]
http://web.archive.org/web/19980215011039/http://java.sun.com/products/jdk/faq/faq-sun-packages.html

Post by Andrew Haley

Post by Peter Levart

Post by Nicolai Parlog
With illegal access being permitted by default much fewer developers
will be aware of the problem and much less pressure will be put on
library and framework maintainers as well as on project management to
invest into paying back this particular form of technical debt. So we
get much less momentum to make the necessary changes in exchange for...
not having to add a flag? That's ridiculous, an Armutszeugnis[2] for the
Java community!

+1
I think that --illegal-access=permit-silently or equivalent is a
reasonable request (in addition to other modes), but please make the
--illegal-access=deny the default!

I don't think that's practical because, as Mark Reinhold put it, "the
strong encapsulation of JDK-internal APIs has, in particular,
triggered many worried expressions of concern that code that works on
JDK 8 today will not work on JDK 9 tomorrow, yet no advance warning of
this change was given in JDK 8."
Andrew.

--
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
http://codefx.org
a blog about software development
https://www.sitepoint.com/java
high-quality Java/JVM content
http://do-foss.de
Free and Open Source Software for the City of Dortmund
https://twitter.com/nipafx

[Weijun Wang]

I don't like this default value, but I also know some people wanting it.

Is it possible to provide the default value in a file inside the conf
directory and also make it overwrite-able on the command line? Maybe
RedHat Linux can make it "permit" out-of-box and other vendors can
choose different values.

Thanks
Max

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[2] http://openjdk.java.net/jeps/260

[Volker Simonis]

Hi Mark,

thanks for this proposal. I think it is pragmatic and well balanced.

While I personally agree with the advocates of a clean solution I also
understand the the needs of big (and slow) companies (including the
one I'm working for:) After all, the new default isn't that bad in my
opinion because it only affects applications/libraries on the class
path which is legacy anyway. If you convert your application/libraries
to modules it will still benefit from the full JPMS features and
restrictions. So altogether I think it is a good compromise.

I have some practical comments/suggestions though:

From my understanding, at run-time, "open" implicates "exports" (i.e.
if a module M1 opens a package P for some other module M2 it also,
implicitly exports P to M2). The "big kill switch" in both, its old
and in the newly proposed form, usually only refers to "enabling
reflective access" but doesn't explicitly mentions that it will also,
implicitly export the respective packages. Also, the usage of the
"kill switch" only produces warnings for reflective accesses which are
enabled by the option itself (and not at the same time, explicitly
allowed by --add-opens directives). But it doesn't warn about the
simple, non-reflective accesses to packages which are implicitly
exported by the kill switch as well.

I think that the "big kill switch" should also print warnings for
these simple, non-reflective accesses to packages which are implicitly
exported by the kill switch. Otherwise, this could lead to a situation
where users claim to be "JPMS-safe" because they did run with
"--illegal-access=warn" and didn't saw any warnings but then, after
the default will change to "--illegal-access=deny", their applications
will break because of simple accesses to non-exported packages.

I'm aware of the initial discussion about "--permit-illegal-access"
and that you decided not to issue warnings for the precise
`--add-opens` and `--add-exports` options [1]. But now, with the new
version of the "--illegal-access" option and the possibility to
enable/disable warnings, I think printing warnings for simple (i.e.
non-reflective) illegal accesses which are implicitly allowed
"--illegal-access" by would be useful.

Regards,
Volker

[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011810.html

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[2] http://openjdk.java.net/jeps/260

[Peter Levart]

Hi Volker,

Post by Volker Simonis
From my understanding, at run-time, "open" implicates "exports" (i.e.
if a module M1 opens a package P for some other module M2 it also,
implicitly exports P to M2). The "big kill switch" in both, its old
and in the newly proposed form, usually only refers to "enabling
reflective access" but doesn't explicitly mentions that it will also,
implicitly export the respective packages. Also, the usage of the
"kill switch" only produces warnings for reflective accesses which are
enabled by the option itself (and not at the same time, explicitly
allowed by --add-opens directives). But it doesn't warn about the
simple, non-reflective accesses to packages which are implicitly
exported by the kill switch as well.

No, you got it wrong. "opens" is just a backdoor for reflection (and
MethodHandles.Lookup). It does not implicitly also mean "exports".
Neither in compile-time nor in run-time. Otherwise everything that is
declared "public" would be accessible to reflection without the need for
.setAccessible(true) - therefore even to code running with
SecurityManager enabled!

Regards, Peter

[Nicolai Parlog]

Hi!

I'm too lazy to try right now but I'm pretty sure I repeatedly observed
the behavior Volker describes:

A module depends on an internal type and is compiled successfully
thanks to --add-exports. At run time it would fail without flags and
would work with --add-exports but it also works with --add-opens.

so long ... Nicolai

Post by Peter Levart
Hi Volker,

Post by Volker Simonis
From my understanding, at run-time, "open" implicates "exports"
(i.e. if a module M1 opens a package P for some other module M2
it also, implicitly exports P to M2). The "big kill switch" in
both, its old and in the newly proposed form, usually only refers
to "enabling reflective access" but doesn't explicitly mentions
that it will also, implicitly export the respective packages.
Also, the usage of the "kill switch" only produces warnings for
reflective accesses which are enabled by the option itself (and
not at the same time, explicitly allowed by --add-opens
directives). But it doesn't warn about the simple, non-reflective
accesses to packages which are implicitly exported by the kill
switch as well.

No, you got it wrong. "opens" is just a backdoor for reflection
(and MethodHandles.Lookup). It does not implicitly also mean
"exports". Neither in compile-time nor in run-time. Otherwise
everything that is declared "public" would be accessible to
reflection without the need for .setAccessible(true) - therefore
even to code running with SecurityManager enabled!
Regards, Peter

--
PGP Key:
http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509

Web:
http://codefx.org
a blog about software development
https://www.sitepoint.com/java
high-quality Java/JVM content
http://do-foss.de
Free and Open Source Software for the City of Dortmund

Twitter:
https://twitter.com/nipafx

[Peter Levart]

Hi Nikolai,

Post by Nicolai Parlog
Hi!
I'm too lazy to try right now

It's easy. Take this example:

public class Test {
public static void main(String[] args) throws Exception {
Object theUnsafe = Class.forName("jdk.internal.misc.Unsafe")
.getMethod("getUnsafe")
.invoke(null);
}
}

Running this on classpath without any additional JVM options produces:

Exception in thread "main" java.lang.IllegalAccessException: class Test
cannot access class jdk.internal.misc.Unsafe (in module java.base)
because module java.base does not export jdk.internal.misc to unnamed
module @63d4e2ba
at
java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:445)
at
java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:436)
at
java.base/jdk.internal.reflect.Reflection.ensureMemberAccess(Reflection.java:112)
at
java.base/java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:407)
at
java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:399)
at java.base/java.lang.reflect.Method.invoke(Method.java:535)
at Test.main(Test.java:9)

(note that the exception is thrown from Method.invoke)

When run with:

--add-exports java.base/jdk.internal.misc=ALL-UNNAMED

...the example works (does not throw exception), but when run with:

--add-opens java.base/jdk.internal.misc=ALL-UNNAMED

...the example still works!!!! OOOPS - this is a bug!!! This is
definitely a bug.


If --illegal-access=permit is an alias for opening all modules to
ALL-UNNAMED, then we definitely have a problem as above example can be
successfully executed even with SecurityManager enabled!!!


Regards, Peter

Post by Nicolai Parlog
but I'm pretty sure I repeatedly observed
A module depends on an internal type and is compiled successfully
thanks to --add-exports. At run time it would fail without flags and
would work with --add-exports but it also works with --add-opens.
so long ... Nicolai

Post by Peter Levart
Hi Volker,

Post by Volker Simonis
From my understanding, at run-time, "open" implicates "exports"
(i.e. if a module M1 opens a package P for some other module M2
it also, implicitly exports P to M2). The "big kill switch" in
both, its old and in the newly proposed form, usually only refers
to "enabling reflective access" but doesn't explicitly mentions
that it will also, implicitly export the respective packages.
Also, the usage of the "kill switch" only produces warnings for
reflective accesses which are enabled by the option itself (and
not at the same time, explicitly allowed by --add-opens
directives). But it doesn't warn about the simple, non-reflective
accesses to packages which are implicitly exported by the kill
switch as well.

No, you got it wrong. "opens" is just a backdoor for reflection
(and MethodHandles.Lookup). It does not implicitly also mean
"exports". Neither in compile-time nor in run-time. Otherwise
everything that is declared "public" would be accessible to
reflection without the need for .setAccessible(true) - therefore
even to code running with SecurityManager enabled!
Regards, Peter

[Alan Bateman]

Post by Peter Levart
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
...the example still works!!!! OOOPS - this is a bug!!! This is
definitely a bug.

No, the behavior is correct as jdk.internal.misc.Unsafe is public and
it's getUnsafe is public. It's different to sun.misc.Unsafe.

However, you do have a point that opening all packages by default means
that the internal Unsafe might need to be looked at again. The long term
plan is to reduce the need for sun.misc.Unsafe but not by having
libraries move to the internal Unsafe.

BTW: For the security manager case then I assume you code fragment would
need to accessClassInPackage.jdk.internal.misc permission.

-Alan

[Peter Levart]

Post by Alan Bateman

Post by Peter Levart
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
...the example still works!!!! OOOPS - this is a bug!!! This is
definitely a bug.

No, the behavior is correct as jdk.internal.misc.Unsafe is public and
it's getUnsafe is public. It's different to sun.misc.Unsafe.
However, you do have a point that opening all packages by default
means that the internal Unsafe might need to be looked at again. The
long term plan is to reduce the need for sun.misc.Unsafe but not by
having libraries move to the internal Unsafe.
BTW: For the security manager case then I assume you code fragment
would need to accessClassInPackage.jdk.internal.misc permission.

Right, but such permission should not be sufficient, since it is
required only to obtain the jdk.internal.misc.Unsafe.class object (i.e.
for Class.forName("jdk.internal.misc.Unsafe"). I can play a trick here
to obtain such Class instance. For example:

public class Test {
public static void main(String[] args) throws Exception {
System.setSecurityManager(new SecurityManager());
Class<?> unsafeClass =
java.io.File.class.getDeclaredField("UNSAFE").getType();
Object theUnsafe = unsafeClass.getMethod("getUnsafe").invoke(null);
}
}


Ok, I need "accessDeclaredMembers" permission here, but you get the
picture...

Regards, Peter

Post by Alan Bateman
-Alan

[Peter Levart]

Post by Peter Levart
public class Test {
public static void main(String[] args) throws Exception {
Object theUnsafe = Class.forName("jdk.internal.misc.Unsafe")
.getMethod("getUnsafe")
.invoke(null);
}
}
Exception in thread "main" java.lang.IllegalAccessException: class
Test cannot access class jdk.internal.misc.Unsafe (in module
java.base) because module java.base does not export jdk.internal.misc
at
java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:445)
at
java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:436)
at
java.base/jdk.internal.reflect.Reflection.ensureMemberAccess(Reflection.java:112)
at
java.base/java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:407)
at
java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:399)
at java.base/java.lang.reflect.Method.invoke(Method.java:535)
at Test.main(Test.java:9)
(note that the exception is thrown from Method.invoke)
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
...the example still works!!!! OOOPS - this is a bug!!! This is
definitely a bug.
If --illegal-access=permit is an alias for opening all modules to
ALL-UNNAMED, then we definitely have a problem as above example can be
successfully executed even with SecurityManager enabled!!!

...Ok, there's still a SecurityManager.checkPackageAccess() performed
when some app class "loads" the jdk.internal.misc.Unsafe class and the
caller needs to have a "accessClassInPackage.jdk.internal.misc" runtime
permission. So this throws exception with SecurityManager enabled (from
Class.forName("...") this time), but this permission should not be
sufficient. I think "opens" should not imply "exports" - not even for
reflection.

Conceptually "exports" should fall in the same category of access
controls as "private", "package-private", "protected" and "public"
modifiers on members/classes. If a member is not accessible to a caller
due to access modifiers on the member or enclosing class or module
lacking "exports", then such member should not be allowed to be accessed:
- during compilation
- during runtime by bytecode instruction
- during runtime by reflection
- during runtime by MethodHandles.Lookup methods

Opening a package to the caller module should only allow the caller to
suppress these access checks, but the caller should still be required to
request this explicitly either by using
AccessibleObject.setAccessible(true) when reflecting; or by using
MethodHandles.Lookup.privateLookupIn() when looking up members to get
MethodHandles or VarHandles. Those two methods require special
"suppressAccessChecks" runtime permission.

Regards, Peter

[David M. Lloyd]

Post by Peter Levart
Hi Nikolai,

Post by Nicolai Parlog
Hi!
I'm too lazy to try right now

public class Test {
public static void main(String[] args) throws Exception {
Object theUnsafe = Class.forName("jdk.internal.misc.Unsafe")
.getMethod("getUnsafe")
.invoke(null);
}
}
Exception in thread "main" java.lang.IllegalAccessException: class Test
cannot access class jdk.internal.misc.Unsafe (in module java.base)
because module java.base does not export jdk.internal.misc to unnamed
at
java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:445)
at
java.base/jdk.internal.reflect.Reflection.throwIllegalAccessException(Reflection.java:436)
at
java.base/jdk.internal.reflect.Reflection.ensureMemberAccess(Reflection.java:112)
at
java.base/java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:407)
at
java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:399)
at java.base/java.lang.reflect.Method.invoke(Method.java:535)
at Test.main(Test.java:9)
(note that the exception is thrown from Method.invoke)
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
...the example still works!!!! OOOPS - this is a bug!!! This is
definitely a bug.

This is seemingly consistent with the code run time code in Module for
addOpens, which always seems to add export when open is added. It would
be quite strange if the descriptor directives and JVM switches didn't
match the run time code; is this something that the EG should talk
about? Or is there a hidden logic to this?

[Alan Bateman]

Post by Nicolai Parlog
A module depends on an internal type and is compiled successfully
thanks to --add-exports. At run time it would fail without flags and
would work with --add-exports but it also works with --add-opens.

That is correct, assuming you mean a reference to a public member of a
public type, and the module declares `requires $MODULE` to establish the
readability (this part is ignored when discussing code on the class path
because unnamed modules read all modules).

-Alan

[Alan Bateman]

Post by Volker Simonis
From my understanding, at run-time, "open" implicates "exports" (i.e.
if a module M1 opens a package P for some other module M2 it also,
implicitly exports P to M2). The "big kill switch" in both, its old
and in the newly proposed form, usually only refers to "enabling
reflective access" but doesn't explicitly mentions that it will also,
implicitly export the respective packages. Also, the usage of the
"kill switch" only produces warnings for reflective accesses which are
enabled by the option itself (and not at the same time, explicitly
allowed by --add-opens directives). But it doesn't warn about the
simple, non-reflective accesses to packages which are implicitly
exported by the kill switch as well.

You are correct. If the package is opened at run-time:

1. All public members of public types in the package are accessible.
This includes both static and reflective access.

2. All members of all types in the package can be reflected on when
using setAccessible or privateLookupIn.

The reason you don't get warnings with the static references is because
access checking in the VM is very low level and we deliberately decided
to avoid the invasive changes needs to report warnings here. The latest
proposal does not change this, it's just too late in JDK 9 to take on
this surgery.

-Alan

[Peter Levart]

Hi Alan,

Post by Alan Bateman

Post by Volker Simonis
From my understanding, at run-time, "open" implicates "exports" (i.e.
if a module M1 opens a package P for some other module M2 it also,
implicitly exports P to M2). The "big kill switch" in both, its old
and in the newly proposed form, usually only refers to "enabling
reflective access" but doesn't explicitly mentions that it will also,
implicitly export the respective packages. Also, the usage of the
"kill switch" only produces warnings for reflective accesses which are
enabled by the option itself (and not at the same time, explicitly
allowed by --add-opens directives). But it doesn't warn about the
simple, non-reflective accesses to packages which are implicitly
exported by the kill switch as well.

1. All public members of public types in the package are accessible.
This includes both static and reflective access.
2. All members of all types in the package can be reflected on when
using setAccessible or privateLookupIn.
The reason you don't get warnings with the static references is
because access checking in the VM is very low level and we
deliberately decided to avoid the invasive changes needs to report
warnings here. The latest proposal does not change this, it's just too
late in JDK 9 to take on this surgery.
-Alan

Ok, I can understand that such interpretation of "opens" is necessary
for JDK 8 compatibility story. If something is "open" then it behaves
exactly like in JDK 8 or before (apart from compilation) regardless of
packages being exported or not. But from consistency perspective,
explicit "exports" should be required just like "public" modifiers are
required to access members without .setAccessible(true) or without
.privateLookupIn().

This would not hinder the ability for frameworks to access members. It
would just require them to call .setAccessible(true)/.privateLookupIn()
in more places, but would be safer too as "suppressAccessChecks"
permission would be required for non-exported package access like it is
required for "private" members.

I'm sorry I haven't noticed this before. It is probably to late to
change this now. Was this deliberate choice (to promote compatibility
with existing code)?

Regards, Peter

[Alan Bateman]

Post by Peter Levart
Ok, I can understand that such interpretation of "opens" is necessary
for JDK 8 compatibility story. If something is "open" then it behaves
exactly like in JDK 8 or before (apart from compilation) regardless of
packages being exported or not. But from consistency perspective,
explicit "exports" should be required just like "public" modifiers are
required to access members without .setAccessible(true) or without
.privateLookupIn().
This would not hinder the ability for frameworks to access members. It
would just require them to call
.setAccessible(true)/.privateLookupIn() in more places, but would be
safer too as "suppressAccessChecks" permission would be required for
non-exported package access like it is required for "private" members.
I'm sorry I haven't noticed this before. It is probably to late to
change this now. Was this deliberate choice (to promote compatibility
with existing code)?

`opens` is intended to open a package for reflective access. Code can
use the reflection APIs to get at public members of public types in the
package, it shouldn't need setAccessible to do that.

The static reference case that Volker brought up is somewhat of a corner
case in this discussion. Readability aside, it would be inconsistent if
reflection APIs could access public members of public types in the
package but bytecode could not. The other corner case is code generation
at run-time which also should also work (assuming both visibility and
readability).

-Alan

[Peter Levart]

Hi Alan,

Post by Alan Bateman

Post by Peter Levart
Ok, I can understand that such interpretation of "opens" is necessary
for JDK 8 compatibility story. If something is "open" then it behaves
exactly like in JDK 8 or before (apart from compilation) regardless
of packages being exported or not. But from consistency perspective,
explicit "exports" should be required just like "public" modifiers
are required to access members without .setAccessible(true) or
without .privateLookupIn().
This would not hinder the ability for frameworks to access members.
It would just require them to call
.setAccessible(true)/.privateLookupIn() in more places, but would be
safer too as "suppressAccessChecks" permission would be required for
non-exported package access like it is required for "private" members.
I'm sorry I haven't noticed this before. It is probably to late to
change this now. Was this deliberate choice (to promote compatibility
with existing code)?

`opens` is intended to open a package for reflective access. Code can
use the reflection APIs to get at public members of public types in
the package, it shouldn't need setAccessible to do that.

Then "opens" is a sharp tool. And the proposal is about applying this
tool to the whole JDK by default!

Why I think this is not good? Because "opens", as it is, caries two
aspects or purposes at the same time. One is the purpose of giving
access to frameworks that introspect classes of a module and also access
them (DI frameworks for example) and this is a desirable purpose. The
other is the undesirable consequence of giving "unrestricted" access to
public types in the opened package during runtime. Introspective access
should be paired with a runtime permission in a secured system while
"unrestricted" access should only be given explicitly - using good old
"exports".

I see the problem with that. "opens" and "exports" are applied to
packages, but "open" can also be applied to the whole module and we
don't have an equivalent for "exports" on the module-level. But is that
really needed? Introspective frameworks should not have a problem with
invoking .setAccessible() as they could do that mechanically.

Opening the whole JDK (--illegal-access=permit by default) means that
all internal "public" APIs are made accessible if by chance someone can
grab an instance of target object and/or an instance of Method/Field
object. Imagine a JDK developer that thought that by putting a public
type into a concealed package was equivalent to making the type
module-private. This is a big surprise from the security perspective and
jdk.internal.misc.Unsafe.getUnsafe() might not be a lone example.

Post by Alan Bateman
The static reference case that Volker brought up is somewhat of a
corner case in this discussion. Readability aside, it would be
inconsistent if reflection APIs could access public members of public
types in the package but bytecode could not. The other corner case is
code generation at run-time which also should also work (assuming both
visibility and readability).

I get that. Generated code would have less access than reflection and
generated code is often used to replace reflection to speed things up.

Huh, not an easy choice, I agree...

Regards, Peter

Post by Alan Bateman
-Alan

[Alan Bateman]

Post by Peter Levart
Opening the whole JDK (--illegal-access=permit by default) means that
all internal "public" APIs are made accessible if by chance someone
can grab an instance of target object and/or an instance of
Method/Field object. Imagine a JDK developer that thought that by
putting a public type into a concealed package was equivalent to
making the type module-private. This is a big surprise from the
security perspective and jdk.internal.misc.Unsafe.getUnsafe() might
not be a lone example.

True although it's no different to JDK 8 and older behavior where all
public members of all public types in all packages were accessible to
code on the class path. Furthermore, setAccessible could be use to hack
everywhere. The proposal is really just giving libraries and tools more
time to sort out their issues.

-Alan

[Peter Levart]

Post by Alan Bateman

Post by Peter Levart
Opening the whole JDK (--illegal-access=permit by default) means that
all internal "public" APIs are made accessible if by chance someone
can grab an instance of target object and/or an instance of
Method/Field object. Imagine a JDK developer that thought that by
putting a public type into a concealed package was equivalent to
making the type module-private. This is a big surprise from the
security perspective and jdk.internal.misc.Unsafe.getUnsafe() might
not be a lone example.

True although it's no different to JDK 8 and older behavior where all
public members of all public types in all packages were accessible to
code on the class path.

Except that in the meanwhile a lot of internal code was written for JDK
9 that assumed the level of privacy provided by concealed packages. This
level is about to be changed by the proposal in the last minute...

Post by Alan Bateman
Furthermore, setAccessible could be use to hack everywhere. The
proposal is really just giving libraries and tools more time to sort
out their issues.
-Alan

Regards, Peter

[Peter Levart]

Hi,

Just a thought...

What about keeping the "opens pkg" and "open module m" as is - there is
a legitimate argument in keeping all accesses (static and reflective) in
sync.

But it does not mean that "--illegal-access=permit" must be equivalent
to --add-opens all-modules/all-packages=ALL-UNNAMED. It could mean a
slightly different thing - just enough to keep the compatibility with
JDK 8. There's no point in making jdk.internal.* packages open for
example - they are mostly new APIs and not relevant to backwards
compatibility.

More precisely, let "--illegal-access=permit" allow
.setAccessible(true) be called by unnamed module(s) on otherwise
inaccessible (private, package-private, protected) members/classes of
exported packages only.

The reasoning is as follows:

- selected JDK modules can explicitly open selected packages if they
feel they should keep runtime compatibility with applications but
prevent compilation with such APIs (like jdk.unsupported module does for
example). In JDK 10 they can close those packages again.
- I don't know what percentage of JDK-internal APIs used by applications
through reflection is in exported packages and what percent in
concealed. It would be interesting to know such statistics first.
- User modules can explicitly open selected packages or entire modules
if they feel so
- User automatic modules already export all packages, but to allow
access to their private members via reflection, they should perhaps be
equivalent to open modules with all packages exported in addition.

What do you think?

Regards, Peter

Post by Peter Levart

Post by Alan Bateman

Post by Peter Levart
Opening the whole JDK (--illegal-access=permit by default) means
that all internal "public" APIs are made accessible if by chance
someone can grab an instance of target object and/or an instance of
Method/Field object. Imagine a JDK developer that thought that by
putting a public type into a concealed package was equivalent to
making the type module-private. This is a big surprise from the
security perspective and jdk.internal.misc.Unsafe.getUnsafe() might
not be a lone example.

True although it's no different to JDK 8 and older behavior where all
public members of all public types in all packages were accessible to
code on the class path.

Except that in the meanwhile a lot of internal code was written for
JDK 9 that assumed the level of privacy provided by concealed
packages. This level is about to be changed by the proposal in the
last minute...

Post by Alan Bateman
Furthermore, setAccessible could be use to hack everywhere. The
proposal is really just giving libraries and tools more time to sort
out their issues.
-Alan

Regards, Peter

[Alan Bateman]

Post by Peter Levart
But it does not mean that "--illegal-access=permit" must be equivalent
to --add-opens all-modules/all-packages=ALL-UNNAMED. It could mean a
slightly different thing - just enough to keep the compatibility with
JDK 8. There's no point in making jdk.internal.* packages open for
example - they are mostly new APIs and not relevant to backwards
compatibility.

This is a good point as it would of course be a big set back to open new
packages (such as jdk.internal.misc as we have been discussing here).
The JDK 9 build has the list of the JDK 8 internal packages (as it is
needed for other tools) so it is feasible to not export the internal
packages that are "new" in JDK 9. We need to give the proposal a few
days to collect feedback before bringing up all the detailed issues
around this.

Post by Peter Levart
- selected JDK modules can explicitly open selected packages if they
feel they should keep runtime compatibility with applications but
prevent compilation with such APIs (like jdk.unsupported module does
for example). In JDK 10 they can close those packages again.

The JDK modules aren't going to export these packages so anyone
compiling code with JDK 9 and static references to types in these
packages will need to compile with `--add-exports`. You are right that a
possible option is to open specific packages but that has lots of
implications, particularly for the standard modules. It would also be
problematic for `--illegal-access=deny`, something that is important to
have in 9 to test how code will behave when the JDK internals are
eventually encapsulated.

Post by Peter Levart
- I don't know what percentage of JDK-internal APIs used by
applications through reflection is in exported packages and what
percent in concealed. It would be interesting to know such statistics
first.

We've presented data on static references to sun.* APIs at conferences
as it's not hard to do static analysis to get this data. It's much
harder to get data on code using reflection to get at JDK internals.
Early builds allowed deep reflection of all members of all types in
exported packages - that was the initial proposal in 2015 to keep as
much existing bad code working as possible. Fixing that issue
(#AwkwardStrongEncapsulation) exposed a lot more hacks with code that is
breaking into protected and private members of public types in exported
packages. There was a notable increase in the hate mail with that
change. There is also the anti-pattern that is
obj.getClass().invoke(...). I've tried of dozens of libraries and
frameworks to identify the hacks. In general they are all over the map
but there is a concentration of hacks on protected and private members
of java.base's exported packages.

Post by Peter Levart
- User modules can explicitly open selected packages or entire modules
if they feel so

Yes.

Post by Peter Levart
- User automatic modules already export all packages, but to allow
access to their private members via reflection, they should perhaps be
equivalent to open modules with all packages exported in addition.

Automatic modules have always opened all packages so I don't think there
is an issue there.

-Alan

[Volker Simonis]

Post by Volker Simonis
From my understanding, at run-time, "open" implicates "exports" (i.e.
if a module M1 opens a package P for some other module M2 it also,
implicitly exports P to M2). The "big kill switch" in both, its old
and in the newly proposed form, usually only refers to "enabling
reflective access" but doesn't explicitly mentions that it will also,
implicitly export the respective packages. Also, the usage of the
"kill switch" only produces warnings for reflective accesses which are
enabled by the option itself (and not at the same time, explicitly
allowed by --add-opens directives). But it doesn't warn about the
simple, non-reflective accesses to packages which are implicitly
exported by the kill switch as well.

1. All public members of public types in the package are accessible. This
includes both static and reflective access.
2. All members of all types in the package can be reflected on when using
setAccessible or privateLookupIn.
The reason you don't get warnings with the static references is because
access checking in the VM is very low level and we deliberately decided to
avoid the invasive changes needs to report warnings here. The latest
proposal does not change this, it's just too late in JDK 9 to take on this
surgery.

I understand. But than we should at least document it more thoroughly
that "--illegal-access=warn" really only warns for reflective accesses
and not for static ones (which are implicitly enabled by
"--illegal-access=permit"). We must make it clear to developers that
running an application warning-free with
"--illegal-access=permit/warn" doesn't necessarily mean that the same
application will also run correctly with "--illegal-access=deny".

Regards,
Volker

-Alan

[Anthony Vanelverdinghe]

As I haven't seen it mentioned yet: JDK 9 introduces a new
JDK_JAVA_OPTIONS environment variable. So I'd propose to:
* have `--illegal-access=deny` be the default
* add a `quiet` possibility
* not resolve the java.se.ee module by default
* add a note to the compatibility guide for JDK 9: if you want maximum
compatibility, set `JDK_JAVA_OPTIONS=--illegal-access=quiet
--add-modules=java.se.ee`, but be aware that these options will
dissapear in a future JDK

This way:
* there's no need to to add command-line parameters or manifest entries
anywhere (and since JDK_JAVA_OPTIONS is new in JDK 9, there's no
compatibility impact)
* the defaults are already what they ought to be

I believe the proposed default is a step backwards, and good arguments
have been given against doing this (a.o. by Peter Levart [1]). So the
above seems like a good middle ground to me: the defaults are what they
should be, and legacy behavior is simply a matter of setting an
environment variable.

[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012739.html

Kind regards,
Anthony

Post by m***@oracle.com
Over time, as we've gotten closer and closer to the JDK 9 GA date, more
and more developers have begun paying attention the actual changes in
this release. The strong encapsulation of JDK-internal APIs has, in
particular, triggered many worried expressions of concern that code that
works on JDK 8 today will not work on JDK 9 tomorrow, yet no advance
warning of this change was given in JDK 8.
To help the entire ecosystem migrate to the modular Java platform at a
more relaxed pace I hereby propose to allow illegal reflective access
from code on the class path by default in JDK 9, and to disallow it in
a future release.
In short, the existing "big kill switch" of the `--permit-illegal-access`
option [1] will become the default behavior of the JDK 9 run-time system,
though without as many warnings. The current behavior of JDK 9, in which
illegal reflective-access operations from code on the class path are not
permitted, will become the default in a future release. Nothing will
change at compile time.
In detail, the recently-introduced `--permit-illegal-access` option will
be replaced by a more-general option, `--illegal-access`. This option
`--illegal-access=permit`
This will be the default mode for JDK 9. It opens every package in
every explicit module to code in all unnamed modules, i.e., code on
the class path, just as `--permit-illegal-access` does today.
The first illegal reflective-access operation causes a warning to be
issued, as with `--permit-illegal-access`, but no warnings are issued
after that point. This single warning will describe how to enable
further warnings.
`--illegal-access=warn`
This causes a warning message to be issued for each illegal
reflective-access operation. This is equivalent to the current
`--permit-illegal-access` option.
`--illegal-access=debug`
This causes both a warning message and a stack trace to be shown
for each illegal reflective-access operation. This is equivalent
to combining today's `--permit-illegal-access` option with
`-Dsun.reflect.debugModuleAccessChecks`.
`--illegal-access=deny`
This disables all illegal reflective-access operations except for
those enabled by other command-line options, such as `--add-opens`.
This will become the default mode in a future release.
- The proposed default mode enables the run-time system to issue a
warning message, possibly at some time long after startup, without
having been explicitly requested to do so. This may be a surprise
in production environments, since it's extremely unusual for the
run-time system to issue any warning messages at all. If the default
mode permits illegal reflective access, however, then it's essential
to make that known so that people aren't surprised when this is no
longer the default mode in a future release.
- Warning messages in any mode can be avoided, as before, by the
judicious use of the `--add-exports` and `--add-opens` options.
- This proposal will, if adopted, require adjustments to JEP 260,
"Encapsulate Most Internal APIs" [2]. APIs that are internal to the
JDK will still be strongly encapsulated from the standpoint of code
in modules, whether those modules are automatic or explicit, but they
will not appear to be encapsulated at run time from the standpoint of
code on the class path.
- When `deny` becomes the default mode then I expect `permit` to remain
supported for at least one release, so that developers can continue
to migrate their code. The `permit`, `warn`, and `debug` modes will,
over time, be removed, as will the `--illegal-access` option itself.
(For launch-script compatibility the unsupported modes will most
likely just be ignored, after issuing a warning to that effect.)
- This change will not magically solve every JDK 9 adoption problem.
The concrete types of the built-in class loaders are still different,
`rt.jar` is still gone, the layout of a system image is still not the
same, and the version string still has a new format.
Comments?
- Mark
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-March/011763.html
[2] http://openjdk.java.net/jeps/260

[Reto Merz]

Post by Uwe Schindler
Hi Mark,
thanks for this proposal. I think it is pragmatic and well balanced.
While I personally agree with the advocates of a clean solution I also
understand the the needs of big (and slow) companies (including the
one I'm working for:) After all, the new default isn't that bad in my
opinion because it only affects applications/libraries on the class
path which is legacy anyway. If you convert your application/libraries
to modules it will still benefit from the full JPMS features and
restrictions. So altogether I think it is a good compromise.

I agree with Volker.

Thanks
Reto