Jonathan Gibbons
2018-07-26 23:42:33 UTC
There has been some recent discussion over the use of the javac and javadoc
`--module-source-path` options (and the lack of adequate documentation
thereof).
To help provide content for documentation, I created some examples,
based on the
use cases in the recent discussions, and came up against some issues
that led me
to some ideas for improving the way that the module source path for `javac`
and `javadoc` can be set.
As background, I note that the `--module-source-path` option was
designed very
early on in what is now the design of Jigsaw. It was designed when the
collective mindset was about single instances of non-repeatable options,
and it
had to support the JDK source structure. It was also designed before the
rewrite
of the JDK build system that allowed much more flexibility in terms of
computing and setting options. The module source path also has to
support the
ability to have a "leaky"[^1] path for the directories that provide the
source
for a single module, while maintaining the strong encapsulation that is
desired
for the overall set of modules on the module path.
Because of all that, it has a relatively complicated syntax. It was
intended to
provide flexibility for a variety of use cases, and for the most part it
succeeds, at the cost of some unfamiliar complexity. That being said, it has
always made one fundamental assumption that prevents a category of use
cases:
the assumption is that there will always be a directory named for the
module in
the overall path for any directory containing source for the module.
For example, the directory `my.module` in the following file tree for a
module
named `my.module`:
/my/library/my.module/src/share/classes/module-info.java
my/classes/C.java
The problem arises when working with a set of modules that do not follow
that
file naming convention, which is now seen to arise for some folk when
working
on modularizing existing libraries. Curiously, the problem is less likely to
arise with `javac`, because when compiling such a set of modules, you can
compile the modules one at a time "bottom-up", using single-module mode and
`--source-path`, placing all previously compiled modules on the module path
with `--module-path`. But with `javadoc`, you are more likely to want to
present the _source_ for all modules together to the tool, to be able to
build
a single documentation bundle for the overall set of modules.
You can almost workaround the problem by (ab)using `--patch-module`, which
allows you to specify a path to override the contents of a module, and for
`javac` and `javadoc`, you can put source in the patch path as well as
classes.
But `--patch-module` cannot be used to add arbitrary modules: any module
being
patched must also exist somewhere on the main module path.
That workaround hints at a solution, and a corresponding follow-up
discussion on
naming.
Proposal
========
Allow the module source path to be specified in a manner similar to the
`--patch-module`. The argument to `--patch-module` is of the form:
<module-name> `=` <file> ( <path-separator> <file> )*
This would allow a specific path to be specified for the source code of any
individual module. In the same way as for `--patch-module`, different
values
could be specified for different modules.
Whether or not to add a new option or modify the existing option is
discussed
separately, later.
Whatever the name of the option, it would be a repeatable option that can be
specified once for each module to be compiled, allowing a standard
(package-oriented) source path to be specified for each module.
This is just a proposal about the syntactic form of how the module
source path
is specified; there is no change to the underlying semantics of the module
source path. In particular, the effect in relation to any other options
specifying the locations of modules (including `patch-module`) is not
affected.
Given the arcane nature of the value of the existing option, it is
possible that
we might wish to eliminate the existing mechanism over a suitable period of
time. It has served its purpose, but the new proposal is a better, simpler
replacement.
The internal implementation for such a proposal is already supported at the
API level: see [StandardJavaFileManager.setLocation][setLocation]. This API
was added relatively late in JDK 9, to solve what was essentially the same
problem, as encountered by IDEs wanting to configure a file manager through
the API. (This proposal can be viewed as surfacing that API to the
command-line
option.)
Dropping the Old Form
---------------------
While the existing form is concise for compilation environments with lots of
modules, and with many source directories contributing to each module
(such as
shared source, platform-specific source, and generated source), that
advantage
is diminished in environments with only a few simple modules, which is
the more
common case. It may be that most users would prefer to always use the
new form:
while more verbose, it is conceptually much simpler and closer to existing
mechanisms, like plain old source paths and the patch module path.
The primary beneficiary of the existing form is the JDK build itself,
but it
should not be a big task to convert it to use the new form of the option
if we
should eventually decide the drop the old form.
Naming
------
How should we provide such new functionality?
1. A new option, alongside `--module-source-path`
2. Allow an alternate syntax for the existing `--module-source-path` option
Using a new option clearly provides the maximum isolation from the existing
option at the cost of giving up on the obvious name for an improved option.
The justification for the second choice is that the name is an obvious
one and
it is highly unlikely that any existing use of the
`--module-source-path` option
begins with `<module-name>=`. The existing syntax and the proposed new
syntax
are sufficiently distinct that we could introduce the new form, and support
both, until such time that we choose to retire the old form.
------
[^1]: "leaky": a convenient term used to describe the semantics of a
traditional
source path or classpath, such that the content of a container early in
the path
may hide like-named content later in the path.
[setLocation]:
https://docs.oracle.com/javase/9/docs/api/javax/tools/StandardJavaFileManager.html#setLocationForModule-javax.tools.JavaFileManager.Location-java.lang.String-java.util.Collection-
`--module-source-path` options (and the lack of adequate documentation
thereof).
To help provide content for documentation, I created some examples,
based on the
use cases in the recent discussions, and came up against some issues
that led me
to some ideas for improving the way that the module source path for `javac`
and `javadoc` can be set.
As background, I note that the `--module-source-path` option was
designed very
early on in what is now the design of Jigsaw. It was designed when the
collective mindset was about single instances of non-repeatable options,
and it
had to support the JDK source structure. It was also designed before the
rewrite
of the JDK build system that allowed much more flexibility in terms of
computing and setting options. The module source path also has to
support the
ability to have a "leaky"[^1] path for the directories that provide the
source
for a single module, while maintaining the strong encapsulation that is
desired
for the overall set of modules on the module path.
Because of all that, it has a relatively complicated syntax. It was
intended to
provide flexibility for a variety of use cases, and for the most part it
succeeds, at the cost of some unfamiliar complexity. That being said, it has
always made one fundamental assumption that prevents a category of use
cases:
the assumption is that there will always be a directory named for the
module in
the overall path for any directory containing source for the module.
For example, the directory `my.module` in the following file tree for a
module
named `my.module`:
/my/library/my.module/src/share/classes/module-info.java
my/classes/C.java
The problem arises when working with a set of modules that do not follow
that
file naming convention, which is now seen to arise for some folk when
working
on modularizing existing libraries. Curiously, the problem is less likely to
arise with `javac`, because when compiling such a set of modules, you can
compile the modules one at a time "bottom-up", using single-module mode and
`--source-path`, placing all previously compiled modules on the module path
with `--module-path`. But with `javadoc`, you are more likely to want to
present the _source_ for all modules together to the tool, to be able to
build
a single documentation bundle for the overall set of modules.
You can almost workaround the problem by (ab)using `--patch-module`, which
allows you to specify a path to override the contents of a module, and for
`javac` and `javadoc`, you can put source in the patch path as well as
classes.
But `--patch-module` cannot be used to add arbitrary modules: any module
being
patched must also exist somewhere on the main module path.
That workaround hints at a solution, and a corresponding follow-up
discussion on
naming.
Proposal
========
Allow the module source path to be specified in a manner similar to the
`--patch-module`. The argument to `--patch-module` is of the form:
<module-name> `=` <file> ( <path-separator> <file> )*
This would allow a specific path to be specified for the source code of any
individual module. In the same way as for `--patch-module`, different
values
could be specified for different modules.
Whether or not to add a new option or modify the existing option is
discussed
separately, later.
Whatever the name of the option, it would be a repeatable option that can be
specified once for each module to be compiled, allowing a standard
(package-oriented) source path to be specified for each module.
This is just a proposal about the syntactic form of how the module
source path
is specified; there is no change to the underlying semantics of the module
source path. In particular, the effect in relation to any other options
specifying the locations of modules (including `patch-module`) is not
affected.
Given the arcane nature of the value of the existing option, it is
possible that
we might wish to eliminate the existing mechanism over a suitable period of
time. It has served its purpose, but the new proposal is a better, simpler
replacement.
The internal implementation for such a proposal is already supported at the
API level: see [StandardJavaFileManager.setLocation][setLocation]. This API
was added relatively late in JDK 9, to solve what was essentially the same
problem, as encountered by IDEs wanting to configure a file manager through
the API. (This proposal can be viewed as surfacing that API to the
command-line
option.)
Dropping the Old Form
---------------------
While the existing form is concise for compilation environments with lots of
modules, and with many source directories contributing to each module
(such as
shared source, platform-specific source, and generated source), that
advantage
is diminished in environments with only a few simple modules, which is
the more
common case. It may be that most users would prefer to always use the
new form:
while more verbose, it is conceptually much simpler and closer to existing
mechanisms, like plain old source paths and the patch module path.
The primary beneficiary of the existing form is the JDK build itself,
but it
should not be a big task to convert it to use the new form of the option
if we
should eventually decide the drop the old form.
Naming
------
How should we provide such new functionality?
1. A new option, alongside `--module-source-path`
2. Allow an alternate syntax for the existing `--module-source-path` option
Using a new option clearly provides the maximum isolation from the existing
option at the cost of giving up on the obvious name for an improved option.
The justification for the second choice is that the name is an obvious
one and
it is highly unlikely that any existing use of the
`--module-source-path` option
begins with `<module-name>=`. The existing syntax and the proposed new
syntax
are sufficiently distinct that we could introduce the new form, and support
both, until such time that we choose to retire the old form.
------
[^1]: "leaky": a convenient term used to describe the semantics of a
traditional
source path or classpath, such that the content of a container early in
the path
may hide like-named content later in the path.
[setLocation]:
https://docs.oracle.com/javase/9/docs/api/javax/tools/StandardJavaFileManager.html#setLocationForModule-javax.tools.JavaFileManager.Location-java.lang.String-java.util.Collection-