ClassLoader.getResources vs Module.getResourceAsStream

[Michał Zegan]

Hello.
When reading docs for jdk9 and jdk10 it seems that those methods work in
a bit different way when it goes to encapsulation:
Module.getResourceAsStream will retrieve the resource without a problem
if a package is opened to the caller module, probably including the fact
that it will find a resource when the calling module is the same as one
represented by the module object.
But, ClassLoader.getResources and other resource methods seem to require
unconditional package open.
Why? I don't quite understand that distinction.

[Alan Bateman]

Post by Michał Zegan
Hello.
When reading docs for jdk9 and jdk10 it seems that those methods work in
Module.getResourceAsStream will retrieve the resource without a problem
if a package is opened to the caller module, probably including the fact
that it will find a resource when the calling module is the same as one
represented by the module object.
But, ClassLoader.getResources and other resource methods seem to require
unconditional package open.
Why? I don't quite understand that distinction.

ClassLoaders, especially in a delegation chain, have no notion of "who"
is trying to locate the resource. The ClassLoader.getResourceXXX methods
are also not final. All told, the ClassLoader.getResourceXXX cannot
reliably support qualified opens so this is why they are specified to
only locate resources in modules when the package is open to all modules.

The general guideline is to use Class or Module getResourceXXX when you
want to locate a resource in your own module or another specific module.
Use ClassLoader.getResourceXXX when you want to search the class path.
If you follow that then it makes it a lot easier to migrate existing
code to modules.

-Alan

[Michał Zegan]

What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.

Post by Alan Bateman

Post by Michał Zegan
Hello.
When reading docs for jdk9 and jdk10 it seems that those methods work in
Module.getResourceAsStream will retrieve the resource without a problem
if a package is opened to the caller module, probably including the fact
that it will find a resource when the calling module is the same as one
represented by the module object.
But, ClassLoader.getResources and other resource methods seem to require
unconditional package open.
Why? I don't quite understand that distinction.

ClassLoaders, especially in a delegation chain, have no notion of "who"
is trying to locate the resource. The ClassLoader.getResourceXXX methods
are also not final. All told, the ClassLoader.getResourceXXX cannot
reliably support qualified opens so this is why they are specified to
only locate resources in modules when the package is open to all modules.
The general guideline is to use Class or Module getResourceXXX when you
want to locate a resource in your own module or another specific module.
Use ClassLoader.getResourceXXX when you want to search the class path.
If you follow that then it makes it a lot easier to migrate existing
code to modules.
-Alan

[Alan Bateman]

Post by Michał Zegan
What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.

Services is the cleaner way to do this kind of thing, esp. if you have
flexibility to change the code and move away from legacy ClassLoader
getResources.

-Alan

[Michał Zegan]

It is a completely new code. It is not modularized for now because some
dependencies are not modularized, but I want it to be compatible to ease
later modularization.
My actual goal is to load xml files defining and configuring some
factories, from all modules that contain them. Not sure how would you do
that with services?

Post by Alan Bateman

Post by Michał Zegan
What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.

Services is the cleaner way to do this kind of thing, esp. if you have
flexibility to change the code and move away from legacy ClassLoader
getResources.
-Alan

[Sander Mak]

In that case you'd expose the factories through the services mechanism (`provides com.acme.api.MyWidgetFactory with com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if you must expose the XML itself to the outside world rather than the factories, you can create a service that offers the XML as String or InputStream. Each module then has an implementation reading its own encapsulated XML. The former approach is preferable IMO.

Note that you can also use the services mechanism without module descriptors by placing text files in META-INF/services (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if your code isn't a reusable library itself, you can also choose to modularize your own code and depend on non-modularized dependencies through their automatic module name (http://branchandbound.net/blog/java/2017/12/automatic-module-name/).

Sander

On 14 Jul 2018, at 17:58, Michał Zegan <***@poczta.onet.pl<mailto:***@poczta.onet.pl>> wrote:

It is a completely new code. It is not modularized for now because some
dependencies are not modularized, but I want it to be compatible to ease
later modularization.
My actual goal is to load xml files defining and configuring some
factories, from all modules that contain them. Not sure how would you do
that with services?

W dniu 14.07.2018 o 17:53, Alan Bateman pisze:
On 14/07/2018 16:38, Michał Zegan wrote:
What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.

Services is the cleaner way to do this kind of thing, esp. if you have
flexibility to change the code and move away fro

[Remi Forax]

----- Mail original -----

Envoyé: Lundi 16 Juillet 2018 08:52:41
Objet: Re: ClassLoader.getResources vs Module.getResourceAsStream
In that case you'd expose the factories through the services mechanism
(`provides com.acme.api.MyWidgetFactory with
com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if you
must expose the XML itself to the outside world rather than the factories, you
can create a service that offers the XML as String or InputStream. Each module
then has an implementation reading its own encapsulated XML. The former
approach is preferable IMO.
Note that you can also use the services mechanism without module descriptors by
placing text files in META-INF/services
(https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if your
code isn't a reusable library itself, you can also choose to modularize your
own code and depend on non-modularized dependencies through their automatic
module name
(http://branchandbound.net/blog/java/2017/12/automatic-module-name/).

You have to do both, each service declared in a module-info MUST be declared in META-INF/services too because you can use a modular jar in the classpath as a plain old jar.

Alan, we should patch jar to warn when there is a module-info.class that declare services with no corresponding META-INF/services.

Sander

Rémi

On 14 Jul 2018, at 17:58, Michał Zegan
It is a completely new code. It is not modularized for now because some
dependencies are not modularized, but I want it to be compatible to ease
later modularization.
My actual goal is to load xml files defining and configuring some
factories, from all modules that contain them. Not sure how would you do
that with services?
What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.
Services is the cleaner way to do this kind of thing, esp. if you have
flexibility to change the code and move away from legacy ClassLoader
getResources.
-Alan

[Alan Bateman]

Post by Remi Forax
Alan, we should patch jar to warn when there is a module-info.class that declare services with no corresponding META-INF/services.

I'll create an issue in JIRA for that. As things stands, the `jar` tool
does some sanity checks but it doesn't catch the case where the legacy
services configuration file doesn't match the module declaration. That
said, it probably needs the equivalent in one or more Maven plugins as
they are likely using the java.util.jar API rather than the `jar` tool.

-Alan

[Michał Zegan]

Well, these are not stateless factories that you register as a service
and then discover to later use them to create some objects.
Those are factories that implement a specific algorithm like crypto
algorithm, or that are adapters for a crypto library like for JCA, in
the second case one factory class can be used to create many factories
for many algorithms depending on properties set on them.
Concrete algo instances are created when needed by connection objects.
That is why I cannot register them as some kind of services, i need
configuration to be able to define what algorithms are supported using
what factory class with what configuration for each algorithm, and this
would be ugly to do in code. And I would actually like for this
mechanism to also load some extensions, like secondary files in other
jars/modules that may register other algorithms that are not registered
by core, using builtin or extension factory classes.
I currently want to put that in META-INF that seems to be non-encapsulated.

Post by Sander Mak
In that case you'd expose the factories through the services mechanism (`provides com.acme.api.MyWidgetFactory with com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if you must expose the XML itself to the outside world rather than the factories, you can create a service that offers the XML as String or InputStream. Each module then has an implementation reading its own encapsulated XML. The former approach is preferable IMO.
Note that you can also use the services mechanism without module descriptors by placing text files in META-INF/services (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if your code isn't a reusable library itself, you can also choose to modularize your own code and depend on non-modularized dependencies through their automatic module name (http://branchandbound.net/blog/java/2017/12/automatic-module-name/).
Sander
It is a completely new code. It is not modularized for now because some
dependencies are not modularized, but I want it to be compatible to ease
later modularization.
My actual goal is to load xml files defining and configuring some
factories, from all modules that contain them. Not sure how would you do
that with services?
What is then a recommendation for searching for all resources with name
x that i can access? Something like load all configuration including
some kind of extensions. I cannot list resources easily.
Services is the cleaner way to do this kind of thing, esp. if you have
flexibility to change the code and move away from legacy ClassLoader
getResources.
-Alan

[Stephen Colebourne]

In my experience (as I've written before), ClassLoader.getResources is
perhaps the biggest pain point I've experienced in trying to move
beyond Java 8. The method seems to have been very widely used, and IMO
was considered to be preferred over Class.getResourceXxx. And it is
very confusing to use once modules come into play.

There is no simple replacement for the ability to search for resources
across jar files. Module.getResourceXxx cannot be used as it only
returns one resource from the unamed module, when there could be many
(and the docs are no particularly clear on what they do). This is very
inconvenient. Returning a stream instead of a URL also typically
involves wider code change to adopt. In addition, many projects are
still on Java 8, so can't use java.lang.Module anyway.

ServiceLoader is completely the wrong solution for config files. Its
far too heavyweight.

My solution (which I think is pretty horrible) has been to move config
files to be under META-INF.

Old location:
org/joda/convert
New location:
META-INF/org/joda/convert

Its backwards incompatible to downstream users, but at least it works
with ClassLoader.getResources

Given there are no good solutions to normal coding problems, I can't
help feeling that Jigsaw didn't get resource access quite right.

Stephen

Post by Michał Zegan
Hello.
When reading docs for jdk9 and jdk10 it seems that those methods work in
Module.getResourceAsStream will retrieve the resource without a problem
if a package is opened to the caller module, probably including the fact
that it will find a resource when the calling module is the same as one
represented by the module object.
But, ClassLoader.getResources and other resource methods seem to require
unconditional package open.
Why? I don't quite understand that distinction.

ClassLoaders, especially in a delegation chain, have no notion of "who" is
trying to locate the resource. The ClassLoader.getResourceXXX methods are
also not final. All told, the ClassLoader.getResourceXXX cannot reliably
support qualified opens so this is why they are specified to only locate
resources in modules when the package is open to all modules.
The general guideline is to use Class or Module getResourceXXX when you want
to locate a resource in your own module or another specific module. Use
ClassLoader.getResourceXXX when you want to search the class path. If you
follow that then it makes it a lot easier to migrate existing code to
modules.
-Alan

[Alan Bateman]

Post by Stephen Colebourne
In my experience (as I've written before), ClassLoader.getResources is
perhaps the biggest pain point I've experienced in trying to move
beyond Java 8. The method seems to have been very widely used, and IMO
was considered to be preferred over Class.getResourceXxx. And it is
very confusing to use once modules come into play.

Class.getResourceXXX is the better choice when locating ones own
resources, ClassLoader.getResourceXX is for searching for a resource in
other components, the ClassLoader provide some context on where to
search. There are of course libraries that were using ClassLoader to
locate their own resources and those cases may need some changes in the
event that the code is migrated to an explicit module and the resources
are encapsulated. If they aren't encapsulated then the existing code
using the ClassLoader methods will work of course.

Post by Stephen Colebourne
There is no simple replacement for the ability to search for resources
across jar files.

The ClassLoader resources and getResources method work as before and
will locate resources in modular JARs when the resources are not
encapsulated.

Post by Stephen Colebourne
Module.getResourceXxx cannot be used as it only
returns one resource from the unamed module, when there could be many
(and the docs are no particularly clear on what they do). This is very
inconvenient.

Module.getResourceXXX works with named modules too.

-Alan

[Bernard Amade]

Post by Stephen Colebourne
ServiceLoader is completely the wrong solution for config files. Its
far too heavyweight.

after java 8
the case of internationalisation is the worst case of resource handling:
- creating diverse resource files (for different cultural contexts) means they might be spread along different jars
(you receive an app in your country - that uses a strange language- ... then you create a specific deployment jar and add it to the app)
- since it is impossible to have the same "directory" (oups I mean "package") in different modules/jars it is required to create a special code to find the real directory
this is super-clumsy!
please find a simpler approach ! (even if it entails exceptions to some principles)

thanks

[Alan Bateman]

Post by Bernard Amade
after java 8
- creating diverse resource files (for different cultural contexts) means they might be spread along different jars
(you receive an app in your country - that uses a strange language- ... then you create a specific deployment jar and add it to the app)
- since it is impossible to have the same "directory" (oups I mean "package") in different modules/jars it is required to create a special code to find the real directory
this is super-clumsy!
please find a simpler approach ! (even if it entails exceptions to some principles)

This sounds like a complaint about ResourceBundle. Best to start a new
thread if you have issues or suggestions for how it can be improved.
Also keep in mind that a huge effort went into getting ResourceBundle to
work with modules and there are several options for deploying
translations in different JAR files or modules (esp. if these
translation are .properties files rather than compiled resources). The
options are detailed in the ResourceBundle javadoc [1].

-Alan

[1]
https://download.java.net/java/early_access/jdk11/docs/api/java.base/java/util/ResourceBundle.html