How does one handle a java method which returns a non-public sub-type via reflection?

[jeffrey kutcher]

This is a good discussion but none of the suggestions satisfy the need I'm looking for.https://stackoverflow.com/questions/450807/how-do-i-make-the-method-return-type-generic

The example code below shows button0 being added to a VBox. It also shows button1 being added to the same VBox only instead of being added directly to the list returned by vbox.getChildren(), that is vbox.getChildren().add(button1), buttton1 is added via reflection/introspection. The end goal is the same in both cases. Add a button. The methodology is different. Regardless, this should be trivial. It's not. Why?
In this example, adding button0 produces no compile time warnings, Attempt-1. As for the code adding button1, Attempt-3 too produces no compile time warning, yet when run produces the WARNINGS shown at the bottom of the code. Commenting out Attempt-3 and uncommenting Attempt-2, produces compile time warnings.
The bottom of the code example shows how to compile and run the example.
As is, this code references no illegal reference code and yet still produces run time WARNINGS.
Is there a way to access the list returned by getChildren() via reflection so a widget can be added without producing the run time WARNINGS? It's not the WARNINGS that I'm concerned about. I'm concerned that when future Java releases become available this code will fail to work. I would like to find a solution so that doesn't happen and this code continues to work.
Maps have similar behavior. Try and use reflection to iterate over a Map, a trivial exercise, and it quickly becomes an impossible task. The internal class of the Map isn't public making it impossible to iterate through the list. I'm assuming that is what we are seeing here as well.
Personally I believe internal classes should never be used. They seem to produce more problems than they solve. They are a quick, lazy fix that usually results in painful long term support. Besides, internal classes defeat write once, run anywhere. If it's internal, it's not being used anywhere else; which means the code has no useful value elsewhere; obviously false. That implies the code will be duplicated elsewhere to leverage it's functionality, meaning the code is now written twice, defeating the montra. Therefore, internal classes should never be used. In the same spirit, classes should never be private for the same reason. The nice thing is, convention easily satisfies this suggestion. Just because you can, doesn't mean you should.
Java has been out for 23 years. Does a solution exist? I would like to see this example code work for future releases of Java.
Is there a solution? Would you please point me at an example.
Thanks
---------- BEGIN SOURCE ----------

public class ReflectAccessBug extends javafx.application.Application {

public static void main(String[] args) {
launch(args);
}

public void start(javafx.stage.Stage stage) {
javafx.scene.control.Button button0 = new javafx.scene.control.Button();
javafx.scene.control.Button button1 = new javafx.scene.control.Button();
javafx.scene.layout.VBox vbox = new javafx.scene.layout.VBox();

javafx.event.EventHandler<javafx.event.ActionEvent> event = e -> {
javafx.scene.control.Button b =
(javafx.scene.control.Button)e.getSource();
System.err.println(b.getText() + " " + e);
};

button0.setText("button0");
button0.setOnAction(event);

button1.setText("button1");
button1.setOnAction(event);

vbox.getChildren().add(button0);

// Attempt-1
// This works as expected.
//vbox.getChildren().add(button1);

try {
// Now let's try adding button1 using introspection ...
java.lang.reflect.Method m =
vbox.getClass().getMethod("getChildren", new Class[0]);
Object o = m.invoke(vbox, new Object[0]);
/*
// Attempt-2
// This causes the same runtime warning as Attempt-3
// and I don't believe it should.
// Attempt-2 produces a compile time warning.
// Attempt-3 eliminates the compile time warning.
javafx.collections.ObservableList<javafx.scene.Node> l =
(javafx.collections.ObservableList<javafx.scene.Node>)o;
m = l.getClass().getMethod("add", new Class[] { Object.class, });
o = m.invoke(l, new Object[] { button1, });
System.err.println("o="+o);
*/
// Attempt-3
// This results with the same runtime warning as Attempt-2.
// I don't believe it should produce this warning and is the
// point of this exercise.
// Attempt-3 produces *NO* compile time warning as in Attempt-2.
// It's not the compile time warning I'm concerned about. It's the
// runtime warning that is the issue.
// This code references no illegal reference code and yet still
// produces a runtime warning. It references
// "illegal reflective access operations". It's not this code
// that is illegally accessing reflective code. Lower level code
// accesses illegal reflective code on behalf of this code.
m = o.getClass().getMethod("add", new Class[] { Object.class, });
o = m.invoke(o, new Object[] { button1, });
System.err.println("o="+o);
} catch (Exception x) {
x.printStackTrace();
}

javafx.scene.Scene scene0 = new javafx.scene.Scene(vbox);
stage.setScene(scene0);
stage.show();
}

}

/*
# compile:
javac ReflectAccessBug.java

# run:
java ReflectAccessBug
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by ReflectAccessBug (...) to method com.sun.javafx.collections.VetoableListDecorator.add(java.lang.Object)
WARNING: Please consider reporting this to the maintainers of ReflectAccessBug
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
*/

---------- END SOURCE ----------

[Peter Levart]

Hi Jeffrey,

See my answer inline...

Post by jeffrey kutcher
This is a good discussion but none of the suggestions satisfy the need I'm looking for.https://stackoverflow.com/questions/450807/how-do-i-make-the-method-return-type-generic
The example code below shows button0 being added to a VBox. It also shows button1 being added to the same VBox only instead of being added directly to the list returned by vbox.getChildren(), that is vbox.getChildren().add(button1), buttton1 is added via reflection/introspection. The end goal is the same in both cases. Add a button. The methodology is different. Regardless, this should be trivial. It's not. Why?
In this example, adding button0 produces no compile time warnings, Attempt-1. As for the code adding button1, Attempt-3 too produces no compile time warning, yet when run produces the WARNINGS shown at the bottom of the code. Commenting out Attempt-3 and uncommenting Attempt-2, produces compile time warnings.
The bottom of the code example shows how to compile and run the example.
As is, this code references no illegal reference code and yet still produces run time WARNINGS.
Is there a way to access the list returned by getChildren() via reflection so a widget can be added without producing the run time WARNINGS? It's not the WARNINGS that I'm concerned about. I'm concerned that when future Java releases become available this code will fail to work. I would like to find a solution so that doesn't happen and this code continues to work.
Maps have similar behavior. Try and use reflection to iterate over a Map, a trivial exercise, and it quickly becomes an impossible task. The internal class of the Map isn't public making it impossible to iterate through the list. I'm assuming that is what we are seeing here as well.
Personally I believe internal classes should never be used. They seem to produce more problems than they solve. They are a quick, lazy fix that usually results in painful long term support. Besides, internal classes defeat write once, run anywhere. If it's internal, it's not being used anywhere else; which means the code has no useful value elsewhere; obviously false. That implies the code will be duplicated elsewhere to leverage it's functionality, meaning the code is now written twice, defeating the montra. Therefore, internal classes should never be used. In the same spirit, classes should never be private for the same reason. The nice thing is, convention easily satisfies this suggestion. Just because you can, doesn't mean you should.
Java has been out for 23 years. Does a solution exist? I would like to see this example code work for future releases of Java.
Is there a solution? Would you please point me at an example.
Thanks
---------- BEGIN SOURCE ----------
public class ReflectAccessBug extends javafx.application.Application {
public static void main(String[] args) {
launch(args);
}
public void start(javafx.stage.Stage stage) {
javafx.scene.control.Button button0 = new javafx.scene.control.Button();
javafx.scene.control.Button button1 = new javafx.scene.control.Button();
javafx.scene.layout.VBox vbox = new javafx.scene.layout.VBox();
javafx.event.EventHandler<javafx.event.ActionEvent> event = e -> {
javafx.scene.control.Button b =
(javafx.scene.control.Button)e.getSource();
System.err.println(b.getText() + " " + e);
};
button0.setText("button0");
button0.setOnAction(event);
button1.setText("button1");
button1.setOnAction(event);
vbox.getChildren().add(button0);
// Attempt-1
// This works as expected.
//vbox.getChildren().add(button1);
try {
// Now let's try adding button1 using introspection ...
java.lang.reflect.Method m =
vbox.getClass().getMethod("getChildren", new Class[0]);
Object o = m.invoke(vbox, new Object[0]);
/*
// Attempt-2
// This causes the same runtime warning as Attempt-3
// and I don't believe it should.
// Attempt-2 produces a compile time warning.
// Attempt-3 eliminates the compile time warning.
javafx.collections.ObservableList<javafx.scene.Node> l =
(javafx.collections.ObservableList<javafx.scene.Node>)o;
m = l.getClass().getMethod("add", new Class[] { Object.class, });

Try this instead of above line:

m = javafx.collections.ObservableList.class.getMethod("add", new Class[]
{ Object.class, });
m.invoke(o, ....);

The problem is that when you say: o.getClass().getMethod(....), you
start searching for a method in the runtime implementation class of the
object 'o' (it doesn't matter if you assign variable 'o' to 'l' with a
cast - 'l' will point to the same runtime object as 'o'). The method
might be found in some class/interface that is not public or not
exported. Class#getMethod always returns the Method object describing
the most specific method - the method declared in the most specific type
(see the javadoc of getMethod() for more accurate description).  What
you may get back from such call is a Method object describing the
declaration of the method in the implementation class. Although such
method is public, the declaring class (implementation class) might not
be accessible (either not public or not exported).

You better start searching for a method in some well known public type
(ObservableList interface for example) that you know declares the method
that is implemented or overridden in the runtime implementation class.
You will get a different Method object, describing the abstract
interface method, but when you invoke the method via this Method object,
the invocation will be dispatched to the most specific implementation of
that method because it obeys the virtual dispatch semantics. Reflection
access checks are unfortunately performed with the declaring type/method
described in the Method object and not with the method that the
invocation actually dispatches to.

Hope this helps.

Regards, Peter

Post by jeffrey kutcher
o = m.invoke(l, new Object[] { button1, });
System.err.println("o="+o);
*/
// Attempt-3
// This results with the same runtime warning as Attempt-2.
// I don't believe it should produce this warning and is the
// point of this exercise.
// Attempt-3 produces *NO* compile time warning as in Attempt-2.
// It's not the compile time warning I'm concerned about. It's the
// runtime warning that is the issue.
// This code references no illegal reference code and yet still
// produces a runtime warning. It references
// "illegal reflective access operations". It's not this code
// that is illegally accessing reflective code. Lower level code
// accesses illegal reflective code on behalf of this code.
m = o.getClass().getMethod("add", new Class[] { Object.class, });
o = m.invoke(o, new Object[] { button1, });
System.err.println("o="+o);
} catch (Exception x) {
x.printStackTrace();
}
javafx.scene.Scene scene0 = new javafx.scene.Scene(vbox);
stage.setScene(scene0);
stage.show();
}
}
/*
javac ReflectAccessBug.java
java ReflectAccessBug
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by ReflectAccessBug (...) to method com.sun.javafx.collections.VetoableListDecorator.add(java.lang.Object)
WARNING: Please consider reporting this to the maintainers of ReflectAccessBug
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
*/
---------- END SOURCE ----------

[mandy chung]

Post by jeffrey kutcher
m = o.getClass().getMethod("add", new Class[] { Object.class, });
o = m.invoke(o, new Object[] { button1, });

o.getClass().getMethod(...) is an anti-pattern for finding a public
method.   Object.getClass() returns the implementation class while you
want to invoke a public method `javafx.collections.ObservableList::add`
in this case.  In this case, the declaring class of the method is known
and so one way to fix it is to use the specific Class:

Class<?> observableListClass = javafx.collections.ObservableList.class;
m = observableListClass.getMethod("add",new Class[] { Object.class, });
o = m.invoke(o,new Object[] { button1, });

Mandy

[Michał Zegan]

Actually, shouldn't reflection be fixed so that you can invoke any
method of a non exported class, if the method overrides one that is
accessible to the class calling invoke... Not sure it makes sense...

Post by mandy chung

             m = o.getClass().getMethod("add", new Class[] {
Object.class, });
             o = m.invoke(o, new Object[] { button1, });

o.getClass().getMethod(...) is an anti-pattern for finding a public
method.   Object.getClass() returns the implementation class while you
want to invoke a public method `javafx.collections.ObservableList::add`
in this case.  In this case, the declaring class of the method is known
Class<?> observableListClass = javafx.collections.ObservableList.class;
m = observableListClass.getMethod("add",new Class[] { Object.class, });
o = m.invoke(o,new Object[] { button1, });
Mandy

[jeffrey kutcher]

The idea is not to be specific, but to be general. This is an example. In practice, I don't know what "o" is and so there has to be a discovery phase. I'm not sure how to make it more general than discovering the class via o.getClass(). As in this example, yes the class object returned is an ObservableList but I'm unable to add an object to this list via reflection. Why? And what can be done to change that? Why should it be changed? Because an object can be added to the list via vbox.getChildren().add(). Therefore an object should be permitted to be added via introspection. The avenue for the addition is different, the end result should be the same.
There are lots of other classes in the Java libraries that have this same issue. 
There's also the issue of what gets returned can be many different types. How do you figure out which type should be returned? Not a trivial question; not so sure there's an answer. In this case, all I need is a List. Not an ArrayList, or an ObservableList, LinkedList, SortedList .... Please don't focus on this ... yet. Original issue first.
On Friday, January 12, 2018, 1:25:13 PM CST, mandy chung <***@oracle.com> wrote:



On 1/12/18 10:26 AM, jeffrey kutcher wrote:

m = o.getClass().getMethod("add", new Class[] { Object.class, });
o = m.invoke(o, new Object[] { button1, });

o.getClass().getMethod(...) is an anti-pattern for finding a public method.   Object.getClass() returns the implementation class while you want to invoke a public method `javafx.collections.ObservableList::add` in this case.  In this case, the declaring class of the method is known and so one way to fix it is to use the specific Class:
Class<?> observableListClass = javafx.collections.ObservableList.class;
m = observableListClass.getMethod("add", new Class[] { Object.class, });
o = m.invoke(o, new Object[] { button1, });

Mandy

[Michał Zegan]

well you rather know what getChildren returns, so you know what the
returned object is, like generally.

Post by jeffrey kutcher
The idea is not to be specific, but to be general. This is an example. In practice, I don't know what "o" is and so there has to be a discovery phase. I'm not sure how to make it more general than discovering the class via o.getClass(). As in this example, yes the class object returned is an ObservableList but I'm unable to add an object to this list via reflection. Why? And what can be done to change that? Why should it be changed? Because an object can be added to the list via vbox.getChildren().add(). Therefore an object should be permitted to be added via introspection. The avenue for the addition is different, the end result should be the same.
There are lots of other classes in the Java libraries that have this same issue. 
There's also the issue of what gets returned can be many different types. How do you figure out which type should be returned? Not a trivial question; not so sure there's an answer. In this case, all I need is a List. Not an ArrayList, or an ObservableList, LinkedList, SortedList .... Please don't focus on this ... yet. Original issue first.
m = o.getClass().getMethod("add", new Class[] { Object.class, });
o = m.invoke(o, new Object[] { button1, });
Class<?> observableListClass = javafx.collections.ObservableList.class;
m = observableListClass.getMethod("add", new Class[] { Object.class, });
o = m.invoke(o, new Object[] { button1, });
Mandy

[Jochen Theodorou]

Post by Michał Zegan
well you rather know what getChildren returns, so you know what the
returned object is, like generally.

but as soon as the method you want to call is not on the statically
declared type, nor within a set of known statically types you are having
a problem. There is no good way (good in terms of usability and
performance) to get an arbitrary valid method object without static
information. You are always forced to traverse the hierarchy to find the
most accessible type. You will have to check access... and then of
course there the two bad scenarios where invocation and search are not
the same class or actual caller and caller are not equal. Which leads to
a lengthy search using reflection to find a method, transfer it to a
method handle and then let the invoker do the actual call.

bye Jochen

[jeffrey kutcher]

Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be.
You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated.
Introspection of vbox.getChildren().add(x) should be just as simple as calling vbox.getChildren().add(x) itself, otherwise why have introspection at all?

Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system.
I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances.
Looking for an Oracle to glean insight (implications of a lock down)
I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work.
I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation.
It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever.

I'm sure there are others.

Post by Michał Zegan
well you rather know what getChildren returns, so you know what the
returned object is, like generally.

but as soon as the method you want to call is not on the statically
declared type, nor within a set of known statically types you are having
a problem. There is no good way (good in terms of usability and
performance) to get an arbitrary valid method object without static
information. You are always forced to traverse the hierarchy to find the
most accessible type. You will have to check access... and then of
course there the two bad scenarios where invocation and search are not
the same class or actual caller and caller are not equal. Which leads to
a lengthy search using reflection to find a method, transfer it to a
method handle and then let the invoker do the actual call.

bye Jochen

[Alan Bateman]

Post by jeffrey kutcher
I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence.

It's always been possible for getClass() to return a non-public class
and for the Method invoke in the code example to fail with
IllegalAccessException. You may have just got lucky in the past.  At
some point then I would expect the JavaFX modules to not be open by
default so maybe now is the time to fix the issues. You can run with
`--illegal-access=warn` to help track down other code that may be
accidentally trying to access members of JDK internal classes.  You can
run with `--illegal-access=deny` to see how that code behaves when the
classes in the JDK modules are not open for illegal access.

-Alan

[jeffrey kutcher]

So developers have to be concerned with how the underlying classes are implemented and code to the implementation. This breaks the contract that is fundamental to Java. For example, the point to garbage collection was don't worry about how garbage collection works. Write your code in the pure sense and over time the garbage collector will get better due to the wizards that are interested in making the garbage collector scream. Leverage the technology and your code will inherently get better because the underlying technology being used is so much better. That holds true for the compiler and every other component of the JDK.
Stay away from native functions. You will break portability.
Again, there should be no concern about the underlying architecture. The example code provided in this post is 100% portable. So either reflection needs to be rewritten or deprecated or classes like <x>Map and <x>Box that can be reflected on needs rewriting so reflection can work properly. If a developer has to worry about how libraries are written, the libraries need to be rewritten.
Maybe those libraries that have internal private classes need to provide the necessary getters and setters so that it won't be necessary to access those internal private classes. Instead of calling vbox.getChildren().add(..) maybe provide vbox.add(..) which will intern call vbox.getChildren().add(..).
Some libraries work as expected. ArrayList, BitSet or JRadioButton are good examples, among many others (proof it has be done).
Just because you can doesn't mean you should.
As a side note which may contribute to resolving this post's issue, internal classes should never be used. First, reflecting on a private class can possibly happen resulting in this post's issue. Second, if you're using an internal class then you're saying that the code will have no use elsewhere which contradicts write once, run anywhere because everyone knows that the point of objects is reuse and if you build something that good, it's a sin to keep it private. Other's will inevitably cut and paste the code elsewhere duplicating the code adding system bloat and now support in more than one location suggesting that the private class should be made public and accessible to others which will contribute to resolving this post's original issue. The internal private class giving us this issue is no longer private.
Just because you can use internal classes doesn't mean you should. From a purist standpoint, this is an easy fix. Just don't do it by convention. The result is all related problems have a probability of zero that they will ever exist. That's pretty good for adhering to simple convention.

Post by jeffrey kutcher
I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence.

It's always been possible for getClass() to return a non-public class
and for the Method invoke in the code example to fail with
IllegalAccessException. You may have just got lucky in the past.  At
some point then I would expect the JavaFX modules to not be open by
default so maybe now is the time to fix the issues. You can run with
`--illegal-access=warn` to help track down other code that may be
accidentally trying to access members of JDK internal classes.  You can
run with `--illegal-access=deny` to see how that code behaves when the
classes in the JDK modules are not open for illegal access.

-Alan

[David Holmes]

Post by jeffrey kutcher
Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be.
You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated.
Introspection of vbox.getChildren().add(x) should be just as simple as calling vbox.getChildren().add(x) itself, otherwise why have introspection at all?

I recall a very similar discussion in the past. You have to start with
(public) statically known types, not with the dynamic type of the
instances involved. In essence you have to reflectively emulate the
method resolution process that occurs with a direct call and which uses
static type information.

David
-----

Post by jeffrey kutcher
Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system.
I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances.
Looking for an Oracle to glean insight (implications of a lock down)
I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work.
I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation.
It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever.
I'm sure there are others.

Post by Michał Zegan
well you rather know what getChildren returns, so you know what the
returned object is, like generally.

but as soon as the method you want to call is not on the statically
declared type, nor within a set of known statically types you are having
a problem. There is no good way (good in terms of usability and
performance) to get an arbitrary valid method object without static
information. You are always forced to traverse the hierarchy to find the
most accessible type. You will have to check access... and then of
course there the two bad scenarios where invocation and search are not
the same class or actual caller and caller are not equal. Which leads to
a lengthy search using reflection to find a method, transfer it to a
method handle and then let the invoker do the actual call.
bye Jochen

[Peter Levart]

Post by David Holmes
I recall a very similar discussion in the past. You have to start with
(public) statically known types, not with the dynamic type of the
instances involved. In essence you have to reflectively emulate the
method resolution process that occurs with a direct call and which
uses static type information.
David
-----

Exactly, and in the example given by Jeffrey:


javafx.scene.control.Button button1 = ...;
javafx.scene.layout.VBox vbox = ...;

vbox.getChildren().add(button1);


The right translation of the last line to reflective invocations would
be this (skipping exception handling):


Method getChildrenMethod =
javafx.scene.layout.VBox.class.getMethod("getChildren");
Object children = getChildrenMethod.invoke(vbox);

Method addMethod = getChildrenMethod.getReturnType().getMethod("add",
javafx.scene.control.Button.class);
addMethod.invoke(children, button1);


You see, the compiler (javac) uses static type information (of the
locals for example) to locate the correct method. In case of method
chaining, it uses the declared return type of the previous method, etc...

Regards, Peter

[jeffrey kutcher]

Trying to find documentation for the method resolution process.

Found this:

https://stackoverflow.com/questions/6021109/java-runtime-method-resolution

which leads to

http://www.cowtowncoder.com/blog/archives/2010/12/entry_436.html

Maybe this is an opportunity for someone to put together a class library that does the right thing all the time regarding the method resolution process. Does the Java library already have this somewhere in it's source? If so, why not expose it making it easier for others so those interested don't have to reinvent the wheel every time.


Is there official documentation explaining the method resolution process somewhere?

Post by jeffrey kutcher
 
Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be.
You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated.
Introspection of vbox.getChildren().add(x) should be just as simple as calling vbox.getChildren().add(x) itself, otherwise why have introspection at all?

I recall a very similar discussion in the past. You have to start with
(public) statically known types, not with the dynamic type of the
instances involved. In essence you have to reflectively emulate the
method resolution process that occurs with a direct call and which uses
static type information.

David

-----

Post by jeffrey kutcher
Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system.
I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances.
Looking for an Oracle to glean insight (implications of a lock down)
I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work.
I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation.
It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever.
I'm sure there are others.
 
 

Post by Michał Zegan
well you rather know what getChildren returns, so you know what the
returned object is, like generally.

but as soon as the method you want to call is not on the statically
declared type, nor within a set of known statically types you are having
a problem. There is no good way (good in terms of usability and
performance) to get an arbitrary valid method object without static
information. You are always forced to traverse the hierarchy to find the
most accessible type. You will have to check access... and then of
course there the two bad scenarios where invocation and search are not
the same class or actual caller and caller are not equal. Which leads to
a lengthy search using reflection to find a method, transfer it to a
method handle and then let the invoker do the actual call.
bye Jochen

[dalibor topic]

On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official
documentation explaining the method resolution process somewhere?
I'd suggest taking a look at the Java Language specification and JVM
specification for details.

cheers,
dalibor topic
--
<http://www.oracle.com> Dalibor Topic | Principal Product Manager
Phone: [phone] <tel:[phone]> | Mobile: [phone]
<tel:[phone]>

ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 22761 Hamburg

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher

<http://www.oracle.com/commitment> Oracle is committed to developing
practices and products that help protect the environment

[jeffrey kutcher]

There's nothing in the JLS by that name however, section 4.5 Parameterized Types, might be close to what I'm looking for.
Even in a single inheritance system, resolving methods and types is a multi-dimensional process.
I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]).
On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic <***@oracle.com> wrote:



On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official
documentation explaining the method resolution process somewhere?
I'd suggest taking a look at the Java Language specification and JVM
specification for details.

cheers,
dalibor topic
--
<http://www.oracle.com> Dalibor Topic | Principal Product Manager
Phone: [phone] <tel:[phone]> | Mobile: [phone]
<tel:[phone]>

ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 22761 Hamburg

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher

<http://www.oracle.com/commitment> Oracle is committed to developing
practices and products that help protect the environment

[Kevin Rushforth]

There are many good reasons for using non-public classes in the
implementation of a class library, the main one being to keep
implementation details from leaking into the public API. So while I
"get" that it causes difficulties in your specific case, I don't agree
that the solution is to avoid using non-public syb-types.

-- Kevin

There's nothing in the JLS by that name however, section 4.5 Parameterized Types, might be close to what I'm looking for.
Even in a single inheritance system, resolving methods and types is a multi-dimensional process.
I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]).
On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official
documentation explaining the method resolution process somewhere?
I'd suggest taking a look at the Java Language specification and JVM
specification for details.
cheers,
dalibor topic

[jeffrey kutcher]

Then in this case, shouldn't getChildren() not be permitted? You're exposing a non-public sub-type.
On Tuesday, January 16, 2018, 8:23:50 AM CST, Kevin Rushforth <***@oracle.com> wrote:

There are many good reasons for using non-public classes in the
implementation of a class library, the main one being to keep
implementation details from leaking into the public API. So while I
"get" that it causes difficulties in your specific case, I don't agree
that the solution is to avoid using non-public syb-types.

-- Kevin

  There's nothing in the JLS by that name however, section 4.5 Parameterized Types, might be close to what I'm looking for.
Even in a single inheritance system, resolving methods and types is a multi-dimensional process.
I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]).
 
 
On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official
documentation explaining the method resolution process somewhere?
I'd suggest taking a look at the Java Language specification and JVM
specification for details.
cheers,
dalibor topic
 

[Kevin Rushforth]

No, the fact that it returns a subtype is irrelevant to a program that
uses it as an instance of the base class. The key here is to know what
the formal declaration of the return type is, either statically or via
reflection (e.g., using Method::getGenericReturnType).

-- Kevin

Post by jeffrey kutcher
Then in this case, shouldn't getChildren() not be permitted? You're
exposing a non-public sub-type.
On Tuesday, January 16, 2018, 8:23:50 AM CST, Kevin Rushforth
There are many good reasons for using non-public classes in the
implementation of a class library, the main one being to keep
implementation details from leaking into the public API. So while I
"get" that it causes difficulties in your specific case, I don't agree
that the solution is to avoid using non-public syb-types.
-- Kevin

Post by jeffrey kutcher
There's nothing in the JLS by that name however, section 4.5

Parameterized Types, might be close to what I'm looking for.

Post by jeffrey kutcher
Even in a single inheritance system, resolving methods and types is

a multi-dimensional process.

Post by jeffrey kutcher
I still say that by not allowing private inner classes, resolving

this issue would be much easier (or providing the access methods in
the parent class of the private internal class would work also ...
calling getChildren() really is a dependency on the underlying classes
implementation which I really don't understand why is declared private
and not accessible. There has to be a good reason otherwise it should
be changed or eliminated [get rid of inner classes]).

Post by jeffrey kutcher
On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic
On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official
documentation explaining the method resolution process somewhere?
I'd suggest taking a look at the Java Language specification and JVM
specification for details.
cheers,
dalibor topic

[jeffrey kutcher]

In this example, if I can access and execute the method vbox.getChildren().add() with no warnings or errors, which is accessing a non-public class, then I should also able to access that very same class and execute the same method using reflection with no warnings or errors and expect the same results. If not, the inconsistency may be a bug, feature or broken. Maybe that's the way it's suppose to work. Inconsistency however tells me something isn't quite right and if ignored will cause more issues later on. In this particular case, a lock down will completely stop currently working code. I'd like to not see that happen. If it does? I'll move on.
I don't disagree that there are good reasons for using non-public classes.
On Tuesday, January 16, 2018, 8:23:50 AM CST, Kevin Rushforth <***@oracle.com> wrote:

There are many good reasons for using non-public classes in the
implementation of a class library, the main one being to keep
implementation details from leaking into the public API. So while I
"get" that it causes difficulties in your specific case, I don't agree
that the solution is to avoid using non-public syb-types.

-- Kevin

  There's nothing in the JLS by that name however, section 4.5 Parameterized Types, might be close to what I'm looking for.
Even in a single inheritance system, resolving methods and types is a multi-dimensional process.
I still say that by not allowing private inner classes, resolving this issue would be much easier (or providing the access methods in the parent class of the private internal class would work also ... calling getChildren() really is a dependency on the underlying classes implementation which I really don't understand why is declared private and not accessible. There has to be a good reason otherwise it should be changed or eliminated [get rid of inner classes]).
 
 
On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official
documentation explaining the method resolution process somewhere?
I'd suggest taking a look at the Java Language specification and JVM
specification for details.
cheers,
dalibor topic
 

[Jochen Theodorou]

Post by jeffrey kutcher
In this example, if I can access and execute the method vbox.getChildren().add() with no warnings or errors, which is accessing a non-public class

In Java the static type is the important one. Of course you can do

VBox vbox = .... // returns implementation from nonexported package
vbox.getChildren().add()

This will work as long as VBox is accessible. the getChildren call is
not done using the implementation type, but VBox. And for this it does
not matter that vbox has a "forbidden" type at runtime at all.
Conclusively you have to do more at runtime if all you have is the
runtime type of vbox. You have to check if this type is accessible to
you and if not go to a super class. You would then maybe also end up in
Vbox and invoke the method you get for VBox, not for the implementation
of VBox. You still do a virtual method call, so the implementation
method is still called

Post by jeffrey kutcher
then I should also able to access that very same class and execute the same method using reflection with no warnings or errors and expect the same results. If not, the inconsistency may be a bug, feature or broken.

design choice I would claim.

Post by jeffrey kutcher
Maybe that's the way it's suppose to work. Inconsistency however tells me something isn't quite right and if ignored will cause more issues later on.

That is because Reflection has been declared as broken in that it has a
tradition of allowing too much. Now this has been restricted, leading to
problems. But in this case you just have to use the type the Java
compiler would, which is VBOX.

Post by jeffrey kutcher
In this particular case, a lock down will completely stop currently working code.

You are not alone here.

bye Jochen

[jeffrey kutcher]

VBox class hierarchy shows getChildren() are implemented in Parentand Pane. vbox.getChildren() returns an instance of Parent which isprotected rather than public found in Pane. Accessing the Listreturned by Parent is an illegal reference because it is protectedwhile accessing the List returned by Pane should be legal, butthis isn't whats returned.
VBox Class Hierarchy:
java.lang.Objectjavafx.scene.Nodejavafx.scene.Parent -> protected ObservableList<Node> getChildren​()javafx.scene.layout.Regionjavafx.scene.layout.Pane -> public ObservableList<Node> getChildren()javafx.scene.layout.VBox
Class instance of vbox.getChildren() is:
vbox.getChildren().getClass() -> class javafx.scene.Parent$3
Pane is widening the access restrictions of Parent. Is this intentional?Does it have any effect? Should Parent's access restrictions of getChildren()be public also or should Pane's be protected?
My thought is public for both.

[jeffrey kutcher]

VBox class hierarchy shows getChildren() is implemented in Parent
and Pane. vbox.getChildren() returns an instance of Parent which is
protected rather than public found in Pane. Accessing the List
returned by Parent is an illegal reference because it is protected
while accessing the List returned by Pane should be legal, but
this isn't whats returned.

VBox Class Hierarchy:

java.lang.Object
javafx.scene.Node
javafx.scene.Parent -> protected ObservableList<Node> getChildren​()
javafx.scene.layout.Region
javafx.scene.layout.Pane -> public ObservableList<Node> getChildren()
javafx.scene.layout.VBox

Class instance of vbox.getChildren() is:

vbox.getChildren().getClass() -> class javafx.scene.Parent$3

Pane is widening the access restrictions of Parent. Is this intentional?
Does it have any effect? Should Parent's access restrictions of getChildren()
be public or should Pane's be protected?

My thought is public for both.

[Kevin Rushforth]

Post by jeffrey kutcher
VBox class hierarchy shows getChildren() is implemented in Parent
and Pane. vbox.getChildren() returns an instance of Parent which is
protected rather than public found in Pane. Accessing the List
returned by Parent is an illegal reference because it is protected
while accessing the List returned by Pane should be legal, but
this isn't whats returned.
java.lang.Object
javafx.scene.Node
javafx.scene.Parent -> protected ObservableList<Node> getChildren​()
javafx.scene.layout.Region
javafx.scene.layout.Pane -> public ObservableList<Node> getChildren()
javafx.scene.layout.VBox
vbox.getChildren().getClass() -> class javafx.scene.Parent$3
Pane is widening the access restrictions of Parent. Is this intentional?
Does it have any effect? Should Parent's access restrictions of getChildren()
be public or should Pane's be protected?
My thought is public for both.

Yes, this is intentional. It is fine for a subclass to widen the access
from protected to public. No, we don't want to make Parent::getChildren
public, since there are some subclasses that do not want to expose
direct access to application.

-- Kevin

[jeffrey kutcher]

Thanks Dalibor.
On Tuesday, January 16, 2018, 7:45:55 AM CST, dalibor topic <***@oracle.com> wrote:

Try
https://docs.oracle.com/javase/specs/jls/se9/html/jls-15.html#jls-15.12.4
for example.

Did that. There's nothing in there by that name.
On Tuesday, January 16, 2018, 7:21:32 AM CST, dalibor topic
On 16.01.2018 14:08, jeffrey kutcher wrote> Is there official
documentation explaining the method resolution process somewhere?
I'd suggest taking a look at the Java Language specification and JVM
specification for details.
cheers,
dalibor topic
--
<http://www.oracle.com> Dalibor Topic | Principal Product Manager
Phone: [phone] <tel:[phone]> | Mobile: [phone]
<tel:[phone]>
ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 22761 Hamburg
ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
<http://www.oracle.com/commitment> Oracle is committed to developing
practices and products that help protect the environment

--
<http://www.oracle.com> Dalibor Topic | Principal Product Manager
Phone: [phone] <tel:[phone]> | Mobile: [phone]
<tel:[phone]>

ORACLE Deutschland B.V. & Co. KG | Kühnehöfe 5 | 22761 Hamburg

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher

<http://www.oracle.com/commitment> Oracle is committed to developing
practices and products that help protect the environment

[David Holmes]

Post by jeffrey kutcher
Trying to find documentation for the method resolution process.

The JVMS defines the rules for method resolution (and subsequent selection).

David
------

Post by jeffrey kutcher
https://stackoverflow.com/questions/6021109/java-runtime-method-resolution
which leads to
http://www.cowtowncoder.com/blog/archives/2010/12/entry_436.html
Maybe this is an opportunity for someone to put together a class library that does the right thing all the time regarding the method resolution process. Does the Java library already have this somewhere in it's source? If so, why not expose it making it easier for others so those interested don't have to reinvent the wheel every time.
Is there official documentation explaining the method resolution process somewhere?

Post by jeffrey kutcher
Thanks for your thoughts Jochen. I have a section of code that does just that; function search and casting return values. Not easy ... nor as easy as it should be.
You would think however, that if a function call would cause no issues, say vbox.getChildren().add(x), that there should similarly be no issues referencing each of the pieces of the same call set via reflection. If not, then something is up with the design. It should be this simple. The point to object oriented programming is to understand the pattern and then leverage that pattern so when another object mimics a prior pattern you know how it works increasing your productivity exponentially. In this case, it's not a different object pattern. It's the very same object/pattern, just referenced differently. If not, then the point of object programming is defeated.
Introspection of vbox.getChildren().add(x) should be just as simple as calling vbox.getChildren().add(x) itself, otherwise why have introspection at all?

I recall a very similar discussion in the past. You have to start with
(public) statically known types, not with the dynamic type of the
instances involved. In essence you have to reflectively emulate the
method resolution process that occurs with a direct call and which uses
static type information.
David
-----

Post by jeffrey kutcher
Simple is hard. Steve Jobs demanded elegance as well as performance and if not it was back to the drawing board until it worked ... and looked good doing it. He was ruthless. Look at the behemoth he created. Yes there are lots of different ways of doing things, especially in programming, but in an object system, if by convention objects are being leveraged improperly, then cracks are being introduced and over time will implode under its own weight. One offs are bad for an object system.
I understand, practically speaking, this isn't how it works, even in Java. There are always nuances. It sometime takes extreme effort to eliminate those nuances.
Looking for an Oracle to glean insight (implications of a lock down)
I've worked with Java since 1995. This example represents an issue that was never an issue up until Java9. Even then, I had no idea it was my code that was the issue since my code never directly referenced illegal classes. I code to write once, run anywhere and never use native method calls to maintain independence. What I can say from this exercise is if this lock down does come to pass, the past 17 years worth of code will no longer work.
I've built special code to handle cases because lists where different from the standard list, for example. That special code should and will go away when Java libraries collapse it's implementation.
It looks like I'll have to make a decision. Never upgrade again or move on and say good-bye to 17 years worth of work. It was a good run. I suppose nothing lasts forever.
I'm sure there are others.

Post by Michał Zegan
well you rather know what getChildren returns, so you know what the
returned object is, like generally.

but as soon as the method you want to call is not on the statically
declared type, nor within a set of known statically types you are having
a problem. There is no good way (good in terms of usability and
performance) to get an arbitrary valid method object without static
information. You are always forced to traverse the hierarchy to find the
most accessible type. You will have to check access... and then of
course there the two bad scenarios where invocation and search are not
the same class or actual caller and caller are not equal. Which leads to
a lengthy search using reflection to find a method, transfer it to a
method handle and then let the invoker do the actual call.
bye Jochen