Print warning message instead of throwing exception for unsupported Java versions

[pan3793]

My local test shows it also works with JDK 25.

[pan3793]

@leerho @tisonkun, this implements my idea proposed at #280 (comment), do you think it's ok?

[leerho]

A whole bunch of internal calls (especially from Unsafe) changed starting with Java 23, and that affects over 100 places in the code.
And after Java 21, the next LTS is Java 25. So no. Permitting any Java version greater than 21 is not permitted, and for good reason. So I don't know what you tested, but you certainly didn't run the unit tests.

[leerho]

This code will absolutely not work for LTS Java versions greater than 21. And what I mean by "work" is that all the functionality works and not just some minor subset.

[pan3793]

So I don't know what you tested, but you certainly didn't run the unit tests.

I do run the unit test, and all unit tests pass under JDK 25.0.3 except for ResourceTest.checkJdkString (the fix is easy, change the method return value to boolean and change the UT to test the return value instead of whether throwing an exception)

[pan3793]

A whole bunch of internal calls (especially from Unsafe) changed starting with Java 23, and that affects over 100 places in the code.

some of the unsafe methods are marked as deprecated for removal in JDK 23, but the removal does not happen, at least in JDK 25, can you point out a concrete example that does not work in JDK 25?

[leerho]

Why do you want this to work in Java 25? You should be using the native FFM features in Java 25.

[tisonkun]

Spark requires a single jar compatible with Java 17, 21, and 25. (for Spark 5 scheduled in early 2027, maybe drop Java 17 and require a single jar compatible with Java 21 and 25

@leerho I think this is the issue. That Spark requires to work on Java 17, 21, and 25 so they cannot depend on the new datasketches-java because it doesn't work on 17/21 (IIUC). And they can't depend on datasketches-memory if this test case fail some of their workflows would fail.

[leerho]

This is new information to me and the first time anyone has informed me about this restriction.

I have just today submitted RC2 of a new version of DS-memory. I will have to cancel and resubmit an RC3.

[pan3793]

Why do you want this to work in Java 25? You should be using the native FFM features in Java 25.

I think I explained it at the beginning #270 (comment)

Spark currently uses Java 17 as a baseline, and is trying its best to be compatible with new Java versions. Before moving the baseline to Java 25, it must at least be compatible with Java 25, so it's a deadlock ...

In practice, upgrading JDK for Hadoop and Spark involves many issues that need to be addressed, including GC, JIT, bugs, etc., so it's a hard requirement to allow users to exercise the software with the newer LTS JDK alongside before escalating the minimum JDK version.

[pan3793]

I made minimal changes to make it runnable with JDK 25, and manually tested all UTs pass with JDK 17, 21, and 25 now.

1. setup maven toolchanins for JDK 11, 17, 21, 25.
2. run mvn clean test with 17, 21, 25, all UTs pass.

[leerho]

@pan3793

Spark currently uses Java 17 as a baseline, and is trying its best to be compatible with new Java versions. Before moving the baseline to Java 25, it must at least be compatible with Java 25, so it's a deadlock ...

Sorry, but that statement did not convey to me what @tisonkun clarified:

Spark requires a single jar compatible with Java 17, 21, and 25.

Even if I fix DS-Memory, it will be a while until I have DS-Java able to simultaneously operate across 17, 21, and 25. I will be lucky if I could get that done this year.

I do have a question: Is DS-Memory being used in Spark independent of its use by DS-Java?

[pan3793]

Is DS-Memory being used in Spark independent of its use by DS-Java?

No, Spark uses DS-Java.

Even if I fix DS-Memory, it will be a while until I have DS-Java able to simultaneously operate across 17, 21, and 25. I will be lucky if I could get that done this year.

If it can be released before 2027.2, Spark is likely to include it in 5.x.

[leerho]

Also, the current version of DS-Memory 7.0.0 that I am releasing is a single source multi-project (one root with two sub-projects) and produces two Jars, one for Java 11 with full capabilities, and a separate jar for Java 17 & 21 but without the capability to create and manage off-heap memory-mapped files. Even if I make it possible for the second jar to operate with Java 25, it still will not be able to create and manage off-heap memory-mapped files.

[pan3793]

Even if I make it possible for the second jar to operate with Java 25, it still will not be able to create and manage off-heap memory-mapped files.

it's also fine, Spark currently does not use DS-memory off-heap mode

[leerho]

Thanks again for new information 🙂