jsonPropertyGeospatialQuery stops working in MarkLogic Java API 5.3.0

[chenfiona]

Version of MarkLogic Java Client API:
Java API v5.3.0

Version of MarkLogic Server:
10.0-4.4

Java version:
Java v11.0.8

ISSUES:
When I plugin v5.3.0:
<dependency> <groupId>com.marklogic</groupId> <artifactId>marklogic-client-api</artifactId> <version>5.3.0</version> </dependency>

Issue One:

cts.jsonPropertyGeospatialQuery JAVA Optic was invalidated:

The method jsonPropertyGeospatialQuery(String, CtsRegionExpr...) in the type CtsExpr is not applicable for the arguments (String, ServerExpression)

Issue Two:

import javax.xml.stream.XMLEventReader; import javax.xml.stream.XMLStreamException; import javax.xml.stream.XMLStreamReader;

System throws error:
The package javax.xml.stream is accessible from more than one module: <unnamed>, java.xml

If I reverted to v5.2.0, aforementioned two problems are resolved.

[georgeajit]

Will work with 5.3.0 tagged release and try to run optic tests to see if it reproduces this issue (looks like compile time).

[chenfiona]

Thanks!
Just updated: There are two issues. Please review!

[georgeajit]

Can you try with Java 1.8 u 261?

[georgeajit]

I did a compile of marklogic-client-api project and do not see gradle (wrapper) complain. Using Java 1.8 update 261 JDK. Here are the results. I do not seem to have the issue.

ageorge@hpzbook-3884 MINGW64 /c/tmp/java-client-api-dev ((5.3.0))
$ ./gradlew marklogic-client-api:clean
Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details
> Task :marklogic-client-api:clean UP-TO-DATE

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 15s
1 actionable task: 1 up-to-date

ageorge@hpzbook-3884 MINGW64 /c/tmp/java-client-api-dev ((5.3.0))
$ ./gradlew marklogic-client-api:compileJava

> Task :marklogic-client-api:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 25s
1 actionable task: 1 executed

ageorge@hpzbook-3884 MINGW64 /c/tmp/java-client-api-dev ((5.3.0))
$ java -version
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)

ageorge@hpzbook-3884 MINGW64 /c/tmp/java-client-api-dev ((5.3.0))
$ ./gradlew marklogic-client-api:compileTestJava
> Task :marklogic-client-api:compileJava UP-TO-DATE
> Task :marklogic-client-api:processResources
> Task :marklogic-client-api:classes

> Task :marklogic-client-api:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 13s
3 actionable tasks: 2 executed, 1 up-to-date

ageorge@hpzbook-3884 MINGW64 /c/tmp/java-client-api-dev ((5.3.0))

[ehennum]

On the 2nd issue, marklogic-client-api/build.gradle should have jackson-dataformat-xml as a testCompile dependency instead of a compile dependency. Rebuilding with that change should solve the problem.

On the 1st issue, can you show the jsonPropertyGeospatialQuery() call that triggers the error? Is it a compile time or runtime issue?

While we of course try to maintain backward compatibility, it's possible that an enhancement mistakenly changed the signature of the method.

I also wonder whether reverting our library to a Java 8 compilation might have invalidated the binding to our method in the compiled calling code of your application. If that wild speculation is correct, recompiling your application would make the issue go away.

[chenfiona]

Interesting point..

1. As it stands, I resolved the latter issue:

Origin pom.xml: ( seems causing the XML problem)
<dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-mllib_2.11</artifactId> <version>2.3.0</version> </dependency>
Updated pom.xml:
<dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-mllib_2.12</artifactId> <version>3.0.1</version> <scope>runtime</scope> </dependency>
I rarely believe it is JDK issue. JDK 11 is LTS.

2. The JAVA Optic issue:

I am fairly certain it is MarkLogic Java API issue. Please check.

[chenfiona]

All of my projects includes the Jackson Dataformat plugins as the modules heavily depend on the Jackson:

<dependency> <groupId>com.fasterxml.jackson.dataformat</groupId> <artifactId>jackson-dataformat-csv</artifactId> <version>2.11.3</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.dataformat</groupId> <artifactId>jackson-dataformat-xml</artifactId> <version>2.11.3</version> </dependency>

The only solution for the 2nd issue, seems to me, is to either remove the Apache Spark or use the updated version + runtime scope. That is the rub if I use Apache web servlet.

As regards the 1st issue, below is one of the core MarkLogic Java Geospatial Optic modules ( I developed the modules back to few months ago and it ran perfectly fine in JDK 11 until I plugin v5.3.0 today) :

CtsQueryExpr jsonGeoQuery = p.cts.jsonPropertyGeospatialQuery("LatitudeLongitude", p.cts.box(geoBox.getsLat(), geoBox.getwLon(), geoBox.getnLat(), geoBox.geteLon()));

I can revert to v5.2.0 only for this project.

But I would like to upgrade other projects to v5.3.0 whenever it is tenable...

[ehennum]

For what it's worth, we've found that Spark doesn't upgrade dependencies very rapidly (including some dependencies with known security vulnerabilities). To use more recent versions of dependencies than Spark, we've used shadow jars sometimes known as shading. Here's a blog post that discusses that approach:

https://imperceptiblethoughts.com/shadow/introduction/

On the Optic issue, thanks for providing the call, which made it possible to identify the problem, which is that the return type of box() changed from CtsBoxExpr (which derives from the CtsRegionExpr expected by the jsonPropertyGeospatialQuery() method) to ServerExpression.

Unfortunately, it doesn't look like casting could provide an interim workaround while we look into the cause and assess the extent of the issue.

[chenfiona]

Fair enough...
This is the first time I encountered cts | Optic problem due to API upgrade. And most of my MarkLogic Java Search depends on the cts | Optic. Please cross-check what other cts might be impacted.
Thank you!

[georgeajit]

Have decided to run Optic tests against previous release (or master branch) to track down these kind of issues. The time to run this verification will be around release time.

[ehennum]

Noting for the sake of a record trail that this commit introduced the Optic issue:

496e588

[ehennum]

Also noting for the sake of a record trail that Subversion commit 324540 under internal bug 52654 modified the optic-defs.json
build module to add the cts:period and cts:region types to the literal (effectively, build time instead of query time) classification
for function return types.

That's consistent with the fact that I've only found significant signature changes on the return types of methods constructing region values and in the removal of the sequence constructors for such values.

By contrast, the return type on constructors for period values haven't changed and the sequence constructor for period values is still provided.

[chenfiona]

Can you not just rollback the cts:region ?
If you can reproduce, confirm the bug and rollback, I can clone and see how it fares.

[ehennum]

The bug was a side effect of an incorrectly implemented enhancement, so we need to figure out the correct implementation so we can restore backward compatibility without regressing on the other parts of the enhancement.

[chenfiona]

Jolly good...

[ehennum]

FYI, the 5.3.1 release with the fixes for both of these issues has been published to Maven Central.

[chenfiona]

Nice going! Both Maven and Gradle bear well.

[ehennum]

@chenfiona , thanks for confirming the point release (and reporting the original issue).

@georgeajit , I've checked in the more strategic fix for the next major release.

[georgeajit]

Not sure how the Git labels went haywire.