Move the Standalone Persistence TCK 3.2+ source to the https://github.com/jakartaee/persistence repo

[scottmarlow]

I can help with a pull request and help with this if this is desired.

Currently the source is on main branch in https://github.com/jakartaee/platform-tck/tree/main/tcks/apis/persistence/persistence-outside-container.

[gavinking]

@scottmarlow We need to wait for @lukasj to get back on deck.

But I'm pretty sure he will be a big fan of this.

[arjantijms]

I mentioned it on the mailinglist, but would like to mention it here too.

We currently have two versions of the persistence TCK.

1. A fork that runs the tests against an EE container like WildFly or GlassFish - https://github.com/jakartaee/platform-tck/tree/main/tcks/apis/persistence/persistence-inside-container
2. The original version that runs the tests against Hibernate and EclipseLink - https://github.com/jakartaee/platform-tck/tree/main/tcks/apis/persistence/persistence-outside-container

Can we look at developing a protocol / container for Arquillian or perhaps a JUnit Extension so that we can run the same (overlapping) set of tests against both an EE container and directly against Hibernate and EclipseLink?

Maybe something along the lines of how the appclient protocol for Arquillian works could be used to run the tests against Hibernate and EclipseLink?

[gavinking]

A fork that runs the tests against an EE container like WildFly or GlassFish - https://github.com/jakartaee/platform-tck/tree/main/tcks/apis/persistence/persistence-inside-container

It seems to me that most of these tests should be deleted.

The platform TCK should contain a few tests to validate the container/provider contracts, integration of the persistence provider, and do some basic sanity checking that JPA works.

There's no good reason for it to contain a whole laundry full of tests for detailed functionality of the Persistence spec that every provider has already passed in order to certify compliance with the Persistence spec.

[arjantijms]

There's no good reason for it to contain a whole laundry full of tests for detailed functionality

I think the difference is that the fork actually runs the tests against WildFly and GlassFish, and not against standalone Hibernate and EclipseLink.

In the case of EclipseLink and GlassFish, we don't just drop the EclipseLink jar into /modules of GF, but we reimplement large parts of it (like parsing persistence.xml) for reasons I guess only you and Lukash would know.

A while back I refactored that parser a little and all of persistence broke. Yet, we were still passing the Persistence TCK.

Upon investigation I found out the original persistence TCK didn't even start GlassFish and was just directly operating against EclipseLink.

[gavinking]

I think the difference is that the fork actually runs the tests against WildFly and GlassFish, and not against standalone Hibernate and EclipseLink.

I don't understand why that would make a difference to functionality which involves only interaction with the database.

In the case of EclipseLink and GlassFish, we don't just drop the EclipseLink jar into /modules of GF, but we reimplement large parts of it (like parsing persistence.xml) for reasons I guess only you and Lukash would know.

Well sure, persistence.xml deals with integration concerns (the information in it gets passed from container to provider), so sure, I expect that sort of stuff to be tested as part of the platform TCK.

I'm talking about all the tests which have nothing to do with integration.

[gavinking]

I'm talking about all the tests which have nothing to do with integration.

I mean, I have not looked closely at what's in any of these directories, but just going by their names, they don't seem to deal with the container/provider contracts:

[arjantijms]

I mean, I have not looked closely at what's in any of these directories, but just going by their names, they don't seem to deal with the container/provider contracts:

I don't know either. But in general I know that when you test something standalone (outside the container / ee runtime) it doesn't necessarily mean it works within it. Even when it does not exactly concern any specific integration.

We used to test Mojarra (JSF/Faces implementation) outside the EE container, and it passed hundreds (or thousands, I forgot) tests using Cactus. But in the end we figured they didn't mean anything really, deleted them all, and now we're solely doing end to end testing.

[gavinking]

But in general I know that when you test something standalone (outside the container / ee runtime) it doesn't necessarily mean it works within it. Even when it does not exactly concern any specific integration.

This seems much to pessimistic to me. And in particular, it's wrong as applied to the things we're talking about out here.

hibernate-core currently has 15 thousand tests, which we run against 16 different databases and various different JDKs. We don't run these tests inside any container, and we have no reason to do so. It would be a pure waste of energy. Running Hibernate inside a container can't just magically break functionality which has nothing to do with container integration.

What we do have is actual meaningful tests which test the sort of things that can actually break when you run Hibernate inside WildFly or Quarkus (stuff like transaction integration, discovery, bytecode processing, datasource lookup, etc, etc, etc).

[arjantijms]

What we do have is actual meaningful tests which test the sort of things that can actually break when you run Hibernate inside WildFly or Quarkus (stuff like transaction integration, discovery, bytecode processing, datasource lookup, etc, etc, etc).

One way or the other we need a bunch of tests that do run inside the EE runtime. If only to prove it's actually there. We may not need the 15 thousand tests for that, but something at least.

I mean, I would now certify GlassFIsh as being Jakarta Persistence compatible by just stating "We are using Hibernate, Hibernate passed the thousands of tests, so we're fine". But supposed I accidentally copied the hibernate jars to /lib and not /modules. Silly example perhaps, but in that case we would be shipping Hibernate and thus would have passed those thousands of tests implicitly, but at runtime nothing would work.

I'm not sure if the TCK now has a 100% fool proof seperation between tests that can absolutely never break no matter how wrong Hibernate is integrated, and tests that could potentially break with a wrong integration.

I understand that your focus is fully on Hibernate, and there's of course nothing wrong with that. It's just that my focus is on GlassFish, so for me I'd like to certify GlassFish.

Putting it the other way around; if the TCK an be run on both Hibernate and GlassFish, would that bother you? Even if you may think I'm a little crazy wanting that?

[gavinking]

One way or the other we need a bunch of tests that do run inside the EE runtime. If only to prove it's actually there. We may not need the 15 thousand tests for that, but something at least.

Which is precisely why I said:

The platform TCK should contain a few tests to validate the container/provider contracts, integration of the persistence provider, and do some basic sanity checking that JPA works.

[gavinking]

Putting it the other way around; if the TCK an be run on both Hibernate and GlassFish, would that bother you? Even if you may think I'm a little crazy wanting that?

I mean it's an absolute hard requirement of the Persistence specification that containers must support any compliant Persistence provider out of the box, via the <provider> element of persistence.xml.

I hope that the platform TCK tests actually validate that this is the case, and that a non-default provider can be selected via use of <provider>.

If GlassFish does not currently pass the TCK with Hibernate as the persistence provider, then GlassFish does not comply with its obligations under the Persistence specification.

You make it sound like you're not sure whether it does, and that's precisely why the platform TCK tests should be focussed on validating container/provider contracts, and not on redundantly retesting shit that's already tested by the Persistence provider TCK tests!

[arjantijms]

Which is precisely why I said:

The platform TCK should contain a few tests to validate the container/provider contracts, integration of the persistence provider, and do some basic sanity checking that JPA works.

And which is why I posed the question of what actually constitutes "some basic sanity checking". Where do we exactly draw the line? It would, from our point of view, be just as easy to run them all. The entire TCK runs in some 15 minutes (https://ci.eclipse.org/jakartaee-tck/view/EFTL-Certification-Jobs-11/job/11/job/tck/job/jakarta-persistence-platform-tck-glassfish/). It's not really a problem for us really. Determining which to run as "the few" would take more time.

If GlassFish does not currently pass the TCK with Hibernate as the persistence provider, then GlassFish does not comply with its obligations under the Persistence specification.

I dunno to be honest if that's tested in the TCK. Do you? Does WildFly pass with EclipseLink as the persistence provider when it runs the Jakarta Persistence TCK?

I know there's a bunch of other ommisions in the TCK, like @Inject EntityManager only tests that it is indeed injected, not that it actually works or corresponds to some specific persistence unit. I would actually like to run "a bunch" of regular tests using this "version" of the entity manager. Which brings us to the point of being able to run the persistence tests inside the EE runtime.

and not on redundantly retesting shit that's already tested by the Persistence provider TCK tests!

I'm not sure. We had this discussion before with other specs.

E.g. I test Mojarra or Soteria independently. Can WildFly just claim Jakarta Faces or Jakarta Security compatibility by just saying that you bundle my Mojarra or Soteria? It's currently (not really) how the certification process works. You want to make sure that whatever is actually in your server and in combination with your server, passes.

Now you could chase down the SHA of whatever jars (Hibernate, Mojarra, Soteria) are bundled, and then check if there's a certification available for those components, using your JDK and OS, etc? Maybe... but it's just not how we really work now.

[gavinking]

The entire TCK runs in some 15 minutes .... It's not really a problem for us really.

That's not the problem. The cost is in maintaining two TCKs with lots of redundant tests across two separate GH repositories.

Do you?

I do not, but I strongly suspect it does not test this.

Does WildFly pass with EclipseLink as the persistence provider when it runs the Jakarta Persistence TCK?

It should do. Support for EclipseLink is a documented feature of WildFly 35.

( Older versions documented support for both EclipseLink and OpenJPA, but I guess OpenJPA was dropped due to the whole jakarta.persistence thing.)

We had this discussion before with other specs.

The persistence spec is not like the other specs. The Persistence spec has a well-defined container/provider contract, which is missing other EE specifications. The persistence provider is treated as a plug-in module, not as part of the container.

Second of all, most of the JPA TCK is testing interaction with the database, which is something almost completely unaffected by the rest of the container environment. And, unlike many of the other specs, the ability to run JPA in Java SE has been a completely well-defined thing since JPA 1.0, and the differences in behavior between the SE and EE environments has always been clearly defined in the spec itself.

You can't just take your intuitions from JSF or whatever and apply them to JPA, because JPA is a whole different beast.

[scottmarlow]

I hope that the platform TCK tests actually validate that this is the case, and that a non-default provider can be selected via use of .

Yes, this is done with a custom persistence provider included in the Jakarta EE 8/9/10/11 TCK in source form. This test was added in Java EE 7 or 8 (forget which) as a replacement for a Persistence 1.0 provider that was included in earlier releases.

[gavinking]

Yes, this is done with a custom persistence provider included in the Jakarta EE 8/9/10/11 TCK in source form.

OK, great!