Get JSONata to operate under Java 8 nashorn JavaScript ES5 engine

[michaelthoward]

Would be good to get JSONata (currently 1.5.3) to operate under the Java 8 nashorn ES5 engine.

Background

The Java 8 nashorn JavaScript engine supports ES5 == ECMA Script 5.
JSONata 1.1.1 can be easily run under Java 8 nashorn.
JSONata 1.2 began using ES6 == ECMA Script 6 features. See this thread.

The babel transpiler generates jsonata-es5.js from ES6 source. However, the transpiled source assumes the presence of a number of node_modules.

#196 discusses a number of issues related to trying to get JSONata 1.5.3 to operate under Java 8.

[xogeny]

I wonder if you could use rollup to address this? I’ve been playing around with rollup in my TypeScript version. But nothing concrete yet.

[michaelthoward]

Be advised that I have almost no exposure-to/experience-with the JavaScript/ES ecosystem.

I googled and got high-level overviews of https://rollupjs.org and https://www.typescriptlang.org

It seems to me like the fundamental problem I am having with the babel-generated jsonata-es5 is the runtime dependency on a large set of node_modules in the transpiled source.

I now understand TypeScript can also generate an ES5 version of the code.
Q: What kind of runtime-library support is needed by a TypeScript-generated jsonata-es5 (or jsonata-es3) version?
Q: Is it any lighter-weight than that generated by babel?

[xogeny]

The point of rollup is to bundle all the JS (including dependencies) together into a single JS file. This is commonly used when deploying web applications, but can be used to deploy any Javascript really.

I don't have much experience with babel, so I cannot make comparisons there. But the point is that rollup gets read of node_modules completely. The node_modules directory is part of the system for importing code as modules. But rollup and other bundlers create bundles in which all used code is pulled out of bundles and inlined. So you definitely shouldn't see any issues with needing files from node_modules when using a bundler.

But I don't really have any working solution for this in my version. Just pointing out that it is a possibility.

[andrew-coleman]

I don't have an immediate answer to this. Adding 'help wanted' flag to open up to anyone with nashorn experience.

[xogeny]

I wonder if you his might be useful...

https://blogs.oracle.com/developers/announcing-graalvm

[s100]

@michaelthoward You said,

The babel transpiler generates jsonata-es5.js from ES6 source. However, the transpiled source assumes the presence of a number of node_modules.

Can you give examples of what node_modules are missing?

So anyway...

I think we have a general problem with the way jsonata is currently being packaged. When the project started the idea was that the project should consist of a single ES5 source file, jsonata.js, which had no external dependencies, needed no building, and could be consumed directly in Node.js or the browser without problems. The introduction of ES6 generators to the source complicated matters, as we can see from the introduction of jsonata-es5.js and my previous attempts to fix this (#51, #58). jsonata-es5.js now bundles a great deal of stuff so that it can still work as a UMD, but doing this seems to have harmed its consumability in other environments as seen here in #202, as well as some, possibly all, of #125, #158, #169, #211 and #214.

I think we need to change the way jsonata is packaged. My hope is that this can be done in a way which has no impact on consumers other than to possibly suddenly cause things to start working when they previously did not, and hence does not entail a major version bump. Here is a rough outline of how I think this should work:

• A new source directory, src. jsonata.js (ES6 code using generators, symbols etc.) moves under here, where it can theoretically be broken out into smaller source files, etc.. This is not distributed.
• A new distribution directory dist.
  • dist/js contains a built pure ES5 version of jsonata.js. Only our own JSONata code is here; dependencies such as core-js polyfills and the generator runtime are not included, but are pulled in using require, and listed as explicit run-time "dependencies" in package.json so they get installed separately. "main" will point to this file. I hope this file will be consumable natively by ES5 environments like older versions of Node.js. It will also hopefully be amenable to uglification and bundling for older browsers e.g. IE. (Unavoidably, such bundles will be quite large...)
  • dist/es contains a built pure ES6 version of jsonata.js. This code will use ES6 syntax, and potentially import and export, but not generators/yield. On current showing this code will have no dependencies. At first this might just be a copied-and-pasted version of the source. In package.json, "module" will point here, enabling ES6-aware bundlers like rollup and webpack to use it.
  • dist/umd will contain UMDs suitable for dropping into the browser, each of which will export jsonata as a global variable window.jsonata:
    • dist/umd/jsonata.js (ES6, with generators and suchlike)
    • dist/umd/jsonata.min.js
    • dist/umd/jsonata-es5.js (ES5, a single huge bundle as mentioned before)
    • dist/umd/jsonata-es5.min.js

I don't know a whole lot about how TypeScript plays into this. I hope that we would be able to convert src to TypeScript without affecting the way dist is structured.

Would appreciate any thoughts people have on this.

[s100]

Hrm, to avoid breaking backwards compatibility, it's okay to change "main" and "module" in package.json to point wherever we like, but I suppose the UMDs may have to continue to live where they currently are in ..

[andrew-coleman]

This is now resolved in v1.6.1 and tested in this example.

The generation of the ES5 library has been rewritten to have no dependencies. If you find any polyfills missing, then please raise a separate issue.