Proposal: Add Windows version of OpenJDK

[StefanScherer]

Here are some Dockerfiles to build Windows docker images with OpenJDK from RedHat. These might be a starting point to get an official openjdk:8-windowsservercore and openjdk:8-nanoserver image on the Docker Hub.

I first started to put Oracle Java into container images which ended in the PR oracle/docker-images#209, but everybody has to build its own version.

The idea for this variant started after I saw this image https://hub.docker.com/r/michaeltlombardi/nanoserveropenjdk/ and wondered that OpenJDK is available for Windows, too.

Yes, it is: http://developers.redhat.com/products/openjdk/overview/

So I started a Dockerfile to download the MSI package, but there is the first hard problem that the download is only available after logging in. So you have to download the MSI package on your host machine.

For the Nano Server variant I also could not use the MSI package, so the build.sh script copies the installed folder to the host and a nanoserver image is then created by COPYing this directory back into the image.

After this worked, I thought this would also help the windowsservercore image to reduce the size of the image as I couldn't remove the MSI package after the first COPY. So the final windowsservercore image is created in the same way as the nanoserver image.

One problem might be if it is allowed to publish such image as RedHat has put that "open" OpenJDK behind a login. The agreement says something about using this only for development and not for production...

[StefanScherer]

After finding appveyor/ci#1154 I searched for other OpenJDK distributions for Windows and found three which can be dockerized, see https://github.com/StefanScherer/dockerfiles-windows/tree/master/java/openjdk

Seems that the most difficult part is which one to choose to make an official image.

[StefanScherer]

Ping @akashche FYI. I started a journey today to build OpenJDK Dockerfile for Windows today and after this PR I found your repo and another distribution from Azul.
Any comments which one could/should be used for an official openjdk Docker image?

[akashche]

@StefanScherer hi, I am a maintainer of https://github.com/ojdkbuild/ojdkbuild/ OpenJDK windows builds. These builds are effectively an "unbranded" version of Red Hat Windows builds available here http://developers.redhat.com/products/openjdk/overview/ - being built from the same sources and updated at the same time.

"ojdkbuild" builds of OpenJDK 8 are going to be maintained during the same period as Red Hat Windows builds will be maintained (according to this https://access.redhat.com/articles/1299013#OpenJDK_Lifecycle_Dates_and_RHEL_versions - at least until October 2020).

These builds are published under GNU GPL v. 2 with classpath exception. I think it would be great, if these builds will be added to Docker Hub. Besides MSI installer (that requires admin rights) there is also a ZIP bundle download that contains the same files as MSI. I can also create an additional "lighter" version of ZIP bundle (without src.zip, jre-only etc) if it will be more convenient for Docker Hub.

Please note, that "ojdkbuild" builds are NOT endorsed by upstream OpenJDK project.

I can answer more questions about these builds.

[StefanScherer]

@akashche Thank you.
Yes, the ZIP installation is much better than MSI if you work with Windows containers. MSI installation only works in windowsservercore, but not in nanoserver.

I've prepared Dockerfiles here https://github.com/StefanScherer/dockerfiles-windows/tree/master/java/openjdk/ojdkbuild and I can update this PR to use the ojdkbuild Dockerfiles.

[StefanScherer]

Ping @arun-gupta FYI, as a Docker Captain and Java Champion, what is your opinion about such a OpenJDK/Java Windows Docker image?

[StefanScherer]

@akashche Just to understand it better. Why doesn't the OpenJDK project build a Windows version by themself? In these days it's not very much more than adding an appveyor.yml to your GitHub repo if you don't have a Windows machine locally.

[akashche]

@StefanScherer , it is hard to answer this question directly, so some overview about OpenJDK development process first:

OpenJDK development is led by Oracle, but there is also a number of other active participants: SAP, Azul, JetBrains, Google, Twitter, Red Hat etc. Most of these companies build customized OpenJDK (for distribution or internal use) including proprietary or open-source "vendor-customizations".

Red Hat OpenJDK builds are done for RHEL/CentOS/Fedora and also most Linux distributions ships OpenJDK packages built upon "RedHat-flavoured" OpenJDK sources, not directly upon upstream ones. These builds are tuned for Linux and majority of changes are about the tighter integration with OS - use OS-provided timezone data, use system libraries (zlib, jpeg-turbo etc) instead of bundled ones etc. "ojdkbuild" Windows builds were created with a goal to be as close as possible to Linux builds of OpenJDK.

Answering directly, IMVHO (and of course I am not speaking for neither Oracle nor upstream OpenJDK project) Oracle holds trademarks/copyrights for OpenJDK and only Oracle can build "official" OpenJDK binaries. But Oracle already has proprietary Oracle Java builds (OpenJDK + number of proprietary customizations + some addons). And OpenJDK project doesn't have "official" binaries for any platform, not only for Windows.

[StefanScherer]

Updated the PR to build it with a single Dockerfile.

Output of the ./build.sh script:

openjdk version "1.8.0_111-3-ojdkbuild"
OpenJDK Runtime Environment (build 1.8.0_111-3-ojdkbuild-b15)
OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)
openjdk version "1.8.0_111-3-ojdkbuild"
OpenJDK Runtime Environment (build 1.8.0_111-3-ojdkbuild-b15)
OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)

[sixeyed]

@StefanScherer - coming to this late, but I've been using the Zulu OpenJDK port from Azul so far (e.g. to run Elasticsearch on Nano Server). Main reasons for choosing it:

• it's stable and well maintained, new versions are out quickly
• ZIP installation works fine on Nano Server
• Microsoft used this a while ago in their HDInsight Emulator to power Hadoop on Windows, so there's a presumption of robustness
• it's freely distributable (checked with the folks at Azul).

I'm not precious though, I'll check your version out with Elasticsearch.

[StefanScherer]

Thanks @sixeyed for your experience. Well I've build all flavors of Java at https://github.com/StefanScherer/dockerfiles-windows/tree/master/java, the Oracle version made it into oracle/docker-images#209 and got merged recently.

I'm open to any of the two public downloadable versions for this Docker image.

[tianon]

I think Azul would probably make sense to have in a separate, Azul-dedicated image, right? (especially since they provide both Linux and Windows builds)

[tianon]

Also, it sounds like these RedHat builds are probably going to be the most similar to our Linux image variants as we're going to get, so I'm generally +1 on those for this image.

[StefanScherer]

Seems like Azul offers images at https://hub.docker.com/r/azul/zulu-openjdk/ - so they should provide a Windows image ;-)

[sixeyed]

Yes they already have a Linux image. It's not an official image though, so that balks using it as a base for other official images.

+1 from me on using the RH build, I'll try it out and get back.

[sixeyed]

@StefanScherer looks like there's a problem with Docker volumes in Windows containers in the JDK - happens with RH OpenJDK and Zulu, so it's not a distro issue. Elasticsearch uses toRealPath in java.nio.file.Path to check the data directory exists, and that call fails if the directory is a volume symlink. Basic repro here: symlink-test. I've submitted a JDK bug report and will update progress. It's not a fault with your image, but it does mean we can't use volumes for Java apps until it's fixed - unless there's a workaround.

[akashche]

@sixeyed , I reproduced this on win2016 using your symlink-test project (thanks for detailed instructions and golang reference). I'll look into this problem and return back.

[akashche]

@sixeyed , I barely scratched the surface of the problem, so notes below may be incorrect, still I am not sure I understand the expected correct behaviour here:

• from windows FS point of view VOLUME command creates a directory symlink that points to non-existed directory
• toRealPath [1] by default follows symlinks, so it tries to resolve volume path using GetFinalPathNameByHandle [2], gets ERROR_PATH_NOT_FOUND and reports error to caller
• if we call toRealPath(LinkOption.NOFOLLOW_LINKS), then it correctly returns non-resolved absolute path to volume dir

What will be the expected result for resolving volume symlink (that points to something like this ContainerMappedDirectories\006DB244-75C2-4ADE-A7A9-6B5EC3F56796)?

Relevant jdk sources: [3], [4], [5], [6].

[1] https://docs.oracle.com/javase/8/docs/api/java/nio/file/Path.html#toRealPath-java.nio.file.LinkOption...-
[2] https://msdn.microsoft.com/en-us/library/windows/desktop/aa364962(v=vs.85).aspx
[3] https://github.com/ojdkbuild/lookaside_java-1.8.0-openjdk/blob/840b6ef32695250a173961bb2211d9fe40c56c65/jdk/src/windows/classes/sun/nio/fs/WindowsLinkSupport.java#L237
[4] https://github.com/ojdkbuild/lookaside_java-1.8.0-openjdk/blob/840b6ef32695250a173961bb2211d9fe40c56c65/jdk/src/windows/classes/sun/nio/fs/WindowsLinkSupport.java#L77
[5] https://github.com/ojdkbuild/lookaside_java-1.8.0-openjdk/blob/840b6ef32695250a173961bb2211d9fe40c56c65/jdk/src/windows/classes/sun/nio/fs/WindowsNativeDispatcher.java#L952
[6] https://github.com/ojdkbuild/lookaside_java-1.8.0-openjdk/blob/840b6ef32695250a173961bb2211d9fe40c56c65/jdk/src/windows/native/sun/nio/fs/WindowsNativeDispatcher.c#L1163

[StefanScherer]

@akashche I just tried it with this simple Dockerfile

FROM microsoft/windowsservercore
VOLUME c:\\data

and then build a test Windows image with it

docker build -t volume .

running it in interactive mode you can normally change to the directory c:\data, so it exists even it is not mounted from the docker host.

$ docker run -it volume cmd

then inside the Windows container

C:\>dir
 Volume in drive C has no label.
 Volume Serial Number is 0668-FEDD

 Directory of C:\

01/09/2017  03:45 PM    <SYMLINKD>     data [\\?\ContainerMappedDirectories\B6744C2A-5FE8-4F30-BA7A-CDC5A16C7C27]
11/22/2016  02:45 PM             1,894 License.txt
07/16/2016  05:18 AM    <DIR>          PerfLogs
01/09/2017  03:45 PM    <DIR>          Program Files
07/16/2016  05:18 AM    <DIR>          Program Files (x86)
01/09/2017  03:45 PM    <DIR>          Users
01/09/2017  03:45 PM    <DIR>          Windows
               1 File(s)          1,894 bytes
               6 Dir(s)  21,232,553,984 bytes free
C:\>cd data

C:\data>dir
 Volume in drive C has no label.
 Volume Serial Number is 0668-FEDD

 Directory of C:\data

01/09/2017  03:45 PM    <DIR>          .
01/09/2017  03:45 PM    <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)  38,212,014,080 bytes free

C:\data>

On Linux in comparison it looks simpler and therefore the applications do not have such problems:

FROM ubuntu
VOLUME /data

and then

$ docker run -it volume bash

and now inside the Linux container:

$ docker run -it volume bash
root@d7e08c154a4a:/# ls -l
total 68
drwxr-xr-x   2 root root 4096 Dec 13 20:24 bin
drwxr-xr-x   2 root root 4096 Apr 12  2016 boot
drwxr-xr-x   2 root root 4096 Jan  9 23:49 data
drwxr-xr-x   5 root root  380 Jan  9 23:49 dev
drwxr-xr-x  45 root root 4096 Jan  9 23:49 etc
drwxr-xr-x   2 root root 4096 Apr 12  2016 home
drwxr-xr-x   8 root root 4096 Sep 13  2015 lib
drwxr-xr-x   2 root root 4096 Dec 13 20:24 lib64
drwxr-xr-x   2 root root 4096 Dec 13 20:23 media
drwxr-xr-x   2 root root 4096 Dec 13 20:23 mnt
drwxr-xr-x   2 root root 4096 Dec 13 20:23 opt
dr-xr-xr-x 109 root root    0 Jan  9 23:49 proc
drwx------   2 root root 4096 Dec 13 20:24 root
drwxr-xr-x   6 root root 4096 Dec 15 17:45 run
drwxr-xr-x   2 root root 4096 Dec 15 17:45 sbin
drwxr-xr-x   2 root root 4096 Dec 13 20:23 srv
dr-xr-xr-x  12 root root    0 Jan  9 23:47 sys
drwxrwxrwt   2 root root 4096 Dec 13 20:24 tmp
drwxr-xr-x  11 root root 4096 Dec 15 17:45 usr
drwxr-xr-x  13 root root 4096 Dec 15 17:45 var

root@d7e08c154a4a:/# cd data

root@d7e08c154a4a:/data# ls -la
total 8
drwxr-xr-x  2 root root 4096 Jan  9 23:49 .
drwxr-xr-x 35 root root 4096 Jan  9 23:49 ..

[StefanScherer]

Linking https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#fully_qualified_vs._relative_paths for reference.

For a program trying to follow symlinks this could not work inside the container. The \\?\ is only a marker for long path names up to 32.767 characters, and afterwards it is not an UNC path but a relative local path to \ContainerMappedDirectories. So all tools like Golang, Java, Node.js, git I found so far I think they try to do it right. I think the symlink misleads them. And that is mostly the "error" I see there that they try to access C:\ContainerMappedDirectories.

It would help a lot all that tools and languages if such volume mounts inside the container just do not show them as symlinks, just like in the Linux container example above. So following symlinks stops there and everything would be fine. I think it will be hard to explain that to all maintainers how to workaround this. Or is there a missing flag in the file attributes I just overlook?

[StefanScherer]

I once tried to propose a simple patch to stop following to that directory for Node.js, but the maintainers answered:

This seems a bit hacky. Wouldn't this prevent any application from using \ContainerMappedDirectories?

I hope there will be a clean way for all languages to solve that volume mount problem.

[sixeyed]

On the volume mount problem, this is an OpenJDK bug now: JDK-8172711. For apps using volumes there is a workaround, as proposed for Elasticsearch

[akashche]

@StefanScherer thanks for detailed explanation!

@sixeyed on the bugreport (this openjdk upstream link may be better [1]):

EXPECTED -
The call to toRealPath should resolve the underlying path to the Docker volume, which is a valid symlink.

Can we say, that this symlink wil be a "dangling symlink" from in-container FS point of view?

I mean resolving VOLUME c:/foo/bar will give us something like c:/foo/ContainerMappedDirectories/006DB244-75C2-4ADE-A7A9-6B5EC3F56796. And such path probably should never be returned to in-container caller.

+1 to drive letter mapping workaround.

PS: as openjdk contributor I have access to its bugtracker [1] - can add comments to that issue if you would like to put some info there.

[1] https://bugs.openjdk.java.net/browse/JDK-8172711

[tianon]

@akashche any plans for ojdkbuild to support more than just Java 1.8, and any notes you can provide about ~how long Java 1.8 will be supported there? 😇 (this is a decent place to start for the purposes of this image, but I'm curious what the future plans will be 👍)

[akashche]

@tianon

any plans for ojdkbuild to support more than just Java 1.8

Yes, jdk9 is in preparation, and going to be added to ojdkbuild repos soon. Though public binaries for it will probably wait until upstream release in July ( http://openjdk.java.net/projects/jdk9/ ). Also jdk8 32-bit is going to be added, but that is probably not relevant to containers as nanoserver doesn't support 32-bit

any notes you can provide about ~how long Java 1.8 will be supported there?

Currently ojdkbuild releases on github are planned to be maintained (updated with security fixes 4 times a year) as long as jdk8 will be supported in RHEL, and, according to public info ( https://access.redhat.com/articles/1299013#OpenJDK_Lifecycle_Dates_and_RHEL_versions ), that is October 2020.

Also a question about this docker image maintenance: I have an updated 8u121 release almost ready to go, what will be a preferred process to update this image with newer jdk version? Will it be convenient if I will create a pull-request to update Dockerfiles? Should I wait for this pull request to be merged first?

[yosifkit]

For version bumps, @tianon updated the update.sh script to scrape the newest release from github so, once merged, our jenkins job should pick it up and update the Dockerfiles automatically. 😄

(just a reminder of the process: this would not cause new images to appear on the hub; that requires a PR to https://github.com/docker-library/official-images)

[sixeyed]

@StefanScherer what are your thoughts on tagging these Windows variants? We're expecting a base image update from Microsoft every month. Would it be preferable to pin to a specific base image version in the Dockerfile and include that version in the tag? It would make the versions clear, but it would give us a tag like: 8u111-jdk-windowsservercore-10.0.14393.693.

[StefanScherer]

@sixeyed Good question. That would be the best way to have exact version even after a new base OS update. I know that the microsoft/iis and microsoft/aspnet images use the exact OS version in the tag names which seems good to me having them there.

How is it done on Linux? I had a quick look at alpine which uses alpine:3.4 and https://hub.docker.com/r/library/buildpack-deps/tags/ which seem to be updated with same tags.
And also the openjdk images are rebuilt only a few days ago, also probably to have latest OS files underneath.

So I think there is no "latest is no version" restriction regarding the underlying OS image for these official images. If someone in prod has such hard constraints he/she also doesn't use any image from Docker Hub, right?

So I think we should not overcomplicate things on Windows side.

It would be more important for me to have current rebuilds available shortly after a base image update from Microsoft.

[sixeyed]

@StefanScherer that makes sense, but also it seems a shame to lose that information when we have it at build time. Maybe include it as a label? I'm thinking of audit scenarios where users want to check if KBxyz is included in their build.

For Ubuntu they add a timestamp as an additional tag - these are all the same image:

ubuntu:16.04
ubuntu:xenial
ubuntu:xenial-20170119

Just a discussion point though.

[yosifkit]

I'm +1 to being more specific on the windows image version for the FROM. We already have an update script and can use it to also scrape for new windows images and bump versions in the Dockerfile. Though I kind of wish the windows images had a microsoft/windowsservercore:10.0 that followed the newest 10.0 or at least something more specific than latest and less specific than 10.0.14393.693.

As far as tagging the image, I think anything more specific than windowsservercore-10.0 will clutter the already cluttered "Supported tags and respective Dockerfile links" section on the Docker Hub and is not going to be generally useful. Users should be using the sha256 id to pin images on their production systems (see also repo-info openjdk:8u11).

[PatrickLang]

Microsoft publishes a list of version numbers, and each one has a corresponding tag in microsoft/windowsservercore repo. I feel that using SHA256 or timestamp in place of the Windows version would obfuscate it rather than making it clear what Windows version was used.

References regarding Windows versions published by Microsoft:

• https://support.microsoft.com/en-us/help/4000825/windows-10-and-windows-server-2016-update-history
• https://github.com/Microsoft/Virtualization-Documentation/blob/live/virtualization/windowscontainers/manage-containers/container-base-images.md

[PatrickLang]

If you don't want to reflect the Windows version in tagging, then I think it would make sense to just use -nanoserver or -windowsservercore in the tag.

microsoft/dotnet follows this approach