Upgrade client to fix build, authentication, and image issues #26

rosscado · 2016-10-14T14:17:11Z

The watchtower build has been broken for several months since a failed attempt to introduce support for private docker registry authentication.

I have addressed these and related issues with the following changes.

Client Upgrade

Migrated the watchtower api client from the deprecated samalba/dockerclient to the officially supported docker/engine-api. The upgraded client achieves support for a private docker registry authentication use-case as described in issue #6, while leveraging .docker/config.json for authentication credentials as described in issue #3.

Dependency Management

watchtower had been using a deprecated version of godep to manage its dependencies on external go packages. I tried migrating godep but godep failed to resolve the project's dependencies, so I abandoned it in favour of the simpler go get command.

Revert Base Image

Pull request #13 had also inflated the size of the watchtower image from <6MB to >190MB by replacing the minimal base image with ubuntu:14.04. This had been introduced as a fix for package dependency errors and was unsuccessfully addressed by pull request #21.
Resolving the dependency management errors allowed me to package the application into a small, self-contained binary with golang-builder as intended, and to revert the base image to centurylink/ca-certs. This reduced the total image size to ~8MB.

Security Changes

Changes in the docker api client have enabled additional use cases such as private repo authentication while simplifying configuration. As a result of the latter I have removed some of the TLS options, e.g. paths to CA keys. It is possible that some more complex use cases are no longer supported as a result. I have not tested this area thoroughly and would appreciate contributions to identify/resolve any lost functionality.

Other Changes

A new cli option --apiversion signals the minimum server api version to work with. This information is required by the new client library and I added the cli option as a way for users with older docker servers to avoid version incompatibility errors. Others may have better suggestions for how to handle this.

…lient upgrade

…instead

Conflicts: CONTRIBUTING.md

rosscado · 2016-10-17T09:31:26Z

Dockerfile

@@ -1,4 +1,6 @@
-FROM ubuntu:14.04


A previous commit had set the base image to ubuntu:14:04 as a workaround for incorrectly packaged dependencies because ubuntu contains many of those dependencies (and much more besides). Proper use of golang-builder to package all of watchtower's dependencies into a single binary executable allows us to revert to a minimal base image.

Load authentication credentials for available credential stores in order of preference: 1. Environment variables REPO_USER, REPO_PASS 2. Docker config files Request image pull with authentication header. Wait until pull request is complete before exiting function.

… on environment variables

…ays used. This causes authentication failures on registries that don't match, including public registries. Fallback to no-authentication to handle the case of public registries.

rosscado · 2016-10-18T15:02:51Z

I found a couple of issues still remained with authentication.

Authentication was failing silently for private registries. The upgraded docker api client was not loading authentication credentials from the docker config by default as I had thought.
I have since implemented authentication directly, with a choice of two credentials stores:

environment variables (REPO_USER and REPO_PASSWORD)
default docker config (e.g. ~/.docker/config.json)

I realised afterwards that the default docker config credential store cannot be supported since the host's docker config files are not available to the container at runtime (and cannot be mounted into a scratch base image). Therefore environment variables are the only supported credential store, and the README.md documentation has been updated to reflect this.

seertenedos · 2016-10-20T19:57:02Z

Any chance of you posting your version of the watchtower till it is merged in? Really need the config.json support

TheCodeDestroyer · 2016-11-07T08:27:10Z

@rosscado It would be awesome if you created a temp hub publish

dolanor · 2016-11-07T16:23:09Z

@seertenedos It is available in his own repo. Or did I get your message wrong?

seertenedos · 2016-11-07T19:38:20Z

@dolanor @rosscado I meant it would be great if it could be put on Docker Hub. Also why is the config.jon not possible? Couldn't the user just copy it to a folder on the host and use volume mapping or something to pass it in at a particular location? Or maybe a small web Ui to configure it. In my case i have 2 private repositories. A dockerhub one and canister.io since it allows multi repositories for free. That said it is really the canister.io that i care about with watchtower

seertenedos · 2016-11-07T20:14:13Z

@rosscado thanks so much it works well. One question. Is there a way to specify repository the password is for like for cloud.canister.io so that it only uses the username and password for containers from that repository and will not try to user the username and password for the containers from other repositories? I know i can just run 2 copies of watch tower and list every container each one supports but i was wondering if i could get away with one watchtower that could support authenticated and un authenticated

rosscado · 2016-11-07T20:50:59Z

@seertenedos there is not, but I appreciate the multiple-registry use case.

The config.json support would have enabled this; The reason I pulled that feature was that the watchtower container lacks its own filesystem into which I could mount a ~/.docker config directory.

There might be a solution if I could mount the ~/.docker/config.json file directly and still know how to resolve its path. Let me see what I can do.

seertenedos · 2016-11-07T21:02:39Z

@rosscado putting the file into the container at any path would not be hard i would think using the -v param when creating the docker container like they do for the docker socket but i assume you are having an issue telling docker where the config file is and to use it. Is that the issue or is it that there is no home directory or even the possibility to create one on the scratch image. I thought it would just use the /root folder as the home directory as it would be running as root by default right so maybe if we mount it under there it would work? An alternative is couldn't we set $HOME using the -e command when creating the container and then put the file in $HOME/.docker/config.json?

Also one other option is to set the config folder with --config param on the docker commandline (https://docs.docker.com/engine/reference/commandline/cli/)

dolanor · 2016-11-07T21:26:00Z

I'm very interested in that patch because I wanted to do it myself to see if it would magically support multiple network from libnetwork by using the official client.
So I'm gonna try your image and see if my use case works out of the box, or if I need to add this specific part.
Thank you for your work already!

…ower hub

The circle.yml version on this branch will automatically trigger a Circle CI build and push the resulting Docker image to the DockerHub repo rosscado/watchtower. This is a temporary DockerHub repo for the rosscado/watchtower GitHub repo, and can be used to pull or test unofficial watchtower builds before they are merged into the official centurylink/watchtower repos.

…b repo

rosscado · 2016-11-08T13:04:47Z

@seertenedos @dolanor I have added config.json support as discussed.

Watchtower will load private registry authentication credentials from the host's docker config file when mapped to the root (/) of the container filesystem

-v ~/.docker/config.json:/config.json

Or to a different location specified by the DOCKER_CONFIG environment variable

-e DOCKER_CONFIG="/home/root" -v ~/.docker/config.json:/home/root/config.json

If REPO_USER and REPO_PASS environment variables are specifed their values will be used instead.

seertenedos · 2016-11-09T10:01:10Z

@rosscado I hit a new issue i did not have before. did you update the docker client version? I get the error below now and since i run on a QNAP NAS i can't change docker versions till they do if that is the issue.
time="2016-11-09T09:58:20Z" level=fatal msg="Error response from daemon: client is newer than server (client API version: 1.25, s
erver API version: 1.23)"

watchtower outputs that and dies right away

rosscado · 2016-11-09T10:31:59Z

@seertenedos a solution is available using the --apiversion option. The default behaviour of the docker client is to not communicate with servers older than itself but you can override this behaviour by explicitly specifying a minimum api version less than or equal to your server's version.

Example

docker run -d \
  --name watchtower \
  -v /var/run/docker.sock:/var/run/docker.sock \
  rosscado/watchtower
  --apiversion 1.23

I refered to the API version problem in my opening post. I don't much like this default behaviour and am open to improvements if you can suggest any?

seertenedos · 2016-11-09T19:30:42Z

@rosscado thanks that worked. I did not realise docker did that. Also i can confirm the config.json is working perfectly! one more question when it is checking to see if there is a new version how much data does it use? as in does it pull just a few kb to check or does it pull the whole image to find out there is no difference? At the minute i have watchtower on default check of every 5 min but wonder if i need to make them further apart

rosscado · 2016-11-10T12:14:07Z

@seertenedos afaict it just pulls a small amount of data to check for changes and pulls the full image only if a difference is found. But that's legacy functionality that I haven't modified in this pull request.

subhashdasyam

was all changes pushed to rosscado/watchtower?

i can't seem to get this work by giving
docker run --name watchtower -v /var/run/docker.sock:/var/run/docker.sock -v ~/.docker/config.json:/config.json rosscado/watchtower

for private registry

EDIT:

Sorry my bad this works !!! thanks :D

dolanor · 2016-11-22T23:59:15Z

So, I checked if my container joined the correct networks after a restart, and it is still not the case. So I guess I have to find where the code is and patch it.
Anyway, thanks again for your work 👍, I would have been forced to pass there first 😄

dolanor

The code should be passed through go fmt.

dolanor · 2016-12-09T00:37:05Z

container/client.go

@@ -110,92 +115,93 @@ func (client dockerClient) StopContainer(c Container, timeout time.Duration) err
 }

 func (client dockerClient) StartContainer(c Container) error {
+	bg := context.Background();


Let's not put some semicolon. It's one of my favourite feature of this language 😛

dolanor · 2016-12-09T00:38:27Z

container/client.go

-		c := Container{containerInfo: containerInfo, imageInfo: imageInfo}
-		if fn(c) {
+		c := Container{containerInfo: &containerInfo, imageInfo: &imageInfo}
+		if (fn(c)) {


Let's remove those parenthesis.
Can you pass your code through go fmt?

Removed hub_mirror deployment, came in with PR #26.

Ross Cadogan added 19 commits October 13, 2016 18:07

Migrate Godeps/_workspace/ to vendor/

936928f

Reinstate MAINTAINER and LABEL, Ubuntu base image required by dockerc…

875efba

…lient upgrade

Improve error handling

c4efc83

Improve error reporting

02bca8d

Private registry authentication distinct from host

25f1fee

Port client lib from samalba/dockerclient to docker/docker/client

99ed959

go fmt

ed035d6

Discard obsolete samalba/dockerclient library and dependent tests

e011d3e

Ignore build output (watchtower binary)

61ac18f

Migrate from codegangsta lib to urfave

b13e6af

Updating dependencies with

da9ba69

Consistent context

97214b2

godep doesn't work, distro required

4771857

godeps doesn't work, go without

69db640

Rework TLS support, remove unsupported options

42fea79

bdehamer/golang-builder doesn't work, use centurylink/golang-builder …

8f13706

…instead

Build instructions for contributors (because it's not obvious)

2386d18

Build instructions for contributors (because it's not obvious)

e8f6db2

Merge branch 'auth' of github.com:rosscado/watchtower into auth

44dbd88

Conflicts: CONTRIBUTING.md

rosscado commented Oct 17, 2016

View reviewed changes

Ross Cadogan added 4 commits October 18, 2016 10:04

Add auth config, registry auth fails silently without

ef430b7

Cannot load host Docker config from container. Remove option and rely…

5412642

… on environment variables

When authentication credentials are supplied as env vars they are alw…

e752364

…ays used. This causes authentication failures on registries that don't match, including public registries. Fallback to no-authentication to handle the case of public registries.

Refactor port mapping functions for build simplicity

6a56597

Ross Cadogan and others added 5 commits November 8, 2016 12:07

Automatically push rosscado/watchtower:auth branch to rosscado/watcht…

5a03b65

…ower hub

Automatically deploy from hub branch to rosscado/watchtower docker hu…

79320bb

…b repo

Support loading authentication credentials from Docker config file

dad5d58

Deploy to official and unofficial hubs

96466db

subhashdasyam approved these changes Nov 22, 2016

View reviewed changes

dolanor suggested changes Dec 9, 2016

View reviewed changes

davidgardner11 merged commit 37f7248 into containrrr:master Dec 22, 2016

stffabi mentioned this pull request Dec 27, 2016

Multiple network support #23

Closed

davidgardner11 pushed a commit that referenced this pull request Dec 27, 2016

Merge pull request #36 from stffabi/RemoveHubMirror

4e7576f

Removed hub_mirror deployment, came in with PR #26.

This was referenced Jan 2, 2017

Does watchtower support a private registry? #32

Closed

Fix Multiple Network Support #23 #40

Merged

stffabi mentioned this pull request Jan 7, 2017

Pass in HostConfig to create operation #15

Closed

This was referenced Jan 15, 2017

revert base image to centurylink/ca-certs #21

Closed

latest release on hub.docker.com #44

Closed

Image not found, private repository #45

Closed

This was referenced Jan 24, 2017

not able to download/update from my private docker registry. #30

Closed

Support .dockercfg / authConfig for using watchtower with images from private registry #3

Closed

Remove apiversion flag #51

Closed

Ostico mentioned this pull request Aug 7, 2023

Unable to install docker-credential-ecr-login using watchtower dockerfile #1719

Closed

Upgrade client to fix build, authentication, and image issues #26

Upgrade client to fix build, authentication, and image issues #26

Uh oh!

Conversation

rosscado commented Oct 14, 2016

Client Upgrade

Dependency Management

Revert Base Image

Security Changes

Other Changes

Uh oh!

rosscado Oct 17, 2016

Choose a reason for hiding this comment

Uh oh!

rosscado commented Oct 18, 2016

Uh oh!

seertenedos commented Oct 20, 2016

Uh oh!

TheCodeDestroyer commented Nov 7, 2016

Uh oh!

dolanor commented Nov 7, 2016

Uh oh!

seertenedos commented Nov 7, 2016

Uh oh!

seertenedos commented Nov 7, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rosscado commented Nov 7, 2016

Uh oh!

seertenedos commented Nov 7, 2016

Uh oh!

dolanor commented Nov 7, 2016

Uh oh!

rosscado commented Nov 8, 2016

Uh oh!

seertenedos commented Nov 9, 2016

Uh oh!

rosscado commented Nov 9, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Example

Uh oh!

seertenedos commented Nov 9, 2016

Uh oh!

rosscado commented Nov 10, 2016

Uh oh!

subhashdasyam left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dolanor commented Nov 22, 2016

Uh oh!

dolanor left a comment

Choose a reason for hiding this comment

Uh oh!

dolanor Dec 9, 2016

Choose a reason for hiding this comment

Uh oh!

dolanor Dec 9, 2016

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

seertenedos commented Nov 7, 2016 •

edited

Loading

rosscado commented Nov 9, 2016 •

edited

Loading

subhashdasyam left a comment •

edited

Loading