NodePublish : Ability to call NodePublish multiple times on the same node

[oritnm]

Handling multiple invocation of NodePublish for a given volume on the same node.

Use cases:

• Security: where the same volume is consumed from different containers running on the same node each with a different user/access mode.
• idempotency requests: The SP can not rely on having a single invocation of NodePublish request such as in timeout flows
  Details can be found in issue NodePublish – Secure access to shared mounted volumes  #17

[jieyu]

I think the key to the problem is: is there any per-container setup for the volume that has to be customized by the plugin? For uid/gid issue, one can argue that the CO can perform that if kernel supports uid/gid shift during mount (currently WIP in kernel I believe)? It'll be nice to see more use cases that absolutely requires per-container plugin specific logic. One I can think of is some fuse mount that requires user credentials which only the plugin understands.

[quolix]

@jieyu I am not aware of any other per-container relationship, but I don't see how access control could be solved in a different way anytime soon. (Your fuse mount with user credentials might be valid though).

Even if uid/gid shift would solve the problem, it might take quite a while until it reaches the kernels of all major distributions. Also the uid/gid shift is probably not the whole story, as the identity is a string (probably even a namespaced one, think tenants), and if you would want to tunnel that through the uid, you'd also need to establish dynamic uid->name mappings in the kernel (which would not easily support namespaces), etc.

By setting the identity from the plug-in over a side channel, we can solve this with today's mechanism and without any further plumbing, as demonstrated with our Quobyte k8s plug-in prototype.

[oritnm]

@jieyu, in addition to what's noted here by @quolix, I don't think that the UID/GID is the entire issue, one could request secret/pwd for the SP to verify you are allowed for the operation from the SP perspective. Nor can you do one mount (while switching users) for several different users each running from a different container using the shared volume.
I don't think we can escape the ability for multiple nodepublish on the same node for these reasons and to be able to resend nodepublish request (as required by the idempotency of the command) in case the CO has not received an ACK

[jieyu]

Raised this issue during the working group sync yesterday. The consensus was that the spec should allow NodePublish to be called multiple times for a volume on a node, one for each "use". Different CO might have different way to interpret "use". For instance, in k8s, "use" probably means pod, while in other CO like Docker, "use" probably means container. This is entirely up to the CO to decide.

We also discussed how to pass per "use" information to the plugin. Again, the CO is responsible for exposing some API allowing users/operators to specify those information. The bottom line is that the API should be general to all plugins because the CO does not want to write plugin specific logic most likely.

[saad-ali]

PR #83 modifies the spec to indicate that NodePublish will be called multiple times (at least once per volume per workload).