script checks: use check ID from group service hook

[tgross]

Script checks are registered in Consul at the group service hook, then executed and heartbeat by the Nomad client in the script check hook. But not all fields in the task environment are available for interpolation at the time the group service hook runs, and the hash we use for the check ID includes all fields post-interpolation. The script check hook intends to use the original uninterpolated check ID, but the service it's reading this value from has been interpolated with the full task environment at this point. Using the uninterpolated check from the task group would be incorrect anyways, as the group service check has interpolated before creating the ID used to register the check.

Ideally we'd interpolate values available at submit-time and generate an immutable service ID and check ID based on that. But for backwards compatibility with existing registered services, we'll need to fix this at the script check instead.

Have the group service check record every check ID it creates in the allochook resources. Thread these down to the script check taskrunner hook and use the stored values as an override of the check ID we use for TTL updates.

Fixes: #26952
Ref: https://hashicorp.atlassian.net/browse/NMD-1054

Testing & Reproduction steps

Run Consul and a dev server, and do the usual nomad setup consul -y configuration. Deploy the following jobspec and observe that both script checks and TCP checks work as expected. Note the NOMAD_ALLOC_IP_www value is key here; just using the NOMAD_JOB_NAME doesn't trigger the bug because that's interpolated at the group level.

jobspec

job "example" {

  group "group" {

    network {
      mode = "bridge"
      port "www" {
        to = 8001
      }
    }

    service {
      name = "${NOMAD_JOB_NAME}-svc"
      port = "www"

      check {
        name      = "${NOMAD_JOB_NAME}-tcp-check"
        type     = "tcp"
        port     = "www"
        interval = "10s"
        timeout  = "2s"
      }

      check {
        name      = "${NOMAD_JOB_NAME}-script-check"
        type      = "script"
        command   = "/bin/sh"
        interval  = "5s"
        timeout   = "1s"
        task      = "httpd"
        args = ["-c", "echo ${NOMAD_ALLOC_IP_www}"]
      }
    }


    task "httpd" {
      driver = "docker"

      config {
        image   = "busybox:1"
        command = "httpd"
        args    = ["-vv", "-f", "-p", "8001", "-h", "/local"]
        ports   = ["www"]
      }
    }
  }
}

Contributor Checklist

• Changelog Entry If this PR changes user-facing behavior, please generate and add a
  changelog entry using the make cl command.
• Testing Please add tests to cover any new functionality or to demonstrate bug fixes and
  ensure regressions will be caught.
• Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
  and job configuration, please update the Nomad product documentation, which is stored in the
  web-unified-docs repo. Refer to the web-unified-docs contributor guide for docs guidelines.
  Please also consider whether the change requires notes within the upgrade
  guide. If you would like help with the docs, tag the nomad-docs team in this PR.

Reviewer Checklist

• Backport Labels Please add the correct backport labels as described by the internal
  backporting document.
• Commit Type Ensure the correct merge method is selected which should be "squash and merge"
  in the majority of situations. The main exceptions are long-lived feature branches or merges where
  history should be preserved.
• Enterprise PRs If this is an enterprise only PR, please add any required changelog entry
  within the public repository.

• If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

[jrasell]

LGTM. I've left a couple of inline comments, but I don't see those as blocking.