Skip to content

Per-endpoint auth keys are sent in the URL query string and printed verbatim by print.epidata_call() / dry-run output — worth redacting #386

Description

@docxology

Summary

Most pvt_* endpoints take a restricted-access key as the auth argument and pass it straight into the request's query string, and print.epidata_call() prints the full URL including that key. Unlike the general DELPHI_EPIDATA_KEY (attached as a Basic-auth header inside do_request() and never present in any URL, print output, or committed snapshot), these per-endpoint keys therefore end up in server/proxy access logs, console transcripts, knitted notebooks, pasted issue reports, and any httr2 error condition embedding req$url. The URL placement itself is dictated by the server protocol, but I think the client can stop amplifying it — by redacting auth= in print.epidata_call(), request_url()-derived user-facing output, and error paths.

Evidence (dev tip 87b10e1; the same pattern is on main)

  • The params list places auth in the query — R/endpoints.R:97-104 (pvt_cdc), same pattern at :2189 (pvt_dengue_sensors), :2652 (pvt_ght), :2747 (pvt_meta_norostat), :2937 (pvt_norostat), :3096 (pvt_quidel), :3164 (pvt_sensors), :3242 (pvt_twitter), plus optional pub_fluview (:2534) — 9 call sites in total:
    create_epidata_call(
      "cdc/",
      list(
        auth = auth,
        locations = locations,
        epiweeks = epiweeks
      ),
      ...
  • The key lands in the committed URL snapshot (tests/testthat/_snaps/endpoint-urls.md:7):
    https://api.delphi.cmu.edu/epidata/cdc/?auth=test-auth-key&locations=fl%2Cca&epiweeks=201501-201601
  • print.epidata_call prints the URL verbatim (R/epidatacall.R:136-142):
    "*" = paste0("Request URL: ", x$request$url)
  • Contrast: the general key never touches the URL — R/request.R:38-45 attaches req_auth_basic("epidata", key) (classic) or req_headers(token = key) (cast) onto a local request copy, and dry_run returns before any attach.
  • Aggravator: the ?pvt_cdc example routes the general key into this URL too — auth = Sys.getenv("DELPHI_EPIDATA_KEY") (R/endpoints.R:68-72).

Maintainer-runnable repro

print(pvt_cdc(
  auth = "SECRETKEY", locations = "fl",
  epiweeks = epirange(201501, 201601),
  fetch_args = fetch_args_list(dry_run = TRUE)
))
# Request URL: https://api.delphi.cmu.edu/epidata/cdc/?auth=SECRETKEY&locations=fl&epiweeks=201501-201601

Impact

  • print.epidata_call() output, request_url(), and dry-run debug output embed the key; these routinely get pasted into issue reports, Slack, notebooks, and teaching material (the package's own examples encourage printing/dry-running).
  • Server- and proxy-side access logs record full URLs by default, so restricted keys leak into log infrastructure the Delphi team may not control.
  • httr2 error conditions (httr2_http_4xx/5xx) carry req$url and surface in rlang::last_trace() output.

Expected vs. actual

  • Expected: secrets never appear in printable/debug output, regardless of how they are transported.
  • Actual: restricted keys are printed verbatim wherever the URL is shown.

Suggested fix

  1. Redact in print.epidata_call() (and any user-facing rendering of x$request$url): rewrite auth=<value> to auth=<redacted> before display.
  2. Apply the same redaction where error messages embed the URL.
  3. Document the log-exposure caveat for auth-style endpoint keys, and stop routing the general DELPHI_EPIDATA_KEY into the example call in ?pvt_cdc.
  4. Longer term, when the server supports it, move restricted-key auth to a header so the key never enters the URL at all.

Related in-flight work

PR #380 reworks the v5/cast request fan-out but does not touch the classic auth-in-URL pattern. No other in-flight branch addresses this.

I checked for prior reports (open and closed issues/PRs mentioning key exposure, URL redaction, or auth handling) and found none.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions