Skip to content

Timeset validation accepts vectors mixing YYYYMMDD and YYYYWW, and pvt_twitter()/pub_wiki() send weekly values on the day parameter without checking `time_type #400

Description

@docxology

Summary

While working through day-vs-week confusion (open issues #286/#278 describe the user-facing symptom: confusingly empty results), I traced it to two concrete validation gaps:

  1. parse_timeset_input() accepts any integer/character vector where every element is 6-or-8 characters — so a vector mixing day values (20200101) and week values (202201) passes validation unchanged and is sent comma-joined on one wire parameter.
  2. pvt_twitter() and pub_wiki() select dates vs epiweeks purely from time_type but never verify the values' format against it — so pvt_twitter(auth, "CA", time_values = epirange(201501, 202001)) with the default time_type = "day" sends a weekly range (201501-202001) as a day value.

I think client-side validation (uniform granularity per vector; nchar cross-check against time_type) would turn these silent misfires into clear errors, complementing the error-message improvements discussed in #286/#278.

Evidence (dev tip 87b10e1; same code on main)

  1. Mixed vector — R/check.R:91-96:
    } else if (test_integerish(value)) {
      if (all(nchar(value) %in% c(6, 8))) {   # per-element: 6 or 8
        return(value)
    c(20200101, 202201) → every element is 8-or-6 → returned unchanged, mixed.
  2. No time_type cross-check — pvt_twitter (R/endpoints.R:3205-3245) and pub_wiki (R/endpoints.R:3290-3330):
    if (time_type == "day") {
      dates <- time_values
      epiweeks <- NULL
      dates <- get_wildcard_equivalent_dates(dates, "day")
    } else {
      dates <- NULL
      epiweeks <- time_values
      ...
    }
    validate_timeset_input() is then applied to dates/epiweeks with no relation to the chosen time_type.

Maintainer-runnable repro

# (1) mixed granularity passes validation:
epidatr:::parse_timeset_input(c(20200101, 202201))
# [1] 20200101      202201   — returned unchanged

# (2) weekly values ride the day parameter:
pvt_twitter(
  auth = "k", locations = "CA",
  time_values = epirange(201501, 202001),   # a *weekly* range
  fetch_args = fetch_args_list(dry_run = TRUE)
)$request$url
# ...&dates=201501-202001   <- sent on 'dates' although time_type defaults to "day"

Impact

Expected vs. actual

  • Expected: client-side rejection (or loud warning) when timeset values don't match the endpoint's expected granularity.
  • Actual: validation passes; the mismatch is only discoverable from an opaque server-side outcome.

Suggested fix

  1. In parse_timeset_input(), require uniform granularity per vector: reject when any(nchar(value) == 6) && any(nchar(value) == 8).
  2. In pvt_twitter()/pub_wiki(), cross-check before the call: for time_type == "day" require nchar 8 (or 10-char dates), for "week" require 6; abort client-side with a message pointing at time_type.
  3. Consider a shared helper so other endpoints taking day-vs-week timesets get the same check (this is the general shape of what Raise error when dates / date range are passed to things expecting weeks?  #278 asks for).

I checked for prior reports: open #286 ("Weekly data should require week numbers") and #278 ("Raise error when dates / date range are passed to things expecting weeks?") describe the user-facing confusion in pub_covidcast/pub_flusurv terms; this report adds the concrete validation gaps (mixed vectors; pvt_twitter/pub_wiki lacking any cross-check). Happy to consolidate into either if preferred.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions