Review automatic weekly data type conversion

[dshemetov]

epidatr automatically converts weeks to dates, using MMWRweeks, but unfortunately we don't have a convention as to what weeks mean in covidcast-indicators, so we should be careful about this.

Either:

• Double check which indicators actually use epiweeks and maybe allow conversions to date markers (e.g. first day of week)
• Don't convert and keep as integer, leave it up to the user

[dsweber2]

https://cmu-delphi.github.io/delphi-epidata/api/covidcast_times.html notionally if its week its an MMWR week

[dajmcdon]

Possibly related, or separate issue:

library(epidatr)
nhsn <- pub_covidcast(
  source = "nhsn",
  signals = "confirmed_admissions_flu_ew",
  geo_type = "state",
  time_type = "epiweek",
  time_values = "*",
  issues = "*"
)
#> Error in create_epidata_field_info("time_value", switch(time_type, day = "date", : is.character(type) && length(type) == 1 && type %in% c("text",  .... is not TRUE

It would be nice if this "worked" (say catching that "epiweek" should be "week") or at minimum, gave a more informative error message.

This particular case is especially confounding since the signal has the _ew suffix indicating "epiweek".

[dshemetov]

A number of actionable tasks here:

• As per Dan's comment: validate the time_type field and give a helpful error in pub_covidcast.
• Error when source = nssp and time_type != week for NSSP in pub_covidcast.
• Go through the rest of the covidcast endpoints, mark which are week only or day only and make these guardrails more comprehensive.
• Add checks to covidcast so that if the time_type is week and the user puts in dates (and vice versa), we either give a helpful error or use reformat_epirange to give a warning and auto-convert.

Ref user confusion: cmu-delphi/delphi-epidata#1653

[dshemetov]

In response to OP concerns:

epidatr automatically converts weeks to dates, using MMWRweek (happens here). covidcast-indicators automatically converts from dates to epiweeks (when appropriate) using epiweeks (happens here). So the automatic conversion we do by default should work fine. Users can also use disable_date_parsing=TRUE to stop this automatic conversion.

[dshemetov]

Following up with a user story: I was just comparing the new hsa_nci NSSP dataset from the API. The original dataset is in date format but gives the end of a given collection week. We convert that to week in the API and convert back in epidatr here. The dates come out off from the original dataset. The problem: our parse_api_week function is hardcoded to pick the start of the week as the canonical date for the week, while the data source uses the end of the week. Considering adding another argument like fetch_args_list(week_reference_day = 7) and threading that through to parse_api_week. Also some warnings about all this.

[dshemetov]

• Need to update these lines in pub_covidcast to include hsa_nci

      create_epidata_field_info(
        "geo_type",
        "categorical",
        categories = c("nation", "msa", "hrr", "hhs", "state", "county")
      ),

[dshemetov]

Hotfixes some of these in #326