Browse Source

API endpoint removal due to loss on CDC portal

pull/18/head
boB Rudis 6 years ago
parent
commit
5deb1f19bf
No known key found for this signature in database GPG Key ID: 1D7529BE14E2BBA9
  1. 6
      .Rbuildignore
  2. 6
      DESCRIPTION
  3. 1
      NAMESPACE
  4. 96
      R/get-mortality-surveillance-data.r
  5. 16
      cran-comments.md
  6. 41
      man/get_mortality_surveillance_data.Rd
  7. 8
      tests/testthat/test-cdcfluview.R

6
.Rbuildignore

@ -1,10 +1,8 @@
^.*\.Rproj$
^\.Rproj\.user$
^\.travis\.yml$
^README\.*Rmd$
^README\.*html$
^NOTES\.*Rmd$
^NOTES\.*html$
^README.*$
^NOTES.*$
^\.codecov\.yml$
^README_files$
^doc$

6
DESCRIPTION

@ -2,8 +2,8 @@ Package: cdcfluview
Type: Package
Encoding: UTF-8
Title: Retrieve 'U.S'. Flu Season Data from the 'CDC' 'FluView' Portal
Version: 0.7.0
Date: 2017-11-04
Version: 0.8.0
Date: 2018-09-16
Authors@R: c(
person("Bob", "Rudis", email = "bob@rud.is", role = c("aut", "cre"),
comment = c(ORCID = "0000-0001-5670-2640")),
@ -39,4 +39,4 @@ Imports:
readr,
MMWRweek,
units (>= 0.4-6)
RoxygenNote: 6.0.1
RoxygenNote: 6.0.1.9000

1
NAMESPACE

@ -5,7 +5,6 @@ export(cdc_basemap)
export(geographic_spread)
export(get_flu_data)
export(get_hosp_data)
export(get_mortality_surveillance_data)
export(get_state_data)
export(get_weekly_flu_report)
export(hospitalizations)

96
R/get-mortality-surveillance-data.r

@ -1,96 +0,0 @@
#' Mortality Surveillance Data from the National Center for Health Statistics (deprecated)
#'
#' The National Center for Health Statistics (NCHS) collects and disseminates the Nation's
#' official vital statistics. These statistics are based on data provided to NCHS through
#' contracts with the vital registration systems operated in the various jurisdictions
#' legally responsible for the registration of deaths (i.e., death certificates) and other
#' vital events. These data have previously only been released as annual final data files
#' 12 months or more after the end of the data year. Recent NCHS efforts to improve the
#' timeliness of jurisdiction reporting and modernize the national vital statistics
#' infrastructure have created a system capable of supporting near real-time surveillance.
#' Capitalizing on these new capabilities, NCHS and CDC’s Influenza Division have
#' partnered to pilot the use of NCHS mortality surveillance data for Pneumonia and
#' Influenza (P&I) mortality surveillance.
#'
#' NCHS mortality surveillance data are presented by the week the death occurred.
#' Nationally P&I percentages are released two weeks after the week of death to allow for
#' collection of enough data to produce a stable P&I percentage at the national level.
#' Collection of complete data is not expected, and reliable P&I ratios are not expected
#' at the region and state level within this two week period. State and Region level
#' counts will be released only after 20% of the expected number of deaths are reported
#' through the system.
#'
#' @references \url{https://www.cdc.gov/flu/weekly/nchs.htm}
#' @return a list of \code{tbl_df}s
#' @export
#' @examples \dontrun{
#' get_mortality_surveillance_data()
#' }
get_mortality_surveillance_data <- function() {
message(
paste0(
c("This function has been deprecated and will be removed in future releases.",
"Use pi_mortality() instead."),
collapse="\n"
)
)
# scrape (ugh) web page to get data file links for state mortality data
pg <- xml2::read_html("https://www.cdc.gov/flu/weekly/nchs.htm")
PREFIX <- "https://www.cdc.gov"
xml2::xml_find_all(pg, ".//select[@id='State']/option[contains(@value, 'csv') and
contains(@value, 'State_')]") %>%
xml2::xml_attr("value") %>%
sprintf("%s%s", PREFIX, .) -> targets
pb <- dplyr::progress_estimated(length(targets))
purrr::map_df(targets, function(x) {
pb$tick()$print()
suppressMessages(readr::read_csv(URLencode(x), col_types="ciidii"))
}) -> influenza_mortality_by_state
# scrape (ugh) web page to get data file links for regional mortality data
xml2::xml_find_all(pg, ".//select[@id='Regional Data']/
option[contains(@value, 'csv') and
not(contains(@value, 'Week_'))]") %>%
xml2::xml_attr("value") %>%
sprintf("%s%s", PREFIX, .) -> targets
pb <- dplyr::progress_estimated(length(targets))
purrr::map_df(targets, function(x) {
pb$tick()$print()
suppressMessages(read_csv(URLencode(x), col_types="ciidii"))
}) -> influenza_mortality_by_region
# scrape (ugh) web page to get data file links for weekly mortality data
xml2::xml_find_all(pg, ".//select[@id='Regional Data']/
option[contains(@value, 'csv') and
contains(@value, 'Week_')]") %>%
xml2::xml_attr("value") %>%
sprintf("%s%s", PREFIX, .) -> targets
pb <- dplyr::progress_estimated(length(targets))
purrr::map_df(targets, function(x) {
pb$tick()$print()
suppressMessages(read_csv(URLencode(x), col_types="ciidii"))
}) -> influenza_mortality_by_week
# if return it all
list(
by_state = influenza_mortality_by_state,
by_region = influenza_mortality_by_region,
by_week = influenza_mortality_by_week
) -> out
class(out) <- c("cfv_mortality", class(out))
out
}

16
cran-comments.md

@ -1,10 +1,9 @@
## Test environments
* local OS X install, R 3.4.2
* local Ubuntu 16.04 R 3.4.1
* local OS X install, R 3.5.1
* local Ubuntu 16.04 R 3.5.1
* ubuntu 14.04 (on travis-ci), R old/current/devel
* win-builder (devel and release)
* rhub Windows (devel and release)
## R CMD check results
@ -18,14 +17,9 @@ None
---
The CDC changed most of their API endpoints to support a new HTML interface and
re-jiggered the back-end API. Craig McGowan updated the old cdcfluview API function
to account for the changes. However, the new API endpoints provided additional
data features and it seemed to make sense to revamp the package to fit more in line
with the way the APIs were structured. Legacy cdcfluview functions have been deprecated
and will display deprecation messages when run. The new cdcfluview package API
changes a few things about how you work with the data but the README and examples
show how to work with it.
The CDC changed their endpoints again and remove support for one of them. The
function had been deprecated in the previous version submitted to CRAN and
had been removed in this version due to the removal from the CDC.
Only some examples run on CRAN due to their time consuming nature and the need
to make external calls. Weekly tests are performed on Travis-CI

41
man/get_mortality_surveillance_data.Rd

@ -1,41 +0,0 @@
% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/get-mortality-surveillance-data.r
\name{get_mortality_surveillance_data}
\alias{get_mortality_surveillance_data}
\title{Mortality Surveillance Data from the National Center for Health Statistics (deprecated)}
\usage{
get_mortality_surveillance_data()
}
\value{
a list of \code{tbl_df}s
}
\description{
The National Center for Health Statistics (NCHS) collects and disseminates the Nation's
official vital statistics. These statistics are based on data provided to NCHS through
contracts with the vital registration systems operated in the various jurisdictions
legally responsible for the registration of deaths (i.e., death certificates) and other
vital events. These data have previously only been released as annual final data files
12 months or more after the end of the data year. Recent NCHS efforts to improve the
timeliness of jurisdiction reporting and modernize the national vital statistics
infrastructure have created a system capable of supporting near real-time surveillance.
Capitalizing on these new capabilities, NCHS and CDC’s Influenza Division have
partnered to pilot the use of NCHS mortality surveillance data for Pneumonia and
Influenza (P&I) mortality surveillance.
}
\details{
NCHS mortality surveillance data are presented by the week the death occurred.
Nationally P&I percentages are released two weeks after the week of death to allow for
collection of enough data to produce a stable P&I percentage at the national level.
Collection of complete data is not expected, and reliable P&I ratios are not expected
at the region and state level within this two week period. State and Region level
counts will be released only after 20% of the expected number of deaths are reported
through the system.
}
\examples{
\dontrun{
get_mortality_surveillance_data()
}
}
\references{
\url{https://www.cdc.gov/flu/weekly/nchs.htm}
}

8
tests/testthat/test-cdcfluview.R

@ -75,11 +75,3 @@ test_that("Old API works", {
invisible(get_weekly_flu_report())
})
test_that("these are potentially time-consuming calls", {
skip_on_cran()
invisible(get_mortality_surveillance_data())
})
Loading…
Cancel
Save