Access and Query Amazon Athena via DBI/JDBC
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

99 lines
2.6 KiB

7 years ago
---
output: rmarkdown::github_document
---
6 years ago
![](https://upload.wikimedia.org/wikipedia/commons/thumb/5/53/Winged_goddess_Louvre_F32.jpg/300px-Winged_goddess_Louvre_F32.jpg)
# `metis`
7 years ago
6 years ago
Helpers for Accessing and Querying Amazon Athena
7 years ago
Including a lightweight RJDBC shim.
6 years ago
In Greek mythology, Metis was Athena's "helper".
7 years ago
6 years ago
## Description
Still fairly beta-quality level but getting there.
7 years ago
The goal will be to get around enough of the "gotchas" that are preventing raw RJDBC Athena
connections from "just working" with `dplyr` v0.6.0+ and also get around the [`fetchSize` problem](https://www.reddit.com/r/aws/comments/6aq22b/fetchsize_limit/) without having to not use `dbGetQuery()`.
7 years ago
6 years ago
The `AthenaJDBC41-1.1.0.jar` JAR file is included out of convenience but that will likely move to a separate package as this gets closer to prime time if this goes on CRAN.
7 years ago
See the **Usage** section for an example.
6 years ago
## What's Inside The Tin?
7 years ago
The following functions are implemented:
6 years ago
Easy-interface connection helper:
- `athena_connect` Make a JDBC connection to Athena
Custom JDBC Classes:
- `Athena`: AthenaJDBC (make a new Athena con obj)
7 years ago
- `AthenaConnection-class`: AthenaJDBC
- `AthenaDriver-class`: AthenaJDBC
- `AthenaResult-class`: AthenaJDBC
6 years ago
Custom JDBC Class Methods:
7 years ago
- `dbConnect-method`: AthenaJDBC
6 years ago
- `dbExistsTable-method`: AthenaJDBC
7 years ago
- `dbGetQuery-method`: AthenaJDBC
6 years ago
- `dbListFields-method`: AthenaJDBC
- `dbListTables-method`: AthenaJDBC
- `dbReadTable-method`: AthenaJDBC
7 years ago
- `dbSendQuery-method`: AthenaJDBC
6 years ago
Pulled in from other `cloudyr` pkgs:
- `read_credentials` Use Credentials from .aws/credentials File
- `use_credentials` Use Credentials from .aws/credentials File
## Installation
7 years ago
```{r eval=FALSE}
devtools::install_github("hrbrmstr/metis")
```
```{r message=FALSE, warning=FALSE, error=FALSE, include=FALSE}
options(width=120)
```
6 years ago
## Usage
7 years ago
```{r message=FALSE, warning=FALSE, error=FALSE}
library(metis)
library(tidyverse)
7 years ago
# current verison
packageVersion("metis")
```
```{r message=FALSE, warning=FALSE, error=FALSE}
6 years ago
use_credentials("default")
6 years ago
athena_connect(
default_schema = "sampledb",
s3_staging_dir = "s3://accessible-bucket",
log_path = "/tmp/athena.log",
log_level = "DEBUG"
) -> ath
6 years ago
dbListTables(ath, schema="sampledb")
dbExistsTable(ath, "elb_logs", schema="sampledb")
dbListFields(ath, "elb_logs", "sampledb")
dbGetQuery(ath, "SELECT * FROM sampledb.elb_logs LIMIT 10") %>%
type_convert() %>%
glimpse()
7 years ago
```
6 years ago
## Code of Conduct
Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). By participating in this project you agree to abide by its terms.