From e8a6172bc087e8e024b4dc3dbd2edafe9bdb70bc Mon Sep 17 00:00:00 2001 From: boB Rudis Date: Sun, 14 Oct 2018 09:35:05 -0400 Subject: [PATCH] docs --- docs/CONDUCT.html | 134 ++++ docs/LICENSE-text.html | 128 ++++ docs/authors.html | 49 +- docs/docsearch.css | 148 ++++ docs/docsearch.js | 85 +++ docs/index.html | 842 +++++++++++---------- docs/news/index.html | 146 +++- docs/pkgdown.css | 88 ++- docs/pkgdown.js | 114 ++- docs/pkgdown.yml | 5 + docs/reference/DrilDriver-class.html | 59 +- docs/reference/Drill.html | 72 +- docs/reference/DrillConnection-class.html | 86 ++- docs/reference/DrillDriver-class.html | 57 +- docs/reference/DrillResult-class.html | 79 +- .../dbDataType-DrillConnection-method.html | 59 +- .../dbDisconnect-DrillConnection-method.html | 143 ++++ docs/reference/drill_active.html | 80 +- docs/reference/drill_cancel.html | 70 +- docs/reference/drill_connection.html | 79 +- docs/reference/drill_custom_functions.html | 142 ++-- docs/reference/drill_metrics.html | 77 +- docs/reference/drill_options.html | 77 +- docs/reference/drill_profile.html | 70 +- docs/reference/drill_profiles.html | 76 +- docs/reference/drill_query.html | 87 ++- docs/reference/drill_set.html | 81 +- docs/reference/drill_settings_reset.html | 76 +- docs/reference/drill_show_files.html | 77 +- docs/reference/drill_show_schemas.html | 70 +- docs/reference/drill_stats.html | 76 +- docs/reference/drill_status.html | 74 +- docs/reference/drill_storage.html | 139 +++- docs/reference/drill_system_reset.html | 75 +- docs/reference/drill_threads.html | 74 +- docs/reference/drill_uplift.html | 54 +- docs/reference/drill_use.html | 70 +- docs/reference/drill_version.html | 76 +- docs/reference/index.html | 409 +++++----- docs/reference/sergeant-exports.html | 48 +- docs/reference/sergeant.html | 53 +- docs/reference/src_drill.html | 140 +++- docs/reference/src_tbls.html | 57 +- 43 files changed, 3594 insertions(+), 1107 deletions(-) create mode 100644 docs/CONDUCT.html create mode 100644 docs/LICENSE-text.html create mode 100644 docs/docsearch.css create mode 100644 docs/docsearch.js create mode 100644 docs/pkgdown.yml create mode 100644 docs/reference/dbDisconnect-DrillConnection-method.html diff --git a/docs/CONDUCT.html b/docs/CONDUCT.html new file mode 100644 index 0000000..b019539 --- /dev/null +++ b/docs/CONDUCT.html @@ -0,0 +1,134 @@ + + + + + + + + +Contributor Code of Conduct • sergeant + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + +
+ +
+
+
+

Contributor Code of Conduct

+
+ +
+ +

As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.

+

We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.

+

Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.

+

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.

+

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.

+

This Code of Conduct is adapted from the Contributor Covenant (http:contributor-covenant.org), version 1.0.0, available at http://contributor-covenant.org/version/1/0/0/

+
+ +
+ +
+ + +
+
+

Developed by Bob Rudis.

+
+ +
+

Site built with pkgdown.

+
+ +
+
+ + + + + + diff --git a/docs/LICENSE-text.html b/docs/LICENSE-text.html new file mode 100644 index 0000000..92d3f79 --- /dev/null +++ b/docs/LICENSE-text.html @@ -0,0 +1,128 @@ + + + + + + + + +License • sergeant + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + + +
+ +
+
+
+

License

+
+ +
YEAR: 2016
+COPYRIGHT HOLDER: Bob Rudis
+
+ +
+ +
+ + +
+
+

Developed by Bob Rudis.

+
+ +
+

Site built with pkgdown.

+
+ +
+
+ + + + + + diff --git a/docs/authors.html b/docs/authors.html index 12f1c40..ae8db06 100644 --- a/docs/authors.html +++ b/docs/authors.html @@ -18,13 +18,22 @@ + + + + + - - - + + + + + + + @@ -47,26 +56,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,19 +94,23 @@ -
-
+
+

Authors

  • -

    Bob Rudis. Author, maintainer. +

    Bob Rudis. Author, maintainer. +

    +
    • +
  • +

    Edward Visel. Contributor.

  • -

    Edward Visel. Contributor. +

    Andy Hine. Contributor.

@@ -102,11 +126,14 @@
-

Site built with pkgdown.

+

Site built with pkgdown.

+ + + diff --git a/docs/docsearch.css b/docs/docsearch.css new file mode 100644 index 0000000..e5f1fe1 --- /dev/null +++ b/docs/docsearch.css @@ -0,0 +1,148 @@ +/* Docsearch -------------------------------------------------------------- */ +/* + Source: https://github.com/algolia/docsearch/ + License: MIT +*/ + +.algolia-autocomplete { + display: block; + -webkit-box-flex: 1; + -ms-flex: 1; + flex: 1 +} + +.algolia-autocomplete .ds-dropdown-menu { + width: 100%; + min-width: none; + max-width: none; + padding: .75rem 0; + background-color: #fff; + background-clip: padding-box; + border: 1px solid rgba(0, 0, 0, .1); + box-shadow: 0 .5rem 1rem rgba(0, 0, 0, .175); +} + +@media (min-width:768px) { + .algolia-autocomplete .ds-dropdown-menu { + width: 175% + } +} + +.algolia-autocomplete .ds-dropdown-menu::before { + display: none +} + +.algolia-autocomplete .ds-dropdown-menu [class^=ds-dataset-] { + padding: 0; + background-color: rgb(255,255,255); + border: 0; + max-height: 80vh; +} + +.algolia-autocomplete .ds-dropdown-menu .ds-suggestions { + margin-top: 0 +} + +.algolia-autocomplete .algolia-docsearch-suggestion { + padding: 0; + overflow: visible +} + +.algolia-autocomplete .algolia-docsearch-suggestion--category-header { + padding: .125rem 1rem; + margin-top: 0; + font-size: 1.3em; + font-weight: 500; + color: #00008B; + border-bottom: 0 +} + +.algolia-autocomplete .algolia-docsearch-suggestion--wrapper { + float: none; + padding-top: 0 +} + +.algolia-autocomplete .algolia-docsearch-suggestion--subcategory-column { + float: none; + width: auto; + padding: 0; + text-align: left +} + +.algolia-autocomplete .algolia-docsearch-suggestion--content { + float: none; + width: auto; + padding: 0 +} + +.algolia-autocomplete .algolia-docsearch-suggestion--content::before { + display: none +} + +.algolia-autocomplete .ds-suggestion:not(:first-child) .algolia-docsearch-suggestion--category-header { + padding-top: .75rem; + margin-top: .75rem; + border-top: 1px solid rgba(0, 0, 0, .1) +} + +.algolia-autocomplete .ds-suggestion .algolia-docsearch-suggestion--subcategory-column { + display: block; + padding: .1rem 1rem; + margin-bottom: 0.1; + font-size: 1.0em; + font-weight: 400 + /* display: none */ +} + +.algolia-autocomplete .algolia-docsearch-suggestion--title { + display: block; + padding: .25rem 1rem; + margin-bottom: 0; + font-size: 0.9em; + font-weight: 400 +} + +.algolia-autocomplete .algolia-docsearch-suggestion--text { + padding: 0 1rem .5rem; + margin-top: -.25rem; + font-size: 0.8em; + font-weight: 400; + line-height: 1.25 +} + +.algolia-autocomplete .algolia-docsearch-footer { + width: 110px; + height: 20px; + z-index: 3; + margin-top: 10.66667px; + float: right; + font-size: 0; + line-height: 0; +} + +.algolia-autocomplete .algolia-docsearch-footer--logo { + background-image: url("data:image/svg+xml;utf8,"); + background-repeat: no-repeat; + background-position: 50%; + background-size: 100%; + overflow: hidden; + text-indent: -9000px; + width: 100%; + height: 100%; + display: block; + transform: translate(-8px); +} + +.algolia-autocomplete .algolia-docsearch-suggestion--highlight { + color: #FF8C00; + background: rgba(232, 189, 54, 0.1) +} + + +.algolia-autocomplete .algolia-docsearch-suggestion--text .algolia-docsearch-suggestion--highlight { + box-shadow: inset 0 -2px 0 0 rgba(105, 105, 105, .5) +} + +.algolia-autocomplete .ds-suggestion.ds-cursor .algolia-docsearch-suggestion--content { + background-color: rgba(192, 192, 192, .15) +} diff --git a/docs/docsearch.js b/docs/docsearch.js new file mode 100644 index 0000000..b35504c --- /dev/null +++ b/docs/docsearch.js @@ -0,0 +1,85 @@ +$(function() { + + // register a handler to move the focus to the search bar + // upon pressing shift + "/" (i.e. "?") + $(document).on('keydown', function(e) { + if (e.shiftKey && e.keyCode == 191) { + e.preventDefault(); + $("#search-input").focus(); + } + }); + + $(document).ready(function() { + // do keyword highlighting + /* modified from https://jsfiddle.net/julmot/bL6bb5oo/ */ + var mark = function() { + + var referrer = document.URL ; + var paramKey = "q" ; + + if (referrer.indexOf("?") !== -1) { + var qs = referrer.substr(referrer.indexOf('?') + 1); + var qs_noanchor = qs.split('#')[0]; + var qsa = qs_noanchor.split('&'); + var keyword = ""; + + for (var i = 0; i < qsa.length; i++) { + var currentParam = qsa[i].split('='); + + if (currentParam.length !== 2) { + continue; + } + + if (currentParam[0] == paramKey) { + keyword = decodeURIComponent(currentParam[1].replace(/\+/g, "%20")); + } + } + + if (keyword !== "") { + $(".contents").unmark({ + done: function() { + $(".contents").mark(keyword); + } + }); + } + } + }; + + mark(); + }); +}); + +/* Search term highlighting ------------------------------*/ + +function matchedWords(hit) { + var words = []; + + var hierarchy = hit._highlightResult.hierarchy; + // loop to fetch from lvl0, lvl1, etc. + for (var idx in hierarchy) { + words = words.concat(hierarchy[idx].matchedWords); + } + + var content = hit._highlightResult.content; + if (content) { + words = words.concat(content.matchedWords); + } + + // return unique words + var words_uniq = [...new Set(words)]; + return words_uniq; +} + +function updateHitURL(hit) { + + var words = matchedWords(hit); + var url = ""; + + if (hit.anchor) { + url = hit.url_without_anchor + '?q=' + escape(words.join(" ")) + '#' + hit.anchor; + } else { + url = hit.url + '?q=' + escape(words.join(" ")); + } + + return url; +} diff --git a/docs/index.html b/docs/index.html index 8968b4f..bf876de 100644 --- a/docs/index.html +++ b/docs/index.html @@ -8,14 +8,21 @@ Tools to Transform and Query Data with 'Apache' 'Drill' • sergeant - - + + + + -
+
-
+
-
-

-

sergeant : Tools to Transform and Query Data with ‘Apache’ ‘Drill’

-

Drill + sergeant is (IMO) a nice alternative to Spark + sparklyr if you don’t need the ML components of Spark (i.e. just need to query “big data” sources, need to interface with parquet, need to combine disparate data source types — json, csv, parquet, rdbms - for aggregation, etc). Drill also has support for spatial queries.

-

I find writing SQL queries to parquet files with Drill on a local linux or macOS workstation to be more performant than doing the data ingestion work with R (for large or disperate data sets). I also work with many tiny JSON files on a daily basis and Drill makes it much easier to do so. YMMV.

-

You can download Drill from https://drill.apache.org/download/ (use “Direct File Download”). I use /usr/local/drill as the install directory. drill-embedded is a super-easy way to get started playing with Drill on a single workstation and most of my workflows can get by using Drill this way. If there is sufficient desire for an automated downloader and a way to start the drill-embedded server from within R, please file an issue.

+
+

+💂 sergeant

+

Tools to Transform and Query Data with ‘Apache’ ‘Drill’

+
+

+NOTE

+

Version 0.7.0 splits off the JDBC interface into a separate package sergeant.caffeinated (GitLab; GitHub).

+
+
+

+Description

+

Drill + sergeant is (IMO) a streamlined alternative to Spark + sparklyr if you don’t need the ML components of Spark (i.e. just need to query “big data” sources, need to interface with parquet, need to combine disparate data source types — json, csv, parquet, rdbms - for aggregation, etc). Drill also has support for spatial queries.

+

Using Drill SQL queries that reference parquet files on a local linux or macOS workstation can often be more performant than doing the same data ingestion & wrangling work with R (especially for large or disperate data sets). Drill can often help further streaming workflows that infolve wrangling many tiny JSON files on a daily basis.

+

Drill can be obtained from https://drill.apache.org/download/ (use “Direct File Download”). Drill can also be installed via Docker. For local installs on Unix-like systems, a common/suggestion location for the Drill directory is /usr/local/drill as the install directory.

+

Drill embedded (started using the $DRILL_BASE_DIR/bin/drill-embedded script) is a super-easy way to get started playing with Drill on a single workstation and most of many workflows can “get by” using Drill this way.

There are a few convenience wrappers for various informational SQL queries (like drill_version()). Please file an PR if you add more.

-

The package has been written with retrieval of rectangular data sources in mind. If you need/want a version of drill_query() that will enable returning of non-rectangular data (which is possible with Drill) then please file an issue.

Some of the more “controlling vs data ops” REST API functions aren’t implemented. Please file a PR if you need those.

-

Finally, I run most of this locally and at home, so it’s all been coded with no authentication or encryption in mind. If you want/need support for that, please file an issue. If there is demand for this, it will change the R API a bit (I’ve already thought out what to do but have no need for it right now).

The following functions are implemented:

-

DBI

+

DBI (REST)

    -
  • As complete of an R DBI driver has been implemented using the Drill REST API, mostly to facilitate the dplyr interface. Use the RJDBC driver interface if you need more DBI functionality.
    • -
  • This also means that SQL functions unique to Drill have also been “implemented” (i.e. made accessible to the dplyr interface). If you have custom Drill SQL functions that need to be implemented please file an issue on GitHub.
    • +
  • A “just enough” feature complete R DBI driver has been implemented using the Drill REST API, mostly to facilitate the dplyr interface. Use the RJDBC driver interface if you need more DBI functionality.
    • +
  • This also means that SQL functions unique to Drill have also been “implemented” (i.e. made accessible to the dplyr interface). If you have custom Drill SQL functions that need to be implemented please file an issue on GitHub. Many should work without it, but some may require a custom interface.
-

RJDBC

+

dplyr: (REST)

  • -drill_jdbc: Connect to Drill using JDBC, enabling use of said idioms. See RJDBC for more info.
    • -
  • NOTE: The DRILL JDBC driver fully-qualified path must be placed in the DRILL_JDBC_JAR environment variable. This is best done via ~/.Renviron for interactive work. i.e. DRILL_JDBC_JAR=/usr/local/drill/jars/drill-jdbc-all-1.9.0.jar -
    • -
-

dplyr:

-
    -
  • -src_drill: Connect to Drill (using dplyr) + supporting functions
    • +src_drill: Connect to Drill (using dplyr) + supporting functions
-

See dplyr for the dplyr operations (light testing shows they work in basic SQL use-cases but Drill’s SQL engine has issues with more complex queries).

+

Note that a number of Drill SQL functions have been mapped to R functions (e.g. grepl) to make it easier to transition from non-database-backed SQL ops to Drill. See the help on drill_custom_functions for more info on these helper Drill custom function mappings.

Drill APIs:

  • @@ -133,425 +151,427 @@
  • drill_version: Identify the version of Drill running
-
-

-Installation

-
devtools::install_github("hrbrmstr/sergeant")
-
+
+

+Installation

+
devtools::install_github("hrbrmstr/sergeant")
+
+
+

+Usage

+

-Experimental dplyr interface

-
library(sergeant)
-
-ds <- src_drill("localhost")  # use localhost if running standalone on same system otherwise the host or IP of your Drill server
-ds
-#> src:  DrillConnection
-#> tbls: INFORMATION_SCHEMA, cp.default, dfs.default, dfs.root, dfs.tmp, sys
-
-db <- tbl(ds, "cp.`employee.json`") 
-
-# without `collect()`:
-count(db, gender, marital_status)
-#> # Source:   lazy query [?? x 3]
-#> # Database: DrillConnection
-#> # Groups:   gender
-#>   marital_status gender     n
-#>            <chr>  <chr> <int>
-#> 1              S      F   297
-#> 2              M      M   278
-#> 3              S      M   276
-#> 4              M      F   304
-
-# ^^ gets translated to:
-# 
-# SELECT *
-# FROM (SELECT  gender ,  marital_status , COUNT(*) AS  n 
-#       FROM  cp.`employee.json` 
-#       GROUP BY  gender ,  marital_status )  govketbhqb 
-# LIMIT 1000
-
-count(db, gender, marital_status) %>% collect()
-#> # A tibble: 4 x 3
-#> # Groups:   gender [2]
-#>   marital_status gender     n
-#> *          <chr>  <chr> <int>
-#> 1              S      F   297
-#> 2              M      M   278
-#> 3              S      M   276
-#> 4              M      F   304
-
-# ^^ gets translated to:
-# 
-# SELECT  gender ,  marital_status , COUNT(*) AS  n 
-# FROM  cp.`employee.json` 
-# GROUP BY  gender ,  marital_status 
-
-group_by(db, position_title) %>% 
-  count(gender) -> tmp2
-
-group_by(db, position_title) %>% 
-  count(gender) %>% 
-  ungroup() %>% 
-  mutate(full_desc=ifelse(gender=="F", "Female", "Male")) %>% 
-  collect() %>% 
-  select(Title=position_title, Gender=full_desc, Count=n)
-#> # A tibble: 30 x 3
-#>                     Title Gender Count
-#>  *                  <chr>  <chr> <int>
-#>  1              President Female     1
-#>  2     VP Country Manager   Male     3
-#>  3     VP Country Manager Female     3
-#>  4 VP Information Systems Female     1
-#>  5     VP Human Resources Female     1
-#>  6          Store Manager Female    13
-#>  7             VP Finance   Male     1
-#>  8          Store Manager   Male    11
-#>  9           HQ Marketing Female     2
-#> 10 HQ Information Systems Female     4
-#> # ... with 20 more rows
-
-# ^^ gets translated to:
-# 
-# SELECT  position_title ,  gender ,  n ,
-#         CASE WHEN ( gender  = 'F') THEN ('Female') ELSE ('Male') END AS  full_desc 
-# FROM (SELECT  position_title ,  gender , COUNT(*) AS  n 
-#       FROM  cp.`employee.json` 
-#       GROUP BY  position_title ,  gender )  dcyuypuypb 
-
-arrange(db, desc(employee_id)) %>% print(n=20)
-#> # Source:     table<cp.`employee.json`> [?? x 16]
-#> # Database:   DrillConnection
-#> # Ordered by: desc(employee_id)
-#>    store_id gender department_id birth_date supervisor_id  last_name          position_title  hire_date
-#>       <int>  <chr>         <int>     <date>         <int>      <chr>                   <chr>     <dttm>
-#>  1       18      F            18 1914-02-02          1140      Stand Store Temporary Stocker 1998-01-01
-#>  2       18      M            18 1914-02-02          1140    Burnham Store Temporary Stocker 1998-01-01
-#>  3       18      F            18 1914-02-02          1139  Doolittle Store Temporary Stocker 1998-01-01
-#>  4       18      M            18 1914-02-02          1139     Pirnie Store Temporary Stocker 1998-01-01
-#>  5       18      M            17 1914-02-02          1140     Younce Store Permanent Stocker 1998-01-01
-#>  6       18      F            17 1914-02-02          1140    Biltoft Store Permanent Stocker 1998-01-01
-#>  7       18      M            17 1914-02-02          1139   Detwiler Store Permanent Stocker 1998-01-01
-#>  8       18      F            17 1914-02-02          1139     Ciruli Store Permanent Stocker 1998-01-01
-#>  9       18      F            16 1914-02-02          1140     Bishop Store Temporary Checker 1998-01-01
-#> 10       18      F            16 1914-02-02          1140  Cutwright Store Temporary Checker 1998-01-01
-#> 11       18      F            16 1914-02-02          1139   Anderson Store Temporary Checker 1998-01-01
-#> 12       18      F            16 1914-02-02          1139  Swartwood Store Temporary Checker 1998-01-01
-#> 13       18      M            15 1914-02-02          1140 Curtsinger Store Permanent Checker 1998-01-01
-#> 14       18      F            15 1914-02-02          1140      Quick Store Permanent Checker 1998-01-01
-#> 15       18      M            15 1914-02-02          1139      Souza Store Permanent Checker 1998-01-01
-#> 16       18      M            15 1914-02-02          1139   Compagno Store Permanent Checker 1998-01-01
-#> 17       18      M            11 1961-09-24          1139  Jaramillo  Store Shift Supervisor 1998-01-01
-#> 18       18      M            11 1972-05-12            17     Belsey Store Assistant Manager 1998-01-01
-#> 19       12      M            18 1914-02-02          1069    Eichorn Store Temporary Stocker 1998-01-01
-#> 20       12      F            18 1914-02-02          1069  Geiermann Store Temporary Stocker 1998-01-01
-#> # ... with more rows, and 8 more variables: management_role <chr>, salary <dbl>, marital_status <chr>, full_name <chr>,
-#> #   employee_id <int>, education_level <chr>, first_name <chr>, position_id <int>
-
-# ^^ gets translated to:
-# 
-# SELECT *
-# FROM (SELECT *
-#       FROM  cp.`employee.json` 
-#       ORDER BY  employee_id  DESC)  lvpxoaejbc 
-# LIMIT 5
-
-mutate(db, position_title=tolower(position_title)) %>%
-  mutate(salary=as.numeric(salary)) %>% 
-  mutate(gender=ifelse(gender=="F", "Female", "Male")) %>%
-  mutate(marital_status=ifelse(marital_status=="S", "Single", "Married")) %>% 
-  group_by(supervisor_id) %>% 
-  summarise(underlings_count=n()) %>% 
-  collect()
-#> # A tibble: 112 x 2
-#>    supervisor_id underlings_count
-#>  *         <int>            <int>
-#>  1             0                1
-#>  2             1                7
-#>  3             5                9
-#>  4             4                2
-#>  5             2                3
-#>  6            20                2
-#>  7            21                4
-#>  8            22                7
-#>  9             6                4
-#> 10            36                2
-#> # ... with 102 more rows
-
-# ^^ gets translated to:
-# 
-# SELECT  supervisor_id , COUNT(*) AS  underlings_count 
-# FROM (SELECT  employee_id ,  full_name ,  first_name ,  last_name ,  position_id ,  position_title ,  store_id ,  department_id ,  birth_date ,  hire_date ,  salary ,  supervisor_id ,  education_level ,  gender ,  management_role , CASE WHEN ( marital_status  = 'S') THEN ('Single') ELSE ('Married') END AS  marital_status 
-#       FROM (SELECT  employee_id ,  full_name ,  first_name ,  last_name ,  position_id ,  position_title ,  store_id ,  department_id ,  birth_date ,  hire_date ,  salary ,  supervisor_id ,  education_level ,  marital_status ,  management_role , CASE WHEN ( gender  = 'F') THEN ('Female') ELSE ('Male') END AS  gender 
-#             FROM (SELECT  employee_id ,  full_name ,  first_name ,  last_name ,  position_id ,  position_title ,  store_id ,  department_id ,  birth_date ,  hire_date ,  supervisor_id ,  education_level ,  marital_status ,  gender ,  management_role , CAST( salary  AS DOUBLE) AS  salary 
-#                   FROM (SELECT  employee_id ,  full_name ,  first_name ,  last_name ,  position_id ,  store_id ,  department_id ,  birth_date ,  hire_date ,  salary ,  supervisor_id ,  education_level ,  marital_status ,  gender ,  management_role , LOWER( position_title ) AS  position_title 
-#                         FROM  cp.`employee.json` )  cnjsqxeick )  bnbnjrubna )  wavfmhkczv )  zaxeyyicxo 
-# GROUP BY  supervisor_id
+dplyr interface +
library(sergeant)
+library(tidyverse)
+
+# use localhost if running standalone on same system otherwise the host or IP of your Drill server
+ds <- src_drill("localhost")  #ds
+db <- tbl(ds, "cp.`employee.json`") 
+
+# without `collect()`:
+count(db, gender, marital_status)
+## # Source:   lazy query [?? x 3]
+## # Database: DrillConnection
+## # Groups:   gender
+##   marital_status gender     n
+##   <chr>          <chr>  <int>
+## 1 S              F        297
+## 2 M              M        278
+## 3 S              M        276
+## 4 M              F        304
+
+count(db, gender, marital_status) %>% collect()
+## # A tibble: 4 x 3
+## # Groups:   gender [2]
+##   marital_status gender     n
+## * <chr>          <chr>  <int>
+## 1 S              F        297
+## 2 M              M        278
+## 3 S              M        276
+## 4 M              F        304
+
+group_by(db, position_title) %>% 
+  count(gender) -> tmp2
+
+group_by(db, position_title) %>% 
+  count(gender) %>% 
+  ungroup() %>% 
+  mutate(full_desc=ifelse(gender=="F", "Female", "Male")) %>% 
+  collect() %>% 
+  select(Title=position_title, Gender=full_desc, Count=n)
+## # A tibble: 30 x 3
+##    Title                  Gender Count
+##  * <chr>                  <chr>  <int>
+##  1 President              Female     1
+##  2 VP Country Manager     Male       3
+##  3 VP Country Manager     Female     3
+##  4 VP Information Systems Female     1
+##  5 VP Human Resources     Female     1
+##  6 Store Manager          Female    13
+##  7 VP Finance             Male       1
+##  8 Store Manager          Male      11
+##  9 HQ Marketing           Female     2
+## 10 HQ Information Systems Female     4
+## # ... with 20 more rows
+
+arrange(db, desc(employee_id)) %>% print(n=20)
+## # Source:     table<cp.`employee.json`> [?? x 20]
+## # Database:   DrillConnection
+## # Ordered by: desc(employee_id)
+##    store_id gender department_id birth_date supervisor_id last_name  position_title hire_date           management_role
+##       <int> <chr>          <int> <date>             <int> <chr>      <chr>          <dttm>              <chr>          
+##  1       18 F                 18 1914-02-02          1140 Stand      Store Tempora… 1998-01-01 00:00:00 Store Temp Sta…
+##  2       18 M                 18 1914-02-02          1140 Burnham    Store Tempora… 1998-01-01 00:00:00 Store Temp Sta…
+##  3       18 F                 18 1914-02-02          1139 Doolittle  Store Tempora… 1998-01-01 00:00:00 Store Temp Sta…
+##  4       18 M                 18 1914-02-02          1139 Pirnie     Store Tempora… 1998-01-01 00:00:00 Store Temp Sta…
+##  5       18 M                 17 1914-02-02          1140 Younce     Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+##  6       18 F                 17 1914-02-02          1140 Biltoft    Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+##  7       18 M                 17 1914-02-02          1139 Detwiler   Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+##  8       18 F                 17 1914-02-02          1139 Ciruli     Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+##  9       18 F                 16 1914-02-02          1140 Bishop     Store Tempora… 1998-01-01 00:00:00 Store Full Tim…
+## 10       18 F                 16 1914-02-02          1140 Cutwright  Store Tempora… 1998-01-01 00:00:00 Store Full Tim…
+## 11       18 F                 16 1914-02-02          1139 Anderson   Store Tempora… 1998-01-01 00:00:00 Store Full Tim…
+## 12       18 F                 16 1914-02-02          1139 Swartwood  Store Tempora… 1998-01-01 00:00:00 Store Full Tim…
+## 13       18 M                 15 1914-02-02          1140 Curtsinger Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+## 14       18 F                 15 1914-02-02          1140 Quick      Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+## 15       18 M                 15 1914-02-02          1139 Souza      Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+## 16       18 M                 15 1914-02-02          1139 Compagno   Store Permane… 1998-01-01 00:00:00 Store Full Tim…
+## 17       18 M                 11 1961-09-24          1139 Jaramillo  Store Shift S… 1998-01-01 00:00:00 Store Manageme…
+## 18       18 M                 11 1972-05-12            17 Belsey     Store Assista… 1998-01-01 00:00:00 Store Manageme…
+## 19       12 M                 18 1914-02-02          1069 Eichorn    Store Tempora… 1998-01-01 00:00:00 Store Temp Sta…
+## 20       12 F                 18 1914-02-02          1069 Geiermann  Store Tempora… 1998-01-01 00:00:00 Store Temp Sta…
+## # ... with more rows, and 7 more variables: salary <dbl>, marital_status <chr>, full_name <chr>, employee_id <int>,
+## #   education_level <chr>, first_name <chr>, position_id <int>
+
+mutate(db, position_title=tolower(position_title)) %>%
+  mutate(salary=as.numeric(salary)) %>% 
+  mutate(gender=ifelse(gender=="F", "Female", "Male")) %>%
+  mutate(marital_status=ifelse(marital_status=="S", "Single", "Married")) %>% 
+  group_by(supervisor_id) %>% 
+  summarise(underlings_count=n()) %>% 
+  collect()
+## # A tibble: 112 x 2
+##    supervisor_id underlings_count
+##  *         <int>            <int>
+##  1             0                1
+##  2             1                7
+##  3             5                9
+##  4             4                2
+##  5             2                3
+##  6            20                2
+##  7            21                4
+##  8            22                7
+##  9             6                4
+## 10            36                2
+## # ... with 102 more rows
-
+

-Usage

-
library(sergeant)
-
-# current verison
-packageVersion("sergeant")
-#> [1] '0.5.0'
-
-dc <- drill_connection("localhost") 
-
-drill_active(dc)
-#> [1] TRUE
-
-drill_version(dc)
-#> [1] "1.10.0"
-
-drill_storage(dc)$name
-#> [1] "cp"    "dfs"   "hbase" "hive"  "kudu"  "mongo" "s3"
-

Working with the built-in JSON data sets:

-
drill_query(dc, "SELECT * FROM cp.`employee.json` limit 100")
-#> Parsed with column specification:
-#> cols(
-#>   store_id = col_integer(),
-#>   gender = col_character(),
-#>   department_id = col_integer(),
-#>   birth_date = col_date(format = ""),
-#>   supervisor_id = col_integer(),
-#>   last_name = col_character(),
-#>   position_title = col_character(),
-#>   hire_date = col_datetime(format = ""),
-#>   management_role = col_character(),
-#>   salary = col_double(),
-#>   marital_status = col_character(),
-#>   full_name = col_character(),
-#>   employee_id = col_integer(),
-#>   education_level = col_character(),
-#>   first_name = col_character(),
-#>   position_id = col_integer()
-#> )
-#> # A tibble: 100 x 16
-#>    store_id gender department_id birth_date supervisor_id last_name         position_title  hire_date   management_role
-#>  *    <int>  <chr>         <int>     <date>         <int>     <chr>                  <chr>     <dttm>             <chr>
-#>  1        0      F             1 1961-08-26             0    Nowmer              President 1994-12-01 Senior Management
-#>  2        0      M             1 1915-07-03             1   Whelply     VP Country Manager 1994-12-01 Senior Management
-#>  3        0      M             1 1969-06-20             1    Spence     VP Country Manager 1998-01-01 Senior Management
-#>  4        0      F             1 1951-05-10             1 Gutierrez     VP Country Manager 1998-01-01 Senior Management
-#>  5        0      F             2 1942-10-08             1   Damstra VP Information Systems 1994-12-01 Senior Management
-#>  6        0      F             3 1949-03-27             1  Kanagaki     VP Human Resources 1994-12-01 Senior Management
-#>  7        9      F            11 1922-08-10             5   Brunner          Store Manager 1998-01-01  Store Management
-#>  8       21      F            11 1979-06-23             5  Blumberg          Store Manager 1998-01-01  Store Management
-#>  9        0      M             5 1949-08-26             1     Stanz             VP Finance 1994-12-01 Senior Management
-#> 10        1      M            11 1967-06-20             5  Murraiin          Store Manager 1998-01-01  Store Management
-#> # ... with 90 more rows, and 7 more variables: salary <dbl>, marital_status <chr>, full_name <chr>, employee_id <int>,
-#> #   education_level <chr>, first_name <chr>, position_id <int>
-
-drill_query(dc, "SELECT COUNT(gender) AS gender FROM cp.`employee.json` GROUP BY gender")
-#> Parsed with column specification:
-#> cols(
-#>   gender = col_integer()
-#> )
-#> # A tibble: 2 x 1
-#>   gender
-#> *  <int>
-#> 1    601
-#> 2    554
-
-drill_options(dc)
-#> # A tibble: 113 x 4
-#>                                              name value   type    kind
-#>  *                                          <chr> <chr>  <chr>   <chr>
-#>  1                 planner.enable_hash_single_key  TRUE SYSTEM BOOLEAN
-#>  2      store.parquet.reader.pagereader.queuesize     2 SYSTEM    LONG
-#>  3             planner.enable_limit0_optimization FALSE SYSTEM BOOLEAN
-#>  4              store.json.read_numbers_as_double FALSE SYSTEM BOOLEAN
-#>  5                planner.enable_constant_folding  TRUE SYSTEM BOOLEAN
-#>  6                      store.json.extended_types FALSE SYSTEM BOOLEAN
-#>  7   planner.memory.non_blocking_operators_memory    64 SYSTEM    LONG
-#>  8                  planner.enable_multiphase_agg  TRUE SYSTEM BOOLEAN
-#>  9                  exec.query_profile.debug_mode FALSE SYSTEM BOOLEAN
-#> 10 planner.filter.max_selectivity_estimate_factor     1 SYSTEM  DOUBLE
-#> # ... with 103 more rows
-
-drill_options(dc, "json")
-#> # A tibble: 7 x 4
-#>                                                    name value   type    kind
-#>                                                   <chr> <chr>  <chr>   <chr>
-#> 1                     store.json.read_numbers_as_double FALSE SYSTEM BOOLEAN
-#> 2                             store.json.extended_types FALSE SYSTEM BOOLEAN
-#> 3                              store.json.writer.uglify FALSE SYSTEM BOOLEAN
-#> 4                store.json.reader.skip_invalid_records FALSE SYSTEM BOOLEAN
-#> 5 store.json.reader.print_skipped_invalid_record_number FALSE SYSTEM BOOLEAN
-#> 6                              store.json.all_text_mode FALSE SYSTEM BOOLEAN
-#> 7                    store.json.writer.skip_null_fields  TRUE SYSTEM BOOLEAN
+REST API +
dc <- drill_connection("localhost") 
+
+drill_active(dc)
+## [1] TRUE
+
+drill_version(dc)
+## [1] "1.13.0"
+
+drill_storage(dc)$name
+## [1] "cp"    "dfs"   "hbase" "hive"  "kudu"  "mongo" "s3"
+
+drill_query(dc, "SELECT * FROM cp.`employee.json` limit 100")
+## Parsed with column specification:
+## cols(
+##   store_id = col_integer(),
+##   gender = col_character(),
+##   department_id = col_integer(),
+##   birth_date = col_date(format = ""),
+##   supervisor_id = col_integer(),
+##   last_name = col_character(),
+##   position_title = col_character(),
+##   hire_date = col_datetime(format = ""),
+##   management_role = col_character(),
+##   salary = col_double(),
+##   marital_status = col_character(),
+##   full_name = col_character(),
+##   employee_id = col_integer(),
+##   education_level = col_character(),
+##   first_name = col_character(),
+##   position_id = col_integer()
+## )
+## # A tibble: 100 x 16
+##    store_id gender department_id birth_date supervisor_id last_name position_title  hire_date           management_role
+##  *    <int> <chr>          <int> <date>             <int> <chr>     <chr>           <dttm>              <chr>          
+##  1        0 F                  1 1961-08-26             0 Nowmer    President       1994-12-01 00:00:00 Senior Managem…
+##  2        0 M                  1 1915-07-03             1 Whelply   VP Country Man… 1994-12-01 00:00:00 Senior Managem…
+##  3        0 M                  1 1969-06-20             1 Spence    VP Country Man… 1998-01-01 00:00:00 Senior Managem…
+##  4        0 F                  1 1951-05-10             1 Gutierrez VP Country Man… 1998-01-01 00:00:00 Senior Managem…
+##  5        0 F                  2 1942-10-08             1 Damstra   VP Information… 1994-12-01 00:00:00 Senior Managem…
+##  6        0 F                  3 1949-03-27             1 Kanagaki  VP Human Resou… 1994-12-01 00:00:00 Senior Managem…
+##  7        9 F                 11 1922-08-10             5 Brunner   Store Manager   1998-01-01 00:00:00 Store Manageme…
+##  8       21 F                 11 1979-06-23             5 Blumberg  Store Manager   1998-01-01 00:00:00 Store Manageme…
+##  9        0 M                  5 1949-08-26             1 Stanz     VP Finance      1994-12-01 00:00:00 Senior Managem…
+## 10        1 M                 11 1967-06-20             5 Murraiin  Store Manager   1998-01-01 00:00:00 Store Manageme…
+## # ... with 90 more rows, and 7 more variables: salary <dbl>, marital_status <chr>, full_name <chr>, employee_id <int>,
+## #   education_level <chr>, first_name <chr>, position_id <int>
+
+drill_query(dc, "SELECT COUNT(gender) AS gender FROM cp.`employee.json` GROUP BY gender")
+## Parsed with column specification:
+## cols(
+##   gender = col_integer()
+## )
+## # A tibble: 2 x 1
+##   gender
+## *  <int>
+## 1    601
+## 2    554
+
+drill_options(dc)
+## # A tibble: 138 x 5
+##    name                                              value    accessibleScopes kind    optionScope
+##  * <chr>                                             <chr>    <chr>            <chr>   <chr>      
+##  1 debug.validate_iterators                          FALSE    ALL              BOOLEAN BOOT       
+##  2 debug.validate_vectors                            FALSE    ALL              BOOLEAN BOOT       
+##  3 drill.exec.functions.cast_empty_string_to_null    FALSE    ALL              BOOLEAN BOOT       
+##  4 drill.exec.hashagg.fallback.enabled               FALSE    ALL              BOOLEAN BOOT       
+##  5 drill.exec.memory.operator.output_batch_size      16777216 SYSTEM           LONG    BOOT       
+##  6 drill.exec.storage.file.partition.column.label    dir      ALL              STRING  BOOT       
+##  7 drill.exec.storage.implicit.filename.column.label filename ALL              STRING  BOOT       
+##  8 drill.exec.storage.implicit.filepath.column.label filepath ALL              STRING  BOOT       
+##  9 drill.exec.storage.implicit.fqn.column.label      fqn      ALL              STRING  BOOT       
+## 10 drill.exec.storage.implicit.suffix.column.label   suffix   ALL              STRING  BOOT       
+## # ... with 128 more rows
+
+drill_options(dc, "json")
+## # A tibble: 9 x 5
+##   name                                                  value accessibleScopes kind    optionScope
+##   <chr>                                                 <chr> <chr>            <chr>   <chr>      
+## 1 store.json.all_text_mode                              FALSE ALL              BOOLEAN BOOT       
+## 2 store.json.extended_types                             FALSE ALL              BOOLEAN BOOT       
+## 3 store.json.read_numbers_as_double                     FALSE ALL              BOOLEAN BOOT       
+## 4 store.json.reader.allow_nan_inf                       TRUE  ALL              BOOLEAN BOOT       
+## 5 store.json.reader.print_skipped_invalid_record_number FALSE ALL              BOOLEAN BOOT       
+## 6 store.json.reader.skip_invalid_records                FALSE ALL              BOOLEAN BOOT       
+## 7 store.json.writer.allow_nan_inf                       TRUE  ALL              BOOLEAN BOOT       
+## 8 store.json.writer.skip_null_fields                    TRUE  ALL              BOOLEAN BOOT       
+## 9 store.json.writer.uglify                              FALSE ALL              BOOLEAN BOOT
+

Working with parquet files

-
drill_query(dc, "SELECT * FROM dfs.`/usr/local/drill/sample-data/nation.parquet` LIMIT 5")
-#> Parsed with column specification:
-#> cols(
-#>   N_COMMENT = col_character(),
-#>   N_NAME = col_character(),
-#>   N_NATIONKEY = col_integer(),
-#>   N_REGIONKEY = col_integer()
-#> )
-#> # A tibble: 5 x 4
-#>              N_COMMENT    N_NAME N_NATIONKEY N_REGIONKEY
-#> *                <chr>     <chr>       <int>       <int>
-#> 1  haggle. carefully f   ALGERIA           0           0
-#> 2 al foxes promise sly ARGENTINA           1           1
-#> 3 y alongside of the p    BRAZIL           2           1
-#> 4 eas hang ironic, sil    CANADA           3           1
-#> 5 y above the carefull     EGYPT           4           4
+

Including multiple parquet files in different directories (note the wildcard support):

-
drill_query(dc, "SELECT * FROM dfs.`/usr/local/drill/sample-data/nations*/nations*.parquet` LIMIT 5")
-#> Parsed with column specification:
-#> cols(
-#>   N_COMMENT = col_character(),
-#>   N_NAME = col_character(),
-#>   N_NATIONKEY = col_integer(),
-#>   N_REGIONKEY = col_integer(),
-#>   dir0 = col_character()
-#> )
-#> # A tibble: 5 x 5
-#>              N_COMMENT    N_NAME N_NATIONKEY N_REGIONKEY      dir0
-#> *                <chr>     <chr>       <int>       <int>     <chr>
-#> 1  haggle. carefully f   ALGERIA           0           0 nationsMF
-#> 2 al foxes promise sly ARGENTINA           1           1 nationsMF
-#> 3 y alongside of the p    BRAZIL           2           1 nationsMF
-#> 4 eas hang ironic, sil    CANADA           3           1 nationsMF
-#> 5 y above the carefull     EGYPT           4           4 nationsMF
-
+ +

-A preview of the built-in support for spatial ops

+Drill has built-in support for spatial ops

Via: https://github.com/k255/drill-gis

A common use case is to select data within boundary of given polygon:

-
drill_query(dc, "
-select columns[2] as city, columns[4] as lon, columns[3] as lat
-    from cp.`sample-data/CA-cities.csv`
-    where
-        ST_Within(
-            ST_Point(columns[4], columns[3]),
-            ST_GeomFromText(
-                'POLYGON((-121.95 37.28, -121.94 37.35, -121.84 37.35, -121.84 37.28, -121.95 37.28))'
-                )
-            )
-")
-#> Parsed with column specification:
-#> cols(
-#>   city = col_character(),
-#>   lon = col_double(),
-#>   lat = col_double()
-#> )
-#> # A tibble: 7 x 3
-#>          city       lon      lat
-#> *       <chr>     <dbl>    <dbl>
-#> 1     Burbank -121.9316 37.32328
-#> 2    San Jose -121.8950 37.33939
-#> 3        Lick -121.8458 37.28716
-#> 4 Willow Glen -121.8897 37.30855
-#> 5 Buena Vista -121.9166 37.32133
-#> 6    Parkmoor -121.9308 37.32105
-#> 7   Fruitdale -121.9327 37.31086
-
-
-

-JDBC

-
library(RJDBC)
-#> Loading required package: rJava
-
-# Use this if connecting to a cluster with zookeeper
-# con <- drill_jdbc("drill-node:2181", "drillbits1") 
-
-# Use the following if running drill-embedded
-con <- drill_jdbc("localhost:31010", use_zk=FALSE)
-#> Using [jdbc:drill:drillbit=localhost:31010]...
-
-drill_query(con, "SELECT * FROM cp.`employee.json`")
-#> # A tibble: 1,155 x 16
-#>    employee_id         full_name first_name last_name position_id         position_title store_id department_id
-#>  *       <dbl>             <chr>      <chr>     <chr>       <dbl>                  <chr>    <dbl>         <dbl>
-#>  1           1      Sheri Nowmer      Sheri    Nowmer           1              President        0             1
-#>  2           2   Derrick Whelply    Derrick   Whelply           2     VP Country Manager        0             1
-#>  3           4    Michael Spence    Michael    Spence           2     VP Country Manager        0             1
-#>  4           5    Maya Gutierrez       Maya Gutierrez           2     VP Country Manager        0             1
-#>  5           6   Roberta Damstra    Roberta   Damstra           3 VP Information Systems        0             2
-#>  6           7  Rebecca Kanagaki    Rebecca  Kanagaki           4     VP Human Resources        0             3
-#>  7           8       Kim Brunner        Kim   Brunner          11          Store Manager        9            11
-#>  8           9   Brenda Blumberg     Brenda  Blumberg          11          Store Manager       21            11
-#>  9          10      Darren Stanz     Darren     Stanz           5             VP Finance        0             5
-#> 10          11 Jonathan Murraiin   Jonathan  Murraiin          11          Store Manager        1            11
-#> # ... with 1,145 more rows, and 8 more variables: birth_date <chr>, hire_date <chr>, salary <dbl>, supervisor_id <dbl>,
-#> #   education_level <chr>, marital_status <chr>, gender <chr>, management_role <chr>
-
-# but it can work via JDBC function calls, too
-dbGetQuery(con, "SELECT * FROM cp.`employee.json`") %>% 
-  tibble::as_tibble()
-#> # A tibble: 1,155 x 16
-#>    employee_id         full_name first_name last_name position_id         position_title store_id department_id
-#>  *       <dbl>             <chr>      <chr>     <chr>       <dbl>                  <chr>    <dbl>         <dbl>
-#>  1           1      Sheri Nowmer      Sheri    Nowmer           1              President        0             1
-#>  2           2   Derrick Whelply    Derrick   Whelply           2     VP Country Manager        0             1
-#>  3           4    Michael Spence    Michael    Spence           2     VP Country Manager        0             1
-#>  4           5    Maya Gutierrez       Maya Gutierrez           2     VP Country Manager        0             1
-#>  5           6   Roberta Damstra    Roberta   Damstra           3 VP Information Systems        0             2
-#>  6           7  Rebecca Kanagaki    Rebecca  Kanagaki           4     VP Human Resources        0             3
-#>  7           8       Kim Brunner        Kim   Brunner          11          Store Manager        9            11
-#>  8           9   Brenda Blumberg     Brenda  Blumberg          11          Store Manager       21            11
-#>  9          10      Darren Stanz     Darren     Stanz           5             VP Finance        0             5
-#> 10          11 Jonathan Murraiin   Jonathan  Murraiin          11          Store Manager        1            11
-#> # ... with 1,145 more rows, and 8 more variables: birth_date <chr>, hire_date <chr>, salary <dbl>, supervisor_id <dbl>,
-#> #   education_level <chr>, marital_status <chr>, gender <chr>, management_role <chr>
+

Test Results

-
library(sergeant)
-library(testthat)
-#> 
-#> Attaching package: 'testthat'
-#> The following object is masked from 'package:dplyr':
-#> 
-#>     matches
-
-date()
-#> [1] "Sat Jun 17 20:47:11 2017"
-
-devtools::test()
-#> Loading sergeant
-#> Testing sergeant
-#> basic functionality: ..
-#> 
-#> DONE ===================================================================================================================
+
-
-

-Code of Conduct

-

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

+
+

+sergeant Metrics

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Lang# Files(%)LoC(%)Blank lines(%)# Lines(%)
R120.926250.921730.755620.87
Rmd10.08550.08580.25860.13
+
+
+

+Code of Conduct

+

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

@@ -561,11 +581,13 @@ devtools::test()
-

Site built with pkgdown.

+

Site built with pkgdown.

+ + diff --git a/docs/news/index.html b/docs/news/index.html index 6f40bd3..8fc118d 100644 --- a/docs/news/index.html +++ b/docs/news/index.html @@ -6,7 +6,7 @@ -All news • sergeant +Changelog • sergeant @@ -18,13 +18,22 @@ + + + + + - - - + + + + + + + @@ -47,26 +56,37 @@ - sergeant + + sergeant + 0.7.0 +
+
@@ -74,48 +94,81 @@ -
- -
+
+
-

Change log All releases

+

Changelog

+ Source: NEWS.md
-
-
-

-sergeant 0.5.0

+
+

+sergeant 0.7.0-BETA Unreleased +

    -
  • make rJava & RJDBC optional (WIP)
    • +
  • separated out Java-based functions into a sibling package
    • +
+
+
+

+sergeant 0.7.0-ALPHA Unreleased +

+
    +
  • enhanced JDBC connectivity
    • +
  • updated REST API to handle new features in Drill 1.14.0
    • +
+
+
+

+sergeant 0.6.0 Unreleased +

+ +
+
+

+sergeant 0.5.2 2017-07-17 +

+
    +
  • Make rJava & RJDBC optional (WIP)
    • +
  • Hack to remove “;” at end of queries sent to drill_query() +
    • +
  • Added dbplyr windows functions to sql_translate_env +
-

-sergeant 0.4.0

+

+sergeant 0.4.0 Unreleased +

  • Getting ready for new dplyr (thx to Edward Visel)
  • Cleaned up roxygen docs so that src_drill is exported now.
-

-sergeant 0.3.2

+

+sergeant 0.3.2 Unreleased +

  • Finally got quoting done. I thought I had before but I guess I hadn’t.
  • Added documnentation for built-in and custom Drill function that are supported.
-

-sergeant 0.3.1.9000

+

+sergeant 0.3.1.9000 Unreleased +

  • fixed src_drill() example
    • -
  • JDBC driver still in github repo but no longer included in pkg builds. See README.md or drill_jdbc() help for more information on using the JDBC driver with sergeant.
    • +
  • JDBC driver still in github repo but no longer included in pkg builds. See README.md or drill_jdbc() help for more information on using the JDBC driver with sergeant.
-

-sergeant 0.3.0.9000

+

+sergeant 0.3.0.9000 Unreleased +

  • New DBI interface (to the REST API)
  • dplyr interface now uses the DBI interace to the REST API
    • @@ -123,61 +176,69 @@
-

-sergeant 0.2.1.9000

+

+sergeant 0.2.1.9000 Unreleased +

-

-sergeant 0.2.0.9000

+

+sergeant 0.2.0.9000 Unreleased +

  • experimental alpha dplyr driver
-

-sergeant 0.1.2.9000

+

+sergeant 0.1.2.9000 Unreleased +

    -
  • can pass RJDBC connections made with drill_jdbc() to drill_query() +
  • can pass RJDBC connections made with drill_jdbc() to drill_query()
  • finally enaled nodes parameter to be a multi-element character vector as it said in the function description
-

-sergeant 0.1.2.9000

+

+sergeant 0.1.2.9000 Unreleased +

  • support embedded drill JDBC connection
-

-sergeant 0.1.1.9000

+

+sergeant 0.1.1.9000 Unreleased +

-

-sergeant 0.1.0.9000

+

+sergeant 0.1.0.9000 Unreleased +

  • Added JDBC connector and included JDBC driver in the package (for now)
  • Changed idiom to piping in a connection object
  • Added a NEWS.md file to track changes to the package.
-
+ + + diff --git a/docs/pkgdown.css b/docs/pkgdown.css index bcc0bd7..6ca2f37 100644 --- a/docs/pkgdown.css +++ b/docs/pkgdown.css @@ -1,13 +1,32 @@ -/* Sticker footer */ +/* Sticky footer */ + +/** + * Basic idea: https://philipwalton.github.io/solved-by-flexbox/demos/sticky-footer/ + * Details: https://github.com/philipwalton/solved-by-flexbox/blob/master/assets/css/components/site.css + * + * .Site -> body > .container + * .Site-content -> body > .container .row + * .footer -> footer + * + * Key idea seems to be to ensure that .container and __all its parents__ + * have height set to 100% + * + */ + +html, body { + height: 100%; +} + body > .container { display: flex; - padding-top: 60px; - min-height: calc(100vh); + height: 100%; flex-direction: column; + + padding-top: 60px; } body > .container .row { - flex: 1; + flex: 1 0 auto; } footer { @@ -16,6 +35,7 @@ footer { border-top: 1px solid #e5e5e5; color: #666; display: flex; + flex-shrink: 0; } footer p { margin-bottom: 0; @@ -38,6 +58,12 @@ img { max-width: 100%; } +/* Typographic tweaking ---------------------------------*/ + +.contents h1.page-header { + margin-top: calc(-60px + 1em); +} + /* Section anchors ---------------------------------*/ a.anchor { @@ -68,7 +94,7 @@ a.anchor { .contents h1, .contents h2, .contents h3, .contents h4 { padding-top: 60px; - margin-top: -60px; + margin-top: -40px; } /* Static header placement on mobile devices */ @@ -100,10 +126,14 @@ a.anchor { margin-bottom: 0.5em; } +.orcid { + height: 16px; + vertical-align: middle; +} + /* Reference index & topics ----------------------------------------------- */ .ref-index th {font-weight: normal;} -.ref-index h2 {font-size: 20px;} .ref-index td {vertical-align: top;} .ref-index .alias {width: 40%;} @@ -137,15 +167,30 @@ pre, code { color: #333; } -pre img { +pre code { + overflow: auto; + word-wrap: normal; + white-space: pre; +} + +pre .img { + margin: 5px 0; +} + +pre .img img { background-color: #fff; display: block; + height: auto; } code a, pre a { color: #375f84; } +a.sourceLine:hover { + text-decoration: none; +} + .fl {color: #1514b5;} .fu {color: #000000;} /* function */ .ch,.st {color: #036a07;} /* string */ @@ -156,3 +201,32 @@ code a, pre a { .error { color: orange; font-weight: bolder;} .warning { color: #6A0366; font-weight: bolder;} +/* Clipboard --------------------------*/ + +.hasCopyButton { + position: relative; +} + +.btn-copy-ex { + position: absolute; + right: 0; + top: 0; + visibility: hidden; +} + +.hasCopyButton:hover button.btn-copy-ex { + visibility: visible; +} + +/* mark.js ----------------------------*/ + +mark { + background-color: rgba(255, 255, 51, 0.5); + border-bottom: 2px solid rgba(255, 153, 51, 0.3); + padding: 1px; +} + +/* vertical spacing after htmlwidgets */ +.html-widget { + margin-bottom: 10px; +} diff --git a/docs/pkgdown.js b/docs/pkgdown.js index c8b38c4..de9bd72 100644 --- a/docs/pkgdown.js +++ b/docs/pkgdown.js @@ -1,8 +1,110 @@ -$(function() { - $("#sidebar").stick_in_parent({offset_top: 40}); - $('body').scrollspy({ - target: '#sidebar', - offset: 60 +/* http://gregfranko.com/blog/jquery-best-practices/ */ +(function($) { + $(function() { + + $("#sidebar") + .stick_in_parent({offset_top: 40}) + .on('sticky_kit:bottom', function(e) { + $(this).parent().css('position', 'static'); + }) + .on('sticky_kit:unbottom', function(e) { + $(this).parent().css('position', 'relative'); + }); + + $('body').scrollspy({ + target: '#sidebar', + offset: 60 + }); + + $('[data-toggle="tooltip"]').tooltip(); + + var cur_path = paths(location.pathname); + var links = $("#navbar ul li a"); + var max_length = -1; + var pos = -1; + for (var i = 0; i < links.length; i++) { + if (links[i].getAttribute("href") === "#") + continue; + var path = paths(links[i].pathname); + + var length = prefix_length(cur_path, path); + if (length > max_length) { + max_length = length; + pos = i; + } + } + + // Add class to parent
  • , and enclosing
  • if in dropdown + if (pos >= 0) { + var menu_anchor = $(links[pos]); + menu_anchor.parent().addClass("active"); + menu_anchor.closest("li.dropdown").addClass("active"); + } }); -}); + function paths(pathname) { + var pieces = pathname.split("/"); + pieces.shift(); // always starts with / + + var end = pieces[pieces.length - 1]; + if (end === "index.html" || end === "") + pieces.pop(); + return(pieces); + } + + function prefix_length(needle, haystack) { + if (needle.length > haystack.length) + return(0); + + // Special case for length-0 haystack, since for loop won't run + if (haystack.length === 0) { + return(needle.length === 0 ? 1 : 0); + } + + for (var i = 0; i < haystack.length; i++) { + if (needle[i] != haystack[i]) + return(i); + } + + return(haystack.length); + } + + /* Clipboard --------------------------*/ + + function changeTooltipMessage(element, msg) { + var tooltipOriginalTitle=element.getAttribute('data-original-title'); + element.setAttribute('data-original-title', msg); + $(element).tooltip('show'); + element.setAttribute('data-original-title', tooltipOriginalTitle); + } + + if(Clipboard.isSupported()) { + $(document).ready(function() { + var copyButton = ""; + + $(".examples, div.sourceCode").addClass("hasCopyButton"); + + // Insert copy buttons: + $(copyButton).prependTo(".hasCopyButton"); + + // Initialize tooltips: + $('.btn-copy-ex').tooltip({container: 'body'}); + + // Initialize clipboard: + var clipboardBtnCopies = new Clipboard('[data-clipboard-copy]', { + text: function(trigger) { + return trigger.parentNode.textContent; + } + }); + + clipboardBtnCopies.on('success', function(e) { + changeTooltipMessage(e.trigger, 'Copied!'); + e.clearSelection(); + }); + + clipboardBtnCopies.on('error', function() { + changeTooltipMessage(e.trigger,'Press Ctrl+C or Command+C to copy'); + }); + }); + } +})(window.jQuery || window.$) diff --git a/docs/pkgdown.yml b/docs/pkgdown.yml new file mode 100644 index 0000000..7908647 --- /dev/null +++ b/docs/pkgdown.yml @@ -0,0 +1,5 @@ +pandoc: 2.2.1 +pkgdown: 1.1.0 +pkgdown_sha: ~ +articles: [] + diff --git a/docs/reference/DrilDriver-class.html b/docs/reference/DrilDriver-class.html index f586ca6..1803621 100644 --- a/docs/reference/DrilDriver-class.html +++ b/docs/reference/DrilDriver-class.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 +
    • +
    @@ -74,20 +97,24 @@ -
    +

    Unload driver

    + Source: R/dbi.r +
    +

    Unload driver

    + 
    # S4 method for DrillDriver
 dbUnloadDriver(drv, ...)
    -

    Arguments

    +

    Arguments

    @@ -100,13 +127,22 @@
    +

    See also

    + +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, + DrillResult-class, Drill, + dbDataType,DrillConnection-method

    +
    @@ -117,11 +153,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/Drill.html b/docs/reference/Drill.html index 8fcc00d..4f7adf8 100644 --- a/docs/reference/Drill.html +++ b/docs/reference/Drill.html @@ -18,13 +18,26 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +60,37 @@ - sergeant + + sergeant + 0.7.0 +
    +
    @@ -74,24 +98,28 @@ -
    +

    Drill

    + Source: R/dbi.r +
    +

    Drill

    Connect to Drill

    + 
    Drill()
 
 # S4 method for DrillDriver
 dbConnect(drv, host = "localhost", port = 8047L,
-  ssl = FALSE, ...)
    + ssl = FALSE, username = NULL, password = NULL, ...) -

    Arguments

    +

    Arguments

    @@ -111,18 +139,37 @@ + + + +

    use ssl?
    username, password

    credentials

    ...

    Extra optional parameters
    +

    See also

    + +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, + DrillResult-class, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, + DrillResult-class, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +
    @@ -133,11 +180,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/DrillConnection-class.html b/docs/reference/DrillConnection-class.html index 34022e6..5261936 100644 --- a/docs/reference/DrillConnection-class.html +++ b/docs/reference/DrillConnection-class.html @@ -18,13 +18,26 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +60,37 @@ - sergeant + + sergeant + 0.7.0 +
    +
    @@ -74,16 +98,20 @@ -
    +

    Drill connection class.

    + Source: R/dbi.r +
    +

    Drill connection class.

    Send a query to Drill

    + 
    # S4 method for DrillConnection
 dbSendQuery(conn, statement, ...)
@@ -93,13 +121,48 @@
 
 # S4 method for DrillConnection,character
 dbListFields(conn, name, ...)
    - + +

    Arguments

    + + + + + + + + + + + + + + +
    conn

    connection

    statement

    SQL statement

    ...

    passed on to methods

    + +

    See also

    + +

    Other Drill REST DBI API: DrillDriver-class, + DrillResult-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +

    Other Drill REST DBI API: DrillDriver-class, + DrillResult-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +

    Other Drill REST DBI API: DrillDriver-class, + DrillResult-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +
    @@ -110,11 +173,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/DrillDriver-class.html b/docs/reference/DrillDriver-class.html index 24efa11..009b557 100644 --- a/docs/reference/DrillDriver-class.html +++ b/docs/reference/DrillDriver-class.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,23 +97,36 @@ -
    +

    Driver for Drill database.

    + Source: R/dbi.r +
    +

    Driver for Drill database.

    +
    +

    See also

    + +

    Other Drill REST DBI API: DrillConnection-class, + DrillResult-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +
    @@ -101,11 +137,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/DrillResult-class.html b/docs/reference/DrillResult-class.html index 0587e4c..a08e795 100644 --- a/docs/reference/DrillResult-class.html +++ b/docs/reference/DrillResult-class.html @@ -18,13 +18,29 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +63,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,12 +101,15 @@ -
    +

    Drill results class.

    + Source: R/dbi.r +
    +

    Drill results class.

    Clear

    @@ -87,6 +117,7 @@

    Completed

    Statement

    + 
    # S4 method for DrillResult
 dbClearResult(res, ...)
@@ -103,7 +134,7 @@
 # S4 method for DrillResult
 dbGetStatement(res, ...)
    -

    Arguments

    +

    Arguments

    @@ -112,13 +143,38 @@
    +

    See also

    + +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, Drill, + dbDataType,DrillConnection-method, + dbUnloadDriver,DrillDriver-method

    +
    @@ -129,11 +185,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/dbDataType-DrillConnection-method.html b/docs/reference/dbDataType-DrillConnection-method.html index 8a76d01..9002c53 100644 --- a/docs/reference/dbDataType-DrillConnection-method.html +++ b/docs/reference/dbDataType-DrillConnection-method.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,20 +97,24 @@ -
    +

    Drill dbDataType

    + Source: R/dbi.r +
    +

    Drill dbDataType

    + 
    # S4 method for DrillConnection
 dbDataType(dbObj, obj, ...)
    -

    Arguments

    +

    Arguments

    @@ -104,13 +131,22 @@
    +

    See also

    + +

    Other Drill REST DBI API: DrillConnection-class, + DrillDriver-class, + DrillResult-class, Drill, + dbUnloadDriver,DrillDriver-method

    +
    @@ -121,11 +157,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/dbDisconnect-DrillConnection-method.html b/docs/reference/dbDisconnect-DrillConnection-method.html new file mode 100644 index 0000000..8ceac14 --- /dev/null +++ b/docs/reference/dbDisconnect-DrillConnection-method.html @@ -0,0 +1,143 @@ + + + + + + + + +Disconnect from Drill — dbDisconnect,DrillConnection-method • sergeant + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
    + + + +
    + +
    +
    +
    +

    Disconnect from Drill

    + Source: R/dbi.r + +
    + +
    + +

    Disconnect from Drill

    + +
    + + 
    # S4 method for DrillConnection
+dbDisconnect(conn, ...)
    + + +
    + +
    + +
    +
    +

    Developed by Bob Rudis.

    +
    + +
    +

    Site built with pkgdown.

    +
    + +
    +
    + + + + + + diff --git a/docs/reference/drill_active.html b/docs/reference/drill_active.html index b5056e7..3e4b527 100644 --- a/docs/reference/drill_active.html +++ b/docs/reference/drill_active.html @@ -6,7 +6,7 @@ -Test whether Drill HTTP REST API server is up — drill_active • sergeant +Test whether Drill HTTP Dill direct REST API Interface server is up — drill_active • sergeant @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,38 +97,60 @@ -
    +
    -

    Test whether Drill HTTP REST API server is up

    +

    Test whether Drill HTTP Dill direct REST API Interface server is up

    + Source: R/sergeant.r +
    +

    This is a very simple test (performs HEAD / request on the Drill server/cluster)

    + 
    drill_active(drill_con)
    -

    Arguments

    +

    Arguments

    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    +

    See also

    + +

    Other Dill direct REST API Interface: drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    +

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_active()
-})
    + 
    try({
+drill_connection() %>% drill_active()
+}, silent=TRUE)
    #> [1] FALSE
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_cancel.html b/docs/reference/drill_cancel.html index c8f21b4..9da71c9 100644 --- a/docs/reference/drill_cancel.html +++ b/docs/reference/drill_cancel.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,24 +97,28 @@ -
    +

    Cancel the query that has the given queryid

    + Source: R/sergeant.r +
    +

    Cancel the query that has the given queryid

    + 
    drill_cancel(drill_con, query_id)
    -

    Arguments

    +

    Arguments

    - + @@ -101,7 +128,23 @@

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    @@ -111,6 +154,8 @@
  • Arguments
  • References
    • + +
  • See also
    • @@ -122,11 +167,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_connection.html b/docs/reference/drill_connection.html index 6ce59ff..e665b87 100644 --- a/docs/reference/drill_connection.html +++ b/docs/reference/drill_connection.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,22 +97,26 @@ -
    +

    Setup a Drill connection

    + Source: R/sergeant.r +
    +

    Setup a Drill connection

    + 
    drill_connection(host = Sys.getenv("DRILL_HOST", "localhost"),
   port = Sys.getenv("DRILL_PORT", 8047), ssl = FALSE,
   user = Sys.getenv("DRILL_USER", ""),
   password = Sys.getenv("DRILL_PASSWORD", ""))
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    query_id
    @@ -106,12 +133,31 @@ - +
    user, password

    NOT IMPLEMENTED YET credentials for username/password auth. -(will pick up the values from DRILL_USER/DRILL_PASSWORD -env vars)

    (will pick up the values from DRILL_USER/DRILL_PASSWORD env vars)
    +

    Note

    + +

    If user/password are set this function will make a POST to the REST +interface immediately to prime the cookie-jar with the session id.

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, drill_metrics, + drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    +

    Examples

    dc <- drill_connection()
    @@ -120,7 +166,11 @@ env vars)

    Contents

    @@ -133,11 +183,14 @@ env vars)

    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_custom_functions.html b/docs/reference/drill_custom_functions.html index 3d01d43..0b23281 100644 --- a/docs/reference/drill_custom_functions.html +++ b/docs/reference/drill_custom_functions.html @@ -6,7 +6,7 @@ -Drill expressions / custom functions <code>dplyr</code> translation: — drill_custom_functions • sergeant +Drill expressions / custom functions <code>dplyr</code> translations — drill_custom_functions • sergeant @@ -18,13 +18,27 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +61,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,23 +99,44 @@ -
    +
    -

    Drill expressions / custom functions dplyr translation:

    +

    Drill expressions / custom functions dplyr translations

    + Source: R/custom.r +
    +
    + +

    One benefit of dplyr is that it provide a nice DSL over datasbase ops but that +means there needs to be knowlege of functions supported by the host database and +then a translation layer so they can be used in R.

    -
      -

    • as.character(x) : CAST( x AS CHARACTER)

      • -

    • as.date(x) : CAST( x AS DATE)

      • +
    + + +

    Details

    + +

    Similarly, there are functions like grepl() in R that don't directly exist in +databases. Yet, one can create a translation for grepl() that maps to a +Drill custom function so you +don't have to think differently or rewrite your pipes when switching from core +tidyverse ops and database ops.

    +

    Many functions translate on their own, but it's handy to provide explicit ones, +especially when you want to use parameters in a different order.

    +

    If you want a particular custom function mapped, file a PR or issue request in +the link found in the DESCRIPTION file.

      +

    • as.character(x) : CAST( x AS CHARACTER )

      • +

    • as.date(x) : CAST( x AS DATE )

    • as.logical(x) : CAST( x AS BOOLEAN)

      • -

    • as.numeric(x) : CAST( x AS DOUBLE)

      • -

    • as.posixct(x) : CAST( x AS TIMESTAMP)

      • +

    • as.numeric(x) : CAST( x AS DOUBLE )

      • +

    • as.posixct(x) : CAST( x AS TIMESTAMP )

    • binary_string(x) : BINARY_STRING( x )

    • cbrt(x) : CBRT( x )

    • char_to_timestamp(x, y) : TO_TIMESTAMP( x, y )

      • -

    • contains(x, y) : CONTAINS x, y )

      • +

    • grepl(y, x) : CONTAINS( x, y )

      • +

    • contains(x, y) : CONTAINS( x, y )

    • convert_to(x, y) : CONVERT_TO( x, y )

    • convert_from(x, y) : CONVERT_FROM( x, y )

    • degrees(x) : DEGREES( x )

      • @@ -105,37 +151,40 @@

    • to_date(x, y) : TO_DATE( x, y )

    • to_number(x, y) : TO_NUMBER( x, y )

    • trunc(x) : TRUNC( x )

      • +

    • double_to_timestamp(x) = TO_TIMESTAMP( x )

      • +

    • char_length(x) = CHAR_LENGTH( x )

      • +

    • flatten(x) = FLATTEN( x )

      • +

    • kvgen(x) = KVGEN( x )

      • +

    • repeated_count(x) = REPEATED_COUNT( x )

      • +

    • repeated_contains(x) = REPEATED_CONTAINS( x )

      • +

    • ilike(x, y) = ILIKE( x, y )

      • +

    • init_cap(x) = INIT_CAP( x )

      • +

    • length(x) = LENGTH( x )

      • +

    • lower(x) = LOWER( x )

      • +

    • tolower(x) = LOWER( x )

      • +

    • ltrim(x, y) = LTRIM( x, y )

      • +

    • nullif(x, y = NULLIF( x, y )

      • +

    • position(x, y) = POSITION( x IN y )

      • +

    • gsub(x, y, z) = REGEXP_REPLACE( z, x, y )

      • +

    • regexp_replace(x, y, z) = REGEXP_REPLACE( x, y, z )

      • +

    • rtrim(x, y) = RTRIM( x, y )

      • +

    • rpad(x, y) = RPAD( x, y )

      • +

    • rpad_with(x, y, z) = RPAD( x, y, z )

      • +

    • lpad(x, y) = LPAD( x, y )

      • +

    • lpad_with(x, y, z) = LPAD( x, y, z )

      • +

    • strpos(x, y) = STRPOS( x, y )

      • +

    • substr(x, y, z) = SUBSTR( x, y, z )

      • +

    • upper(x) = UPPER(1)

      • +

    • toupper(x) = UPPER(1)

    +

    You can get a compact list of these with:

    +

    sql_translate_env(src_drill()$con)

    +

    as well.

    +

    See also

    - -

    Details

    - -

    I'll get these converted into ^^ format:

      -

    • double_to_timestamp = sql_prefix("TO_TIMESTAMP", 1),

      • -

    • char_length = sql_prefix("CHAR_LENGTH", 1),

      • -

    • flatten = sql_prefix("FLATTEN", 1),

      • -

    • kvgen = sql_prefix("KVGEN", 1),

      • -

    • repeated_count = sql_prefix("REPEATED_COUNT", 1),

      • -

    • repeated_contains = sql_prefix("REPEATED_CONTAINS", 1),

      • -

    • ilike = sql_prefix("ILIKE", 2),

      • -

    • init_cap = sql_prefix("INIT_CAP", 1),

      • -

    • length = sql_prefix("LENGTH", 1),

      • -

    • lower = sql_prefix("LOWER", 1),

      • -

    • ltrim = sql_prefix("LTRIM", 2),

      • -

    • nullif = sql_prefix("NULLIF", 2),

      • -

    • position = function(x, y) build_sql("POSITION(", x, " IN ", y, ")"),

      • -

    • regexp_replace = sql_prefix("REGEXP_REPLACE", 3),

      • -

    • rtrim = sql_prefix("RTRIM", 2),

      • -

    • rpad = sql_prefix("RPAD", 2),

      • -

    • rpad_with = sql_prefix("RPAD", 3),

      • -

    • lpad = sql_prefix("LPAD", 2),

      • -

    • lpad_with = sql_prefix("LPAD", 3),

      • -

    • strpos = sql_prefix("STRPOS", 2),

      • -

    • substr = sql_prefix("SUBSTR", 3),

      • -

    • trim = function(x, y, z) build_sql("TRIM(", x, " ", y, " FROM ", z, ")"),

      • -

    • upper = sql_prefix("UPPER", 1)

      • -
    +

    Other Drill REST `dplyr` API: src_drill, + src_tbls.src_drill

    @@ -144,6 +193,8 @@
    @@ -155,11 +206,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_metrics.html b/docs/reference/drill_metrics.html index 48501b4..5da4ad7 100644 --- a/docs/reference/drill_metrics.html +++ b/docs/reference/drill_metrics.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,38 +97,61 @@ -
    +

    Get the current memory metrics

    + Source: R/sergeant.r +
    +

    Get the current memory metrics

    + 
    drill_metrics(drill_con)
    -

    Arguments

    +

    Arguments

    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    +

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_metrics()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_metrics()
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_options.html b/docs/reference/drill_options.html index fb968df..2b9b5ec 100644 --- a/docs/reference/drill_options.html +++ b/docs/reference/drill_options.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,24 +97,28 @@ -
    +

    List the name, default, and data type of the system and session options

    + Source: R/sergeant.r +
    +

    List the name, default, and data type of the system and session options

    + 
    drill_options(drill_con, pattern = NULL)
    -

    Arguments

    +

    Arguments

    - + @@ -101,13 +128,30 @@

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_options()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_options()
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_profile.html b/docs/reference/drill_profile.html index 645f3d9..aa23eed 100644 --- a/docs/reference/drill_profile.html +++ b/docs/reference/drill_profile.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,24 +97,28 @@ -
    +

    Get the profile of the query that has the given queryid

    + Source: R/sergeant.r +
    +

    Get the profile of the query that has the given queryid

    + 
    drill_profile(drill_con, query_id)
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    pattern
    - + @@ -101,7 +128,23 @@

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    @@ -111,6 +154,8 @@
  • Arguments
  • References
    • + +
  • See also
    • @@ -122,11 +167,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_profiles.html b/docs/reference/drill_profiles.html index 740ea28..33cce6b 100644 --- a/docs/reference/drill_profiles.html +++ b/docs/reference/drill_profiles.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,36 +97,56 @@ -
    +

    Get the profiles of running and completed queries

    + Source: R/sergeant.r +
    +

    Get the profiles of running and completed queries

    + 
    drill_profiles(drill_con)
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    query_id
    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_profiles()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_profiles()
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_query.html b/docs/reference/drill_query.html index 8e62803..0ba52a9 100644 --- a/docs/reference/drill_query.html +++ b/docs/reference/drill_query.html @@ -18,13 +18,29 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +63,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,12 +101,15 @@ -
    +

    Submit a query and return results

    + Source: R/query.r +
    +

    This function can handle REST API connections or JDBC connections. There is a benefit to calling this function for JDBC connections vs a straight call to dbGetQuery() in @@ -87,15 +117,16 @@ that the function result is a `tbl_df` vs a plain data.frame so you default printing (which can be helpful if you accidentally execute a query and the result set is huge).

    + 
    drill_query(drill_con, query, uplift = TRUE, .progress = interactive())
    -

    Arguments

    +

    Arguments

    - @@ -104,27 +135,44 @@ set is huge).

    - +httr::POST to display a progress bar

    drill_con

    drill server connection object setup by drill_connection() or +

    drill server connection object setup by drill_connection() or drill_jdbc())
    uplift

    automatically run drill_uplift() on the result? (default: TRUE, +

    automatically run drill_uplift() on the result? (default: TRUE, ignored if drill_con is a JDBCConnection created by drill_jdbc())
    .progress

    if TRUE (default if in an interactive session) then ask -httr::POST to display a progress bar

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>%
-    drill_query("SELECT * FROM cp.`employee.json` limit 5")
-})
    + 
    try({
+drill_connection() %>%
+  drill_query("SELECT * FROM cp.`employee.json` limit 5")
+}, silent=TRUE)
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_set.html b/docs/reference/drill_set.html index ff6d9fe..ec2e49d 100644 --- a/docs/reference/drill_set.html +++ b/docs/reference/drill_set.html @@ -18,13 +18,27 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +61,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,26 +99,30 @@ -
    +

    Set Drill SYSTEM or SESSION options

    + Source: R/set.R +
    +

    Helper function to make it more R-like to set Drill SESSION or SYSTEM optons. It handles the conversion of R types (like TRUE) to SQL types and automatically quotes parameter values (when necessary).

    + 
    drill_set(drill_con, ..., type = c("session", "system"))
    -

    Arguments

    +

    Arguments

    - + @@ -115,14 +144,31 @@ quotes parameter values (when necessary).

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>%
-    drill_set(exec.errors.verbose=TRUE, store.format="parquet", web.logs.max_lines=20000)
-})
    + 
    # NOT RUN {
+drill_connection() %>%
+  drill_set(exec.errors.verbose=TRUE, store.format="parquet", web.logs.max_lines=20000)
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_settings_reset.html b/docs/reference/drill_settings_reset.html index b4c7aa5..1a55076 100644 --- a/docs/reference/drill_settings_reset.html +++ b/docs/reference/drill_settings_reset.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,24 +97,28 @@ -
    +

    Changes (optionally, all) session settings back to system defaults

    + Source: R/set.R +
    +

    Changes (optionally, all) session settings back to system defaults

    + 
    drill_settings_reset(drill_con, ...)
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    ...
    - + @@ -101,13 +128,29 @@

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_settings_reset(exec.errors.verbose)
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_settings_reset(exec.errors.verbose)
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_show_files.html b/docs/reference/drill_show_files.html index fe8a75c..2d98199 100644 --- a/docs/reference/drill_show_files.html +++ b/docs/reference/drill_show_files.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,24 +97,28 @@ -
    +

    Show files in a file system schema.

    + Source: R/schemas.R +
    +

    Show files in a file system schema.

    + 
    drill_show_files(drill_con, schema_spec)
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    ...
    - + @@ -101,13 +128,30 @@

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_show_files("dfs.tmp")
-})
    + 
    try({
+drill_connection() %>% drill_show_files("dfs.tmp")
+}, silent=TRUE)
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_show_schemas.html b/docs/reference/drill_show_schemas.html index 3e25061..ff8d801 100644 --- a/docs/reference/drill_show_schemas.html +++ b/docs/reference/drill_show_schemas.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,30 +97,50 @@ -
    +

    Returns a list of available schemas.

    + Source: R/schemas.R +
    +

    Returns a list of available schemas.

    + 
    drill_show_schemas(drill_con)
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    schema_spec
    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    @@ -107,6 +150,8 @@
  • Arguments
  • References
    • + +
  • See also
    • @@ -118,11 +163,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_stats.html b/docs/reference/drill_stats.html index af99563..9daf275 100644 --- a/docs/reference/drill_stats.html +++ b/docs/reference/drill_stats.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,36 +97,56 @@ -
    +

    Get Drillbit information, such as ports numbers

    + Source: R/sergeant.r +
    +

    Get Drillbit information, such as ports numbers

    + 
    drill_stats(drill_con)
    -

    Arguments

    +

    Arguments

    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_status, drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_stats()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_stats()
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_status.html b/docs/reference/drill_status.html index 5d177de..b06d211 100644 --- a/docs/reference/drill_status.html +++ b/docs/reference/drill_status.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,24 +97,28 @@ -
    +

    Get the status of Drill

    + Source: R/sergeant.r +
    +

    Get the status of Drill

    + 
    drill_status(drill_con)
    -

    Arguments

    +

    Arguments

    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    @@ -99,11 +126,27 @@

    The output of this is in a "viewer" window

    +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_storage, + drill_system_reset, + drill_threads, drill_use, + drill_version

    +

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_status()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_status()
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_storage.html b/docs/reference/drill_storage.html index d3707f1..94ed5e3 100644 --- a/docs/reference/drill_storage.html +++ b/docs/reference/drill_storage.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,40 +97,121 @@ -
    +

    Get the list of storage plugin names and configurations

    + Source: R/sergeant.r +
    +

    Get the list of storage plugin names and configurations

    +
    + + 
    drill_storage(drill_con, plugin = NULL)
 
-    
    drill_storage(drill_con, plugin = NULL)
    
+drill_mod_storage(drill_con, name, config)
+
+drill_rm_storage(drill_con, name)
    -

    Arguments

    +

    Arguments

    - + + + + + + + + +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    plugin

    the assigned name in the storage plugin definition.
    name

    name of the storage plugin configuration to create/update/remove

    config

    a list or raw character, valid JSON of a complete storage +spec

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_system_reset, + drill_threads, drill_use, + drill_version

    +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_system_reset, + drill_threads, drill_use, + drill_version

    +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_system_reset, + drill_threads, drill_use, + drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_storage()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_storage()
+
+drill_connection() %>%
+  drill_mod_storage(
+    name = "drilldat",
+    config = '
+{
+  "config" : {
+    "connection" : "file:///",
+    "enabled" : true,
+    "formats" : null,
+    "type" : "file",
+    "workspaces" : {
+      "root" : {
+        "location" : "/Users/hrbrmstr/drilldat",
+        "writable" : true,
+        "defaultInputFormat": null
+      }
+    }
+  },
+  "name" : "drilldat"
+}
+')
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_system_reset.html b/docs/reference/drill_system_reset.html index 412344e..5a82b67 100644 --- a/docs/reference/drill_system_reset.html +++ b/docs/reference/drill_system_reset.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,24 +97,28 @@ -
    +

    Changes (optionally, all) system settings back to system defaults

    + Source: R/set.R +
    +

    Changes (optionally, all) system settings back to system defaults

    + 
    drill_system_reset(drill_con, ..., all = FALSE)
    -

    Arguments

    +

    Arguments

    - + @@ -105,13 +132,28 @@

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, drill_threads, + drill_use, drill_version

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_system_reset(all=TRUE)
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_system_reset(all=TRUE)
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_threads.html b/docs/reference/drill_threads.html index a1841b9..56c033d 100644 --- a/docs/reference/drill_threads.html +++ b/docs/reference/drill_threads.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,24 +97,28 @@ -
    +

    Get information about threads

    + Source: R/sergeant.r +
    +

    Get information about threads

    + 
    drill_threads(drill_con)
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    ...
    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    @@ -99,11 +126,27 @@

    The output of this is in a "viewer" window

    +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_use, drill_version

    +

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_threads()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_threads()
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_uplift.html b/docs/reference/drill_uplift.html index 1e814ff..37e1915 100644 --- a/docs/reference/drill_uplift.html +++ b/docs/reference/drill_uplift.html @@ -18,13 +18,27 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +61,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,21 +99,25 @@ -
    +

    Turn columnar query results into a type-converted tbl

    + Source: R/query.r +
    +

    If you know the result of `drill_query()` will be a data frame, then you can pipe it to this function to pull out `rows` and automatically type-convert it.

    + 
    drill_uplift(query_result)
    -

    Arguments

    +

    Arguments

    @@ -100,11 +129,11 @@ type-convert it.

    Details

    Not really intended to be called directly, but useful if you accidentally ran -drill_query() without `uplift=TRUE` but want to then convert the structure.

    +drill_query() without `uplift=TRUE` but want to then convert the structure.

    References

    -

    Drill documentation

    +

    Drill documentation

    @@ -127,11 +156,14 @@ type-convert it.

    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_use.html b/docs/reference/drill_use.html index 4488b68..aae3601 100644 --- a/docs/reference/drill_use.html +++ b/docs/reference/drill_use.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,24 +97,28 @@ -
    +

    Change to a particular schema.

    + Source: R/schemas.R +
    +

    Change to a particular schema.

    + 
    drill_use(drill_con, schema_name)
    -

    Arguments

    +

    Arguments

    - + @@ -102,7 +129,23 @@ storage plugin, such as hive, or a storage plugin and workspace.

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_version

    @@ -112,6 +155,8 @@ storage plugin, such as hive, or a storage plugin and workspace.

  • Arguments
  • References
    • + +
  • See also
    • @@ -123,11 +168,14 @@ storage plugin, such as hive, or a storage plugin and workspace.

    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/drill_version.html b/docs/reference/drill_version.html index 5e2264f..17de196 100644 --- a/docs/reference/drill_version.html +++ b/docs/reference/drill_version.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,36 +97,56 @@ -
    +

    Identify the version of Drill running

    + Source: R/sergeant.r +
    +

    Identify the version of Drill running

    + 
    drill_version(drill_con)
    -

    Arguments

    +

    Arguments

    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()
    schema_name
    - +
    drill_con

    drill server connection object setup by drill_connection()

    drill server connection object setup by drill_connection()

    References

    -

    Drill documentation

    +

    Drill documentation

    + +

    See also

    + +

    Other Dill direct REST API Interface: drill_active, + drill_cancel, + drill_connection, + drill_metrics, drill_options, + drill_profiles, + drill_profile, drill_query, + drill_settings_reset, + drill_set, drill_show_files, + drill_show_schemas, + drill_stats, drill_status, + drill_storage, + drill_system_reset, + drill_threads, drill_use

    Examples

    - 
    not_run({
-  drill_connection() %>% drill_version()
-})
    + 
    # NOT RUN {
+drill_connection() %>% drill_version()
+# }
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/index.html b/docs/reference/index.html index c7a4446..ce1e7d2 100644 --- a/docs/reference/index.html +++ b/docs/reference/index.html @@ -18,13 +18,22 @@ + + + + + - - - + + + + + + + @@ -47,26 +56,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,202 +94,192 @@ -
    -
    +
    +
    -

    - Reference - version 0.5.0 -

    +

    Reference

    -
    - +
    - - - - + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    -

    All functions

    -

    -
    -

    dbDataType

    -

    Drill dbDataType

    -

    dbUnloadDriver

    -

    Unload driver

    -

    drill_active

    -

    Test whether Drill HTTP REST API server is up

    -

    drill_cancel

    -

    Cancel the query that has the given queryid

    -

    drill_connection

    -

    Setup a Drill connection

    -

    drill_custom_functions

    -

    Drill expressions / custom functions dplyr translation:

    -

    drill_jdbc

    -

    Connect to Drill using JDBC

    -

    drill_metrics

    -

    Get the current memory metrics

    -

    drill_options

    -

    List the name, default, and data type of the system and session options

    -

    drill_profile

    -

    Get the profile of the query that has the given queryid

    -

    drill_profiles

    -

    Get the profiles of running and completed queries

    -

    drill_query

    -

    Submit a query and return results

    -

    drill_set

    -

    Set Drill SYSTEM or SESSION options

    -

    drill_settings_reset

    -

    Changes (optionally, all) session settings back to system defaults

    -

    drill_show_files

    -

    Show files in a file system schema.

    -

    drill_show_schemas

    -

    Returns a list of available schemas.

    -

    drill_stats

    -

    Get Drillbit information, such as ports numbers

    -

    drill_status

    -

    Get the status of Drill

    -

    drill_storage

    -

    Get the list of storage plugin names and configurations

    -

    drill_system_reset

    -

    Changes (optionally, all) system settings back to system defaults

    -

    drill_threads

    -

    Get information about threads

    -

    drill_uplift

    -

    Turn columnar query results into a type-converted tbl

    -

    drill_use

    -

    Change to a particular schema.

    -

    drill_version

    -

    Identify the version of Drill running

    -

    Drill dbConnect

    -

    Drill

    -

    sergeant-exports

    -

    sergeant exported operators

    -

    sergeant

    -

    Tools to Transform and Query Data with 'Apache' 'Drill'

    -

    src_drill tbl

    -

    Connect to Drill (dplyr)

    -
    + + + +

    All functions

    +

    + + + + + +

    dbUnloadDriver(<DrillDriver>)

    + +

    Unload driver

    + + + +

    Drill() dbConnect(<DrillDriver>)

    + +

    Drill

    + + + +

    dbDataType(<DrillConnection>)

    + +

    Drill dbDataType

    + + + +

    drill_active()

    + +

    Test whether Drill HTTP Dill direct REST API Interface server is up

    + + + +

    drill_cancel()

    + +

    Cancel the query that has the given queryid

    + + + +

    drill_connection()

    + +

    Setup a Drill connection

    + + + +

    drill_custom_functions

    + +

    Drill expressions / custom functions dplyr translations

    + + + +

    drill_metrics()

    + +

    Get the current memory metrics

    + + + +

    drill_options()

    + +

    List the name, default, and data type of the system and session options

    + + + +

    drill_profile()

    + +

    Get the profile of the query that has the given queryid

    + + + +

    drill_profiles()

    + +

    Get the profiles of running and completed queries

    + + + +

    drill_query()

    + +

    Submit a query and return results

    + + + +

    drill_set()

    + +

    Set Drill SYSTEM or SESSION options

    + + + +

    drill_settings_reset()

    + +

    Changes (optionally, all) session settings back to system defaults

    + + + +

    drill_show_files()

    + +

    Show files in a file system schema.

    + + + +

    drill_show_schemas()

    + +

    Returns a list of available schemas.

    + + + +

    drill_stats()

    + +

    Get Drillbit information, such as ports numbers

    + + + +

    drill_status()

    + +

    Get the status of Drill

    + + + +

    drill_storage() drill_mod_storage() drill_rm_storage()

    + +

    Get the list of storage plugin names and configurations

    + + + +

    drill_system_reset()

    + +

    Changes (optionally, all) system settings back to system defaults

    + + + +

    drill_threads()

    + +

    Get information about threads

    + + + +

    drill_uplift()

    + +

    Turn columnar query results into a type-converted tbl

    + + + +

    drill_use()

    + +

    Change to a particular schema.

    + + + +

    drill_version()

    + +

    Identify the version of Drill running

    + + + +

    sergeant-exports

    + +

    sergeant exported operators

    + + + +

    sergeant

    + +

    Tools to Transform and Query Data with 'Apache' 'Drill'

    + + + +

    src_drill() tbl(<src_drill>)

    + +

    Connect to Drill (dplyr)

    + + +
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/sergeant-exports.html b/docs/reference/sergeant-exports.html index d4e6602..287fd79 100644 --- a/docs/reference/sergeant-exports.html +++ b/docs/reference/sergeant-exports.html @@ -18,13 +18,27 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +61,37 @@ - sergeant + + sergeant + 0.7.0 +
    +
    @@ -74,17 +99,21 @@ -
    +

    sergeant exported operators

    + Source: R/sergeant-package.r +
    +

    The following functions are imported and then re-exported from the sergeant package to enable use of the magrittr pipe operator with no additional library calls

    +
    @@ -103,11 +132,14 @@ pipe operator with no additional library calls

    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/sergeant.html b/docs/reference/sergeant.html index 62112f9..5c24e88 100644 --- a/docs/reference/sergeant.html +++ b/docs/reference/sergeant.html @@ -18,13 +18,28 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +62,37 @@ - sergeant + + sergeant + 0.7.0 +
    + @@ -74,18 +100,22 @@ -
    +

    Tools to Transform and Query Data with 'Apache' 'Drill'

    + Source: R/sergeant-package.r +
    +

    Drill is an innovative low-latency distributed query engine designed to enable data exploration and analytics on both relational and non-relational datastores, scaling to petabytes of data. Users can query the data using standard SQL and BI tools without having to create and manage schemas. Some of the key features are:

    +

    Details

    @@ -105,12 +135,12 @@ distributed cluster environment. When a Drillbit runs on each data node in a clu Drill can maximize data locality during query execution without moving data over the network or between nodes. Drill uses ZooKeeper to maintain cluster membership and health check information.

    -

    Methods are provided to work with Drill via the native JDBC & REST APIs along with R +

    Methods are provided to work with Drill via the REST APIs along with R DBI and dplyr interfaces.

    References

    -

    Drill documentation

    +

    Drill documentation

    @@ -136,11 +166,14 @@ Bob Rudis (bob@rud.is)
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/src_drill.html b/docs/reference/src_drill.html index 2a20279..8261de7 100644 --- a/docs/reference/src_drill.html +++ b/docs/reference/src_drill.html @@ -18,13 +18,28 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +62,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,26 +100,31 @@ -
    +

    Connect to Drill (dplyr)

    + Source: R/dplyr.r +
    +
    -

    Use src_drill() to connect to a Drill cluster and `tbl()` to connect to a +

    Use src_drill() to connect to a Drill cluster and tbl() to connect to a fully-qualified "table reference". The vast majority of Drill SQL functions have also been made available to the dplyr interface. If you have custom Drill SQL functions that need to be implemented please file an issue on GitHub.

    + 
    src_drill(host = Sys.getenv("DRILL_HOST", "localhost"),
-  port = as.integer(Sys.getenv("DRILL_PORT", 8047L)), ssl = FALSE)
+  port = as.integer(Sys.getenv("DRILL_PORT", 8047L)), ssl = FALSE,
+  username = NULL, password = NULL)
 
 # S3 method for src_drill
-tbl(src, from, ...)
    +tbl(src, from, ...) -

    Arguments

    +

    Arguments

    @@ -109,6 +140,10 @@ SQL functions that need to be implemented please file an issue on GitHub.

    + + + + @@ -124,30 +159,64 @@ SQL functions that need to be implemented please file an issue on GitHub.

    Note

    -

    This is a DBI wrapper around the Drill REST API. TODO username/password support

    +

    This is a DBI wrapper around the Drill REST API.

    + +

    See also

    + +

    Other Drill REST `dplyr` API: drill_custom_functions, + src_tbls.src_drill

    +

    Other Drill REST `dplyr` API: drill_custom_functions, + src_tbls.src_drill

    Examples

    - 
    not_run({
-  db <- src_drill("localhost", "8047")
-
-  print(db)
-
-  emp <- tbl(db, "cp.`employee.json`")
-
-  count(emp, gender, marital_status)
-
-  # Drill-specific SQL functions are also available
-  select(emp, full_name) %>%
-    mutate(        loc = strpos(full_name, "a"),
-           first_three = substr(full_name, 1L, 3L),
-                   len = length(full_name),
-                    rx = regexp_replace(full_name, "[aeiouAEIOU]", "*"),
-                   rnd = rand(),
-                   pos = position("en", full_name),
-                   rpd = rpad(full_name, 20L),
-                  rpdw = rpad_with(full_name, 20L, "*"))
-})
    + 
    try({
+db <- src_drill("localhost", 8047L)
+
+print(db)
+## src:  DrillConnection
+## tbls: INFORMATION_SCHEMA, cp.default, dfs.default, dfs.root, dfs.tmp, sys
+
+emp <- tbl(db, "cp.`employee.json`")
+
+count(emp, gender, marital_status)
+## # Source:   lazy query [?? x 3]
+## # Database: DrillConnection
+## # Groups:   gender
+##   marital_status gender     n
+##            <chr>  <chr> <int>
+## 1              S      F   297
+## 2              M      M   278
+## 3              S      M   276
+
+# Drill-specific SQL functions are also available
+select(emp, full_name) %>%
+  mutate(        loc = strpos(full_name, "a"),
+         first_three = substr(full_name, 1L, 3L),
+                 len = length(full_name),
+                  rx = regexp_replace(full_name, "[aeiouAEIOU]", "*"),
+                 rnd = rand(),
+                 pos = position("en", full_name),
+                 rpd = rpad(full_name, 20L),
+                rpdw = rpad_with(full_name, 20L, "*"))
+## # Source:   lazy query [?? x 9]
+## # Database: DrillConnection
+##      loc         full_name   len                 rpdw   pos                rx
+##    <int>             <chr> <int>                <chr> <int>             <chr>
+##  1     0      Sheri Nowmer    12 Sheri Nowmer********     0      Sh*r* N*wm*r
+##  2     0   Derrick Whelply    15 Derrick Whelply*****     0   D*rr*ck Wh*lply
+##  3     5    Michael Spence    14 Michael Spence******    11    M*ch**l Sp*nc*
+##  4     2    Maya Gutierrez    14 Maya Gutierrez******     0    M*y* G*t**rr*z
+##  5     7   Roberta Damstra    15 Roberta Damstra*****     0   R*b*rt* D*mstr*
+##  6     7  Rebecca Kanagaki    16 Rebecca Kanagaki****     0  R*b*cc* K*n*g*k*
+##  7     0       Kim Brunner    11 Kim Brunner*********     0       K*m Br*nn*r
+##  8     6   Brenda Blumberg    15 Brenda Blumberg*****     3   Br*nd* Bl*mb*rg
+##  9     2      Darren Stanz    12 Darren Stanz********     5      D*rr*n St*nz
+## 10     4 Jonathan Murraiin    17 Jonathan Murraiin***     0 J*n*th*n M*rr***n
+## # ... with more rows, and 3 more variables: rpd <chr>, rnd <dbl>, first_three <chr>
+}, silent=TRUE)
    #> src:  DrillConnection
+#> tbls: cp.default, dfs.d, dfs.default, dfs.root, dfs.tmp, INFORMATION_SCHEMA,
+#>   sys
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + + diff --git a/docs/reference/src_tbls.html b/docs/reference/src_tbls.html index 640ca0b..83c0ce4 100644 --- a/docs/reference/src_tbls.html +++ b/docs/reference/src_tbls.html @@ -18,13 +18,25 @@ + + + + + - - - + + + + + + + + + + @@ -47,26 +59,37 @@ - sergeant + + sergeant + 0.7.0 + + @@ -74,15 +97,19 @@ -
    +

    src tbls

    + Source: R/dplyr.r +
    +

    "SHOW DATABASES"

    + 
    # S3 method for src_drill
 src_tbls(x)
@@ -108,7 +135,7 @@
 # S3 method for DrillConnection
 sql_translate_env(con)
    -

    Arguments

    +

    Arguments

    use ssl?
    username, password

    if not NULL the credentials for the Drill service.

    src

    A Drill "src" created with src_drill()
    @@ -117,13 +144,20 @@
    +

    See also

    + +

    Other Drill REST `dplyr` API: drill_custom_functions, + src_drill

    +
    @@ -134,11 +168,14 @@
    -

    Site built with pkgdown.

    +

    Site built with pkgdown.

    + + +