Browse Source

final tweaks & README

master
boB Rudis 4 years ago
parent
commit
6065d96f0e
No known key found for this signature in database GPG Key ID: 1D7529BE14E2BBA9
  1. 5
      .Rbuildignore
  2. 4
      DESCRIPTION
  3. 2
      NAMESPACE
  4. 2
      R/caa-package.R
  5. 38
      README.Rmd
  6. 87
      README.md
  7. 25
      src/go/src/main/main.c
  8. 3
      src/go/src/main/main.go
  9. 0
      src/init.c

5
.Rbuildignore

@ -20,3 +20,8 @@
^appveyor\.yml$
^tools$
^LICENSE\.md$
dnscaa\.a$
\.codecov\.yml$
\.travis\.yml$
src/go/pkg$
src/go/src/golang\.org/x/crypto/sha3/testdata$

4
DESCRIPTION

@ -6,7 +6,9 @@ Date: 2020-03-06
Authors@R: c(
person("Bob", "Rudis", email = "bob@rud.is", role = c("aut", "cre"),
comment = c(ORCID = "0000-0001-5670-2640")),
person("Romain", "Francois", role = "ctb", comment = "Original researcher.")
person("Romain", "Francois", role = "ctb", comment = "Original Go-R interface research"),
person("Simone", "Carletti", role = "ctb", comment = "dnscaa Go library https://github.com/weppos/dnscaa"),
person("Miek", "Gieben", role = "ctb", comment = "dns Go library https://github.com/miekg/dns")
)
Maintainer: Bob Rudis <bob@rud.is>
Description: Experimental R wrapper for the Go dnscall library. Builds off of work done by

2
NAMESPACE

@ -2,4 +2,4 @@
export(caa_dig)
importFrom(tibble,as_tibble)
useDynLib(caa)
useDynLib(caa, .registration = TRUE)

2
R/caa-package.R

@ -8,5 +8,5 @@
#' @importFrom tibble as_tibble
#' @keywords internal
#' @author Bob Rudis (bob@@rud.is)
#' @useDynLib caa
#' @useDynLib caa, .registration = TRUE
"_PACKAGE"

38
README.Rmd

@ -23,6 +23,44 @@ The following functions are implemented:
hrbrpkghelpr::describe_ingredients()
```
## WAT?!
Grabbing & parsing DNS CAA records isn't the actual purpose of this pkg/repo. Romain did a fantastic job cracking the Go nut for R, but not many R packages seem to be based on using Go (C, Rcpp, Fortran, and even Rust seems to be more en vogue) and the existing examples work with basic types.
This package uses Go libraries and makes a data frame from the results of Go-side network queries for DNS CAA records. It relies 100% on Go for networking and DNS record parsing, which only leaves getting the results back into R.
`main.go` contains some useful idioms, such as the comment block at the top right before the first `import` which makes "unsafe" C things there available to Go (so we get `SEXP`!) Midway down Go String slices are turned into C-compatible character arrays, along with the code to free up those bits of heap memory at the end (`defer`s). We use the external reference to `MakeDF()` (`C.MakeDF()`) in `main.c` to make the `SEXP` data frame we're returning. We could have tried to keep that in Go but it's cleaner (IMO) to just let C do the work. NOTE that it's important to use `free()` on each element of the 2D (`char **`) character array on the C side since they were allocated on the heap on the Go side and Go's memory manager won't auto-free those for you.
`main.go` also has:
```{go eval=FALSE}
func init() {
log.SetOutput(ioutil.Discard)
}
```
in it to stop modules that use the Go logger from outputting things (Go programmers tend to be "systems programmers" and, as a result, log the heck out of everything).
Also on the C side is the `R_caa_dig()` function which we `.Call()` from the R side (along with the package registration code for it). Given that I had to stick a `.c` file in `src/` I'll eventually (likely) modify the `Makefile` and put the required R registration code there.
The `.Rbuildignore` removes some Go module repository cruft.
Ultimately, it still does not pass CRAN checks. The lib dir is YUGE, `cgo`'s preferred/required compilation settings are against the CRAN rules, and the requirement for GNU Make is also somewhat verbotten. However, a package check with CRAN checks enabled only results in 1 warning and 2 notes (on macOS).
For this particular package, the `dnscaa` Go library had a call to `fmt.Println()` in `dnscaa.go` when there was an NXDOMAIN result which I had to turn into `log.Println()`. Unfortunately, the way `go get` works with git repos, they're submodules, so those changes will get overwritten and you'll have to add them manuall (keep them as git submodules so you can more easily update the underlying libraries if need be).
As a result of ^^ you'll need to do this to clone this repo effectively:
```
$ git clone --recurse-submodules -j8 ssh://git@git.rud.is:7777/hrbrmstr/caa.git
```
Replace `git clone --recurse-submodules -j8 ssh://git@git.rud.is:7777/hrbrmstr/caa.git` with the `https` or `ssh` URL of where you're viewing this and have access.
There are comments peppered throughout `main.go` and especially `main.c`, but if anything needs clarification, drop an issue.
This is my first time through the `cgo` interface (Go itself is straightforward, `cgo` has lots of gotchas), so if I missed anything there def lemme know.
## Installation
```{r install-ex, results='asis', echo=FALSE, cache=FALSE}

87
README.md

@ -27,6 +27,79 @@ The following functions are implemented:
- `caa_dig`: Retrieve the CAA record values for a domain (if any)
## WAT?\!
Grabbing & parsing DNS CAA records isn’t the actual purpose of this
pkg/repo. Romain did a fantastic job cracking the Go nut for R, but not
many R packages seem to be based on using Go (C, Rcpp, Fortran, and even
Rust seems to be more en vogue) and the existing examples work with
basic types.
This package uses Go libraries and makes a data frame from the results
of Go-side network queries for DNS CAA records. It relies 100% on Go for
networking and DNS record parsing, which only leaves getting the results
back into R.
`main.go` contains some useful idioms, such as the comment block at the
top right before the first `import` which makes “unsafe” C things there
available to Go (so we get `SEXP`\!) Midway down Go String slices are
turned into C-compatible character arrays, along with the code to free
up those bits of heap memory at the end (`defer`s). We use the external
reference to `MakeDF()` (`C.MakeDF()`) in `main.c` to make the `SEXP`
data frame we’re returning. We could have tried to keep that in Go but
it’s cleaner (IMO) to just let C do the work. NOTE that it’s important
to use `free()` on each element of the 2D (`char **`) character array on
the C side since they were allocated on the heap on the Go side and Go’s
memory manager won’t auto-free those for you.
`main.go` also has:
``` go
func init() {
log.SetOutput(ioutil.Discard)
}
```
in it to stop modules that use the Go logger from outputting things (Go
programmers tend to be “systems programmers” and, as a result, log the
heck out of everything).
Also on the C side is the `R_caa_dig()` function which we `.Call()` from
the R side (along with the package registration code for it). Given that
I had to stick a `.c` file in `src/` I’ll eventually (likely) modify the
`Makefile` and put the required R registration code there.
The `.Rbuildignore` removes some Go module repository cruft.
Ultimately, it still does not pass CRAN checks. The lib dir is YUGE,
`cgo`’s preferred/required compilation settings are against the CRAN
rules, and the requirement for GNU Make is also somewhat verbotten.
However, a package check with CRAN checks enabled only results in 1
warning and 2 notes (on macOS).
For this particular package, the `dnscaa` Go library had a call to
`fmt.Println()` in `dnscaa.go` when there was an NXDOMAIN result which I
had to turn into `log.Println()`. Unfortunately, the way `go get` works
with git repos, they’re submodules, so those changes will get
overwritten and you’ll have to add them manuall (keep them as git
submodules so you can more easily update the underlying libraries if
need be).
As a result of ^^ you’ll need to do this to clone this repo effectively:
$ git clone --recurse-submodules -j8 ssh://git@git.rud.is:7777/hrbrmstr/caa.git
Replace `git clone --recurse-submodules -j8
ssh://git@git.rud.is:7777/hrbrmstr/caa.git` with the `https` or `ssh`
URL of where you’re viewing this and have access.
There are comments peppered throughout `main.go` and especially
`main.c`, but if anything needs clarification, drop an issue.
This is my first time through the `cgo` interface (Go itself is
straightforward, `cgo` has lots of gotchas), so if I missed anything
there def lemme know.
## Installation
``` r
@ -63,9 +136,9 @@ caa_dig("www.comodo.com")
## # A tibble: 3 x 2
## tag value
## <chr> <chr>
## 1 issue digicert.com
## 2 iodef mailto:sslabuse@comodoca.com
## 3 issue comodoca.com
## 1 iodef mailto:sslabuse@comodoca.com
## 2 issue comodoca.com
## 3 issue digicert.com
# none (lookup error)
caa_dig("www.comodo.comm")
@ -77,19 +150,19 @@ caa_dig("www.comodo.comm")
| Lang | \# Files | (%) | LoC | (%) | Blank lines | (%) | \# Lines | (%) |
| :----------------- | -------: | ---: | -----: | ---: | ----------: | ---: | -------: | ---: |
| Go | 1293 | 0.92 | 374722 | 0.95 | 40522 | 0.95 | 39892 | 0.95 |
| Assembly | 69 | 0.05 | 10426 | 0.03 | 1258 | 0.03 | 1424 | 0.03 |
| Go | 1293 | 0.92 | 376551 | 0.95 | 40523 | 0.95 | 38063 | 0.95 |
| Assembly | 69 | 0.05 | 10426 | 0.03 | 1258 | 0.03 | 1424 | 0.04 |
| XML | 1 | 0.00 | 4780 | 0.01 | 228 | 0.01 | 13 | 0.00 |
| HTML | 10 | 0.01 | 2214 | 0.01 | 441 | 0.01 | 16 | 0.00 |
| Bourne Shell | 5 | 0.00 | 798 | 0.00 | 113 | 0.00 | 429 | 0.01 |
| C | 6 | 0.00 | 351 | 0.00 | 111 | 0.00 | 80 | 0.00 |
| C | 6 | 0.00 | 359 | 0.00 | 117 | 0.00 | 83 | 0.00 |
| Dockerfile | 3 | 0.00 | 91 | 0.00 | 27 | 0.00 | 23 | 0.00 |
| Bourne Again Shell | 2 | 0.00 | 72 | 0.00 | 12 | 0.00 | 6 | 0.00 |
| C/C++ Header | 1 | 0.00 | 48 | 0.00 | 28 | 0.00 | 10 | 0.00 |
| YAML | 2 | 0.00 | 45 | 0.00 | 0 | 0.00 | 0 | 0.00 |
| make | 2 | 0.00 | 25 | 0.00 | 7 | 0.00 | 4 | 0.00 |
| R | 3 | 0.00 | 15 | 0.00 | 8 | 0.00 | 25 | 0.00 |
| Rmd | 1 | 0.00 | 11 | 0.00 | 18 | 0.00 | 33 | 0.00 |
| Rmd | 1 | 0.00 | 14 | 0.00 | 34 | 0.00 | 52 | 0.00 |
## Code of Conduct

25
src/go/src/main/main.c

@ -1,11 +1,11 @@
#include "_cgo_export.h"
#include <R_ext/Rdynload.h>
#include <stdlib.h>
#include <string.h>
// define R bridge functions here
#define SHORT_VEC_LENGTH(x) (((VECSEXP) (x))->vecsxp.length)
// This gets called from Go and returns a data frame
SEXP MakeDF(int n, char** tag, char** val) {
@ -53,14 +53,31 @@ SEXP MakeDF(int n, char** tag, char** val) {
}
// The thing we're going to call from a .R file
// It calls the Go `caa_dig()` function which, in turn, calls MakeDF() above
SEXP R_caa_dig(SEXP x) {
SEXP sx = STRING_ELT(x, 0);
GoString h = { (char*) CHAR(sx), SHORT_VEC_LENGTH(sx) };
GoString h = { (char*) CHAR(sx), Rf_xlength(sx) };
SEXP out = caa_dig(h);
return(out);
}
// Required "registration" code by CRAN
static const R_CallMethodDef CallEntries[] = {
{"R_caa_dig", (DL_FUNC) &R_caa_dig, 1},
{NULL, NULL, 0}
};
void R_init_caa(DllInfo *dll) {
R_registerRoutines(dll, NULL, CallEntries, NULL, NULL);
R_useDynamicSymbols(dll, FALSE);
}

3
src/go/src/main/main.go

@ -56,9 +56,10 @@ func caa_dig(hostname string) C.SEXP {
}
// how many things
n := C.int(len(val))
n := C.int(len(val)) // don't need to free https://blog.golang.org/c-go-cgo
// clean up after calling the data frame maker
defer C.free(unsafe.Pointer(val_arr))
defer C.free(unsafe.Pointer(tag_arr))

0
src/init.c

Loading…
Cancel
Save