Browse Source

some tweaks before dinner

master
boB Rudis 7 years ago
parent
commit
7965e8d624
No known key found for this signature in database GPG Key ID: 2A514A4997464560
  1. 1
      .Rbuildignore
  2. 4
      DESCRIPTION
  3. 1
      NAMESPACE
  4. 11
      README.Rmd
  5. 45
      README.md
  6. BIN
      img/cap.png
  7. 2
      man/get_response_body.Rd
  8. 19
      man/json_fromb64.Rd
  9. 5
      man/splashr.Rd

1
.Rbuildignore

@ -10,3 +10,4 @@
^README_files$ ^README_files$
^appveyor\.yml$ ^appveyor\.yml$
^codecov\.yml$ ^codecov\.yml$
^docs$

4
DESCRIPTION

@ -4,7 +4,9 @@ Title: Tools to Work with the 'Splash' 'JavaScript' Rendering Service
Version: 0.4.0 Version: 0.4.0
Date: 2017-08-26 Date: 2017-08-26
Encoding: UTF-8 Encoding: UTF-8
Author: Bob Rudis (bob@rud.is) Authors@R: c(
person("Bob", "Rudis", email = "bob@rud.is", role = c("aut", "cre"))
)
Maintainer: Bob Rudis <bob@rud.is> Maintainer: Bob Rudis <bob@rud.is>
Description: 'Splash' <https://github.com/scrapinghub/splash> is a 'JavaScript' rendering service. Description: 'Splash' <https://github.com/scrapinghub/splash> is a 'JavaScript' rendering service.
It’s a lightweight web browser with an 'HTTP' API, implemented in 'Python' using 'Twisted' It’s a lightweight web browser with an 'HTTP' API, implemented in 'Python' using 'Twisted'

1
NAMESPACE

@ -35,6 +35,7 @@ export(is_post)
export(is_svg) export(is_svg)
export(is_xhr) export(is_xhr)
export(is_xml) export(is_xml)
export(json_fromb64)
export(killall_splash) export(killall_splash)
export(renderHARviewer) export(renderHARviewer)
export(render_har) export(render_har)

11
README.Rmd

@ -247,17 +247,6 @@ render_file(url = "/tmp/diag.html", output="html")
stop_splash(splash_vm) stop_splash(splash_vm)
``` ```
### Test Results
```{r message=FALSE, warning=FALSE, error=FALSE}
library(splashr)
library(testthat)
date()
test_dir("tests/")
```
### Code of Conduct ### Code of Conduct
Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). By participating in this project you agree to abide by its terms. Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). By participating in this project you agree to abide by its terms.

45
README.md

@ -151,13 +151,13 @@ splash_debug()
## List of 7 ## List of 7
## $ active : list() ## $ active : list()
## $ argcache: int 0 ## $ argcache: int 0
## $ fds : int 14 ## $ fds : int 21
## $ leaks :List of 4 ## $ leaks :List of 4
## ..$ Deferred : int 50 ## ..$ Deferred : int 50
## ..$ LuaRuntime: int 1 ## ..$ LuaRuntime: int 1
## ..$ QTimer : int 1 ## ..$ QTimer : int 1
## ..$ Request : int 1 ## ..$ Request : int 1
## $ maxrss : int 76308 ## $ maxrss : int 228260
## $ qsize : int 0 ## $ qsize : int 0
## $ url : chr "http://localhost:8050" ## $ url : chr "http://localhost:8050"
## - attr(*, "class")= chr [1:2] "splash_debug" "list" ## - attr(*, "class")= chr [1:2] "splash_debug" "list"
@ -171,7 +171,7 @@ render_html(url = "http://marvel.com/universe/Captain_America_(Steve_Rogers)")
## {xml_document} ## {xml_document}
## <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr"> ## <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">
## [1] <head>\n<script type="text/javascript" async="async" src="http://dpm.demdex.net/id?d_rtbd=json&amp;d_ver=2&amp;d_ ... ## [1] <head>\n<script type="text/javascript" async="" charset="utf-8" src="http://ds.serving-sys.com/SemiCachedScripts/ ...
## [2] <body id="index-index" class="index-index" onload="findLinks('myLink');">\n\n\t<div id="page_frame" style="overfl ... ## [2] <body id="index-index" class="index-index" onload="findLinks('myLink');">\n\n\t<div id="page_frame" style="overfl ...
``` r ``` r
@ -202,21 +202,16 @@ print(har)
## --------HAR PAGES-------- ## --------HAR PAGES--------
## Page id: 1 , Page title: Poynter – A global leader in journalism. Strengthening democracy. ## Page id: 1 , Page title: Poynter – A global leader in journalism. Strengthening democracy.
## --------HAR ENTRIES-------- ## --------HAR ENTRIES--------
## Number of entries: 42 ## Number of entries: 6
## REQUESTS: ## REQUESTS:
## Page: 1 ## Page: 1
## Number of entries: 42 ## Number of entries: 6
## - http://www.poynter.org/ ## - http://www.poynter.org/
## - http://www.poynter.org/wp-content/plugins/easy-author-image/css/easy-author-image.css?ver=2016_06_24.1 ## - http://www.googletagservices.com/tag/js/gpt.js
## - http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css?ver=2016_06_24.1 ## - http://s.ntv.io/serve/load.js?ver=4.6.1
## - http://cloud.webtype.com/css/162ac332-3b31-4b73-ad44-da375b7f2fe3.css?ver=2016_06_24.1 ## - http://www.googletagservices.com/tag/js/gpt.js
## - http://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css?ver=2016_06_24.1 ## - http://www.google-analytics.com/analytics.js
## ........ ## - http://static.parsely.com/p.js
## - http://cloud.webtype.com/webtype/ff2/3/4ac7f809-9bdf-4acc-8bd5-a922f05f271b?ec_token=8f7c4c4997246fd7fa920171c994...
## - http://cloud.webtype.com/webtype/ff2/3/c6608520-1978-45ac-9061-74ada664cae4?ec_token=8f7c4c4997246fd7fa920171c994...
## - http://cloud.webtype.com/webtype/ff2/3/380e3672-840d-462a-83ee-2ea85a43504a?ec_token=8f7c4c4997246fd7fa920171c994...
## - http://cloud.webtype.com/webtype/ff2/3/c6369fc5-fc59-4a12-ac92-25afa6c567a0?ec_token=8f7c4c4997246fd7fa920171c994...
## - http://static.chartbeat.com/js/chartbeat.js
You can use [`HARtools::HARviewer`](https://github.com/johndharrison/HARtools/blob/master/R/HARviewer.R) — which this pkg import/exports — to get view the HAR in an interactive HTML widget. You can use [`HARtools::HARviewer`](https://github.com/johndharrison/HARtools/blob/master/R/HARviewer.R) — which this pkg import/exports — to get view the HAR in an interactive HTML widget.
@ -275,26 +270,6 @@ splash_local %>%
stop_splash(splash_vm) stop_splash(splash_vm)
``` ```
### Test Results
``` r
library(splashr)
library(testthat)
date()
```
## [1] "Sun Aug 27 09:01:57 2017"
``` r
test_dir("tests/")
```
## testthat results ========================================================================================================
## OK: 8 SKIPPED: 0 FAILED: 0
##
## DONE ===================================================================================================================
### Code of Conduct ### Code of Conduct
Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). By participating in this project you agree to abide by its terms. Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). By participating in this project you agree to abide by its terms.

BIN
img/cap.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 521 KiB

After

Width:  |  Height:  |  Size: 522 KiB

2
man/get_response_body.Rd

@ -4,7 +4,7 @@
\alias{get_response_body} \alias{get_response_body}
\title{Retrieve the body content of a HAR entry} \title{Retrieve the body content of a HAR entry}
\usage{ \usage{
get_response_body(har_resp_obj) get_response_body(har_resp_obj, type = c("raw", "text"))
} }
\arguments{ \arguments{
\item{har_resp_obj}{HAR response object} \item{har_resp_obj}{HAR response object}

19
man/json_fromb64.Rd

@ -0,0 +1,19 @@
% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/utils.r
\name{json_fromb64}
\alias{json_fromb64}
\title{Convert a Base64 encoded string into an R object}
\usage{
json_fromb64(x, flatten = TRUE, ...)
}
\arguments{
\item{x}{a string}
\item{flatten}{flatten JSON structures upon conversion?}
\item{...}{passed on to \code{jsonlite::fromJSON()}}
}
\description{
A simple wrapper around calls to \code{openssl::base64_decode()} and
\code{jsonlite::fromJSON()}.
}

5
man/splashr.Rd

@ -9,7 +9,10 @@
'Splash' \url{https://github.com/scrapinghub/splash} is a 'JavaScript' rendering service. 'Splash' \url{https://github.com/scrapinghub/splash} is a 'JavaScript' rendering service.
It’s a lightweight web browser with an 'HTTP' API, implemented in 'Python' using 'Twisted' It’s a lightweight web browser with an 'HTTP' API, implemented in 'Python' using 'Twisted'
and 'QT' and provides some of the core functionality of the 'RSelenium' or 'seleniumPipes' and 'QT' and provides some of the core functionality of the 'RSelenium' or 'seleniumPipes'
R pacakges in a lightweight footprint. Some of 'Splash' features include the ability to process R pacakges in a lightweight footprint.
}
\details{
Some of 'Splash' features include the ability to process
multiple webpages in parallel; retrieving 'HTML' results and/or take screenshots; disabling multiple webpages in parallel; retrieving 'HTML' results and/or take screenshots; disabling
images or use 'Adblock Plus' rules to make rendering faster; executing custom 'JavaScript' in images or use 'Adblock Plus' rules to make rendering faster; executing custom 'JavaScript' in
page context; getting detailed rendering info in 'HAR' format. page context; getting detailed rendering info in 'HAR' format.

Loading…
Cancel
Save