Browse Source

appveyor tests

master
boB Rudis 3 years ago
parent
commit
e54dc2bbcb
No known key found for this signature in database GPG Key ID: 2A514A4997464560
10 changed files with 111 additions and 16 deletions
  1. +2
    -0
      .Rbuildignore
  2. +3
    -0
      .travis.yml
  3. +2
    -1
      DESCRIPTION
  4. +6
    -6
      R/render-json.r
  5. +3
    -1
      README.Rmd
  6. +8
    -8
      README.md
  7. +84
    -0
      appveyor.yml
  8. +1
    -0
      codecov.yml
  9. BIN
      img/cap.png
  10. +2
    -0
      tests/testthat/test-splash.R

+ 2
- 0
.Rbuildignore View File

@@ -8,3 +8,5 @@
^img$
^CONDUCT\.md$
^README_files$
^appveyor\.yml$
^codecov\.yml$

+ 3
- 0
.travis.yml View File

@@ -25,3 +25,6 @@ r:
- oldrel
- release
- devel

after_success:
- Rscript -e 'covr::codecov()'

+ 2
- 1
DESCRIPTION View File

@@ -22,7 +22,8 @@ Suggests:
testthat,
tibble,
jpeg,
png
png,
covr
Depends:
R (>= 3.2.0)
Imports:


+ 6
- 6
R/render-json.r View File

@@ -76,12 +76,12 @@ render_json <- function(splash_obj = splash_local, url, base_url=NULL, quality=7
if ("har" %in% names(out)) {

sphar <- list(
log=list(
version=out$har$log$version,
creator=out$har$log$creator,
browser=out$har$log$browser,
pages=out$har$log$pages,
entries=out$har$log$entries
log = list(
version = out$har$log$version,
creator = out$har$log$creator,
browser = out$har$log$browser,
pages = out$har$log$pages,
entries = out$har$log$entries
)
)



+ 3
- 1
README.Rmd View File

@@ -2,12 +2,14 @@
output: rmarkdown::github_document
---
[![Travis-CI Build Status](https://travis-ci.org/hrbrmstr/splashr.svg?branch=master)](https://travis-ci.org/hrbrmstr/splashr)
[![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/hrbrmstr/splashr?branch=master&svg=true)](https://ci.appveyor.com/project/hrbrmstr/splashr)
<!-- [![Coverage Status](https://img.shields.io/codecov/c/github/hrbrmstr/splashr/master.svg)](https://codecov.io/github/hrbrmstr/splashr?branch=master) -->

`splashr` : Tools to Work with the 'Splash' JavaScript Rendering Service

TL;DR: This package works with Splash rendering servers which are really just a REST API & `lua` scripting interface to a QT browser. It's an alternative to the Selenium ecosystem which was really engineered for application testing & validation.

Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do _everything_ Selenium can, but if you're just trying to get a page back that needs javascript rendering, this is a nice alternative.
Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do _everything_ Selenium can in pure R (the Lua interface is equally as powerful and accessible via R), but if you're just trying to get a page back that needs javascript rendering, this is a nice, lightweight, consistent alternative.

It's also an alternative to `phantomjs` (which you can use in R within or without a Selenium context as it's it's own webdriver) and it may be useful to compare renderings between this package & `phantomjs`.



+ 8
- 8
README.md View File

@@ -1,11 +1,11 @@

[![Travis-CI Build Status](https://travis-ci.org/hrbrmstr/splashr.svg?branch=master)](https://travis-ci.org/hrbrmstr/splashr)
[![Travis-CI Build Status](https://travis-ci.org/hrbrmstr/splashr.svg?branch=master)](https://travis-ci.org/hrbrmstr/splashr) [![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/hrbrmstr/splashr?branch=master&svg=true)](https://ci.appveyor.com/project/hrbrmstr/splashr) <!-- [![Coverage Status](https://img.shields.io/codecov/c/github/hrbrmstr/splashr/master.svg)](https://codecov.io/github/hrbrmstr/splashr?branch=master) -->

`splashr` : Tools to Work with the 'Splash' JavaScript Rendering Service

TL;DR: This package works with Splash rendering servers which are really just a REST API & `lua` scripting interface to a QT browser. It's an alternative to the Selenium ecosystem which was really engineered for application testing & validation.

Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do *everything* Selenium can, but if you're just trying to get a page back that needs javascript rendering, this is a nice alternative.
Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do *everything* Selenium can in pure R (the Lua interface is equally as powerful and accessible via R), but if you're just trying to get a page back that needs javascript rendering, this is a nice, lightweight, consistent alternative.

It's also an alternative to `phantomjs` (which you can use in R within or without a Selenium context as it's it's own webdriver) and it may be useful to compare renderings between this package & `phantomjs`.

@@ -153,13 +153,13 @@ splash_debug()
## List of 7
## $ active : list()
## $ argcache: int 0
## $ fds : int 19
## $ fds : int 14
## $ leaks :List of 4
## ..$ Deferred : int 50
## ..$ LuaRuntime: int 1
## ..$ QTimer : int 1
## ..$ Request : int 1
## $ maxrss : int 197264
## $ maxrss : int 75912
## $ qsize : int 0
## $ url : chr "http://localhost:8050"
## - attr(*, "class")= chr [1:2] "splash_debug" "list"
@@ -204,15 +204,15 @@ print(har)
## --------HAR PAGES--------
## Page id: 1 , Page title: Poynter – A global leader in journalism. Strengthening democracy.
## --------HAR ENTRIES--------
## Number of entries: 29
## Number of entries: 42
## REQUESTS:
## Page: 1
## Number of entries: 29
## Number of entries: 42
## - http://www.poynter.org/
## - http://www.poynter.org/wp-content/plugins/easy-author-image/css/easy-author-image.css?ver=2016_06_24.1
## - http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css?ver=2016_06_24.1
## - http://cloud.webtype.com/css/162ac332-3b31-4b73-ad44-da375b7f2fe3.css?ver=2016_06_24.1
## - http://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css?ver=2016_06_24.1
## - http://www.poynter.org/wp-content/themes/poynter_timber/assets/scrollbar/jquery.mCustomScrollbar.min.css?ver=2016...
## ........
## - http://cloud.webtype.com/webtype/ff2/3/4ac7f809-9bdf-4acc-8bd5-a922f05f271b?ec_token=8f7c4c4997246fd7fa920171c994...
## - http://cloud.webtype.com/webtype/ff2/3/c6608520-1978-45ac-9061-74ada664cae4?ec_token=8f7c4c4997246fd7fa920171c994...
@@ -286,7 +286,7 @@ library(testthat)
date()
```

## [1] "Sun Aug 27 07:51:03 2017"
## [1] "Sun Aug 27 08:16:01 2017"

``` r
test_dir("tests/")


+ 84
- 0
appveyor.yml View File

@@ -0,0 +1,84 @@
# DO NOT CHANGE the "init" and "install" sections below

image: Visual Studio 2017

environment:
NOT_CRAN: true
USE_RTOOLS: true
matrix:
- PYTHON: "C:\\Python27"
RETICULATE_PYTHON: "C:\\Python27"
- PYTHON: "C:\\Python34"
RETICULATE_PYTHON: "C:\\Python34"
- PYTHON: "C:\\Python35"
RETICULATE_PYTHON: "C:\\Python35"
- PYTHON: "C:\\Python36"
RETICULATE_PYTHON: "C:\\Python36"

matrix:
fast_finish: true

# Download script file from GitHub
init:
ps: |
$ErrorActionPreference = "Stop"
Invoke-WebRequest http://raw.github.com/krlmlr/r-appveyor/master/scripts/appveyor-tool.ps1 -OutFile "..\appveyor-tool.ps1"
Import-Module '..\appveyor-tool.ps1'

install:
ps: Bootstrap

cache:
- C:\RLibrary

# Adapt as necessary starting from here

build_script:
- docker pull microsoft/nanoserver
- docker pull scrapinghub/splash:3.0
- "%PYTHON%\\python.exe -m pip install setuptools numpy pypiwin32 docker"
- travis-tool.sh install_github rstudio/reticulate
- travis-tool.sh install_deps

before_test:
- echo %RETICULATE_PYTHON%
- echo %PATH%
- R -q -e "reticulate::py_discover_config('docker')"
- docker run -p 5023:5023 -p 8050:8050 -p 8051:8051 -d scrapinghub/splash:3.0
- docker ps -a

test_script:
- travis-tool.sh run_tests

on_failure:
- 7z a failure.zip *.Rcheck\*
- appveyor PushArtifact failure.zip

artifacts:
- path: '*.Rcheck\**\*.log'
name: Logs

- path: '*.Rcheck\**\*.out'
name: Logs

- path: '*.Rcheck\**\*.fail'
name: Logs

- path: '*.Rcheck\**\*.Rout'
name: Logs

- path: '\*_*.tar.gz'
name: Bits

- path: '\*_*.zip'
name: Bits

notifications:
- provider: Email
to:
- bob@rud.is
subject: 'r-appveyor build {{status}}'
message: "https://ci.appveyor.com/project/hrbrmstr/splashr"
on_build_success: false
on_build_failure: true
on_build_status_changed: true

+ 1
- 0
codecov.yml View File

@@ -0,0 +1 @@
comment: false

BIN
img/cap.png View File

Before After
Width: 1024  |  Height: 1451  |  Size: 521KB Width: 1024  |  Height: 1451  |  Size: 521KB

+ 2
- 0
tests/testthat/test-splash.R View File

@@ -1,6 +1,8 @@
context("basic functionality")
test_that("we can do something", {

testthat::skip_on_cran()

test_url <- "http://localhost:8050/"

xpct <- function(x) {


Loading…
Cancel
Save