Browse Source

appveyor tests

master
boB Rudis 7 years ago
parent
commit
e54dc2bbcb
No known key found for this signature in database GPG Key ID: 2A514A4997464560
  1. 2
      .Rbuildignore
  2. 3
      .travis.yml
  3. 3
      DESCRIPTION
  4. 12
      R/render-json.r
  5. 4
      README.Rmd
  6. 16
      README.md
  7. 84
      appveyor.yml
  8. 1
      codecov.yml
  9. BIN
      img/cap.png
  10. 2
      tests/testthat/test-splash.R

2
.Rbuildignore

@ -8,3 +8,5 @@
^img$
^CONDUCT\.md$
^README_files$
^appveyor\.yml$
^codecov\.yml$

3
.travis.yml

@ -25,3 +25,6 @@ r:
- oldrel
- release
- devel
after_success:
- Rscript -e 'covr::codecov()'

3
DESCRIPTION

@ -22,7 +22,8 @@ Suggests:
testthat,
tibble,
jpeg,
png
png,
covr
Depends:
R (>= 3.2.0)
Imports:

12
R/render-json.r

@ -76,12 +76,12 @@ render_json <- function(splash_obj = splash_local, url, base_url=NULL, quality=7
if ("har" %in% names(out)) {
sphar <- list(
log=list(
version=out$har$log$version,
creator=out$har$log$creator,
browser=out$har$log$browser,
pages=out$har$log$pages,
entries=out$har$log$entries
log = list(
version = out$har$log$version,
creator = out$har$log$creator,
browser = out$har$log$browser,
pages = out$har$log$pages,
entries = out$har$log$entries
)
)

4
README.Rmd

@ -2,12 +2,14 @@
output: rmarkdown::github_document
---
[![Travis-CI Build Status](https://travis-ci.org/hrbrmstr/splashr.svg?branch=master)](https://travis-ci.org/hrbrmstr/splashr)
[![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/hrbrmstr/splashr?branch=master&svg=true)](https://ci.appveyor.com/project/hrbrmstr/splashr)
<!-- [![Coverage Status](https://img.shields.io/codecov/c/github/hrbrmstr/splashr/master.svg)](https://codecov.io/github/hrbrmstr/splashr?branch=master) -->
`splashr` : Tools to Work with the 'Splash' JavaScript Rendering Service
TL;DR: This package works with Splash rendering servers which are really just a REST API & `lua` scripting interface to a QT browser. It's an alternative to the Selenium ecosystem which was really engineered for application testing & validation.
Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do _everything_ Selenium can, but if you're just trying to get a page back that needs javascript rendering, this is a nice alternative.
Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do _everything_ Selenium can in pure R (the Lua interface is equally as powerful and accessible via R), but if you're just trying to get a page back that needs javascript rendering, this is a nice, lightweight, consistent alternative.
It's also an alternative to `phantomjs` (which you can use in R within or without a Selenium context as it's it's own webdriver) and it may be useful to compare renderings between this package & `phantomjs`.

16
README.md

@ -1,11 +1,11 @@
[![Travis-CI Build Status](https://travis-ci.org/hrbrmstr/splashr.svg?branch=master)](https://travis-ci.org/hrbrmstr/splashr)
[![Travis-CI Build Status](https://travis-ci.org/hrbrmstr/splashr.svg?branch=master)](https://travis-ci.org/hrbrmstr/splashr) [![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/hrbrmstr/splashr?branch=master&svg=true)](https://ci.appveyor.com/project/hrbrmstr/splashr) <!-- [![Coverage Status](https://img.shields.io/codecov/c/github/hrbrmstr/splashr/master.svg)](https://codecov.io/github/hrbrmstr/splashr?branch=master) -->
`splashr` : Tools to Work with the 'Splash' JavaScript Rendering Service
TL;DR: This package works with Splash rendering servers which are really just a REST API & `lua` scripting interface to a QT browser. It's an alternative to the Selenium ecosystem which was really engineered for application testing & validation.
Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do *everything* Selenium can, but if you're just trying to get a page back that needs javascript rendering, this is a nice alternative.
Sometimes, all you need is a page scrape after javascript has been allowed to roam wild and free over your meticulously crafted HTML tags. So, this package does not do *everything* Selenium can in pure R (the Lua interface is equally as powerful and accessible via R), but if you're just trying to get a page back that needs javascript rendering, this is a nice, lightweight, consistent alternative.
It's also an alternative to `phantomjs` (which you can use in R within or without a Selenium context as it's it's own webdriver) and it may be useful to compare renderings between this package & `phantomjs`.
@ -153,13 +153,13 @@ splash_debug()
## List of 7
## $ active : list()
## $ argcache: int 0
## $ fds : int 19
## $ fds : int 14
## $ leaks :List of 4
## ..$ Deferred : int 50
## ..$ LuaRuntime: int 1
## ..$ QTimer : int 1
## ..$ Request : int 1
## $ maxrss : int 197264
## $ maxrss : int 75912
## $ qsize : int 0
## $ url : chr "http://localhost:8050"
## - attr(*, "class")= chr [1:2] "splash_debug" "list"
@ -204,15 +204,15 @@ print(har)
## --------HAR PAGES--------
## Page id: 1 , Page title: Poynter – A global leader in journalism. Strengthening democracy.
## --------HAR ENTRIES--------
## Number of entries: 29
## Number of entries: 42
## REQUESTS:
## Page: 1
## Number of entries: 29
## Number of entries: 42
## - http://www.poynter.org/
## - http://www.poynter.org/wp-content/plugins/easy-author-image/css/easy-author-image.css?ver=2016_06_24.1
## - http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css?ver=2016_06_24.1
## - http://cloud.webtype.com/css/162ac332-3b31-4b73-ad44-da375b7f2fe3.css?ver=2016_06_24.1
## - http://maxcdn.bootstrapcdn.com/font-awesome/4.4.0/css/font-awesome.min.css?ver=2016_06_24.1
## - http://www.poynter.org/wp-content/themes/poynter_timber/assets/scrollbar/jquery.mCustomScrollbar.min.css?ver=2016...
## ........
## - http://cloud.webtype.com/webtype/ff2/3/4ac7f809-9bdf-4acc-8bd5-a922f05f271b?ec_token=8f7c4c4997246fd7fa920171c994...
## - http://cloud.webtype.com/webtype/ff2/3/c6608520-1978-45ac-9061-74ada664cae4?ec_token=8f7c4c4997246fd7fa920171c994...
@ -286,7 +286,7 @@ library(testthat)
date()
```
## [1] "Sun Aug 27 07:51:03 2017"
## [1] "Sun Aug 27 08:16:01 2017"
``` r
test_dir("tests/")

84
appveyor.yml

@ -0,0 +1,84 @@
# DO NOT CHANGE the "init" and "install" sections below
image: Visual Studio 2017
environment:
NOT_CRAN: true
USE_RTOOLS: true
matrix:
- PYTHON: "C:\\Python27"
RETICULATE_PYTHON: "C:\\Python27"
- PYTHON: "C:\\Python34"
RETICULATE_PYTHON: "C:\\Python34"
- PYTHON: "C:\\Python35"
RETICULATE_PYTHON: "C:\\Python35"
- PYTHON: "C:\\Python36"
RETICULATE_PYTHON: "C:\\Python36"
matrix:
fast_finish: true
# Download script file from GitHub
init:
ps: |
$ErrorActionPreference = "Stop"
Invoke-WebRequest http://raw.github.com/krlmlr/r-appveyor/master/scripts/appveyor-tool.ps1 -OutFile "..\appveyor-tool.ps1"
Import-Module '..\appveyor-tool.ps1'
install:
ps: Bootstrap
cache:
- C:\RLibrary
# Adapt as necessary starting from here
build_script:
- docker pull microsoft/nanoserver
- docker pull scrapinghub/splash:3.0
- "%PYTHON%\\python.exe -m pip install setuptools numpy pypiwin32 docker"
- travis-tool.sh install_github rstudio/reticulate
- travis-tool.sh install_deps
before_test:
- echo %RETICULATE_PYTHON%
- echo %PATH%
- R -q -e "reticulate::py_discover_config('docker')"
- docker run -p 5023:5023 -p 8050:8050 -p 8051:8051 -d scrapinghub/splash:3.0
- docker ps -a
test_script:
- travis-tool.sh run_tests
on_failure:
- 7z a failure.zip *.Rcheck\*
- appveyor PushArtifact failure.zip
artifacts:
- path: '*.Rcheck\**\*.log'
name: Logs
- path: '*.Rcheck\**\*.out'
name: Logs
- path: '*.Rcheck\**\*.fail'
name: Logs
- path: '*.Rcheck\**\*.Rout'
name: Logs
- path: '\*_*.tar.gz'
name: Bits
- path: '\*_*.zip'
name: Bits
notifications:
- provider: Email
to:
- bob@rud.is
subject: 'r-appveyor build {{status}}'
message: "https://ci.appveyor.com/project/hrbrmstr/splashr"
on_build_success: false
on_build_failure: true
on_build_status_changed: true

1
codecov.yml

@ -0,0 +1 @@
comment: false

BIN
img/cap.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 521 KiB

After

Width:  |  Height:  |  Size: 521 KiB

2
tests/testthat/test-splash.R

@ -1,6 +1,8 @@
context("basic functionality")
test_that("we can do something", {
testthat::skip_on_cran()
test_url <- "http://localhost:8050/"
xpct <- function(x) {

Loading…
Cancel
Save