You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
27 lines
889 B
27 lines
889 B
Package: spiderbar
|
|
Type: Package
|
|
Title: Parse and Test Robots Exclusion Protocol Files and Rules
|
|
Version: 0.2.3
|
|
Date: 2020-05-29
|
|
Author: Bob Rudis (bob@rud.is) [aut, cre], SEOmoz, Inc [aut]
|
|
Maintainer: Bob Rudis <bob@rud.is>
|
|
Description: The 'Robots Exclusion Protocol' <https://www.robotstxt.org/orig.html> documents
|
|
a set of standards for allowing or excluding robot/spider crawling of different areas of
|
|
site content. Tools are provided which wrap The 'rep-cpp' <https://github.com/seomoz/rep-cpp>
|
|
C++ library for processing these 'robots.txt' files.
|
|
SystemRequirements: C++11
|
|
NeedsCompilation: yes
|
|
URL: https://gitlab.com/hrbrmstr/spiderbar
|
|
BugReports: https://gitlab.com/hrbrmstr/spiderbar/issues
|
|
License: MIT + file LICENSE
|
|
Suggests:
|
|
covr,
|
|
robotstxt,
|
|
tinytest
|
|
Depends:
|
|
R (>= 3.2.0)
|
|
Encoding: UTF-8
|
|
Imports:
|
|
Rcpp
|
|
RoxygenNote: 7.1.0
|
|
LinkingTo: Rcpp
|
|
|