BSD Ports

www/py-robot-detection 0.4 DragonFly

Detect if a HTTP User-Agent header is likely to be a bot

System
DragonFly
Repository
DPorts
Maintainer
[email protected] (as in FreeBSD)
License
GPLv3+
Categories
python, www
Package size
11.3 KiB, 40.2 KiB installed

Description

Library for detecting if a HTTP User Agent header is likely to be a bot. It uses the list of registered robots from https://www.robotstxt.org/.

In the other systems

The newest version among them is 0.4, as upstream numbers it.

SystemPortVersionVulnerabilitiesStatusMaintainerLast update
DragonFlywww/py-robot-detection0.4 newestnone known-[email protected]-
FreeBSDwww/py-robot-detection0.4 newestnone known-[email protected]-
NetBSDwww/py-robot-detection0.4 newestnone known-[email protected]-
HardenedBSDwww/py-robot-detection0.4 newestno data-[email protected]-
GhostBSDwww/py-robot-detection0.4 newestno data-[email protected]-

Compare across BSDs

Version history 1

  1. 0.4

Upstream's version as each commit to the port's Makefile set it, or as this site first saw it in the system's packages; the tree's own revisions are not versions here.

Install

Binary package:

pkg install py312-robot-detection

From DPorts:

cd /usr/dports/www/py-robot-detection && make install clean

Dependency graph

Ports that need it on the left, what it needs to run on the right; at most 8 of each. "Dependencies" below lists them all.

Dependencies

Run dependencies 2
Required by 1

Configuration

No options to configure.

Service scripts
none

Message after installing

You can download a new version of the Robot Database from https://www.robotstxt.org/db/all.txt.

Download the database dump, and run the file robot_detection.py with the file as first argument.

	$ fetch https://www.robotstxt.org/db/all.txt
	$ python robot_detection.py all.txt

If the database has changed, it'll print out the new version of robot_useragents variable that you need to put into the source code.