BSD Ports

www/py-robot-detection 0.4 FreeBSD

Detect if a HTTP User-Agent header is likely to be a bot

System
FreeBSD
Repository
FreeBSD ports
Maintainer
[email protected]
License
GPLv3+
Categories
www, python
Package size
12.3 KiB, 40.2 KiB installed

Description

Library for detecting if a HTTP User Agent header is likely to be a bot. It uses the list of registered robots from https://www.robotstxt.org/.

In the other systems

SystemPortVersion
FreeBSDwww/py-robot-detection0.4
NetBSDwww/py-robot-detection0.4
HardenedBSDwww/py-robot-detection0.4
GhostBSDwww/py-robot-detection0.4

Install

Binary package:

pkg install py312-robot-detection

From the ports tree:

cd /usr/ports/www/py-robot-detection/ && make install clean

To depend on this port in another port's Makefile:

RUN_DEPENDS=	py312-robot-detection>0:www/py-robot-detection

Packages

The version each package repository carries. “-” means the repository builds no package of it, “n/a” that there is no such repository.

Repositoryaarch64amd64armv6armv7i386
FreeBSD:14:latest0.40.4--0.4
FreeBSD:14:quarterly0.40.4n/a-0.4
FreeBSD:15:latest0.40.4n/a--
FreeBSD:15:quarterly0.40.4n/an/an/a
FreeBSD:16:latest0.40.4n/an/an/a
FreeBSD:16:quarterlyn/an/an/an/an/a

Dependencies

As this port's Makefile names them.

Runtime 1

The full lists below include those, and their dependencies in turn.

Build dependencies 11
Run dependencies 8
Required by 1

Configuration

No options to configure.

USES
python
Service scripts
none

Message after installing

You can download a new version of the Robot Database from https://www.robotstxt.org/db/all.txt.

Download the database dump, and run the file robot_detection.py with the file as first argument.

	$ fetch https://www.robotstxt.org/db/all.txt
	$ python robot_detection.py all.txt

If the database has changed, it'll print out the new version of robot_useragents variable that you need to put into the source code.

Files

distinfo 1
FileSizeSHA256
robot-detection-0.4.tar.gz63873d8fb72ca47164b8ce55e33bdda93742f62c348def7d3cc3b42b0ceb4795a2f5
Master sites 1
  • PYPI

Commit history 0

No commits in the history this site was built from.