BSD Ports

www/py-protego 0.6.2 NetBSD

Pure-Python robots.txt parser with support for modern conventions

System
NetBSD
Repository
pkgsrc
Maintainer
[email protected]
Last update
Commit
adbf782f5e
License
modified-bsd
Categories
www, python
Package size
20.1 KiB, 82.2 KiB installed

Description

Protego is a pure-Python robots.txt parser with support for modern conventions.

In the other systems

SystemPortVersion
NetBSDwww/py-protego0.6.2
FreeBSDwww/py-protego0.1.16_1
DragonFlywww/py-protego0.1.16_1
HardenedBSDwww/py-protego0.1.16_1
GhostBSDwww/py-protego0.1.16_1

Install

Binary package:

pkgin install py311-protego

From pkgsrc:

cd /usr/pkgsrc/www/py-protego && make install clean

Packages

The version each package repository carries. “-” means the repository builds no package of it, “n/a” that there is no such repository.

RepositoryVersion
NetBSD 10.1 x86_640.6.2
NetBSD 10.1 aarch640.6.0
SmartOS / illumos x86_640.7.0
macOS 15 arm640.6.2

Dependencies

Run dependencies 1
Required by 1

Configuration

No options to configure.

Service scripts
none
Flavors
py312-protego-0.6.2, py313-protego-0.6.2, py314-protego-0.6.2

Commit history 1

DateCommitCreditsLog message
adbf782f5e adam py-protego: updated to 0.7.0
0.7.0 (2026-09-21) - **Backward-incompatible:** ``Protego.parse()`` now raises a more suitable ``TypeError`` instead of a ``ValueError`` when ``content`` is not a string. - Added official support for Python 3.15. - ``can_fetch()`` now allows ``/robots.txt`` whatever the rules say, as required by RFC 9309. - A group now applies to a user agent only if its product token appears in that user agent at a token boundary, so that ``User-agent: bot`` no longer applies to ``mybot``. A product token within a whole ``User-Agent`` header value, e.g. ``Mozilla/5.0 (compatible; mybot/1.0)``, still matches. - Percent-encoding is now normalized in the query string and the parameters of a URL, and not only in its path, so that a rule matches whichever …