www/py-scrapy 2.18.0 FreeBSD

High level scraping and web crawling framework

System: FreeBSD
Repository: FreeBSD ports
Maintainer: [email protected]
Last update: 2026-09-30
Commit: 4b13291392
License: BSD3CLAUSE
Categories: www, python
Package size: 657.9 KiB, 3922.9 KiB installed

Description

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

In the other systems

System

Port

Version

FreeBSD

www/py-scrapy

2.18.0

NetBSD

www/py-scrapy

2.18.0

DragonFly

www/py-scrapy

2.11.1_1

GhostBSD

www/py-scrapy

2.18.0

Install

Binary package:

pkg install py312-scrapy

From the ports tree:

cd /usr/ports/www/py-scrapy/ && make install clean

To depend on this port in another port's Makefile:

RUN_DEPENDS=	py312-scrapy>0:www/py-scrapy

Packages

The version each package repository carries. “-” means the repository builds no package of it, “n/a” that there is no such repository.

Repository	aarch64	amd64	armv6	armv7	i386
FreeBSD:14:latest	2.18.0	2.18.0	2.5.1	2.13.3	2.18.0
FreeBSD:14:quarterly	2.18.0	2.18.0	n/a	2.11.1_1	2.18.0
FreeBSD:15:latest	2.18.0	2.18.0	n/a	2.13.3	2.13.3
FreeBSD:15:quarterly	2.18.0	2.18.0	n/a	n/a	n/a
FreeBSD:16:latest	2.18.0	2.18.0	n/a	n/a	n/a
FreeBSD:16:quarterly	n/a	n/a	n/a	n/a	n/a

Dependencies

As this port's Makefile names them.

Build 1

${PYTHON_PKGNAMEPREFIX}hatchling>=1.27.0 : devel/py-hatchling

Runtime 18

${PYTHON_PKGNAMEPREFIX}twisted>=21.7.0 : devel/py-twisted
${PYTHON_PKGNAMEPREFIX}cssselect>=0.9.1 : www/py-cssselect
${PYTHON_PKGNAMEPREFIX}defusedxml>=0.7.1 : devel/py-defusedxml
${PYTHON_PKGNAMEPREFIX}itemloaders>=1.0.1 : devel/py-itemloaders
${PYTHON_PKGNAMEPREFIX}parsel>=1.5.0 : textproc/py-parsel
${PYTHON_PKGNAMEPREFIX}pyopenssl>=24.3.0 : security/py-pyopenssl
${PYTHON_PKGNAMEPREFIX}cryptography>=41.0.5 : security/py-cryptography
${PYTHON_PKGNAMEPREFIX}queuelib>=1.6.1 : sysutils/py-queuelib
${PYTHON_PKGNAMEPREFIX}service-identity>=24.2.0 : security/py-service-identity
${PYTHON_PKGNAMEPREFIX}w3lib>=2.1.1 : www/py-w3lib
${PYTHON_PKGNAMEPREFIX}zope.interface>=5.1.0 : devel/py-zope.interface
${PYTHON_PKGNAMEPREFIX}protego>=0.1.15 : www/py-protego
${PYTHON_PKGNAMEPREFIX}pydispatcher>=2.0.5 : devel/py-pydispatcher
${PYTHON_PKGNAMEPREFIX}itemadapter>=0.1.0 : devel/py-itemadapter
${PYTHON_PKGNAMEPREFIX}lxml>=4.6.4 : devel/py-lxml
${PYTHON_PKGNAMEPREFIX}brotli>=1.2.0 : archivers/py-brotli
${PYTHON_PKGNAMEPREFIX}backports.zstd>=1.3.0 : devel/py-backports.zstd
${PYTHON_PKGNAMEPREFIX}tldextract>0 : dns/py-tldextract

The full lists below include those, and their dependencies in turn.

Build dependencies 19

Run dependencies 67

No port depends on this one.

Configuration

No options to configure.

USES: cpe python:3.11+
Service scripts: none

Files

distinfo 1

File

Size

SHA256

scrapy-2.18.0.tar.gz

1.407695e+06

2445f8b5bf87ba105d239cb230878646e29dbc6a6cae10037ba0550d8fe7fc73

Master sites 1

PYPI

Commit history 2

Date Commit Credits Log message

2026-09-30 4b13291392 Rene Ladan all: bump USES=python version to 3.11+ in preparation of removing lang/python310

2026-09-07

f1d9a80798

Steven Kreuzer

www/py-scrapy: Update to 2.18.0

Changelog: https://github.com/scrapy/scrapy/releases/tag/2.18.0 Sync RUN_DEPENDS with the upstream requirements: - Raise the minimum versions of w3lib (2.1.1), pyopenssl (24.3.0), queuelib (1.6.1), service-identity (24.2.0) and lxml (4.6.4); the stale floors previously allowed pkg to pair this scrapy with an incompatible w3lib 1.x at runtime - Add direct dependencies on cryptography, brotli and backports.zstd (zstd response support on Python < 3.14), which upstream declares explicitly since 2.18