BSD Portsdns/py-dns-crawler 1.6.5_1 FreeBSD
Crawler for getting info about DNS domains and services
- System
- FreeBSD
- Repository
- FreeBSD ports
- Maintainer
- [email protected]
- License
- GPLv3
- Categories
- dns, python
- Package size
- 74.4 KiB, 351.8 KiB installed
Description
Despite the name, the crawler gets info for more services than just DNS:
- DNS:
- all A/AAAA records (for the 2nd level domain and www.subdomain), annotated with GeoIP
- TXT records (with SPF and DMARC parsed for easier filtering)
- TLSA (for the 2nd level domain and www.subdomain)
- MX
- DNSSEC validation
- nameservers:
- each server IP annotated with GeoIP
- HOSTNAME.BIND, VERSION.BIND, AUTHORS.BIND and fortune (also for all IPs)
- users can add custom additional RRs in the config file
- E-mail (for every server from MX):
- SMTP server banners (optional, ports are configurable)
- TLSA records
- Web:
- HTTP status & headers (inc. parsed cookies) for ports 80 & 443 on each IP from A/AAAA records
- certificate info for HTTPS (optionally with an entire cert chain)
- webpage content (optional)
- everything of the above is saved for each step in the redirect history -- the crawler follows redirects until it gets a non-redirecting status or hits a configurable limit
In the other systems
| System | Port | Version |
|---|---|---|
| FreeBSD | dns/py-dns-crawler | 1.6.5_1 |
| DragonFly | dns/py-dns-crawler | 1.6.5 |
| HardenedBSD | dns/py-dns-crawler | 1.6.5_1 |
| GhostBSD | dns/py-dns-crawler | 1.6.5_1 |
Install
Binary package:
pkg install py312-dns-crawlerFrom the ports tree:
cd /usr/ports/dns/py-dns-crawler/ && make install cleanTo depend on this port in another port's Makefile:
RUN_DEPENDS= py312-dns-crawler>0:dns/py-dns-crawlerPackages
The version each package repository carries. “-” means the repository builds no package of it, “n/a” that there is no such repository.
| Repository | aarch64 | amd64 | armv6 | armv7 | i386 |
|---|---|---|---|---|---|
| FreeBSD:14:latest | 1.6.5_1 | 1.6.5_1 | 1.4.9 | 1.6.5 | 1.6.5_1 |
| FreeBSD:14:quarterly | 1.6.5_1 | 1.6.5_1 | n/a | 1.6.5 | 1.6.5_1 |
| FreeBSD:15:latest | 1.6.5_1 | 1.6.5_1 | n/a | 1.6.5 | 1.6.5 |
| FreeBSD:15:quarterly | 1.6.5_1 | 1.6.5_1 | n/a | n/a | n/a |
| FreeBSD:16:latest | 1.6.5_1 | 1.6.5_1 | n/a | n/a | n/a |
| FreeBSD:16:quarterly | n/a | n/a | n/a | n/a | n/a |
Dependencies
As this port's Makefile names them.
Build 3
${PYTHON_PKGNAMEPREFIX}setuptools>=41: devel/py-setuptools${PYTHON_PKGNAMEPREFIX}setuptools-git-versioning>=0: devel/py-setuptools-git-versioning${PYTHON_PKGNAMEPREFIX}wheel>=0: devel/py-wheel
Runtime 17
${PYTHON_PKGNAMEPREFIX}asn1crypto>=1.5.1: devel/py-asn1crypto${PYTHON_PKGNAMEPREFIX}cert-human>=1.0.7: security/py-cert-human${PYTHON_PKGNAMEPREFIX}checkdmarc>=4.3.1: mail/py-checkdmarc${PYTHON_PKGNAMEPREFIX}dnspython>=2.3.0,1: dns/py-dnspython${PYTHON_PKGNAMEPREFIX}ecdsa>=0.18.0: security/py-ecdsa${PYTHON_PKGNAMEPREFIX}forcediphttpsadapter>=1.0.2: www/py-forcediphttpsadapter${PYTHON_PKGNAMEPREFIX}GeoIP2>=3.0.0: net/py-GeoIP2${PYTHON_PKGNAMEPREFIX}hstspreload>=0: www/py-hstspreload${PYTHON_PKGNAMEPREFIX}idna>=2.10: dns/py-idna${PYTHON_PKGNAMEPREFIX}pyaml>=21.10.1: textproc/py-pyaml${PYTHON_PKGNAMEPREFIX}pyicu>=2.13.1: devel/py-pyicu${PYTHON_PKGNAMEPREFIX}pyopenssl>=23.1.1,1: security/py-pyopenssl${PYTHON_PKGNAMEPREFIX}python-magic>=0.4.27: devel/py-python-magic${PYTHON_PKGNAMEPREFIX}redis>=4.5.4: databases/py-redis${PYTHON_PKGNAMEPREFIX}requests>=2.23.0: www/py-requests${PYTHON_PKGNAMEPREFIX}requests-toolbelt>=0.10.1: www/py-requests-toolbelt${PYTHON_PKGNAMEPREFIX}rq1>=1.13.0: devel/py-rq1
The full lists below include those, and their dependencies in turn.
Build dependencies 17
Run dependencies 79
- archivers/liblz4
- archivers/zstd
- databases/py-redis
- devel/gettext-runtime
- devel/icu
- devel/libffi
- devel/py-aiosignal
- devel/py-anyio
- devel/py-asn1crypto
- devel/py-async_generator
- devel/py-attrs
- devel/py-cffi
- devel/py-click
- devel/py-expiringdict
- devel/py-frozenlist
- devel/py-importlib-resources
- devel/py-outcome
- devel/py-packaging
- devel/py-propcache
- devel/py-pycparser
- devel/py-pyicu
- devel/py-pyleri
- devel/py-python-magic
- devel/py-pyyaml
- devel/py-rq1
- devel/py-six
- devel/py-sniffio
- devel/py-sortedcontainers
- devel/py-typing-extensions
- devel/py-xmltodict
- devel/readline
- dns/py-dnspython
- dns/py-idna
- dns/py-publicsuffixlist
- lang/python312
- lang/python315
- mail/py-checkdmarc
- math/gmp
- math/mpc
- math/mpdecimal
- math/mpfr
- math/py-gmpy2
- net/libmaxminddb
- net/py-GeoIP2
- net/py-aiohappyeyeballs
- net/py-h11
- net/py-maxminddb
- net/py-pysocks
- net/py-socksio
- net/py-trio
- net/py-urllib3
- print/indexinfo
- security/py-cert-human
- security/py-certifi
- security/py-cryptography
- security/py-ecdsa
- security/py-pem
- security/py-pyopenssl
- security/py-service-identity
- security/py-truststore
- textproc/expat2
- textproc/libyaml
- textproc/py-charset-normalizer
- textproc/py-pyaml
- www/llhttp
- www/py-aiohttp
- www/py-aioquic
- www/py-forcediphttpsadapter
- www/py-h2
- www/py-hpack
- www/py-hstspreload
- www/py-httpcore
- www/py-httpx
- www/py-hyperframe
- www/py-multidict
- www/py-pylsqpack
- www/py-requests
- www/py-requests-toolbelt
- www/py-yarl
No port depends on this one.
Configuration
No options to configure.
- USES
- python
- Service scripts
- none
Files
distinfo 1
| File | Size | SHA256 |
|---|---|---|
| dns_crawler-1.6.5.tar.gz | 45203 | f4ee4e98b7fae84e71e51422736ff1cdcf3f01653b5c6f73cb190f79b46115e4 |
Master sites 1
- PYPI
Commit history 0
No commits in the history this site was built from.