www/colly 2.3.0_2 FreeBSD
Elegant scraper and crawler framework for Golang
- System
- FreeBSD
- Repository
- FreeBSD ports
- Maintainer
- [email protected]
- Last update
- Commit
d81b7ba3c3- License
- APACHE20
- Categories
- www
- Package size
- 38663.1 KiB, 128422.9 KiB installed
Description
With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.
Features: * Clean API * Fast (>1k request/sec on a single core) * Manages request delays and maximum concurrency per domain * Automatic cookie and session handling * Sync/async/parallel scraping * Distributed scraping * Caching * Automatic encoding of non-unicode responses * Robots.txt support * Google App Engine support
In the other systems
| System | Port | Version |
|---|---|---|
| FreeBSD | www/colly | 2.3.0_2 |
| DragonFly | www/colly | 2.1.0_23 |
Install
Binary package:
pkg install colly
From the ports tree:
cd /usr/ports/www/colly/ && make install clean
To depend on this port in another port's Makefile:
RUN_DEPENDS= colly>0:www/colly
Packages
The version each package repository carries. “-” means the repository builds no package of it, “n/a” that there is no such repository.
| Repository | aarch64 | amd64 | armv6 | armv7 | i386 |
|---|---|---|---|---|---|
| FreeBSD:14:latest | 2.3.0_1 | 2.3.0_1 | - | 2.1.0_28 | 2.3.0_1 |
| FreeBSD:14:quarterly | 2.3.0_1 | 2.3.0_1 | n/a | 2.1.0_26 | 2.3.0_1 |
| FreeBSD:15:latest | 2.3.0_1 | 2.3.0_1 | n/a | - | 2.1.0_28 |
| FreeBSD:15:quarterly | 2.3.0_1 | 2.3.0_1 | n/a | n/a | n/a |
| FreeBSD:16:latest | 2.3.0 | 2.3.0_1 | n/a | n/a | n/a |
| FreeBSD:16:quarterly | n/a | n/a | n/a | n/a | n/a |
Dependencies
The Makefile names none itself; they come from USES and USE_*.
The full lists below include those, and their dependencies in turn.
Build dependencies 1
No port depends on this one.
Configuration
No options to configure.
- USES
- go:modules
- Service scripts
- none
Files
distinfo 2
| File | Size | SHA256 |
|---|---|---|
| go/www_colly/colly-v2.3.0/v2.3.0.mod | 905 | a64462c41ce3b20598ad9fca23e7f5f13fa00699b1f37d895f4dfd5fdc51e236 |
| go/www_colly/colly-v2.3.0/v2.3.0.zip | 122628 | ebdaad1d7ed2876081063d05af8b5c36cb5267710882c0bb828c2aad58bc252d |
Commit history 7
The latest 5 of 7 in the history this site was built from.