BSD PortsFive ports trees on one site
Until today this site knew one tree, FreeBSD's. It now reads five: FreeBSD's ports, OpenBSD's ports, NetBSD's pkgsrc, DragonFly BSD's DPorts and MidnightBSD's mports. That is about 105,000 ports and 110,000 pages, rebuilt every 30 minutes.
Each tree, read the way it is published
No two of them publish their tree the same way, so each is read from what its project already ships:
| System | Read from | Gives |
|---|---|---|
| FreeBSD | the INDEX, the ports tree, pkg.freebsd.org |
versions, options, files, history, package versions per release |
| OpenBSD | sqlports, the SQLite database of the whole tree, shipped as a package |
every Makefile variable, dependencies by kind |
| NetBSD | pkg_summary of the binary packages, the pkgsrc tree |
versions, licences, run dependencies; maintainers and history from the tree |
| DragonFly | packagesite.pkg, as pkg reads it |
versions, sizes, run dependencies |
| MidnightBSD | the mports tree, Makefile by Makefile | everything a Makefile states |
Two of these needed more than a parser. OpenBSD signs its packages with
signify, which keeps the signature, a hash per 64 KB block, in the gzip
header's comment field. For the 46 MB sqlports package that comment runs to
tens of kilobytes, and Go's compress/gzip refuses any header field longer
than 512 bytes. The header is now skipped by hand and the body inflated as
it is. MidnightBSD's package index names packages but not their origins, so
mports is read as a tree, the same way an overlay repository is.
The same software, side by side
A port is now identified by its system and its origin, so www/nginx is five
ports with five pages: FreeBSD's at /www/nginx/, as before, the others
under the system's own path, such as /openbsd/www/nginx/. Each page lists
the same software in the other systems:
| System | Port | Version |
|---|---|---|
| FreeBSD | www/nginx |
1.30.6_1,3 |
| OpenBSD | www/nginx |
1.30.6 |
| NetBSD | www/nginx |
1.30.5 |
| DragonFly | www/nginx |
1.26.3_3,3 |
| MidnightBSD | www/nginx |
1.30.1,3 |
The match is the same origin first, else the one port of the same package
name in that system. Where a system has two ports of one name, lua51 and
lua54 say, nothing is matched, since either could be the wrong one.
What the data turned up
Reading five trees at once finds what reading one never does:
- MidnightBSD's maintainer lines spell its ports address about a dozen
ways, among them
[email protected]andports@MidnightBSD>org. Spellings that make the same page now share one maintainer page. - DPorts copies FreeBSD's maintainer lines, so a FreeBSD maintainer would appear to maintain every port twice. DragonFly's ports show the maintainer but stay off the maintainer pages.
- More than a thousand DragonFly packages give their homepage as
UNKNOWN. A homepage is now linked only when it is a URL.
Every system has its own tab in the header, its own page of categories and
its own install commands on each port page: pkg install, pkg_add,
pkgin install, mport install. The search box searches all five, and the
benchmarks page shows what a refresh of them costs.