BSD Ports

textproc/py-tokenizers 0.23.1p0 OpenBSD

tokenizers with a focus on performance

System
OpenBSD
Repository
OpenBSD ports
Maintainer
The OpenBSD ports mailing-list <[email protected]>
Port added
Last update
Commit
5fbf97e5f7
Categories
textproc, lang/python, lang/rust

Description

An implementation of today's most used tokenizers, with a focus on performance.

In the other systems

SystemPortVersion
OpenBSDtextproc/py-tokenizers0.23.1p0
FreeBSDtextproc/py-tokenizers0.23.3_1
DragonFlytextproc/py-tokenizers0.21.1
MidnightBSDtextproc/py-tokenizers0.23.1_2

Install

Binary package:

pkg_add py3-tokenizers

From the ports tree:

cd /usr/ports/textproc/py-tokenizers && make install

Dependencies

Build dependencies 6
Run dependencies 2

No port depends on this one.

Commit history 3

DateCommitCreditsLog message
5fbf97e5f7 sthen py-tokenizers ports fixes, based on part of a 20k line email from David Uhden Collado: - add missing LDEP and WANTLIB - override setting --manifest-path, otherwise Maturin misses the Python metadata outside the Rust workspace and doesn't install properly
4cbaf6b116 kmos py-tokenizers uses C++11 so move to ports-gcc on base-gcc arches to fix build on sparc64
52ae587db3 daniel import py-tokenizers; ok kirill@