BSD Portstextproc/py-tokenizer 3.6.4 DragonFly
Tokenizer for Icelandic text
- System
- DragonFly
- Repository
- DPorts
- Maintainer
- [email protected] (as in FreeBSD)
- License
- MIT
- Categories
- python, textproc
- Package size
- 120.0 KiB, 648.8 KiB installed
Description
Tokenizer: A tokenizer for Icelandic text
Tokenization is a necessary first step in many natural language processing tasks, such as word counting, parsing, spell checking, corpus generation, and statistical analysis of text.
In the other systems
The newest version among them is 3.6.4, as upstream numbers it.
| System | Port | Version | Vulnerabilities | Status | Maintainer | Last update |
|---|---|---|---|---|---|---|
| DragonFly | textproc/py-tokenizer | 3.6.4 newest | none known | - | [email protected] | - |
| FreeBSD | textproc/py-tokenizer | 3.6.4 newest | none known | - | [email protected] | 2026-07-12 |
| HardenedBSD | textproc/py-tokenizer | 3.6.4 newest | no data | - | [email protected] | - |
| GhostBSD | textproc/py-tokenizer | 3.6.4 newest | no data | - | [email protected] | - |
Version history 1
- 3.6.4
Upstream's version as each commit to the port's Makefile set it, or as this site first saw it in the system's packages; the tree's own revisions are not versions here.
Install
Binary package:
pkg install py312-tokenizerFrom DPorts:
cd /usr/dports/textproc/py-tokenizer && make install cleanDependency graph
Ports that need it on the left, what it needs to run on the right; at most 8 of each. "Dependencies" below lists them all.
Dependencies
Run dependencies 1
No port depends on this one.