BSD Portstextproc/sentencepiece 0.2.1 MidnightBSD
Unsupervised text tokenizer for Neural Network-based text generation
- System
- MidnightBSD
- Repository
- mports
- Maintainer
- [email protected]
- Port added
- Last update
- Commit
8af06753b2- License
- Apache-2.0
- Categories
- textproc
Description
SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training. SentencePiece implements subword units (e.g., byte-pair-encoding (BPE)) and unigram language model with the extension of direct training from raw sentences. SentencePiece allows us to make a purely end-to-end system that does not depend on language-specific pre/postprocessing.
In the other systems
The newest version among them is 0.2.2, as upstream numbers it.
| System | Port | Version | Vulnerabilities | Status | Maintainer | Last update |
|---|---|---|---|---|---|---|
| MidnightBSD | textproc/sentencepiece | 0.2.1 behind since 2026-07-13 | no data | - | [email protected] | 2026-07-05 |
| FreeBSD | textproc/sentencepiece | 0.2.2 newest | none known | - | [email protected] | 2026-07-13 |
| NetBSD | textproc/sentencepiece | 0.2.1 behind since 2026-07-13 | none known | - | - | - |
| DragonFly | textproc/sentencepiece | 0.2.0_1 behind since 2026-07-05 | none known | - | [email protected] | - |
| HardenedBSD | textproc/sentencepiece | 0.2.2 newest | no data | - | [email protected] | - |
| GhostBSD | textproc/sentencepiece | 0.2.2 newest | no data | - | [email protected] | - |
Version history 1
Upstream's version as each commit to the port's Makefile set it, or as this site first saw it in the system's packages; the tree's own revisions are not versions here.
Install
Binary package:
mport install sentencepieceFrom mports:
cd /usr/mports/textproc/sentencepiece && make install cleanTo depend on this port in another port's Makefile:
RUN_DEPENDS= sentencepiece>0:textproc/sentencepiece
LIB_DEPENDS= libsentencepiece.so:textproc/sentencepieceDependency graph
Ports that need it on the left, what it needs to run on the right; at most 8 of each. "Dependencies" below lists them all.
Dependencies
Required by 1
Configuration
Options name: textproc_sentencepiece
| Option | Default | Description |
|---|---|---|
| GPERFTOOLS | off |
- USES
- cmake:testing compiler:c++17-lang
- Service scripts
- none
Files
pkg-plist 16
- bin/spm_decode
- bin/spm_encode
- bin/spm_export_vocab
- bin/spm_normalize
- bin/spm_train
- include/sentencepiece_processor.h
- include/sentencepiece_trainer.h
- lib/libsentencepiece.a
- lib/libsentencepiece.so
- lib/libsentencepiece.so.0
- lib/libsentencepiece.so.0.0.0
- lib/libsentencepiece_train.a
- lib/libsentencepiece_train.so
- lib/libsentencepiece_train.so.0
- lib/libsentencepiece_train.so.0.0.0
- libdata/pkgconfig/sentencepiece.pc
distinfo 1
| File | Size | SHA256 |
|---|---|---|
| google-sentencepiece-v0.2.1_GH0.tar.gz | 1.3485527e+07 | c1a59e9259c9653ad0ade653dadff074cd31f0a6ff2a11316f67bee4189a8f1b |
Commit history 1
| Date | Commit | Credits | Log message |
|---|---|---|---|
8af06753b2 |
Lucas Holt | databases/rubygem-flipper-active_record-rails70: update to 1.3.6 Match the flipper-gitlab provider series so fake-qa can satisfy the gemspec flipper dependency.
AI-Assisted-by: OpenAI Codex <[email protected]>
Signed-off-by: Lucas Holt <[email protected]> |