mbpe channel

Byte Pair Encoding tokenizer for Mojo — train, encode, decode, save, load.

A binary package from the modular-community channel — latest version 0.1.1. Maintained by ratulb. Repository / details: ratulb/mbpe.

⭐ 8 · last commit 8d ago · 23 commits last month · 235 last year

Install — pick one

with the shelf extension

pixi shelf add mbpe

or plain pixi (channel in your channel list)

pixi add mbpe

Dependencies are resolved by conda — no registry pinning; your pixi.lock records the solved version. Note: channel packages carry their own mojo-compiler requirement, which may conflict with a workspace pinned to a different Mojo era — the solver reports it if so.