jmoney/go-bpe-tokenizer

BPE Tokenization in go. Meant as a code for learning and not performance or production.