Count every adjacent pair, fuse the most frequent one, throw
the counts away, count again — until the vocabulary is as big as you asked for.
That loop is the whole of byte-pair encoding.
It starts from the 256 byte values rather than from letters, so
anything at all can be encoded — an accented character simply begins life as
two separate symbols. Text is cut into chunks GPT-2 style first, so a space
belongs to the word after it and ␣the is a different token from
the. Encoding, at the bottom, replays the merges in the order
they were learned and never counts anything, and then hands on the one thing
the model actually receives: a row of numbers.
Try text with different habits — code, another language, your own writing — and watch which pairs rise.