ChipokiaTech news, without the noise
← All stories

ttok 1.0

Release: ttok 1.0 I released ttok 0.4 , ran uv tool upgrade ttok , piped a file into the new version... and realized that it was defaulting to the GPT-4 tokenizer when it should very clearly default to GPT-5/GPT-6 instead! I figured switching the default was…

Preview courtesy of Simon Willison. The full article opens on their site.

More from Simon Willison