Tiktokenizer

Online playground for openai/tiktoken, calculating the correct number of tokens for a given prompt.

Special thanks to Diagram for sponsorship and guidance.

Acknowledgments

• T3 Stack
• shadcn/ui
• openai/tiktoken