README.md

metadata

title: Token Detective
emoji: 👁
colorFrom: red
colorTo: red
sdk: gradio
sdk_version: 6.0.1
app_file: app.py
pinned: false
license: mit
short_description: Explore how different tokenisers handle rare symbols

For local usage, clone the repository and run:

uv venv
uv pip install -r requirements.txt
uv run app.py

Then open your browser by clicking the link provided in the terminal (default: http://localhost:7860).