The basic unit of text that LLMs process, roughly equivalent to three-fourths of a word in English. Token limits affect how much context can be provided to and generated by an AI model.
— Vibe Coding, Gene Kim, Steve Yegge
Token
2 of our books define this term — and their definitions are worth reading side by side.
a physical token, stick, marker, Zoom icon, etc. that shows who has the designation/right to speak at the moment
— Practice Makes Culture, in “Chapter 6: Invite Others to Practice”, Christine Hudson, Ronica Roth
See also Token Window · Agent · Agentic coding · Chat coding · ChatGPT · CHOP (Chat-Oriented Programming)
Filed under AI & Software Engineering