The world of AI tokens — and why they matter

Google has only one way to measure the phenomenal AI growth it’s seen: in tokens. The company processes 3.2 quadrillion tokens per month, Google CEO Sundar Pichai said during this week’s I/O keynote , adding, “never imagined I’d say quadrillion…, but here we are.” Basically, tokens are a unit of measure used by large language models (LLMs) to process data. Tokens, which have been called the “new oil” fueling the AI revolution , are also a way AI vendors can meter usage and price their services. Enterprises are lusting for tokens, and spending billions of them to grab compute time. As with oil, the demand for tokens is seemingly insatiable — and it is straining an already short GPU supply, which in turn is increasing the cost of running AI tools. What exactly is a token? Similar to the way humans think, LLMs grasp the meaning of a sentence by breaking words down into tokens. Pichai described them as “the fundamental units of data our models process, many representing a problem being sol