Token In natural language processing, a token is the smallest unit of text that a computer processes. The unit of text may correspond to a word, a syllable, a character, or—in languages like Korean—a morpheme. For example, the word “사과” (“apple”) may be treated as one token, while a word like “unhappy” may be split into two tokens: “un” and “happy.” The method of tokenization varies by model, so the same sentence can produce different numbers of tokens depending on the system. Since many generative AI services measure usage in terms of tokens, token counts often directly affect costs for users. Embedding Embedding refers to a technique (or the resulting numerical representation) that converts data such as text, images, or audio into an array of numbers—a vector—that a computer can process. AI models transform words, sentences, images, and other inputs into embeddings when recognizing or generating content. Embeddings act as a kind of translator, simplifying complex data patterns and clarifying relationships for the model. They are also essential in multimodal models, serving as the foundation for tasks such as converting between text and images. Data represented as embeddings are numerical expressions of their meanings or features. This allows computations such as determining that the distance between the embeddings for “dog” and “cat” is smaller than 26 27