What is a language model

In short#

A large language model (LLM) is a neural network trained on a huge amount of text and code. Its core skill is predicting the next piece of text. Everything else grows from that simple skill: answering questions, summarizing, translating, writing code, reasoning step by step.

A model does not «look the answer up in a database» and does not «remember the internet» word for word. It picks the most likely continuation of what it sees right now: your question, the rules, the files passed to it.

Tokens#

A model reads and writes not in letters but in tokens — pieces of words. In English one token is about three or four letters; Russian text usually splits into smaller tokens, so the same meaning costs a few more tokens in Russian.

Tokens are also money: providers charge separately for the tokens the model reads (input) and writes (output). Output is usually several times more expensive than input.

Context#

The context (context window) is how many tokens the model sees at once: the rules, the conversation history, files and the question itself. Current models have windows from tens of thousands to a million tokens. Whatever does not fit, the model does not see.

In SKILLOK the window is capped at 128 thousand tokens: a long history is compressed into a summary, and large tool results are trimmed. That keeps answers fast and inexpensive.

What a model does well#

• Explaining and summarizing: a topic, an article, someone else's code.
• Writing drafts: text, code, tests, self-check questions.
• Working out structure: finding a flaw in reasoning, proposing a plan.
• Using tools: reading repository files, searching, opening issues, proposing changes.

Where a model makes mistakes#

• It makes things up («hallucinates»): it confidently names a fact, a link or a function that does not exist. Check what matters against the source.
• It does not know recent events: a model's knowledge ends at its training date. Fresh data has to be passed to it — as a file or through a web search.
• It counts inaccurately: long arithmetic is better left to code than to the model.
• It depends on wording: the more precise the question and the more relevant context, the better the answer.

«Thinking» models and ordinary ones#

Some models reason before answering — they write a hidden chain of steps. They solve math, logic and code problems better but answer more slowly and cost more: the reasoning is paid for in tokens too. A fast model without reasoning is enough for simple questions.

What next#

• How to choose
• Connecting in SKILLOK

Was this helpful?

Yes No