AI glossary: 30 terms for decision-makers, briefly explained

This glossary explains 30 AI terms that come up in quotes, contracts and project meetings. Each entry describes what the term means and what you should look out for as a decision-maker. The terms are grouped by topic, not in alphabetical order.

Models and technology

Artificial intelligence (AI)

An umbrella term for software that performs tasks that previously required human judgement, such as understanding text, recognising images or preparing decisions. When businesses talk about AI today, they usually mean language models.

Language model (LLM)

A large language model is a model that has been trained on a very large amount of text and generates text on that basis. Well-known model families are GPT from OpenAI, Claude from Anthropic, Gemini from Google and open models such as Llama or Mistral. For your project, the name matters less than where the model runs and who can see your inputs.

Prompt

The instruction to a language model: task, context and desired format. In a finished AI solution, prompts are part of the software. They should be versioned, tested and handed over to you like code.

Token

The unit in which language models process and bill text. A token roughly corresponds to part of a word; in German it is often three to four characters. Model costs are calculated per million tokens for input and output.

Context window

The amount of text a model can take into account at once in a single request, measured in tokens. A large context window makes it possible to pass long contracts or entire files in one request. More context costs more, though, and does not automatically make answers more accurate.

Hallucination

An answer that sounds convincing but is wrong or made up. Hallucinations cannot be prevented entirely, but they can be contained through source references, validation rules and human approval before steps with consequences.

Inference

Running a fully trained model, for example for a single request. When a provider talks about inference in Germany, it means that requests are processed in a German data centre.

Open-weight model

A model whose weights have been published, so that you can run it on your own servers. The weights are the model parameters learned during training. Running it yourself helps with sensitive data but requires suitable hardware and someone to operate and update the model.

Fine-tuning

Further training of a model with your own examples so that it matches a style or format more closely. It is not needed for most SME projects. Good prompts and RAG are almost always enough and are easier to change.

Embedding

A series of numbers that describes the meaning of a section of text. Similar content has similar embeddings. This allows a search to find relevant passages even when different words are used.

RAG (retrieval-augmented generation)

Before the model answers, a system finds relevant sections from your own documents and passes them to the model. The answer is then based on your knowledge and can name the source. Most knowledge assistants in businesses work this way.

OCR and document processing

OCR stands for optical character recognition and recognises text in scans and photos. Modern solutions also capture the structure of a document and work out which figure is the invoice amount and which is the order number. This is the basis for invoice checking and order entry.

Applications

AI assistant

A chat tool for individuals, such as ChatGPT, Claude or Microsoft 365 Copilot. It helps with writing, summarising and searching. An assistant does not take over a process, though; the person steers the work.

AI solution

Software that takes over or prepares a specific task in the business, for example creating drafts from order emails in the ERP system, the software used to plan and manage business operations. It is connected to your systems, has checkpoints and is tested on your real cases.

AI agent

A program that uses a language model to plan intermediate steps itself and uses tools, such as a search, a database or an interface to customer management (CRM). The more an agent is allowed to do on its own, the more important tightly limited permissions, logs and approvals before externally visible steps become.

Skill

A package of instructions, examples and templates that describes a recurring task to an AI assistant, such as "create a quote in the company format". Skills are described in a file named SKILL.md. Claude reads this format, and Microsoft and OpenAI now support it as well. Our Skill Hub shows how businesses collect and approve skills.

MCP (Model Context Protocol)

An open standard through which AI assistants access tools and data, such as a ticketing system or a database. An MCP server provides these tools. For every server, check what permissions it has and who operates it.

Copilot agent

An agent built into Microsoft 365 Copilot with fixed instructions and selected knowledge sources, such as a SharePoint folder. It is approved for the organisation in the Microsoft 365 admin centre.

Operation and control

Human-in-the-loop

A person reviews and decides at defined points before a result is sent out or booked. Our article on human-in-the-loop explains how to define these points sensibly.

Guardrails

Rules that limit an AI solution, for example permitted topics, mandatory fields, amount limits or blocked actions. They catch known errors but do not replace review by people.

Evaluation

Systematic testing of an AI solution against a fixed collection of real cases with a known correct answer. An evaluation shows how often the solution is right and whether a model update makes anything worse.

On-premises

Operation on the company's own servers instead of in a cloud. This gives the most control over data but requires hardware and effort to operate.

Shadow AI

AI tools that employees use without approval, often with private accounts. Bans rarely help. An approved tool, clear rules and training work better.

Law and data protection

AI Act (KI-Verordnung)

Regulation (EU) 2024/1689 regulates AI systems according to risk. For most office applications in SMEs, the main obligations concern AI literacy and transparency. Strict obligations apply to high-risk systems.

AI literacy

Under Art. 4 of the AI Act, providers and deployers of AI systems take measures to promote the AI literacy of their staff. Since the amendment by Regulation (EU) 2026/1744 in July 2026, this is an obligation to make an effort, not a prescribed level. In practice, this means training suited to each role and an internal record. A template for this is in the article on the AI training record.

High-risk AI

AI systems in areas that the AI Act names explicitly, for example selecting job applicants, creditworthiness or critical infrastructure. They are subject to obligations on risk management, documentation and human oversight. Typical back-office processes such as invoice checking or order entry are generally not among them.

AVV (Auftragsverarbeitungsvertrag, data processing agreement)

A contract under Art. 28 GDPR with every service provider that processes personal data on your behalf, which includes the provider of the language model. Without an AVV, no personal data should enter the system.

Data residency

A provider's commitment to store and process data only in a specific region, such as the EU or Germany. Ask whether the commitment applies to stored data, to the processing of requests, or to both.

DSFA (Datenschutz-Folgenabschätzung, data protection impact assessment)

An assessment under Art. 35 GDPR that is required when processing is likely to result in a high risk to the people concerned. For AI systems involving personal data, your data protection officer should check early on whether a DSFA is needed.

The works council and AI

If a company introduces a technical system that is capable of monitoring the behaviour or performance of employees, the works council (Betriebsrat) has a right of co-determination under § 87(1) no. 6 BetrVG (Works Constitution Act). This applies to many AI tools. Involve the works council early and avoid evaluating individuals where it is not necessary.

If a term from a quote is missing, write to us via the contact form. We will add it to the glossary.