A Concise Glossary of Artificial Intelligence: The Terms You Need

Why can a conversation about artificial intelligence fail to reach agreement? Because a word like "model" or "inference" means two things to two people. This is a concise glossary of the field's terms in five groups; for the big concepts see "Artificial Intelligence, Really: What Is It, and What Is It Not?" and "Large Language Models".

Foundations

Artificial intelligence: a machine-based system that infers from its input how to generate outputs such as predictions, content, recommendations or decisions, per the OECD definition revised in 2023. Machine learning: systems extracting their patterns from data instead of from rules written in advance. Deep learning: machine learning that uses multi-layered neural networks. Neural network: a mathematical structure of linked units whose idea was inspired by nerve cells, not a copy of them. Learning comes in three kinds: supervised, on examples labelled with their answers; unsupervised, finding structure in unlabelled data; and reinforcement, which refines behaviour through reward and penalty.

Building Models

Transformer: an architecture introduced by the 2017 paper "Attention Is All You Need", relying on attention, a mechanism that weighs how relevant each part of the input is to the rest without processing it in sequence. Parameters: the numerical weights that training adjusts and that hold what the model has learned. Token: the unit of text the model processes, which may be a word or a part of one. Context window: how much the model can bring to mind at once. Embedding: a numerical representation (a vector) of a word or image in which similar meanings lie close together.

Pre-training: general learning from a large amount of data. Fine-tuning: adapting a model to a task with narrower data. Reinforcement learning from human feedback (RLHF): refining a model's behaviour with the preferences of human raters; a 2022 study found raters preferred the outputs of a small model refined this way over those of a model about a hundred times larger that had not been. Alignment: the effort to make a system's behaviour match human goals and values. Scaling laws: empirical relations linking better performance to more size, data and computing.

Use

Prompt: the request with which we address the model. Zero-shot prompting: a task with no examples, and few-shot prompting: a task with samples to follow. Chain of thought: the model laying out its intermediate reasoning steps before its answer. Temperature: a setting that controls the randomness of outputs. Hallucination: an output that sounds confident but is wrong or unsupported. Retrieval-augmented generation (RAG): combines the model's memory with an index of documents it searches, as the 2020 paper described, making the answer easier to source and the knowledge easier to update, without eliminating error.

Agent: a system that plans and carries out steps towards a goal with minimal human intervention. Tool calling: letting the model run other software. Multimodal: handling text, image and sound together. Inference is ambiguous: it is either systematic logic, or running a trained model to produce an answer.

Measurement and Efficiency

Benchmark: a standard test against which models are compared, with results affected by how it is run. Emergent abilities: skills that appear in large models and are absent in small ones, according to a 2022 paper; a 2023 study argued that this sudden appearance may stem from a non-gradual metric rather than a real phenomenon, so the matter is disputed. Distillation: transferring what a large model has learned to a smaller one. Quantisation: compressing weights so they run on limited hardware. Edge computing: running the model on the device itself rather than on a distant server. Graphics processing unit: a chip for parallel computation on which training rests.

Risks and Governance

Bias: outputs skewed towards unjust patterns the model inherited from its data. Drift: accuracy falling when reality changes from what the model was trained on. Black box: a system whose decision is hard to explain, hence explainable AI. Adversarial attack: inputs designed to deceive the model. Prompt injection: instructions slipped into text the model reads in order to hijack its behaviour. Deepfake: a convincing synthetic voice or image. For detail see "Algorithmic Bias" and "AI and Cybersecurity".

Artificial general intelligence: a hypothetical system matching humans at any mental task, and superintelligence: what exceeds it; both are matters of debate, not measured reality. Open-weight models have weights available to download, as against closed ones behind commercial interfaces.

These definitions do not replace the most important question when someone speaks with them: by what benchmark is the claim measured? Perhaps the most useful thing in a glossary is that it helps us ask with precision.