What Is Generative AI? A Complete Beginner's Guide

Generative AI is software that produces new content, such as text, images, audio, video or code, in response to an instruction. It does this by learning the patterns in very large amounts of existing content and then generating output that follows those patterns. When you ask a chatbot to draft an email, or an image tool to draw "a lighthouse in watercolour", you are using generative AI.

The US National Institute of Standards and Technology uses a more precise definition in its Generative AI Profile (NIST AI 600-1, July 2024): "the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content." The key words are "emulate" and "derived". A generative model does not look answers up. It produces output that resembles what it learned from. That single fact explains both what these systems are good at and where they fail.

Generative versus the AI that came before

Most machine learning before generative AI was discriminative: it assigns a label or a number to an input. A spam filter decides "spam" or "not spam". A credit model outputs a risk score. The output is chosen from a small, fixed set of answers.

A generative model's output is open-ended. Asked to "summarise this contract", it has to produce a sequence of words that did not exist before, one piece at a time. That makes it far more flexible, and far harder to check, because there is no fixed list of correct answers to compare against. Our comparison of generative AI and machine learning goes further into how the two relate.

How a text model generates an answer

Large language models (LLMs), the models behind chat assistants, work in a loop:

1. The input text is split into tokens: words or pieces of words, each mapped to a number.
2. The model reads all the tokens so far and computes a probability for every token that could come next.
3. One token is chosen from that distribution, usually by sampling rather than always taking the most likely one.
4. The chosen token is added to the input, and the loop repeats until the answer is complete.

You can see the principle in a few lines of Python. The model below is a toy: it only counts which word follows which in a two-sentence text. Real LLMs are vastly more capable, but the generation loop is the same idea.

import random
from collections import Counter, defaultdict

text = """the model reads the prompt . the model predicts the next word .
the next word is added to the prompt . then the model predicts again ."""
tokens = text.split()

# Count which token follows which: a tiny "bigram" language model.
follows = defaultdict(Counter)
for current, nxt in zip(tokens, tokens[1:]):
    follows[current][nxt] += 1

def probabilities(token):
    counts = follows[token]
    total = sum(counts.values())
    return {t: round(c / total, 2) for t, c in counts.items()}

def generate(start, steps, seed):
    rng = random.Random(seed)
    out = [start]
    for _ in range(steps):
        options = follows[out[-1]]
        out.append(rng.choices(list(options), weights=list(options.values()))[0])
    return " ".join(out)

print(probabilities("the"))
print(generate("the", 8, seed=1))
print(generate("the", 8, seed=2))

{'model': 0.43, 'prompt': 0.29, 'next': 0.29}
the model predicts again . the prompt . then
the next word . the next word is added

Three things in that output carry over to real models. After "the", the model has a probability distribution over possible next words, not a single answer (the probabilities are rounded). Two runs from the same starting point produce different text, because each word is sampled. And the output is locally fluent but can drift into nonsense, because the model only follows statistical patterns and has no idea what it is talking about.

What separates an LLM from this toy is the context it can use and what it has learned. The transformer architecture, introduced in the 2017 paper "Attention Is All You Need", lets a model weigh every earlier token when predicting the next one, not just the previous word. Trained on enormous amounts of text, that is enough to capture grammar, facts, reasoning patterns and style. Our explainers on how LLMs work under the hood and why tokenization matters go deeper into both steps.

How the models are trained

Training a chat model usually happens in stages:

• Pretraining. The model learns to predict the next token across a very large text corpus. This is where it picks up language and general knowledge. It is also the most expensive stage.
• Instruction tuning and feedback. A pretrained model continues text; it doesn't follow instructions well. It is then fine-tuned on examples of instructions and good responses, and further adjusted using human preference ratings. OpenAI's InstructGPT paper describes this approach, reinforcement learning from human feedback (RLHF). Tuning of this kind is what makes an assistant answer a question instead of just continuing your sentence.

Two consequences follow. A model knows nothing about events after its training data was collected, its knowledge cutoff, unless an application gives it newer information. And its behaviour reflects its training data, including that data's gaps and biases.

Images, audio and video

Image generators mostly use a different technique, diffusion. A diffusion model is trained to remove noise from images step by step. To generate a picture, it starts from pure noise and repeatedly denoises it, guided by your text prompt, until an image emerges. The method was popularised by the paper Denoising Diffusion Probabilistic Models (2020). Our diffusion models explainer walks through the process. Audio and video generation build on the same families of ideas, adapted to sound and moving images.

What it is good at, and what it gets wrong

The central weakness has a formal name. NIST calls it confabulation: "the production of confidently stated but erroneous or false content", colloquially known as hallucinations. It happens because the model generates what is plausible, and a plausible-sounding wrong answer is still plausible. The model has no built-in way to tell a fact it learned from a pattern it is continuing.

The practical rules follow from that. Verify claims that matter against a primary source. Give the model the source material, rather than relying on what it remembers. And treat its output as a draft for a person to check, not a final answer. Our guide to reducing hallucinations covers the techniques applications use.

How applications build on generative models

Most applications built on generative AI don't train their own models. They combine an existing model with other components:

• Prompting: careful instructions and examples in the input. See prompt engineering techniques.
• Retrieval-augmented generation (RAG): search the organisation's own documents first and give the relevant passages to the model, so answers are grounded in real, current sources.
• Fine-tuning: further training on domain examples when prompting is not enough.
• Tools and agents: letting the model call functions, such as a search API, a calculator or a database, and act across several steps.

The risks worth knowing about

NIST's profile lists twelve risks that generative AI creates or makes worse. They include confabulation, data privacy (leaking or misusing personal information), harmful bias, information integrity (convincing misinformation at scale), information security, intellectual property, and over-reliance by people who trust outputs too readily. For everyday users, three habits cover most of it:

• Don't paste confidential data into tools your organisation hasn't approved.
• Check facts and sources before you act on or publish an output.
• Keep a person accountable for any decision the output informs.

Where to go from here

If you want to use generative AI better, practise prompting on real tasks and learn to spot confabulation. If you want to build with it, learn Python, then how to call a model API, then retrieval. That sequence is a common path into generative AI application work. If you decide to build with it, the GloryTecks Generative AI course works through LLM applications with LangChain and retrieval-augmented generation.