Hugging Face Transformers: A Practical Getting-Started Guide

Thousands of trained models are free to download, and most of them are published on the Hugging Face Hub. Hugging Face Transformers is the Python library that loads and runs them. Its documentation describes it as the model-definition framework for current machine learning models across text, vision and audio, and counts over a million model checkpoints on the Hub (Hugging Face: Transformers).

You can use it at two levels. A pipeline does a whole task in one line. Underneath, a tokenizer and a model do the work, and knowing them is what lets you go beyond the defaults. This guide runs both on a laptop.

The pieces

Each model is used with its own tokenizer. Loading both from the same name on the Hub is what keeps them matched.

Install

pip install transformers torch

Models are downloaded on first use and cached, so the second run starts quickly and works offline.

Running models

The script does three things. It classifies three short reviews with a sentiment pipeline. It repeats the first classification by hand, to show what the pipeline does inside. And it uses a second pipeline for a different task, text generation with GPT-2 (Hugging Face: GPT-2).

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline

MODEL = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
reviews = ["The course projects were genuinely useful.",
           "The instructions were confusing and the video kept freezing.",
           "It was fine, nothing special."]

# 1. The one-line way: a pipeline downloads the model and its tokenizer and runs both.
classify = pipeline("sentiment-analysis", model=MODEL)
for review, result in zip(reviews, classify(reviews)):
    print(f"{result['label']:8} {result['score']:.3f}  {review}")

# 2. The same thing by hand, which is what the pipeline does inside.
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL)
batch = tokenizer(reviews[0], return_tensors="pt")
print()
print("token ids:", batch["input_ids"][0].tolist())
print("tokens:   ", tokenizer.convert_ids_to_tokens(batch["input_ids"][0]))
with torch.no_grad():
    scores = model(**batch).logits[0]
probabilities = torch.softmax(scores, dim=-1)
print("raw scores:", [round(s, 3) for s in scores.tolist()],
      " probabilities:", {model.config.id2label[i]: round(p, 4) for i, p in enumerate(probabilities.tolist())})
print(f"parameters: {sum(p.numel() for p in model.parameters()):,}")

# 3. A different task is a different model behind the same interface.
generate = pipeline("text-generation", model="openai-community/gpt2")
print()
print(generate("Hugging Face makes it easy to", max_new_tokens=12, do_sample=False)[0]["generated_text"].strip())

POSITIVE 1.000  The course projects were genuinely useful.
NEGATIVE 0.999  The instructions were confusing and the video kept freezing.
NEGATIVE 0.958  It was fine, nothing special.

token ids: [101, 1996, 2607, 3934, 2020, 15958, 6179, 1012, 102]
tokens:    ['[CLS]', 'the', 'course', 'projects', 'were', 'genuinely', 'useful', '.', '[SEP]']
raw scores: [-3.785, 4.047]  probabilities: {'NEGATIVE': 0.0004, 'POSITIVE': 0.9996}
parameters: 66,955,010

Hugging Face makes it easy to see how the game is going to play out.

What the output shows

• The pipeline handled everything, from downloading the model to returning a label and a confidence.
• The third review exposes a limit of the model, not of the code. "It was fine, nothing special" is neutral, but this model knows only two labels, so it called the review negative with 95.8% confidence. Confidence is not correctness.
• By hand, the sentence became nine numbers, with special tokens marking its start and end. The tokens are lower case because this model is uncased.
• The model returned two raw scores. A softmax turned them into probabilities of 0.04% negative and 99.96% positive. Those are the numbers the pipeline reported, rounded.
• The classifier has 67 million parameters. It is a distilled model, built to keep most of a larger model's ability at a fraction of the size.
• The generator continued the prompt plausibly and pointlessly. GPT-2 is small and old. Choosing a better model is the subject of the next section.

The pipeline, and when to drop below it

A pipeline is named by its task, such as sentiment-analysis, text-generation, summarization or automatic-speech-recognition. Every task has a default model, and you should not rely on it: pass the model you have chosen explicitly, as the script does, so that results do not change when the default does (Hugging Face: pipeline). Inputs can be passed as a list, as the reviews were.

Drop below the pipeline when you need what it hides:

• The raw scores, for your own thresholds or for combining models.
• Batching and padding control for throughput.
• Embeddings from inside the model.
• Training, which starts from the model class, not the pipeline.

The classes used by hand, AutoTokenizer and AutoModelFor..., load the right architecture for any model name through from_pretrained (Hugging Face: quickstart).

Choosing a model

The model card is where you decide. For the sentiment model used here, the card states that it was fine-tuned on a sentiment dataset called SST-2, reaches 91.3% accuracy on its validation set against 92.7% for the larger BERT model, is licensed under Apache 2.0, and has a section on risks and biases (Hugging Face: DistilBERT SST-2 model card). The distillation that produced it cut BERT's size by 40% while keeping 97% of its language understanding (Sanh et al.: DistilBERT).

Read every card for:

1. The task and labels. A two-label sentiment model cannot say "neutral", as the output showed.
2. The training data, which decides the language and domain it understands.
3. Reported scores, and on what data.
4. The licence. Some models may not be used commercially.
5. Size, which decides whether it runs on your hardware at all.
6. Stated limitations.

Where this fits

Transformers is the common layer. Models defined in it can be fine-tuned with the PEFT library, see LoRA and QLoRA, and served at scale by engines such as vLLM, see vLLM and model serving. Sentence embedding models from the Hub power search and RAG, see embeddings explained. For running chat models locally with less code, see the Ollama tutorial.

Common mistakes

• Relying on a task's default model.
• Pairing a model with a different model's tokenizer.
• Skipping the model card, and discovering the licence or the label set too late.
• Reading confidence as accuracy.
• Downloading a large model to a machine that cannot hold it. Check the size first.
• Running the pipeline once per item in a large job, instead of passing batches.

Hugging Face Transformers is one of the frameworks taught in the Gen AI Frameworks and Tools module of the Generative AI course.

Sources

• Transformers (Hugging Face documentation) — Hugging Face
• Pipeline (Hugging Face documentation) — Hugging Face
• Quickstart (Hugging Face documentation) — Hugging Face
• DistilBERT base uncased finetuned SST-2 model card (Hugging Face) — Hugging Face
• DistilBERT, a distilled version of BERT (arXiv) — Sanh et al.
• GPT-2 model card (Hugging Face) — OpenAI community on Hugging Face