Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Save and Load Local Models with Python Transformers

Use save_pretrained and from_pretrained in Python to save and reload fine-tuned Hugging Face models from a local directory, including what gets written and common failure modes.

~6 min readPython
#transformers#save_pretrained#from_pretrained#Hugging Face
Illustration of a Python transformers model being saved to a local directory and reloaded, with weight files and configuration represented as labeled blocks.

After you fine-tune a Hugging Face transformers model, the trained weights stay in memory until you persist them. The two methods that handle local persistence are save_pretrained and from_pretrained: the first writes a model and its configuration to a directory on disk, and the second reconstructs the model from that directory. Knowing what each method writes, reads, and skips is what makes a local model workflow reproducible when you move directories or change machines.

What save_pretrained Actually Writes to Disk

Calling model.save_pretrained('./my-model') writes a directory containing the files needed to reload the model weights:

  • config.json — the model configuration, including the architecture type, hidden dimensions, vocabulary size, and any custom attributes you set when constructing the model.
  • model.safetensors — the serialized weight tensors. With the safetensors package installed, this is the default format in current releases of transformers. If you pass safe_serialization=False, the weights are written as pytorch_model.bin in the older pickle-based format instead.

Checkpoints that exceed the shard size are split into shards. A large checkpoint may produce files named model-00001-of-00003.safetensors, model-00002-of-00003.safetensors, and so on, controlled by the max_shard_size argument. The directory also includes a shard index file such as model.safetensors.index.json; from_pretrained uses that index, not config.json, to reassemble the shards.

Generation-capable models may also produce a generation_config.json in the same directory.

Crucially, save_pretrained on the model does not save the tokenizer. The directory becomes a complete, loadable checkpoint only when the tokenizer files are written alongside the weights.

Saving a Model and Tokenizer to a Local Directory

The standard pattern after fine-tuning is to save both the model and its tokenizer into the same directory:

python
from transformers import AutoModelForSequenceClassification, AutoTokenizer model = AutoModelForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2) tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased') # ... fine-tuning happens here ... model.save_pretrained('./my-finetuned-model') tokenizer.save_pretrained('./my-finetuned-model')

The tokenizer writes files such as tokenizer_config.json, vocab.txt, tokenizer.json, special_tokens_map.json, and others, depending on the tokenizer type. Keeping them in the same directory as the weights makes the directory self-contained: you can copy it to another machine or upload it to the Hub and load everything from one path.

If you save the model without the tokenizer, loading the directory later still works for the model itself, but you will need to reconstruct the tokenizer from its original source or from a separate saved copy.

Loading the Model Back with from_pretrained

Loading from a local directory uses the same API as loading from the Hub:

python
from transformers import AutoModelForSequenceClassification, AutoTokenizer model = AutoModelForSequenceClassification.from_pretrained('./my-finetuned-model') tokenizer = AutoTokenizer.from_pretrained('./my-finetuned-model')

The loader reads config.json first, then instantiates the requested model class and loads the weight tensors. The class you call from_pretrained on must be compatible with the saved architecture. For example, loading a checkpoint saved from BertForSequenceClassification with BertModel ignores the task-specific classification head weights and usually emits warnings about unused weights. Conversely, loading a base BertModel checkpoint with BertForSequenceClassification leaves the classification head randomly initialized because those weights were never saved.

When the weights are sharded, from_pretrained reads the shard index file and loads each shard in sequence. By default, when both model.safetensors and pytorch_model.bin are present, the safetensors file takes precedence.

What Is Not Saved by save_pretrained

save_pretrained persists the model weights and configuration only. Several pieces of state that matter in a training workflow are deliberately left out:

  • Optimizer state — Adam momentum and variance buffers are not written. If you need to resume training exactly where you left off, use the Trainer's checkpoint mechanism (output_dir/checkpoint-*), which saves the optimizer, scheduler, and training arguments alongside the model.
  • Tokenizer — as noted above, this requires a separate tokenizer.save_pretrained call.
  • Training arguments and hyperparameters — the Trainer stores these in trainer_state.json and training_args.bin in its output directory, not in the directory created by model.save_pretrained.
  • Custom model code — if you defined a custom architecture in a Python script, the weights and config are saved, but the class definition is not. Loading such a checkpoint requires the class definition to be importable at load time. If the repository contains the custom modeling code and the config has auto_map entries, you also need trust_remote_code=True to load that code.

This distinction matters when you plan to share a checkpoint. A directory containing only config.json and model.safetensors is a weights checkpoint, not a complete reproducible artifact.

How from_pretrained Resolves Its Argument

The first positional argument to from_pretrained is either a local directory path or a Hub repository identifier. The method checks whether the argument points to an existing local directory first; if it does, it loads from disk without contacting the Hub. If not, it treats the argument as a Hub repo ID and downloads the files into the Hugging Face cache.

For offline environments, local_files_only=True prevents network access: if the file is not already local or cached, the method raises instead of attempting a download.

python
model = AutoModel.from_pretrained('./my-finetuned-model', local_files_only=True)

This is useful in CI pipelines or air-gapped deployments where an accidental network call would hang or raise a connection error. The same flag applies when loading from the Hub cache: if the model is already cached, local_files_only=True loads it from cache without checking for updates.

Common Failure Modes and How to Diagnose Them

The most frequent errors with local model loading fall into a few categories.

Architecture mismatch. Loading a checkpoint with a model class that is incompatible with the saved architecture produces warnings like Some weights of the model checkpoint were not used or shape mismatches. Load with the same class that saved the checkpoint, or use a compatible auto class for the task.

Missing tokenizer files. If the directory contains weights but no vocab.txt or tokenizer.json, AutoTokenizer.from_pretrained raises an error. The directory is incomplete; re-save the tokenizer into it.

Relative path issues. save_pretrained('./model') writes relative to the current working directory. If a script later runs from a different directory, from_pretrained('./model') fails because the path no longer resolves. Using an absolute path or resolving the path at runtime avoids this.

Custom code without trust_remote_code. When you use an auto class, a checkpoint whose config.json contains auto_map entries requires trust_remote_code=True. Without it, transformers raises an error rather than executing arbitrary code from the repository.

Neither safetensors nor bin present. If the directory has a config.json but no weight files, from_pretrained fails with a clear error about missing weights. This usually happens when only the config was saved, or when the weights were written to a different directory.

Operational Considerations: Format, Disk, and Memory

The choice between safetensors and pickle is not cosmetic. pytorch_model.bin is pickle-based, so loading a checkpoint from an untrusted source carries code-execution risk if the pickle is processed without safety restrictions. model.safetensors uses a flat binary format that only stores tensors, making it the safer default for shared or downloaded checkpoints.

Disk usage is straightforward: a checkpoint occupies roughly the size of the model weights in float32, or half that for fp16/bf16. A 7B-parameter model in fp16 is about 14 GB on disk when unsharded; with sharding, the same data is split across multiple files but the total is unchanged.

Memory during loading deserves attention for large models. from_pretrained materializes the full weight tensors in RAM before moving them to the device. For a model that exceeds available RAM, loading fails with an out-of-memory error. Two options exist: low_cpu_mem_usage=True avoids the default full in-RAM materialization and reduces peak CPU memory, and device_map='auto' (with the accelerate library) distributes layers across available devices. Both are relevant when loading a local checkpoint that is large relative to the host machine.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.