Load DeepSeek-V3 Model from Local Repo with Hugging Face Transformers

Direct Answer

Yes, you can load a DeepSeek-V3 model from a local directory using the Hugging Face Transformers library (version 4.51.0 or later) by passing the filesystem path directly to the model parameter in the pipeline function. Instead of using the model name "deepseek-ai/DeepSeek-R1", provide the absolute or relative path to the folder containing the downloaded model files, such as "/my-models/deepseek-r1". The pipeline function will first attempt to load from the local filesystem, and if the path is not found, it will fall back to searching the Hugging Face Hub.

The Full Answer

Prerequisites

Before you can load a DeepSeek-V3 model from a local repository, you need the following:

• Hugging Face Transformers library version 4.51.0 or later. This version introduced support for DeepSeek-V3 model architectures. You can check your installed version with pip show transformers and upgrade if needed using pip install --upgrade transformers.
• The model files downloaded from Hugging Face. This includes all files from the model repository, such as:
  • config.json (model configuration)
  • model.safetensors or pytorch_model.bin (model weights)
  • tokenizer.json or tokenizer_config.json (tokenizer files)
  • generation_config.json (generation parameters)
  • Any additional JSON files that describe the model architecture for loading (e.g., modeling_deepseek.py or custom code files)
• Trust remote code enabled. DeepSeek models often require custom modeling code that is not part of the standard Transformers library. You must set trust_remote_code=True when loading the model.

Step-by-Step: Loading from a Local Path

1. Download the model repository to a local directory. For example, if you have cloned or downloaded the DeepSeek-R1 repository from Hugging Face, it might be located at /my-models/deepseek-r1. Ensure the folder contains all necessary files. The official repository is deepseek-ai/DeepSeek-R1, but the same approach works for DeepSeek-V3 variants.

2. Use the pipeline function with the local path. The following code snippet demonstrates how to load the model from a local directory and run inference:

from transformers import pipeline

# Define your conversation messages
messages = [
    {"role": "user", "content": "Who are you?"},
]

# Load the pipeline using a local filesystem path
pipe = pipeline(
    "text-generation",
    model="/my-models/deepseek-r1",
    trust_remote_code=True
)

# Generate a response
output = pipe(messages)
print(output)

What happens under the hood:

• The pipeline function checks if the string passed to model is a valid filesystem path. If it is, it loads the model from that path.
• If the path does not exist on the filesystem, the function falls back to searching the Hugging Face Hub for a model with that name.
• The trust_remote_code=True flag is required because DeepSeek models use custom modeling code that is not included in the standard Transformers library. This flag allows the library to execute code from the model repository (e.g., modeling_deepseek.py).

3. Run inference. The pipeline object handles tokenization, model inference, and decoding automatically. You can pass a list of messages (as shown) or a single string.

Alternative: Using the AutoModel and AutoTokenizer Directly

If you need more control over the loading process (e.g., for custom generation parameters or batch processing), you can use the AutoModelForCausalLM and AutoTokenizer classes directly:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "/my-models/deepseek-r1"

# Load tokenizer and model from local path
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)

# Tokenize input
inputs = tokenizer("Who are you?", return_tensors="pt")

# Generate output
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

This approach gives you direct access to the model and tokenizer objects, which is useful for debugging or when you need to modify generation parameters (e.g., temperature, top_p, repetition_penalty).

When to Use Each Approach

• Use the pipeline approach when you want a quick, high-level interface for text generation. It handles all the details (tokenization, device placement, decoding) automatically. This is ideal for prototyping or simple applications.
• Use the AutoModel/AutoTokenizer approach when you need fine-grained control over the generation process, such as setting specific generation parameters, handling multiple sequences, or integrating with custom inference loops.

Important Notes

• Path must point to the repository root, not to individual files. The folder should contain all the files from the Hugging Face repository, including JSON configuration files and any custom Python modules.
• The model files must be complete. Missing files (e.g., config.json or tokenizer.json) will cause loading errors. The error message typically indicates which file is missing.
• DeepSeek-V3 vs DeepSeek-R1: The question specifically mentions DeepSeek-V3, but the accepted solution uses the DeepSeek-R1 repository as an example. Both models follow the same loading pattern. If you have downloaded DeepSeek-V3 files, replace the path accordingly.