How to import custom GGUF model files into Ollama

What this does

Registers a downloaded GGUF model file as a runnable model in Ollama's local library. After completing this guide the custom model will appear in ollama list and respond to inference requests alongside library-sourced models.

Steps

1. Place the GGUF file in a working directory. Creates a clean directory for the Modelfile and model blob.

   mkdir -p ~/ollama-models/my-model && cp /path/to/model.gguf ~/ollama-models/my-model/
   

2. Create a Modelfile referencing the GGUF file. The FROM directive must point to the GGUF relative path.

   cat > ~/ollama-models/my-model/Modelfile << 'EOF'
   FROM ./model.gguf
   TEMPLATE """{{ .Prompt }}"""
   PARAMETER temperature 0.7
   PARAMETER num_ctx 4096
   EOF
   

3. Register the model with Ollama. Analyzes the GGUF metadata and adds the model to the local registry.

   cd ~/ollama-models/my-model && ollama create my-custom-model -f Modelfile
   

   Expected output: transferring model data followed by success.

4. Verify the model is available. Confirms the model registered correctly.

   ollama list
   

   Expected output: A row with name my-custom-model and a valid size.

Verification

ollama run my-custom-model "Hello, describe yourself in one sentence."
# Expected: a coherent text response generated by the imported model

Common failures

• FROM path error - Relative path in FROM is incorrect; use an absolute path to the GGUF file.
• unable to determine architecture - GGUF format is not recognized; verify the file is a valid GGUF and not a PyTorch or Safetensors checkpoint.
• disk full during create - Ollama copies the GGUF internally; free space before retrying.
• template mismatch - Output is garbled or repetitive; adjust the TEMPLATE directive for the specific model architecture.

Operator checkpoint

Before treating this as solved, write down the local runtime, model or package version, hardware/backend if relevant, and the verification output. This keeps the guide useful as a Will-It-Run style decision instead of a one-off command transcript.

Related guides

• How to create a Modelfile to customize model parameters
• How to verify model integrity after downloading