Instructions to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="mistralai/Ministral-3-14B-Instruct-2512-GGUF",
	filename="Ministral-3-14B-Instruct-2512-BF16-mmproj.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with Ollama:
```
ollama run hf.co/mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M
```

Unsloth Studio

How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mistralai/Ministral-3-14B-Instruct-2512-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mistralai/Ministral-3-14B-Instruct-2512-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mistralai/Ministral-3-14B-Instruct-2512-GGUF to start chatting

How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with Docker Model Runner:
```
docker model run hf.co/mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M
```

Lemonade

How to use mistralai/Ministral-3-14B-Instruct-2512-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mistralai/Ministral-3-14B-Instruct-2512-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Ministral-3-14B-Instruct-2512-GGUF-Q4_K_M

List all available models

lemonade list

Ministral 3 14B Instruct 2512 GGUF

The largest model in the Ministral 3 family, Ministral 3 14B offers frontier capabilities and performance comparable to its larger Mistral Small 3.2 24B counterpart. A powerful and efficient language model with vision capabilities.

This model includes different quantization levels of the instruct post-trained version in GGUF, fine-tuned for instruction tasks, making it ideal for chat and instruction based use cases.

The Ministral 3 family is designed for edge deployment, capable of running on a wide range of hardware. Ministral 3 14B can even be deployed locally, capable of fitting in 24GB of VRAM in FP8, and less if further quantized.

Learn more in our blog post and paper.

Key Features

Ministral 3 14B consists of two main architectural components:

13.5B Language Model
0.4B Vision Encoder

The Ministral 3 14B Instruct model offers the following capabilities:

Vision: Enables the model to analyze images and provide insights based on visual content, in addition to text.
Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
System Prompt: Maintains strong adherence and support for system prompts.
Agentic: Offers best-in-class agentic capabilities with native function calling and JSON outputting.
Edge-Optimized: Delivers best-in-class performance at a small scale, deployable anywhere.
Apache 2.0 License: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
Large Context Window: Supports a 256k context window.

Recommended Settings

We recommend deploying with the following best practices:

System Prompt: Define a clear environment and use case, including guidance on how to effectively leverage tools in agentic systems.
Sampling Parameters: Use a temperature below 0.1 for daily-driver and production environments ; Higher temperatures may be explored for creative use cases - developers are encouraged to experiment with alternative settings.
Tools: Keep the set of tools well-defined and limit their number to the minimum required for the use case - Avoiding overloading the model with an excessive number of tools.
Vision: When deploying with vision capabilities, we recommend maintaining an aspect ratio close to 1:1 (width-to-height) for images. Avoiding the use of overly thin or wide images - crop them as needed to ensure optimal performance.

License

This model is licensed under the Apache 2.0 License.

You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.