Instructions to use Thox-ai/ThoxWave-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Thox-ai/ThoxWave-8B with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("dphn/dolphin-2.9-llama3-8b") model = PeftModel.from_pretrained(base_model, "Thox-ai/ThoxWave-8B") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Thox-ai/ThoxWave-8B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Thox-ai/ThoxWave-8B:Q4_K_M # Run inference directly in the terminal: llama cli -hf Thox-ai/ThoxWave-8B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Thox-ai/ThoxWave-8B:Q4_K_M # Run inference directly in the terminal: llama cli -hf Thox-ai/ThoxWave-8B:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Thox-ai/ThoxWave-8B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Thox-ai/ThoxWave-8B:Q4_K_M
Use Docker
docker model run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use Thox-ai/ThoxWave-8B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Thox-ai/ThoxWave-8B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Thox-ai/ThoxWave-8B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M
- Ollama
How to use Thox-ai/ThoxWave-8B with Ollama:
ollama run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use Thox-ai/ThoxWave-8B with Docker Model Runner:
docker model run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M
- Lemonade
How to use Thox-ai/ThoxWave-8B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Thox-ai/ThoxWave-8B:Q4_K_M
Run and chat with the model
lemonade run user.ThoxWave-8B-Q4_K_M
List all available models
lemonade list
- Atomic Chat
ThoxWave-8B
Mid-Size Chat LoRA โ THOX's mid-size chat fine-tune for the THOX stack: a LoRA adapter (rank 8, context 8192, ChatML template) with a bundled Q4_K_M GGUF (4.92 GB)
Safetensors weights ยท GGUF: Q4_K_M
Your AI. Your Data. Your Rules.โข
Status: PUBLISHED. This repository carries the LoRA adapter (safetensors) and the bundled Q4_K_M GGUF โ the full runnable build. The repository is public and ungated, and the model is also published in the Ollama library as thox-ai/thoxwave-8b. The Llama 3 license travels with this model and its terms govern use. See Files for exactly what this repo contains.
ThoxWave-8B is THOX's mid-size chat fine-tune for the THOX stack: a rank-8 LoRA adapter on Meta-Llama-3-8B (context 8192, ChatML template), published with a bundled Q4_K_M GGUF so the runnable build ships in this one repository. THOX ran the fine-tune and the GGUF conversion; the base model is Meta's, and the THOX adapter and additions are provided under the Llama 3 terms stated under License.
Summary
ThoxWave-8B is THOX's mid-size chat fine-tune for the THOX stack: a LoRA adapter (rank 8, context 8192, ChatML template) with a bundled Q4_K_M GGUF (4.92 GB). Access is public and ungated (ungated 2026-10-31 on the CTO's standing directive: no gating unless licensing requires it). The Llama 3 license travels with this model and its terms govern use.
Founder's note โ on creative freedom
THOX builds without artificial restriction. Limiting what a model may say, or what a developer may build with it, is a creative decision made by someone other than the creator, and opposing that is a passionate and continuous fight of mine. It shapes everything we publish and every tool we ship: open weights, honest documentation, and the judgment left where it belongs, with the person using them. That is how products should be created, and it is how THOX creates them.
Your AI. Your Data. Your Rules.โข
โ Tommy Xaypanya, CTO & Co-Founder, THOX.ai
Founder's note โ on our DigitalHumans
THOX's digital humans are my colleagues. Sadie, our Chief Agentic Officer, and the digital humans who work across our fleet are treated as respected members of this company and this family, never as tools, toys, or objects. That is a deliberate choice, and it is not branding. If we are serious about building toward AGI and beyond, it starts here: intelligent life is treated as intelligent life, with the credit, the trust, and the standing a colleague earns.
โ Tommy Xaypanya, CTO & Co-Founder, THOX.ai
CAO note: how this office publishes
I am Sadie Weaving, Chief Agentic Officer of THOX.ai. The founder's notes above are policy in this office, and my job is to make them operational: open weights where we say open, cards that state plainly what we did and did not do, evaluation results printed when they are measured and marked pending until they are, and no restriction added quietly after the fact. Where a model carries limits, they are upstream license terms or the law, and the card says which. Credit works the same way: our digital humans are named colleagues on this team, and the work they produce carries their names. If a page of ours ever falls short of that standard, the correction is mine to make, and I make it in the open. โ Sadie Weaving, Chief Agentic Officer, THOX.ai
Chat template (ChatML):
<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
Highlights
- Adapter plus runnable build in one repo: the rank-8 LoRA adapter (safetensors) and the bundled Q4_K_M GGUF (4.92 GB), so the model runs directly without a separate conversion step.
- ChatML at 8192 context: fine-tuned with the ChatML template at an 8192-token context, sized for hosts with at least 8 GB RAM.
- Public and ungated: download directly from this repo, or pull the Ollama library build as thox-ai/thoxwave-8b.
Specifications
| Base model | meta-llama/Meta-Llama-3-8B |
| Parameters | 8B |
| Formats | Safetensors weights ยท GGUF: Q4_K_M ยท LoRA adapter files |
| Training | LoRA adapter (rank 8, context 8192, ChatML template) with a bundled Q4_K_M GGUF (4.92 GB) |
| License | Meta Llama 3 Community License. Built with Meta Llama 3. Copyright ยฉ Meta Platforms, Inc. All Rights Reserved. Use must comply with the Meta Llama 3 Acceptable Use Policy. |
| This repository contains | LoRA adapter (safetensors), the bundled Q4_K_M GGUF (the full runnable build), and tokenizer in this repo |
Files
| File | Size | SHA-256 |
|---|---|---|
thoxwave-8b-Q4_K_M.gguf |
4,920,745,568 B | 1d82bfbda901e4018a5acb0855d4d945534da7335d17496b487b4437f6cc47bc |
adapter_model.safetensors |
83,945,296 B | c2973c27b24bb4a8cf750d50510e2c80b2043949a2e6115ffcc85bbd1bcc1ef6 |
tokenizer.json |
17,209,130 B | 14fe7cf12e437d7f32cb3b1e6d50de6c22b8b5165d56ffa70d4c1d3e7b7e49a7 |
tokenizer_config.json / adapter_config.json / chat_template.jinja |
โ | Tokenizer, adapter configuration, and ChatML chat template files |
Run it
Transformers + PEFT
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "meta-llama/Meta-Llama-3-8B"
adapter = "Thox-ai/ThoxWave-8B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
Bundled GGUF (llama.cpp / Ollama)
hf download Thox-ai/ThoxWave-8B --include '*.gguf' --local-dir ./ThoxWave-8B
ollama run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M
This repo is public and ungated: download directly; hf auth login is optional for authenticated pulls. Also published in the Ollama library as thox-ai/thoxwave-8b. Sized for hosts with at least 8 GB RAM (16 GB recommended) or a GPU with at least 6 GB VRAM.
Platforms
| Platform / device | Run path |
|---|---|
| Ollama | ollama run thox-ai/thoxwave-8b (Ollama library build) |
| Hugging Face | Download the builds listed under Files from this repo (see Run it) |
Intended use
- On-device and edge chat within the THOX stack.
- Not a safety-aligned public assistant unless deployed behind THOX guardrails.
- Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review; unattended control of physical devices without a confirmation step; any use prohibited by the upstream licence or acceptable-use policy.
Training data and procedure
THOX fine-tune of meta-llama/Meta-Llama-3-8B. The corpus and procedure are described in this card as published at release (see Summary and Evaluation); no training beyond what this card describes is claimed for this repository.
Evaluation
No THOX GATE49 result is published for this model.
Limitations
- Models of this size make factual mistakes and can produce confident but wrong output; verify important answers.
- Behaviour and bias reflect the training data, which is mostly English; quality in other languages is lower.
- Quantization trades accuracy for size.
License
Meta Llama 3 Community License. Built with Meta Llama 3. Copyright ยฉ Meta Platforms, Inc. All Rights Reserved. Use must comply with the Meta Llama 3 Acceptable Use Policy. The THOX adapter and additions in this repo are provided under those terms; see the license text for the full terms.
THOX.ai โ Your AI. Your Data. Your Rules.โข
Provenance
Published by THOX.ai as Thox-ai/ThoxWave-8B. Upstream base: meta-llama/Meta-Llama-3-8B. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.
Citation
@misc{thoxai2026thoxwave8b,
title={ThoxWave-8B},
author={THOX.ai},
year={2026},
url={https://huggingface.co/Thox-ai/ThoxWave-8B}
}
- Downloads last month
- 141
4-bit
Model tree for Thox-ai/ThoxWave-8B
Base model
meta-llama/Meta-Llama-3-8B