EgorKodin/Huihui-gemma-4-12B-it-abliterated-mlx-4bit

Huihui-gemma-4-12B-it-abliterated MLX 4-bit

This repository contains an MLX 4-bit quantized conversion of huihui-ai/Huihui-gemma-4-12B-it-abliterated.

Model details

• Base model: huihui-ai/Huihui-gemma-4-12B-it-abliterated
• Format: MLX
• Quantization: 4-bit
• Quantization group size: 64
• Quantization mode: affine
• Compute dtype: bfloat16
• Library: mlx-vlm
• Task: image-text-to-text / vision-language generation

Conversion command

This MLX version was created with:

mlx_vlm.convert \
  --hf-path ./Huihui-gemma-4-12B-it-abliterated-hf \
  --mlx-path ./Huihui-gemma-4-12B-it-abliterated-mlx-4bit \
  -q \
  --q-bits 4 \
  --q-group-size 64 \
  --q-mode affine \
  --dtype bfloat16

Notes

This is an abliterated model. Its refusal and safety behavior may differ from the original model.

Use responsibly. Test the model carefully before using it in public-facing, automated, or production environments.

License

This model follows the license terms of the original base model:

huihui-ai/Huihui-gemma-4-12B-it-abliterated

Please review the original model card and license before use.