PaddleOCR-VL-1.6 — 6-bit (MLX)
A 6-bit, group-size-64 MLX quantization of
PaddlePaddle/PaddleOCR-VL-1.6, for
fast on-device OCR on Apple Silicon. The architecture is unchanged — this repo only re-quantizes
the original weights.
This is the size/quality sweet spot: in side-by-side OCR benchmarking it matches the 8-bit variant's quality, and — unlike 4-bit — it does not slip into repetition / hallucination loops on hard scripts, at a smaller footprint than 8-bit.
| Base model | PaddlePaddle/PaddleOCR-VL-1.6 |
| Quantization | 6-bit, group size 64, affine (≈7.9 bits/weight) |
| Format | MLX safetensors |
| Size | ~0.9 GB |
Other tiers: 8-bit (slightly larger, same quality) and 4-bit (smallest, but prone to loops on hard/rare scripts).
License & attribution
Apache-2.0, inherited from the base model. All credit for the model goes to the PaddlePaddle / PaddleOCR team — this repository only provides an MLX-quantized copy.
- Downloads last month
- 278
Model size
0.3B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
6-bit
Model tree for translate-studio/PaddleOCR-VL-1.6-6bit
Base model
baidu/ERNIE-4.5-0.3B-Paddle Finetuned
PaddlePaddle/PaddleOCR-VL-1.5 Finetuned
PaddlePaddle/PaddleOCR-VL-1.6
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("translate-studio/PaddleOCR-VL-1.6-6bit") config = load_config("translate-studio/PaddleOCR-VL-1.6-6bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output)