This is just a reupload of the gemma-3-4b-it-qat-q4_0 model. It can be installed by ollama run hf.co/kreier/gemma3.
The reason for this reupload is the conflicting login requirements for huggingface when downloading the original model and the use of huggingface-cli. I'm logged in in both and have access, but the combination does not work.
It is different from the general gemma3 model in that it is a 4B instruction-tuned version of the Gemma 3 model in GGUF format using Quantization Aware Training (QAT). The GGUF corresponds to Q4_0 quantization.
See more details here: https://huggingface.co/google/gemma-3-4b-it-qat-q4_0-gguf. It states:
Thanks to QAT, the model is able to preserve similar quality as
bfloat16while significantly reducing the memory requirements to load the model.
- Downloads last month
- 4
4-bit