ggml-org

/

GLM-4.6V-GGUF

like 6

Follow

ggml.ai 1.2k

I'm using latest llama.cpp docker image,
and getting this error:

clip_model_loader: has vision encoder
clip_ctx: CLIP using CUDA0 backend
clip_init: failed to load model '/root/.cache/llama.cpp/router/local-GLM4.6V-106b/mmproj-GLM-4.6V-Q8_0.gguf': load_hparams: unknown projector type: glm4v
mtmd_init_from_file: error: Failed to load CLIP model from /root/.cache/llama.cpp/router/local-GLM4.6V-106b/mmproj-GLM-4.6V-Q8_0.gguf

Running this:

docker run -it --rm --gpus all --name llama1
-v /home/slavik/.cache:/root/.cache
-p 8080:8080
--entrypoint ./llama-server
ghcr.io/ggml-org/llama.cpp:full-cuda --mmproj /root/.cache/llama.cpp/router/local-GLM4.6V-106b/mmproj-GLM-4.6V-Q8_0.gguf --model /root/.cache/llama.cpp/router/local-GLM4.6V-106b/ggml-org_GLM-4.6V-GGUF_GLM-4.6V-Q4_K_M.gguf --host 0.0.0.0 --port 8080

If I run it without mmproj - everything works.