Timon

At 4 bit or lower use IQ-type quant. The math is more advanced and quantization error is lower. You can double the context with -ctk q8_0 -ctv q8_0for virtually no loss of quality and speed.

New activity in unsloth/gemma-4-26B-A4B-it-GGUF 6 months ago

Gemma 4 seems to work best with high temperature for coding

👍 1

#21 opened 6 months ago by

Reverger

New activity in mistralai/Mistral-Small-4-119B-2603 7 months ago

Recommended sampler?

#4 opened 7 months ago by

mratsim

New activity in bartowski/Qwen_Qwen3.5-397B-A17B-GGUF 7 months ago

Older quants get in the way

#1 opened 7 months ago by

KeyboardMasher

New activity in ggml-org/gpt-oss-120b-GGUF about 1 year ago

Error with built-in Web UI

#3 opened about 1 year ago by

KeyboardMasher

New activity in legraphista/Llama-3.2-3B-Instruct-IMat-GGUF over 1 year ago

Thanks for IQ4_NL

❤️ 1

#1 opened over 1 year ago by

KeyboardMasher

Timon

AI & ML interests

Recent Activity

Organizations

KeyboardMasher's activity

Thanks for the V2 lora.

Unsloth fork only

KeyboardMasher/Step-3.5-Flash-MTP-GGUF-backup

KeyboardMasher/Step-3.5-Flash-MTP-GGUF-backup

Any idea when is the PR for llama.cpp comming?

Update metadata (tokenizer.ggml.model)

Poor performance

Temperature is last for Llama.cpp

Correct Samplers Order?

mmproj files are missing

version for 16 GB VRAM

token_embd.weight is quantized too much

Gemma 4 seems to work best with high temperature for coding

Recommended sampler?

Older quants get in the way

Error with built-in Web UI

Thanks for IQ4_NL