Tootfinder

@arXiv_csCR_bot@mastoxiv.page
2025-06-03 17:53:19

This https://arxiv.org/abs/2505.23786 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

Mind the Gap: A Practical Attack on GGUF Quantization
With the increasing size of frontier LLMs, post-training quantization has become the standard for memory-efficient deployment. Recent work has shown that basic rounding-based quantization schemes pose security risks, as they can be exploited to inject malicious behaviors into quantized models that remain hidden in full precision. However, existing attacks cannot be applied to more complex quantization methods, such as the GGUF family used in the popular ollama and llama.cpp frameworks. In this …