Search results

ggml-quants : weighted rounding algorithms with cumulative search

generation qualityQuality of model outputQuality of model output

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Less than 4 bitsEfforts related to viable quantized models using <4 bitsEfforts related to viable quantized models using <4 bits

research 🔬

Review Complexity : MediumGenerally require more time to grok but manageable by beginner to medium expertise levelGenerally require more time to grok but manageable by beginner to medium expertise level

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Draft (not ready).

ggml-org/llama.cppnumber 12557

#12557 In ggml-org/llama.cpp;

· compilade opened on Mar 25, 2025

Introduce New Lookup-Table(LUT)-Based Matrix Multiplication Method

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

pythonpython script changespython script changes

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Open (in progress).

ggml-org/llama.cppnumber 10181

#10181 In ggml-org/llama.cpp;

· QingtaoLi1 opened on Nov 5, 2024

add FP8 support to gguf/llama:

buildCompilation issuesCompilation issues

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

scriptScript relatedScript related

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

testingEverything test relatedEverything test related

Status: Draft (not ready).

ggml-org/llama.cppnumber 10055

#10055 In ggml-org/llama.cpp;

· Djip007 opened on Oct 26, 2024

Introduce Q8_0 and Q4_0 with Bf16 delta values

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

pythonpython script changespython script changes

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Open (in progress).

ggml-org/llama.cppnumber 7497

#7497 In ggml-org/llama.cpp;

· Srihari-mcw opened on May 23, 2024