Quantization - Python SDK

Quantization method reference

The Python SDK and docs are currently in beta. Report issues on GitHub.

Values

NameValue
INT4int4
INT8int8
FP4fp4
FP6fp6
FP8fp8
FP16fp16
BF16bf16
FP32fp32
UNKNOWNunknown